Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] HVU CN README #738

Merged
merged 34 commits into from
Mar 24, 2021
Merged
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
c5f00a8
resolve comments
Oct 16, 2020
05575c1
update changelog
Oct 16, 2020
eb09070
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Oct 19, 2020
81302a4
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Oct 22, 2020
43a649a
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Oct 27, 2020
755809e
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Oct 29, 2020
d478c9d
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 2, 2020
08bbc06
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 7, 2020
ff958e6
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 8, 2020
d0e192d
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 11, 2020
a52c536
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 11, 2020
81a2029
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 16, 2020
e03d2a9
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 17, 2020
2a9b57f
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 27, 2020
28001ff
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Nov 30, 2020
46cc5dd
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Dec 1, 2020
667818a
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Dec 18, 2020
34398a8
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Dec 18, 2020
9928127
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Jan 5, 2021
b633443
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Jan 5, 2021
a891176
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
Jan 5, 2021
611ae31
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Jan 13, 2021
96662d4
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Jan 15, 2021
ef5ab3c
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Jan 25, 2021
0b63bd0
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Jan 27, 2021
d4450fc
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Jan 27, 2021
467b8d8
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Feb 1, 2021
1e2e566
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Feb 3, 2021
f91fe5e
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Mar 3, 2021
c2067b1
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Mar 5, 2021
0b923cf
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Mar 11, 2021
c181802
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Mar 12, 2021
8e59233
Merge branch 'master' of https://github.com/open-mmlab/mmaction2
kennymckormick Mar 18, 2021
d5218d4
update
kennymckormick Mar 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions tools/data/hvu/README_zh_CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# 准备 HVU 数据集

## 简介

[DATASET]

```BibTeX
@article{Diba2019LargeSH,
title={Large Scale Holistic Video Understanding},
author={Ali Diba and M. Fayyaz and Vivek Sharma and Manohar Paluri and Jurgen Gall and R. Stiefelhagen and L. Gool},
journal={arXiv: Computer Vision and Pattern Recognition},
year={2019}
}
```

请参照 [官方项目](https://github.com/holistic-video-understanding/HVU-Dataset/) 及 [原论文](https://arxiv.org/abs/1904.11451) 以获取数据集基本信息。
在开始之前,用户需确保当前目录为 `$MMACTION2/tools/data/hvu/`。

## 1. 准备标注文件

首先,用户可以使用如下脚本下载标注文件并进行预处理:

```shell
bash download_annotations.sh
```

此外,用户可使用如下命令解析 HVU 的标签列表:

```shell
python parse_tag_list.py
```

## 2. 准备视频

用户可以使用以下脚本准备视频,视频准备代码修改自 [ActivityNet 爬虫](https://github.com/activitynet/ActivityNet/tree/master/Crawler/Kinetics)。
注意这一步骤将花费较长时间。

```shell
bash download_videos.sh
```

## 3. 提取 RGB 帧和光流

如果用户仅使用 video loader,则可以跳过本步。

在提取之前,请参考 [安装教程](/docs_zh_CN/install.md) 安装 [denseflow](https://github.com/open-mmlab/denseflow)。

用户可使用如下脚本同时抽取 RGB 帧和光流:

```shell
bash extract_frames.sh
```

该脚本默认生成短边长度为 256 的帧,可参考 [数据准备](/docs_zh_CN/data_preparation.md) 获得更多细节。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

长度为 256 像素


## 4. 生成文件列表

用户可以使用以下两个脚本分别为视频和帧文件夹生成文件列表:

```shell
bash generate_videos_filelist.sh
# 为帧文件夹生成文件列表
bash generate_rawframes_filelist.sh
```

## 5. 为每个 tag 种类生成文件列表

若用户需要为 HVU 数据集的每个 tag 种类训练识别模型,则需要进行此步骤。

步骤 4 中生成的文件列表包含不同类型的标签,仅支持使用 HVUDataset 进行涉及多个标签种类的多任务学习。加载数据的过程中需要使用 `LoadHVULabel` 类进行多类别标签的加载,训练过程中使用 `HVULoss` 作为损失函数。

如果用户仅需训练某一特定类别的标签,例如训练一识别模型用于识别 HVU 中 `action` 类别的标签,则建议使用如下脚本为特定标签种类生成文件列表。新生成的列表将只含有特定类别的标签,因此可使用 `VideoDataset` 或 `RawframeDataset` 进行加载。训训练过程中使用 `BCELossWithLogits` 作为损失函数。

以下脚本为类别为 ${category} 的标签生成文件列表,注意仅支持 HVU 数据集包含的 6 种标签类别: action, attribute, concept, event, object, scene。

```shell
python generate_sub_file_list.py path/to/filelist.json ${category}
```

对于类别 ${category},生成的标签列表文件名中将使用 `hvu_${category}` 替代 `hvu`。例如,若原指定文件名为 `hvu_train.json`,则对于类别 action,生成的文件列表名为 `hvu_action_train.json`。

## 6. 目录结构

在完整完成 HVU 的数据处理后,将得到帧文件夹(RGB 帧和光流帧),视频以及标注文件。

在整个项目目录下(仅针对 HVU),完整目录结构如下所示:

```
mmaction2
├── mmaction
├── tools
├── configs
├── data
│ ├── hvu
│ │ ├── hvu_train_video.json
│ │ ├── hvu_val_video.json
│ │ ├── hvu_train.json
│ │ ├── hvu_val.json
│ │ ├── annotations
│ │ ├── videos_train
│ │ │ ├── OLpWTpTC4P8_000570_000670.mp4
│ │ │ ├── xsPKW4tZZBc_002330_002430.mp4
│ │ │ ├── ...
│ │ ├── videos_val
│ │ ├── rawframes_train
│ │ ├── rawframes_val

```

关于 HVU 数据集上的训练与测试,请参照 [基础教程](/docs_zh_CN/getting_started.md)。