Skip to content

Commit 0575058

Browse files
committed
docs: 🗃️ remove autodownload of metainfo and put them to huggingface
1 parent b718f54 commit 0575058

File tree

2 files changed

+12
-12
lines changed

2 files changed

+12
-12
lines changed

README.md

+12-4
Original file line numberDiff line numberDiff line change
@@ -136,26 +136,34 @@ Please refer to the [results calibration](./ResultsCalibra/ResultsCalibra.md) to
136136

137137
### ⏬ Download Benchmark Datasets
138138

139-
For convenience, we upload all related datasets to [huggingface](https://huggingface.co/datasets/chaofengc/IQA-Toolbox-Datasets/tree/main).
140-
Here are example codes to download datasets from huggingface:
139+
For convenience, we upload all related datasets to [huggingface IQA-Toolbox-Dataset](https://huggingface.co/datasets/chaofengc/IQA-Toolbox-Datasets), and corresponding meta information files to [huggingface IQA-Toolbox-Dataset-metainfo](https://huggingface.co/datasets/chaofengc/IQA-Toolbox-Datasets-metainfo).
140+
Here are example codes to download them from huggingface:
141141

142142
>[!CAUTION]
143143
> we only collect the datasets for academic, research, and educational purposes. It is important for the users to adhere to the usage guidelines, licensing terms, and conditions set forth by the original creators or owners of each dataset.
144144
145-
146145
```python
147146
import os
148147
from huggingface_hub import snapshot_download
149148

150149
save_dir = './datasets'
151150
os.makedirs(save_dir, exist_ok=True)
152151

153-
filename = "meta_info.tgz"
152+
filename = "koniq10k.tgz"
154153
snapshot_download("chaofengc/IQA-Toolbox-Datasets", repo_type="dataset", local_dir=save_dir, allow_patterns=filename, local_dir_use_symlinks=False)
155154

156155
os.system(f"tar -xzvf {save_dir}/{filename} -C {save_dir}")
157156
```
158157

158+
Download meta information from Huggingface with `git clone` or update with `git pull`:
159+
```
160+
cd ./datasets
161+
git clone https://huggingface.co/datasets/chaofengc/IQA-Toolbox-Datasets-metainfo meta_info
162+
163+
cd ./datasets/meta_info
164+
git pull
165+
```
166+
159167
Examples to specific dataset options can be found in `./options/default_dataset_opt.yml`. Details of the dataloader interface and meta information files can be found in [Dataset Preparation](docs/Dataset_Preparation.md)
160168

161169
### Performance Evaluation Protocol

pyiqa/train.py

-8
Original file line numberDiff line numberDiff line change
@@ -31,14 +31,6 @@ def init_tb_loggers(opt):
3131

3232

3333
def create_train_val_dataloader(opt, logger):
34-
# download meta informatioin for datasets if needed
35-
if not os.path.exists(f"{opt['root_path']}/datasets/meta_info"):
36-
logger.info('Downloading meta information for datasets.')
37-
os.makedirs(f"{opt['root_path']}/datasets", exist_ok=True)
38-
file_path = load_file_from_url('https://github.com/chaofengc/IQA-PyTorch/releases/download/v0.1-weights/meta_info.tgz', f"{opt['root_path']}/datasets")
39-
metainfo_file = tarfile.open(file_path, mode='r|gz')
40-
metainfo_file.extractall(f"{opt['root_path']}/datasets")
41-
4234
# create train and val dataloaders
4335
train_loader, val_loaders = None, []
4436
for phase, dataset_opt in opt['datasets'].items():

0 commit comments

Comments
 (0)