Refine codes and docs (PaddlePaddle#61)

* refine codes docs

README.md

@@ -32,31 +32,10 @@ PaddleGAN 是一个基于飞桨的生成对抗网络开发工具包.
 
 ## 安装
 
-### 1. 安装 paddlepaddle
-
-PaddleGAN 所需的版本:
-* PaddlePaddle >= 2.0.0-rc
-* Python >= 3.5+
-
-```
-pip install -U paddlepaddle-gpu
-```
-
-### 2. 安装ppgan
-```
-python -m pip install 'git+https://github.com/PaddlePaddle/PaddleGAN.git'
-```
-
-或者通过将项目克隆到本地
-```
-git clone https://github.com/PaddlePaddle/PaddleGAN
-cd PaddleGAN
-
-pip install -v -e .  # or "python setup.py develop"
-```
+请参考[安装文档](./docs/install.md)来进行PaddlePaddle和ppgan的安装
 
 ## 数据准备
-请参考 [数据准备](./docs/data_prepare.md) 来准备对应的数据.
+请参考[数据准备](./docs/data_prepare.md) 来准备对应的数据.
 
 
 ## 快速开始

README_en.md

@@ -1,4 +1,4 @@
-English | [简体中文](./README_cn.md)
+English | [简体中文](./README.md)
 
 # PaddleGAN
 
@@ -35,35 +35,13 @@ changes.
 
 ## Install
 
-### 1. install paddlepaddle
-
-PaddleGAN work with:
-* PaddlePaddle >= 2.0.0-rc
-* Python >= 3.5+
-
-```
-pip install -U paddlepaddle-gpu
-```
-
-### 2. install ppgan
-
-```
-python -m pip install 'git+https://github.com/PaddlePaddle/PaddleGAN.git'
-```
-
-Or install it from a local clone
-```
-git clone https://github.com/PaddlePaddle/PaddleGAN
-cd PaddleGAN
-
-pip install -v -e .  # or "python setup.py develop"
-```
+Please refer to [install](./docs/install_en.md).
 
 ## Data Prepare
-Please refer to [data prepare](./docs/data_prepare.md) for dataset preparation.
+Please refer to [data prepare](./docs/data_prepare_en.md) for dataset preparation.
 
 ## Get Start
-Please refer [get started](./docs/get_started.md) for the basic usage of PaddleGAN.
+Please refer [get started](./docs/get_started_en.md) for the basic usage of PaddleGAN.
 
 ## Model tutorial
 * [Pixel2Pixel and CycleGAN](./docs/tutorials/pix2pix_cyclegan.md)

data/download_cyclegan_data.py

@@ -0,0 +1,30 @@
+import os
+import argparse
+
+from ppgan.utils.download import get_path_from_url
+
+CYCLEGAN_URL_ROOT = 'https://people.eecs.berkeley.edu/~taesung_park/CycleGAN/datasets/'
+
+parser = argparse.ArgumentParser(description='download datasets')
+parser.add_argument('--name',
+                    type=str,
+                    required=True,
+                    help='dataset name, \
+                    support dataset name: apple2orange, summer2winter_yosemite, \
+                    horse2zebra, monet2photo, cezanne2photo, ukiyoe2photo, \
+                    vangogh2photo, maps, cityscapes, facades, iphone2dslr_flower, \
+                    ae_photos, cityscapes')
+
+if __name__ == "__main__":
+    args = parser.parse_args()
+
+    data_url = CYCLEGAN_URL_ROOT + args.name + '.zip'
+
+    if args.name == 'cityscapes':
+        data_url = 'https://paddlegan.bj.bcebos.com/datasets/cityscapes.zip'
+
+    path = get_path_from_url(data_url)
+
+    dst = os.path.join('data', args.name)
+    print('symlink {} to {}'.format(path, dst))
+    os.symlink(path, dst)

data/download_pix2pix_data.py

@@ -0,0 +1,25 @@
+import os
+import argparse
+
+from ppgan.utils.download import get_path_from_url
+
+PIX2PIX_URL_ROOT = 'http://efrosgans.eecs.berkeley.edu/pix2pix/datasets/'
+
+parser = argparse.ArgumentParser(description='download datasets')
+parser.add_argument('--name',
+                    type=str,
+                    required=True,
+                    help='dataset name, \
+                    support dataset name: cityscapes, night2day, edges2handbags, \
+                    edges2shoes, facades, maps')
+
+if __name__ == "__main__":
+    args = parser.parse_args()
+
+    data_url = PIX2PIX_URL_ROOT + args.name + '.tar.gz'
+
+    path = get_path_from_url(data_url)
+
+    dst = os.path.join('data', args.name)
+    print('symlink {} to {}'.format(path, dst))
+    os.symlink(path, dst)

docs/data_prepare.md

@@ -1,6 +1,6 @@
 ## 数据准备
 
-推荐把数据集软链接到 `$PaddleGAN/data`. 软链接后的目录结构如下图所示：
+现有的配置默认数据集的路径是在`$PaddleGAN/data`下，目录结构如下图所示。如果你已经下载好数据集了，建议将数据集软链接到 `$PaddleGAN/data`。
 
 ```
 PaddleGAN
@@ -28,8 +28,65 @@ PaddleGAN
 
 ```
 
-### cyclegan 相关的数据集下载
+如果将数据集放在其他位置，比如 ```your/data/path```，
+你可以修改配置文件中的 ```dataroot``` 参数:
+
+```
+dataset:
+  train:
+    name: PairedDataset
+    dataroot: your/data/path
+    num_workers: 4
+```
+
+### CycleGAN模型相关的数据集下载
+
+#### 已有的数据集下载
+
+##### 从网页下载
 cyclgan模型相关的数据集可以在[这里](https://people.eecs.berkeley.edu/~taesung_park/CycleGAN/datasets/)下载
 
-### pix2pix 相关的数据集下载
+##### 使用脚本下载
+我们在 ```PaddleGAN/data``` 文件夹下提供了一个脚本 ```download_cyclegan_data.py``` 方便下载CycleGAN相关的
+数据集。执行如下命令可以下载相关的数据集，目前支持的数据集名称有：apple2orange, summer2winter_yosemite,horse2zebra, monet2photo, cezanne2photo, ukiyoe2photo, vangogh2photo, maps, cityscapes, facades, iphone2dslr_flower, ae_photos, cityscapes。
+
+执行如下命令，可以下载对应的数据集到 ```~/.cache/ppgan``` 并软连接到 ```PaddleGAN/data/``` 下。
+```
+python data/download_cyclegan_data.py --name horse2zebra
+```
+#### 使用自己的数据集
+如果你使用自己的数据集，需要构造成如下目录的格式。注意 ```xxxA```，```xxxB```文件数量，文件内容无需一一对应。
+```
+custom_datasets
+├── testA
+├── testB
+├── trainA
+└── trainB
+```
+
+### Pix2Pix相关的数据集下载
+
+#### 已有的数据集下载
+
+##### 从网页下载
 pixel2pixel模型相关的数据集可以在[这里](hhttps://people.eecs.berkeley.edu/~tinghuiz/projects/pix2pix/datasets/)下载
+
+##### 使用脚本下载
+我们在 ```PaddleGAN/data``` 文件夹下提供了一个脚本 ```download_pix2pix_data.py``` 方便下载pix2pix模型相关的数据集。执行如下命令可以下载相关的数据集，目前支持的数据集名称有：apple2orange, summer2winter_yosemite,horse2zebra, monet2photo, cezanne2photo, ukiyoe2photo, vangogh2photo, maps, cityscapes, facades, iphone2dslr_flower, ae_photos, cityscapes。
+
+执行如下命令，可以下载对应的数据集到 ```~/.cache/ppgan``` 并软连接到 ```PaddleGAN/data/``` 下。
+```
+python data/download_pix2pix_data.py --name cityscapes
+```
+
+#### 使用自己的数据集
+如果你使用自己的数据集，需要构造成如下目录的格式。同时图片应该制作成下图的样式，即左边为一种风格，另一边为相应转换的风格。
+
+```
+facades
+├── test
+├── train
+└── val
+```
+
+![](./imgs/1.jpg)

docs/data_prepare_en.md

@@ -1,6 +1,6 @@
 ## data prepare
 
-It is recommended to symlink the dataset root to `$PaddleGAN/data`.
+The config will suppose your data put in `$PaddleGAN/data`. You can symlink your datasets to `$PaddleGAN/data`.
 
 ```
 PaddleGAN
@@ -28,8 +28,65 @@ PaddleGAN
 
 ```
 
-### cyclegan datasets
-more dataset for cyclegan you can download from [here](https://people.eecs.berkeley.edu/~taesung_park/CycleGAN/datasets/)
+if you put your datasets on other place，for example ```your/data/path```,
+you can also change ```dataroot``` in config file:
 
-### pix2pix datasets
-more dataset for pix2pix you can download from [here](hhttps://people.eecs.berkeley.edu/~tinghuiz/projects/pix2pix/datasets/)
+```
+dataset:
+  train:
+    name: PairedDataset
+    dataroot: your/data/path
+    num_workers: 4
+```
+
+### Datasets of CycleGAN
+
+#### download existed datasets
+
+##### download form website
+datasets for CycleGAN you can download from [here](https://people.eecs.berkeley.edu/~taesung_park/CycleGAN/datasets/)
+
+##### download by script
+You can use ```download_cyclegan_data.py``` in ```PaddleGAN/data``` to download datasets you wanted. Supported datasets are: apple2orange, summer2winter_yosemite,horse2zebra, monet2photo, cezanne2photo, ukiyoe2photo, vangogh2photo, maps, cityscapes, facades, iphone2dslr_flower, ae_photos, cityscapes。
+
+run following command. Dataset will be downloaded to ```~/.cache/ppgan``` and symlink to ```PaddleGAN/data/``` .
+```
+python data/download_cyclegan_data.py --name horse2zebra
+```
+
+#### custom dataset
+Data should be arranged in following way if you use custom dataset.
+```
+custom_datasets
+├── testA
+├── testB
+├── trainA
+└── trainB
+```
+
+### Datasets of Pix2Pix
+
+#### download existed datasets
+
+##### download from website
+dataset for pix2pix you can download from [here](hhttps://people.eecs.berkeley.edu/~tinghuiz/projects/pix2pix/datasets/)
+
+##### download by script
+You can use ```download_pix2pix_data.py``` in ```PaddleGAN/data``` to download datasets you wanted. Supported datasets are: apple2orange, summer2winter_yosemite,horse2zebra, monet2photo, cezanne2photo, ukiyoe2photo, vangogh2photo, maps, cityscapes, facades, iphone2dslr_flower, ae_photos, cityscapes.
+
+run following command. Dataset will be downloaded to ```~/.cache/ppgan``` and symlink to ```PaddleGAN/data/``` .
+```
+python data/download_pix2pix_data.py --name cityscapes
+```
+
+#### custom datasets
+Data should be arranged in following way if you use custom dataset. And image content shoubld be same with example image.
+
+```
+facades
+├── test
+├── train
+└── val
+```
+
+![](./imgs/1.jpg)

docs/get_started.md

@@ -1,22 +1,71 @@
 
 ## 快速开始使用PaddleGAN
 
+注意：
+* 开始使用PaddleGAN前请确保已经阅读过[安装文档](./install.md)，并根据[数据准备文档](./data_prepare.md)准备好数据集。
+* 以下教程以CycleGAN模型在Cityscapes数据集上的训练预测作为示例。
+
+
 ### 训练
+
+#### 单卡训练
 ```
 python -u tools/main.py --config-file configs/cyclegan_cityscapes.yaml
 ```
+#### 参数
 
-continue train from last checkpoint
+- `--config-file (str)`: 配置文件的路径。
+
+ 输出的日志，权重，可视化结果会默认保存在```./output_dir```中，可以通过配置文件中的```output_dir```参数修改：
+ ```
+ output_dir: output_dir
+ ```
+
+ 保存的文件夹会根据模型名字和时间戳自动生成一个新目录，目录示例如下：
+```
+output_dir
+└── CycleGANModel-2020-10-29-09-21
+    ├── epoch_1_checkpoint.pkl
+    ├── log.txt
+    └── visual_train
+        ├── epoch001_fake_A.png
+        ├── epoch001_fake_B.png
+        ├── epoch001_idt_A.png
+        ├── epoch001_idt_B.png
+        ├── epoch001_real_A.png
+        ├── epoch001_real_B.png
+        ├── epoch001_rec_A.png
+        ├── epoch001_rec_B.png
+        ├── epoch002_fake_A.png
+        ├── epoch002_fake_B.png
+        ├── epoch002_idt_A.png
+        ├── epoch002_idt_B.png
+        ├── epoch002_real_A.png
+        ├── epoch002_real_B.png
+        ├── epoch002_rec_A.png
+        └── epoch002_rec_B.png
+```
+
+#### 恢复训练
+
+训练过程中默认会保存上一个epoch的checkpoint，方便恢复训练
 ```
 python -u tools/main.py --config-file configs/cyclegan_cityscapes.yaml --resume your_checkpoint_path
 ```
+#### 参数
 
-multiple gpus train:
+- `--resume (str)`: 用来恢复训练的checkpoint路径。
+
+#### 多卡训练:
 ```
-CUDA_VISIBLE_DEVICES=0,1 python -m paddle.distributed.launch tools/main.py --config-file configs/pix2pix_cityscapes.yaml
+CUDA_VISIBLE_DEVICES=0,1 python -m paddle.distributed.launch tools/main.py --config-file configs/cyclegan_cityscapes.yaml
 ```
 
 ### 预测
 ```
 python tools/main.py --config-file configs/cyclegan_cityscapes.yaml --evaluate-only --load your_weight_path
 ```
+
+#### 参数
+- `--evaluate-only`: 是否仅进行预测。
+- `--load (str)`: 训练好的权重路径。