update code to the final ICCP version and update dataset link

Author: Guocheng Qian

Evaluate_PSNR_SSIM.m

@@ -4,10 +4,10 @@ function Evaluate_PSNR_SSIM()
 
 %%
 % div2k 
-% pretrain_dataset = 'pixelshift'; 
-% dataset = 'pixelshift'; 
-pretrain_dataset = 'div2k'; 
-dataset = 'urban100'; 
+pretrain_dataset = 'pixelshift'; 
+dataset = 'pixelshift'; 
+% pretrain_dataset = 'div2k'; 
+% dataset = 'urban100'; 
 % dataset = 'cbsd68'; 
 % dataset = 'set14'; 
 % dataset = 'div2k'; 
@@ -20,7 +20,9 @@ function Evaluate_PSNR_SSIM()
 
 
 % % tasks = {'resnet-dn+dm+sr-SR2', 'resnet-dn+sr-dm-SR2'}; 
-tasks = {'e2e-dn+dm+sr-SR2', 'e2e-dn+sr-dm-SR2'}; 
+% tasks = {'e2e-dn+dm+sr-SR2', 'e2e-dn+sr-dm-SR2'}; 
+% tasks = {'e2e-dn+sr-dm-SR2'}; 
+tasks = {'e2e-dn-sr-dm-SR2'}; 
 
 gt_path = fullfile('/home/qiang/codefiles/low_level/ISP/ispnet/data/benchmark/', dataset, 'gt');
 pred_dir = fullfile(fullfile('/home/qiang/codefiles/low_level/ISP/ispnet/pretrain/', pretrain_dataset, 'pipeline'), ['result_', dataset]);

README.md

@@ -1,51 +1,48 @@
 # TENet <a href="https://arxiv.org/abs/1905.02538" target="_blank">[PDF]</a> <a href="http://gcqian.com/project/pixelshift200">[pixelshift200]</a> 
 
 ### Rethink the Pipeline of Demosaicking, Denoising, and Super-resolution
-By [Guocheng Qian](https://guochengqian.github.io/), [Yuanhao Wang](https://github.com/yuanhaowang1213), Chao Dong, [Jimmy S. Ren](http://www.jimmyren.com/), Wolfgang Heidrich, Bernard Ghanem, [Jinjin Gu](http://www.jasongt.com/)
+By [Guocheng Qian*](https://guochengqian.github.io/), [Yuanhao Wang*](https://github.com/yuanhaowang1213), [Jinjin Gu](http://www.jasongt.com/), Chao Dong, Wolfgang Heidrich, Bernard Ghanem, [Jimmy S. Ren](http://www.jimmyren.com/)
 
 The original name of this project is: "Trinity of Pixel Enhancement: a Joint Solution for Demosaicking, Denoising and Super-Resolution"
 
 
 
-## pipeline DN -> SR -> DM
-
-![pipeline](misc/pipeline_result.png)
-
 ## TENet
 
 We insert the proposed pipeline DN -> SR -> DM into an end-to-end network constructed by RRDB for the joint DN, DM and SR. We leverage the detachable branch to provide the middle stage supervision. 
 
+
 <p align="center">
   <img height="300" src="misc/Network.png">
 </p>
 
 
-## PixelShift200 dataset
-
-![pixelshift](misc/PixelShift.png)
-
 
 
-## Resources
+## PixelShift200 dataset
 
-* pretrained models
-* Pixelshift200: 
-* Real-shot raw images:
+We employ advanced pixel shift technology to perform a full color sampling of the image.
+Pixel shift technology takes four samples of the same image, and physically controls the camera sensor to move one pixel horizontally or vertically at each sampling to capture all color information at each pixel.
+The pixel shift technology ensures that the sampled images follow the distribution of natural images sampled by the camera, and the full information of the color is completely obtained.
+In this way, the collected images are artifacts-free, which leads to better training results for demosaicing related tasks.
 
-Will be available soon. 
+<p align="center">
+  <img height="200" src="misc/PixelShift.png">
+</p>  
 
 
 
+Download dataset from [pxielshift200 website](http://gcqian.com/project/pixelshift200).
 
 
-### Enviroment installnation
+### Environment installation
 
 Clone this github repo and install the environment by:
 
-```shell
+```bash
 git clone https://github.com/guochengqian/TENet
 cd TENet
-source env_install.sh
+source install.sh
 conda activate tenet
 ```
 
@@ -57,108 +54,115 @@ conda activate tenet
 
    1. Download ([DIV2K](https://drive.google.com/file/d/1vXPPr2hVaMewz2JA1lFfI5uHB4ENwRXQ/view?usp=sharing)) dataset
 
-   2. create data directory in TENet folder: `mkdir data && cd data`
+   2. `mkdir data && cd data`
 
    3. Link DIV2K data into ./data/DIV2K, e.g. `ln -s /data/lowlevel/DIV2K ./` 
 
    4. Crop DIV2K
+      
       ```bash
       cd ../datasets
+      python crop_imgs.py # crop train images
       python crop_imgs.py --src_dir ../data/DIV2K/DIV2K_val5_HR --save_dir ../data/DIV2K/DIV2K_val5_HR_sub # crop val5 images
-      python generate_datalist_div2k.py  # generate div2k training and val dataset
       ```
-      The generated .txt file  `train_div2k.txt` and `val_div2k.txt` are used for training on DIV2K.     
-   
+
 2. PixelShift200 data preparation 
 
-   1. Download [Pixelshift200]. They are .mat format, having 4 channels (R, Gr, Gb, B). Unzip the .zip file and put all folders inside into one folder called pixelshift200. For example, put here `/data/pixelshift200`. 
+   1. Download [Pixelshift200](http://guochengqian.com/pixelshift200). They are .mat format, having 4 channels (R, Gr, Gb, B). Unzip the .zip file and put all folders inside into one folder called pixelshift200. For example, put here `/data/lowlevel/pixelshift200`. 
+
+   2. `cd TENet && mkdir data && cd data`
 
-   3. Link PixelShift200 data into ./data/pixelshift200, e.g. `cd TENet/data && ln -s /data/pixelshift200 pixelshift200` 
+   3. Link PixelShift200 data into ./data/pixelshift200, e.g. `ln -s /data/lowlevel/pixelshift200 pixelshift200` 
 
    4. Crop images into 512*512, and generate the text file contains the location of each image:
+
       ```bash
       cd ../datasets
       python crop_pixelshift200.py 
       python generate_datalist_pixelshift.py 
       ```
-      The generated .txt file  `train_pixelshift.txt` (9444 Lines) and `val_pixelshift.txt` (20 Lines) are used for training. check them. 
 
 
 ## Training
-#### Train  joint  models: 
+#### Train  joint  models
 
 * DN+DM+SR (end to end without pipeline)
 
-  ```shell
-  python train.py --in_type noisy_lr_raw --mid_type None --out_type linrgb --model tenet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block_type rrdb --n_blocks 6 --imgs_per_gpu 8
+  ```bash
+  python train.py --in_type noisy_lr_raw --mid_type None --out_type linrgb --model tenet --n_gpus 4 --block rrdb --n_blocks 12
   ```
 
-* DN+SR->DM (our TENet)
+* DN+SR->DM (our **TENet**)
 
-  ```SHELL
-  python train.py --in_type noisy_lr_raw --mid_type raw --out_type linrgb --model tenet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block_type rrdb  --n_blocks 6  --imgs_per_gpu 8
+  ```bash
+  python train.py --in_type noisy_lr_raw --mid_type raw --out_type linrgb --model tenet --n_gpus 4 --block rrdb  --n_blocks 12
   ```
 
   Note: 
 
-  1.  `--mid_type raw` is to activate the auxiliary mid stage supervision. Here we add `raw` as the supervision, therefore, the pipeline will be DN+SR->DM.
+  1.  `--mid_type raw` is to activate the auxiliary mid stage supervision. Here we add `raw` as the supervision, therefore, the pipeline will be DN+SR->DM
 
-  2.  for training on a different dataset, like DIV2K, change `--train_list datasets/train_div2k.txt --val_list datasets/val_div2k.txt` 
+  2.  for training on a different dataset, like DIV2K, change `--dataset div2k` 
+   
+  3.  for training with Gaussian noise model, add `--noise_model g`
 
-  3.  for using a different building block, such as NLSA `--block_type nlsa`, or EAM `--block_type eam` , or RRG `--block_type rrg`, or DRLM `--block_type drlm` or RRDB `--block_type rrdb`
+  4.  for using a different building block, such as NLSA `--block nlsa`, or EAM `--block eam` , or RRG `--block rrg`, or DRLM `--block drlm` or RRDB `--block rrdb`
 
-  4.  Monitor your jobs using Tensorboard (log saved in ./log folder by default, `tensorboard --logdir=./ ` ) or using wandb (online website) by set `--use_wandb`. 
+  5.   [`wandb`](https://wandb.ai/) is used by default. Set `--no_wandb` to not using wandb 
 
 
 
-#### Train sequential models: 
+#### Train sequential models (ablation study) 
 
-```shell
+```bash
 # RawDN
-python train.py --in_type noisy_raw --out_type raw --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block_type rrdb --n_blocks 6
+python train.py --in_type noisy_raw --out_type raw --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block rrdb --n_blocks 12
 
 # RawSR
-python train.py --in_type lr_raw --out_type raw --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block_type rrdb --n_blocks 6
+python train.py --in_type lr_raw --out_type raw --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block rrdb --n_blocks 12
 
 # DM
-python train.py --in_type raw --out_type linrgb --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block_type rrdb --n_blocks 6
+python train.py --in_type raw --out_type linrgb --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block rrdb --n_blocks 12
 
 # RGBDN
-python train.py --in_type noisy_linrgb --out_type linrgb --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block_type rrdb --n_blocks 6
+python train.py --in_type noisy_linrgb --out_type linrgb --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block rrdb --n_blocks 12
 
 # RGBSR
-python train.py --in_type lr_linrgb --out_type linrgb --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block_type rrdb --n_blocks 6
+python train.py --in_type lr_linrgb --out_type linrgb --model resnet --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block rrdb --n_blocks 12
 ```
 
 
-
-
 ### Train SOTA models
 * JDSR
-  ```SHELL
-  python train.py --in_type noisy_lr_raw --mid_type None --out_type linrgb --model jdsr --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block_type res --n_blocks 12 --channels 256
+  ```bash
+  python train.py --in_type noisy_lr_raw --mid_type None --out_type linrgb --model jdsr --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4  --use_wandb --block res --n_blocks 12 --channels 256
   ```
 
 * JDnDmSR
 
-  ```SHELL
+  ```bash
   python train.py --in_type noisy_lr_raw --mid_type lr_raw --out_type linrgb --model jdndmsr --scale 2 --train_list datasets/train_pixelshift.txt --val_list datasets/val_pixelshift.txt --n_gpus 4 --n_blocks 2 --block rcab
   ```
 
 
-
 ## Testing
 
-```shell
+```bash
 bash script_all_pipelines.sh 
+# this script supports evaluation on all benchmarking datasets as well as the real-shot images for all possible pipelines
 ```
 
-Note: 
-1. this script supports evaluation on all benchmarking datasets as well as the real-shot images for all possible pipelines
-2. for the real shot images testing, you have to:
-    * save the real-shot image as a readable raw image (like in .RAW, .ARW, .DNG format).  For example,  we use Lightroom mobile version to shot images on iPhone and save the photo in .DNG format.
-    * Read the general metadata using RawPy and read the noise profiling metadata using [Jeffrey's Image Metadata Viewer](http://exif.regex.info/exif.cgi) or [metapicz](http://metapicz.com/). 
+Note: for the real shot images testing, you have to:
 
+1.  save the real-shot image as a readable raw image (like in .RAW, .ARW, .DNG format).  For example,  we use Lightroom mobile version to shot images on iPhone and save the photo in .DNG format.
+2. Read the general metadata using RawPy and read the noise profiling metadata using [Jeffrey's Image Metadata Viewer](http://exif.regex.info/exif.cgi) or [metapicz](http://metapicz.com/). 
+
+
+
+## Result
+<p align="center">
+  <img width="800" src="misc/pipeline_result.png">
+</p>
 
 
 ### Citation