Bounding-box deep calibration for high performance face detection

IET Computer Vision 2022 https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/cvi2.12122

Introduction

This repository includes the BDC codes for calibrating misaligned bounding-boxes on WIDER FACE training set and offering calibrated annotations for training high performance face detectors. To visualize misaligned detection results, the misaligned and replaced bounding-boxes are drawn on their corresponding training images seperately. All the source codes are implemented in python.

Specifically, we firstly utilize TinaFace as predicting detector to generate detection results on WIDER FACE training set. Then, calibrated annotations are generated by our proposed BDC method. The core idea of BDC is to replace misaligned annotations with predicted bounding-boxes. Finally, we adopt calibrated annotations to train high performance face detectors.

During prediction stage, our goal is to obtain high confidence detection results (HCDRs), where their predicted bounding-boxes are prepared for replacement operation in the following calibration stage. During calibration stage, our purpose is to determine the misaligned detection results (MDRs) between misaligned annotations and predicted HCDRs and replace misaligned bounding-boxes with their best matching predicted bounding-boxes.

The main purpose for replacement operation is to reasonably decrease bounding-box regression loss for these misaligned faces in the training phase. As a result, we can reduce bounding-box regression loss and later the total loss of detection models, which can further improve the detection performance.

Welcome to report the performance with BDC method on different face detectors and let us know it. If it is useful, please light up a star for this repository. Many thanks !

(Email: 1093945458@qq.com or shiluo1990@hotmail.com)

Contents

1. Clone Repository
2. Preparation
3. Run BDC Algorithm
4. Training High Performance Face Detectors from Calibrated Annotations
5. Visualization Misaligned Detection Results

1. Clone Repository:

git clone --recursive https://github.com/shiluo1990/BDC.git

2. Preparation

2.1. Detection Results Download

To run the BDC algorithm, you need to offer detection results on WIDER FACE training set. Here, you can download TinaFace predicted detection results from Baidu Yun via the following link and extraction code :

link: https://pan.baidu.com/s/1EDvczs2BHIOqXhuHdp1TSQ
extraction code: kqn7

By default, the detection results should be saved into a folder named ./prediction.

|--prediction
         |--0--Parade
         |--...
         |--61--Street_Battle

2.2. Detection Results Formulation

As the format of detection results generated by different predicting detectors is different, you need to formate the detection results.

Detection Results Format

image_path
the number of detection results
xmin ymin weigh height detection_confidence_score
...

Example

0--Parade/0_Parade_marchingband_1_5.jpg
3942
497 178 37 48 0.8394013
...
1008 233 11 18 0.010000908

Specifically, we should run following codes to formulate detection results

python abstract_image.py
python formulation.py

If detection results are offered by yourself and meet the formulated format, you can skip 2.1 and 2.2 and just place the detection results in folder ./formulation.

Before starting to run BDC algorithm, you should have a directory structure as follows:

|--formulation
        |--0--Parade
        |--...
        |--61--Street_Battle

3. Run BDC Algorithm

3.1. Calculate Average Detection Confidence (ADC)

Run adc.py to output the average detection confidence (ADC) score

$python adc.py

3.2. Generate Calibrated Annotations for WIDER FACE Training Set

Modify the setting of ADC score in line 25 of bdc.py, and run bdc.py.

$python bdc.py

Therefore, BDC code generates calibrated annotation file of WIDER FACE training set in current directory: wider_face_train_bbx_gt_calibrated.txt.

Besides, wider_face_train_misaligned_bbx.txt and wider_face_train_replaced_bbx.txt record the misaligned and corresponding replaced bounding-boxes, which facilitate to visualize misaligned detection results (MDRs) below.

4. Training High Performance Face Detectors from Calibrated Annotations

You can directly utilize calibrated annotations wider_face_train_bbx_gt_calibrated.txt to train high performance face detectors.

5. Visualization Misaligned Detection Results

5.1. Prepare WIDER FACE training images

By default, the WIDER FACE training images should be saved into a folder named ./WIDER_train.

|--WIDER_train
         |--0--Parade
         |--...
         |--61--Street_Battle

5.2. Draw misaligned bounding-boxes, run draw_misaligned_bbx.py

$python draw_misaligned_bbx.py

Generate training images with misaligned bounding-boxes in folder ./misaligned/.

5.2. Draw replaced bounding-boxes, run draw_replaced_bbx.py

$python draw_replaced_bbx.py

Generate training images with misaligned detection results (misaligned and corresponding replaced bounding-boxes) in folder ./calibrated/.

Citation

If this project helps your research, please consider citing the following papers:

@article{luo2022bdc,
  title={Bounding-box deep calibration for high performance face detection},
  author={Luo, Shi and Li, Xiongfei and Zhang, Xiaoli},
  journal={IET Computer Vision},
  year={2022}
}

@article{zhu2020tinaface,
  title={Tinaface: Strong but simple baseline for face detection},
  author={Zhu, Yanjia and Cai, Hongxiang and Zhang, Shuhan and Wang, Chenhao and Xiong, Yichao},
  journal={arXiv preprint arXiv:2011.13183},
  year={2020}
}

Contact

To ask questions or report issues, please open an issue on the issues tracker.