OpenCV_4.2.0/opencv_contrib-4.2.0/modules/dnn_superres/README.md

5.8 KiB

Super Resolution using Convolutional Neural Networks

This module contains several learning-based algorithms for upscaling an image.

Usage

Run the following command to build this module:

cmake -DOPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules -Dopencv_dnn_superres=ON <opencv_source_dir>

Refer to the tutorials to understand how to use this module.

Models

There are four models which are trained.

EDSR

Trained models can be downloaded from here.

ESPCN

Trained models can be downloaded from here.

FSRCNN

Trained models can be downloaded from here.

LapSRN

Trained models can be downloaded from here.

  • Size of the model: between 1-5Mb
  • This model was trained for ~50 iterations with a batch size of 32
  • Link to implementation code: https://github.com/fannymonori/TF-LAPSRN
  • x2, x4, x8 trained models available
  • Advantage: The model can do multi-scale super-resolution with one forward pass. It can now support 2x, 4x, 8x, and [2x, 4x] and [2x, 4x, 8x] super-resolution.
  • Disadvantage: It is slower than ESPCN and FSRCNN, and the accuracy is worse than EDSR.
  • Speed: < 0.1 sec for every scaling factor on 256x256 images on an Intel i7-9700K CPU.
  • Original paper: Deep laplacian pyramid networks for fast and accurate super-resolution [4]

Benchmarks

Comparing different algorithms. Scale x4 on monarch.png (768x512 image).

Inference time in seconds (CPU) PSNR SSIM
ESPCN 0.01159 26.5471 0.88116
EDSR 3.26758 29.2404 0.92112
FSRCNN 0.01298 26.5646 0.88064
LapSRN 0.28257 26.7330 0.88622
Bicubic 0.00031 26.0635 0.87537
Nearest neighbor 0.00014 23.5628 0.81741
Lanczos 0.00101 25.9115 0.87057

Refer to the benchmarks located in the tutorials for more detailed benchmarking.

References

[1] Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution", 2nd NTIRE: New Trends in Image Restoration and Enhancement workshop and challenge on image super-resolution in conjunction with CVPR 2017. [PDF] [arXiv] [Slide]

[2] Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A., Bishop, R., Rueckert, D. and Wang, Z., "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network", Proceedings of the IEEE conference on computer vision and pattern recognition CVPR 2016. [PDF] [arXiv]

[3] Chao Dong, Chen Change Loy, Xiaoou Tang. "Accelerating the Super-Resolution Convolutional Neural Network", in Proceedings of European Conference on Computer Vision ECCV 2016. [PDF] [arXiv] [Project Page]

[4] Lai, W. S., Huang, J. B., Ahuja, N., and Yang, M. H., "Deep laplacian pyramid networks for fast and accurate super-resolution", In Proceedings of the IEEE conference on computer vision and pattern recognition CVPR 2017. [PDF] [arXiv] [Project Page]