Selur's Little Message Board

Full Version: New Feature: RealSR (Real-World Super-Resolution)
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
Hello Selur,

   I was impressed by the resize quality of RealSR resize method.
   In this link there is a version which is using ncnn Vulkan

    https://github.com/nihui/realsr-ncnn-vulkan

   I don't know if it is possible to integrate this process in Hybrid, could you provide your opinion ?

Thanks
I planned to add support for https://github.com/Kiyamou/VapourSynth-R...cnn-Vulkan, but I'm totally busy in real live atm. and spend my free time playing around with VSGAN (https://forum.selur.net/Thread-Why-VSGAN...-supported) and some models that can be used with it.

Cu Selur
Did a small test and know I know why I dropped it last I had time.
Code:
# Imports
import vapoursynth as vs
core = vs.get_core()
import ctypes
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/ResizeFilter/RealSR/rsnv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# source: 'F:\TestClips&Co\files\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading source using FFMS2
clip = core.ffms2.Source(source="F:/TestClips&Co/files/test.avi",cachefile="E:/Temp/avi_078c37f69bb356e7b5fa040c71584c40_853323747.ffindex",format=vs.YUV420P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting color space from YUV420P8 to RGBS for RealSR
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# resizing using RealSR
clip = core.rsnv.RealSR(clip=clip, scale=4,model=1)
# adjusting resizing to hit target resolution
clip = core.fmtc.resample(clip=clip, w=1920, h=1056, kernel="lanczos", interlaced=False, interlacedd=False)
# adjusting output color from: RGB48 to YUV420P10 for x265Model (i420@8)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")
# set output frame rate to 25.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()
works for the first three frames,
Code:
[0 NVIDIA GeForce GTX 1070 Ti]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce GTX 1070 Ti]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce GTX 1070 Ti]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
[0 NVIDIA GeForce GTX 1070 Ti]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
then I get:
Code:
vkWaitForFences failed -4
vkQueueSubmit failed -4
vkQueueSubmit failed -4
vkQueueSubmit failed -4
vkQueueSubmit failed -4
vkQueueSubmit failed -4
vkQueueSubmit failed -4
not sure whether it's my card running out of memory or what,... (same happens with model=0)
using a lower tilesize_x value (used 32) helps, so this might be me graphic card,..

Cu Selur
Send you a link to a dev version for testing.

Cu Selur
I tried to perform a resize of a 3min video. I have a GTX 1070 which should have enough memory to perform the resize.
The starting encoding speed is very low, about 0.08fps, after some time the speed increase, but probably because the encoding is skipped.
Even if the encoding end without errors, was encoded only the first 2 seconds of the movie, then the video image is frozen.

I will try to look at this problem tomorrow.

Thanks in any case for providing this test version.
Best test in the preview a bit what settings are usable on your hardware.
Try different gpu_thread options and forcing tile_size to 32x32, might be slower, but more stable.

Cu Selur
I was finally able to encode using https://github.com/jixiaozhong/RealSR

The problem is that the encoding speed is too slow, the resizing of a 30sec movie took 57min.

To perform the test I used the following script (assuming that all the executable are in the script's directory)

Code:
# 1) extract all the frames in a temporary directory
.\ffmpeg.exe -i input.mkv -q:v 1 .\frames\input_%05d.jpg

# 2) resize the frames
.\realsr-ncnn-vulkan.exe -v -m .\models\models-DF2K_JPEG -f jpg -i .\frames -o .\fresize

# 3) encode the resized frames back in the movie
.\ffmpeg.exe -y -framerate 23.976 -i "fresize\input_%05d.jpg" -vcodec libx264 -vf "scale=1920:800:flags=lanczos" -crf 18 output.mp4
 
The result is very good, but unfortunately is too slow, to encode a movie with a length of 1h will be necessary about 5 days Sad

Probably google, using its GPU farm will be able to use these methods, but for a standard user they are unusable.
I also tested the implementation using the test image original.png

but the results are quite different, respect to the ones published

The version with TTA is the following

output-x.png

While the version not using TTA (better but equal to the one published)

output.png

So the version realeased is different from the one used by the author. But what is worse is that TTA (4x slower) is even worse.
-> complain to the author or the plugin.
I experienced similar results when testing it on 23 Nov 2020, see:
https://github.com/Kiyamou/VapourSynth-R...n/issues/2


Cu Selur
I followed your suggestion

I cropped the test image to keep only the high resolution part, then I resized the image by 25%.

For the comparison I resized by 4x the downsized image with RealSR and Lanczos. I obtained the following result

[Image: 0935-half-4-test.png]

This is not exactly the same image shown by the author, but I think is still the best resizer I tested so far.
The problem is that is really slow, maybe with a new NVIDIA Card I could get better results, but in these days (and probably for the next 2 years) the NVIDIA cards are untouchable.
Pages: 1 2