Selur's Little Message Board

Pages: 1 2

Hello Selur,

   I was impressed by the resize quality of RealSR resize method.
   In this link there is a version which is using ncnn Vulkan

    https://github.com/nihui/realsr-ncnn-vulkan

   I don't know if it is possible to integrate this process in Hybrid, could you provide your opinion ?

Thanks

I planned to add support for https://github.com/Kiyamou/VapourSynth-R...cnn-Vulkan, but I'm totally busy in real live atm. and spend my free time playing around with VSGAN (https://forum.selur.net/Thread-Why-VSGAN...-supported) and some models that can be used with it.

Cu Selur

Did a small test and know I know why I dropped it last I had time.

Code:
# Imports

import vapoursynth as vs

core = vs.get_core()

import ctypes

# Loading Plugins

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/ResizeFilter/RealSR/rsnv.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")

# source: 'F:\TestClips&Co\files\test.avi'

# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive

# Loading source using FFMS2

clip = core.ffms2.Source(source="F:/TestClips&Co/files/test.avi",cachefile="E:/Temp/avi_078c37f69bb356e7b5fa040c71584c40_853323747.ffindex",format=vs.YUV420P8,alpha=False)

# making sure input color matrix is set as 470bg

clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")

# making sure frame rate is set to 25

clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)

# Setting color range to TV (limited) range.

clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

# adjusting color space from YUV420P8 to RGBS for RealSR

clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")

# resizing using RealSR

clip = core.rsnv.RealSR(clip=clip, scale=4,model=1)

# adjusting resizing to hit target resolution 

clip = core.fmtc.resample(clip=clip, w=1920, h=1056, kernel="lanczos", interlaced=False, interlacedd=False)

# adjusting output color from: RGB48 to YUV420P10 for x265Model (i420@8)

clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")

# set output frame rate to 25.000fps

clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)

# Output

clip.set_output()

works for the first three frames,

Code:
[0 NVIDIA GeForce GTX 1070 Ti]  queueC=2[8]  queueG=0[16]  queueT=1[2]

[0 NVIDIA GeForce GTX 1070 Ti]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0

[0 NVIDIA GeForce GTX 1070 Ti]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1

[0 NVIDIA GeForce GTX 1070 Ti]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1

then I get:

Code:
vkWaitForFences failed -4

vkQueueSubmit failed -4

vkQueueSubmit failed -4

vkQueueSubmit failed -4

vkQueueSubmit failed -4

vkQueueSubmit failed -4

vkQueueSubmit failed -4

not sure whether it's my card running out of memory or what,... (same happens with model=0)
using a lower tilesize_x value (used 32) helps, so this might be me graphic card,..

Cu Selur

Send you a link to a dev version for testing.

Cu Selur

I tried to perform a resize of a 3min video. I have a GTX 1070 which should have enough memory to perform the resize.
The starting encoding speed is very low, about 0.08fps, after some time the speed increase, but probably because the encoding is skipped.
Even if the encoding end without errors, was encoded only the first 2 seconds of the movie, then the video image is frozen.

I will try to look at this problem tomorrow.

Thanks in any case for providing this test version.

Best test in the preview a bit what settings are usable on your hardware.
Try different gpu_thread options and forcing tile_size to 32x32, might be slower, but more stable.

Cu Selur

I was finally able to encode using https://github.com/jixiaozhong/RealSR

The problem is that the encoding speed is too slow, the resizing of a 30sec movie took 57min.

To perform the test I used the following script (assuming that all the executable are in the script's directory)

Code:
# 1) extract all the frames in a temporary directory

.\ffmpeg.exe -i input.mkv -q:v 1 .\frames\input_%05d.jpg

# 2) resize the frames

.\realsr-ncnn-vulkan.exe -v -m .\models\models-DF2K_JPEG -f jpg -i .\frames -o .\fresize

# 3) encode the resized frames back in the movie

.\ffmpeg.exe -y -framerate 23.976 -i "fresize\input_%05d.jpg" -vcodec libx264 -vf "scale=1920:800:flags=lanczos" -crf 18 output.mp4

The result is very good, but unfortunately is too slow, to encode a movie with a length of 1h will be necessary about 5 days Sad

Probably google, using its GPU farm will be able to use these methods, but for a standard user they are unusable.

I also tested the implementation using the test image original.png

but the results are quite different, respect to the ones published

The version with TTA is the following

output-x.png

While the version not using TTA (better but equal to the one published)

output.png

So the version realeased is different from the one used by the author. But what is worse is that TTA (4x slower) is even worse.

-> complain to the author or the plugin.
I experienced similar results when testing it on 23 Nov 2020, see:
https://github.com/Kiyamou/VapourSynth-R...n/issues/2

Cu Selur

I followed your suggestion

I cropped the test image to keep only the high resolution part, then I resized the image by 25%.

For the comparison I resized by 4x the downsized image with RealSR and Lanczos. I obtained the following result

[Image: 0935-half-4-test.png]

This is not exactly the same image shown by the author, but I think is still the best resizer I tested so far.
The problem is that is really slow, maybe with a new NVIDIA Card I could get better results, but in these days (and probably for the next 2 years) the NVIDIA cards are untouchable.

Pages: 1 2

Dan64

Selur

Selur

Selur

Dan64

Selur

Dan64

Dan64

Selur

Dan64