08.02.2024, 00:41
Hello Selur,
Usually CodeFormer is used with a Resizer.
In your example I noted that you used SwinIR, which is a fast and good resizer, but RealESRGAN is better.
In Hybrid it is possible to apply the RealESRGAN resizer and the CodeFormer.
But CodeFormer allows to use the RealESRGAN resizer directly in the filter.
This approach is not supported (are supported only simple resizer like Lanczos).
I compared the result obtained using RealESRGAN directly in the filter and the output image is better
https://imgsli.com/MjM4NTgx
I used the following script
It is possible to add this type of resize in Hybrid ?
Thanks
Dan
P.S.
Given the special combination of Resizer and CodeFormer, could be better add this capability in the Resizer page of RealESRGAN, because after CodeFormer the video is upscaled
Usually CodeFormer is used with a Resizer.
In your example I noted that you used SwinIR, which is a fast and good resizer, but RealESRGAN is better.
In Hybrid it is possible to apply the RealESRGAN resizer and the CodeFormer.
But CodeFormer allows to use the RealESRGAN resizer directly in the filter.
This approach is not supported (are supported only simple resizer like Lanczos).
I compared the result obtained using RealESRGAN directly in the filter and the output image is better
https://imgsli.com/MjM4NTgx
I used the following script
# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import os
import site
core = vs.core
# Adding torch dependencies to PATH
path = site.getsitepackages()[0]+'/torch_dependencies/bin/'
ctypes.windll.kernel32.SetDllDirectoryW(path)
path = path.replace('\\', '/')
os.environ["PATH"] = path + os.pathsep + os.environ["PATH"]
# Loading Plugins
core.std.LoadPlugin(path="D:/Programs/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="D:/Programs/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# source: 'E:\VideoTest\CodeFormer\Selur\suzie_lossless.mp4'
# current color space: YUV444P8, bit depth: 8, resolution: 176x144, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading E:\VideoTest\CodeFormer\Selur\suzie_lossless.mp4 using LWLibavSource
clip = core.lsmas.LWLibavSource(source="E:/VideoTest/CodeFormer/Selur/suzie_lossless.mp4", format="YUV444P8", stream_index=0, cache=0, fpsnum=30000, fpsden=1001, prefer_hw=0)
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer (to 470bg), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
clip = core.std.SetFrameProps(clip, _Transfer=5)
# Setting color primaries info (to 5), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
clip = core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
from vsrealesrgan import realesrgan as RealESRGAN
# adjusting color space from YUV444P8 to RGBH for vsRealESRGAN
clipBG = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# resizing using RealESRGAN
clipBG = RealESRGAN(clip=clipBG, model=2, device_index=0, trt=True, trt_cache_path=r"E:\VideoTest\CodeFormer\Selur") # 704x576
# changing range from limited to full range
clipBG = core.resize.Bicubic(clipBG, range_in_s="limited", range_s="full")
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# adjusting color space from RGBH to RGB24 for vsCodeFormer
clipBG = core.resize.Bicubic(clip=clipBG, format=vs.RGB24, range_s="full", dither_type="error_diffusion")
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, range_s="full", dither_type="error_diffusion")
# Blind Face Restoration using CodeFormer
#clipBG = core.resize.Lanczos(clip, width=704, height=576) # 704x576
from vscodeformer import codeformer as CodeFormer
clip = CodeFormer(clip=clip, upscale=4, weight=1.000, only_center_face=False, num_streams=4, bg_clip=clipBG) # 704x576
# changing range from full to limited range
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
# adjusting output color from: RGB24 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")
# adjusted resizing to achieve PAR 1:1 for preview
# resizing for preview to square pixel to 770x576
clip = core.resize.Bicubic(clip=clip, width=770, height=576)
# set output frame rate to 29.97fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Output
clip.set_output()It is possible to add this type of resize in Hybrid ?
Thanks
Dan
P.S.
Given the special combination of Resizer and CodeFormer, could be better add this capability in the Resizer page of RealESRGAN, because after CodeFormer the video is upscaled