+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: Improving CodeFormer videos (/thread-3574.html)
RE: Improving CodeFormer videos - Dan64 - 14.02.2024
(14.02.2024, 09:37)Selur Wrote:Quote:In any case it could be a big enhancement to introduce "ffmpeg" as Video encoder like "nvenc"a. Hybrid already supports this on Linux
b. don't see any gain in it compared to NVEncC
Quote: In this case it will be possible to skip "vspipe" providing to Hybrid a more robust alternative in case of "unknown" problems with filters.In my eyes, ffmpegs Vapoursynth support is not more stable than vspipe.
=> not planning to support directly loading the a vapoursynth script through ffmpeg
Cu Selur
I do agree with you that "ffmpeg" is not introducing any significant gain compared to NVEncC and that ffmpegs Vapoursynth support is not more stable than vspipe.
The problem is that the "pipe" it is the weakest link in the encoding chain, the "ffmpeg" (Vapoursynth enabled version) is able to avoid the "pipe" in this sense is more robust than "vspipe".
Dan
RE: Improving CodeFormer videos - Selur - 14.02.2024
Quote:The problem is that the "pipe" it is the weakest link in the encoding chain, the "ffmpeg" (Vapoursynth enabled version) is able to avoid the "pipe" in this sense is more robust thana. compiling ffmpeg with Vapoursynth support requires compiling Vapoursynth beforehand
b. whenever a dependecy of ffmpeg fails building ffmpeg fails
c. whenever an api change happens the vapoursynth library for ffmpeg needs adjustment
adding another dependecy doesn't really sound appealing to me.
What do you call more robust?
Have you tried using rawvideo pipes instead of yuv4mpegpipe, which isn't supported by all encoders.
x264 does support both, does using something like:
"D:\Programs\Hybrid\64bit\Vapoursynth\ffmpeg.exe" -y -loglevel fatal -f vapoursynth -i CodeFormerTest6-001-BAD-1.vpy -noautorotate -nostdin -threads 8 -map 0:0 -an -sn -color_primaries bt470bg -color_trc smpte170m -colorspace bt470bg -color_range tv -pix_fmt yuv420p -vsync 0 -f rawvideo - | x264 --preset veryfast --crf 18.00 --profile high --level 5.1 --ref 3 --direct auto --b-adapt 0 --sync-lookahead 17 --qcomp 0.50 --rc-lookahead 40 --qpmax 51 --partitions i4x4,p8x8,b8x8 --no-fast-pskip --subme 5 --aq-mode 0 --vbv-maxrate 300000 --vbv-bufsize 300000 --sar 1:1 --non-deterministic --range tv --colormatrix bt470bg --demuxer raw --input-res 640x352 --input-csp i420 --input-range tv --input-depth 8 --fps 25/1 --output-depth 8 --output "2024-02-14@15_36_30_3410_01.264" -(not that the paths, vui settings, bitdepth, fps, etc need to be adjusted)
If not then ffmpeg only works in your example since it has a special handling for yuv4mpegpipe output.
Cu Selur
RE: Improving CodeFormer videos - Selur - 14.02.2024
I tried to get your cuda dlib version working, but failed.
Here's what I did:
- I copied 'dlib-19.24.1-cp311-cp311-win_amd64.whl' to 'Hybrid\64bit\Vapoursynth'
- inside the Hybrid\64bit\Vapoursynth-folder I then called:
which worked fine:python -m pip install dlib-19.24.1-cp311-cp311-win_amd64.whl
Processing f:\hybrid\64bit\vapoursynth\dlib-19.24.1-cp311-cp311-win_amd64.whl Installing collected packages: dlib Successfully installed dlib-19.24.1 - I then extracted the content of the vs-codeformer.zip into the 'Hybrid\64bit\Vapoursynth\Lib\site-packages\vscodeformer'-folder.
# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# source: 'G:\TestClips&Co\files\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg
# Loading G:\TestClips&Co\files\test.avi using LWLibavSource
clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer (170), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
clip = core.std.SetFrameProps(clip, _Transfer=6)
# Setting color primaries info (to 470), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
clip = core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
# changing range from limited to full range
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# adjusting color space from YUV420P8 to RGB24 for vsCodeFormer
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="full")
# Blind Face Restoration using CodeFormer
from vscodeformer import codeformer as CodeFormer
clip = CodeFormer(clip=clip, upscale=1, detector=1, weight=1.000) # 640x352
# changing range from full to limited range
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
# no resizing since resolution is already archived
# adjusting output color from: RGB24 to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()Failed to evaluate the script:
Python exception: cannot access local variable 'dlib' where it is not associated with a value
Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 3115, in vapoursynth._vpy_evaluate
File "src\cython\vapoursynth.pyx", line 3116, in vapoursynth._vpy_evaluate
File "J:\tmp\tempPreviewVapoursynthFile17_04_54_105.vpy", line 34, in
clip = CodeFormer(clip=clip, upscale=1, detector=1, weight=1.000) # 640x352
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vscodeformer\__init__.py", line 102, in codeformer
face_helper = [
^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vscodeformer\__init__.py", line 103, in
FaceRestoreHelper(upscale, det_model=detection_model, use_parse=True, device=device) for _ in range(num_streams)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vscodeformer\face_restoration_helper.py", line 113, in __init__
self.face_detector, self.shape_predictor_5 = self.init_dlib()
^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vscodeformer\face_restoration_helper.py", line 173, in init_dlib
face_detector = dlib.cnn_face_detection_model_v1(detection_path)
^^^^
UnboundLocalError: cannot access local variable 'dlib' where it is not associated with a valuereplacing
if 'ON' == 'ON':
add_lib_to_dll_path('C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/lib/x64/cudnn.lib')
add_lib_to_dll_path('C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/lib/x64/cudart.lib')if 'ON' == 'ON':
add_lib_to_dll_path('F:/Hybrid/64bit/Vapoursynth/torch_dependencies/cudnn.lib')
add_lib_to_dll_path('F:/Hybrid/64bit/Vapoursynth/torch_dependencies/cudart.lib')in Hybrid\64bit\Vapoursynth\Lib\site-packages\dlib\__init__.py didn't help.
(script works fine when dlib isn't used)
Cu Selur
RE: Improving CodeFormer videos - Dan64 - 14.02.2024
(14.02.2024, 18:52)Selur Wrote: I then extracted the content of the vs-codeformer.zip into the 'Hybrid\64bit\Vapoursynth\Lib\site-packages\vscodeformer'-folder.
Did you cleared the folder: "Hybrid\64bit\Vapoursynth\Lib\site-packages\vscodeformer\__pycache__" ?
RE: Improving CodeFormer videos - Selur - 14.02.2024
To be sure I just deleted all the '__pycache__'-folders and tried again -> sadly I still get the same error.
(going to bed now)
Cu Selur
RE: Improving CodeFormer videos - Dan64 - 14.02.2024
(14.02.2024, 18:52)Selur Wrote: replacing
with:if 'ON' == 'ON': add_lib_to_dll_path('C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/lib/x64/cudnn.lib') add_lib_to_dll_path('C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/lib/x64/cudart.lib')
replacing these also does not change anything.if 'ON' == 'ON': add_lib_to_dll_path('F:/Hybrid/64bit/Vapoursynth/torch_dependencies/cudnn.lib') add_lib_to_dll_path('F:/Hybrid/64bit/Vapoursynth/torch_dependencies/cudart.lib')
in Hybrid\64bit\Vapoursynth\Lib\site-packages\dlib\__init__.py didn't help.
(script works fine when dlib isn't used)
Cu Selur
Did you cleared the foldr: "Hybrid\64bit\Vapoursynth\Lib\site-packages\dlib\__pycache__" ?
RE: Improving CodeFormer videos - Selur - 14.02.2024
yes, I deleted all __pycache__ folders
RE: Improving CodeFormer videos - Dan64 - 14.02.2024
What happen if inside python command you execute the following commands:
import dlib
print(dlib.DLIB_USE_CUDA)
print(dlib.cuda.get_num_devices()))RE: Improving CodeFormer videos - Selur - 14.02.2024
F:\Hybrid\64bit\Vapoursynth>python
Python 3.11.3 (tags/v3.11.3:f3909b8, Apr 4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)] on win32
>>> import dlib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\dlib\__init__.py", line 20, in <module>
from _dlib_pybind11 import *
ImportError: DLL load failed while importing _dlib_pybind11: Das angegebene Modul wurde nicht gefunden.
>>>
>>> print(dlib.DLIB_USE_CUDA)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'dlib' is not defined
>>>
>>> print(dlib.cuda.get_num_devices()))
File "<stdin>", line 1
print(dlib.cuda.get_num_devices()))
^
SyntaxError: unmatched ')'
>>>RE: Improving CodeFormer videos - Dan64 - 14.02.2024
Try to check your folder: Hybrid\64bit\Vapoursynth\Lib\site-packages
should be only one copy of: _dlib_pybind11.cp311-win_amd64.pyd
In the file __init__.py there is the following code:
def add_lib_to_dll_path(path):
""" On windows you must call os.add_dll_directory() to allow linking to external DLLs. See
https://docs.python.org/3.8/whatsnew/3.8.html#bpo-36085-whatsnew. This function adds the folder
containing path to the dll search path.
"""
try:
import os
os.add_dll_directory(os.path.join(os.path.dirname(path), '../../bin'))
except (AttributeError,KeyError,FileNotFoundError):
pass
if 'ON' == 'ON':
add_lib_to_dll_path('C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/lib/x64/cudnn.lib')
add_lib_to_dll_path('C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/lib/x64/cudart.lib')
from _dlib_pybind11 import *
from _dlib_pybind11 import __version__, __time_compiled__What is relevant in windows are the DLLs. The path where are stored is recovered from the path of the libraries.
If the path for lib is: 'C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/lib/x64/cudart.lib'
The DLLs are searched in: 'C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/bin'
(see path contruction in the function: add_lib_to_dll_path).
This imply that in your case the DLLs are searched in: 'F:/Hybrid/64bit/bin'
Please check and move the libraries in: 'F:/Hybrid/64bit/Vapoursynth/torch_dependencies/lib/x64'
But in any case the DLLs will be searched in: 'F:/Hybrid/64bit/Vapoursynth/torch_dependencies/bin'
If you want to keep 'F:/Hybrid/64bit/Vapoursynth/torch_dependencies' you must change the function add_lib_to_dll_path()
Dan