+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: Deoldify Vapoursynth filter (/thread-3595.html)
Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
RE: Deoldify Vapoursynth filter - Selur - 22.09.2024
using:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)You can get the file here.
using:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)2024-09-22 09:44:34.029
Failed to evaluate the script:
Python exception: HAVC_deepex: no reference frames found in 'J:/tmp', allowed format is: ref_nnnnnn.[png|jpg]
Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate
File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate
File "J:\tmp\tempPreviewVapoursynthFile09_44_12_474.vpy", line 44, in
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 280, in HAVC_main
clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 510, in HAVC_deepex
clip = SceneDetectFromDir(clip_ref, sc_framedir=sc_framedir, merge_ref_frame=merge_ref_frame,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\vsslib\vsutils.py", line 156, in SceneDetectFromDir
raise vs.Error(
vapoursynth.Error: HAVC_deepex: no reference frames found in 'J:/tmp', allowed format is: ref_nnnnnn.[png|jpg]2024-09-22 09:46:45.803
Failed to evaluate the script:
Python exception: HAVC_deepex: method != 0 but sc_framedir is unset
Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate
File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate
File "C:\Users\Selur\Desktop\Vapoursynth Scripts\test.vpy", line 44, in
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 280, in HAVC_main
clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 425, in HAVC_deepex
raise vs.Error("HAVC_deepex: method != 0 but sc_framedir is unset")
vapoursynth.Error: HAVC_deepex: method != 0 but sc_framedir is unsetCu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 22.09.2024
(22.09.2024, 09:47)Selur Wrote: using:
coloring isn't good, but the chroma lag seems to be fixed.clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
You can get the file here.
using:
I get:clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
not setting 'ScFrameDir' I get:2024-09-22 09:44:34.029
Failed to evaluate the script:
Python exception: HAVC_deepex: no reference frames found in 'J:/tmp', allowed format is: ref_nnnnnn.[png|jpg]
2024-09-22 09:46:45.803
Failed to evaluate the script:
Python exception: HAVC_deepex: method != 0 but sc_framedir is unset
The error messages should be clear.
In the first case has been provided sc_framedir but in this folder are not available reference images in the format: ref_nnnnnn.[png|jpg].
For example valid ref frames are ref_000000.png or ref_000310.jpg. It is mandatory to prefix all the reference images by the text "ref_", the frame number must be 6 digit long padded on the left with zeros, and the extension can be ".png" or ".jpg".
In the second case, has been specified DeepExMethod=1, which implies "HAVC + RF same as video", but not reference frame directory has been provided.
Dan
RE: Deoldify Vapoursynth filter - Selur - 22.09.2024
Quote: In the second case, has been specified DeepExMethod=1, which implies "HAVC + RF same as video", but not reference frame directory has been provided.No.
First I used:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)Then I used:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)and DeOlddify complained about
HAVC_deepex: no reference frames found in 'J:/tmp', allowed format is: ref_nnnnnn.[png|jpg]HAVC_deepex: method != 0 but sc_framedir is unsetSo complaining about the format seems wrong.
Cu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 22.09.2024
The feature that you are referring to is enabled only if the parameter "only_ref_frames" is set to True.
With the current GUI this parameter can be set only as shown in the picture below (see check box "Reference frames only")
Currently your GUI don't allow to set "Ref FrameDir" when Method is in: 0, 1, 2
So I had, first set method = 3, set the reference dir and then change the method to 0 (please allow always the setting of this field, with the option to clear it)
I attached a new RC12 where now it is possible to set the flag "only_ref_frames" using a command like
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="ref_ext_stable", DeepExModel=1, VideoTune="Stable", DeepExOnlyRefFrames=True, ScThreshold=0.035)As explained this option has been enabled to check the reference frames that are generated when is selected the method = 0 (HAVC).
The best way to use this feature is to open the preview window, open with vsedit or veViewer the script generated automatically and press "F7" or select "Script->Benchmark".
There is not reason to encode the generated movie, the Benchmark will run the script and the reference frames will be generated without encoding.
For example if you run the command described previously you will see that the following frames were not colored properly
You can create a new directory called "ref_fix" and put inside this directory these pictures
Then you can use a command like this
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="ref_fix", DeepExModel=1, VideoTune="Stable", DeepExMethod=1, ScThreshold=0.035)to use DeepExMethod=1 -> HAVC + RF same as video
In this case the reference frames will be generated using HAVC but the 2 frames added in the directory "ref_fix" will override the frames generated automatically by HAVC, thus providing an improved colored image.
Dan
RE: Deoldify Vapoursynth filter - Selur - 22.09.2024
Quote:Currently your GUI don't allow to set "Ref FrameDir" when Method is in: 0, 1, 2I adjusted the code to always allow changing the "Ref FrameDir". (If nothing is set, Hybrid will use the temp folder)
So I had, first set method = 3, set the reference dir and then change the method to 0 (please allow always the setting of this field, with the option to clear it)
Updated the deoldify test version.
I have no clue what the screenshot was to do with my settings and the error I got, since those are totally different settings.
Quote:now it is possible to set the flag "only_ref_frames" using a command likeThis is driving me nuts. You are changing the logic while the GUI does not reflect the old logic properly.
So in which cases should be set now? atm. it only gets set for HAVC_deepex if method is 0.
At this point I have no clue, whether the GUI sets things properly or not.
Cu Selur
RE: Deoldify Vapoursynth filter - Selur - 22.09.2024
Since you are the only one out there who knows how to use it, I need feedback on what to change so that:
a. one can enable the option without the preview or the encoding crashing
b. one can not set stuff that causes crashes. If one specifies to demanding settings for one's hardware that is 'okay' to crash. But enabling feature X should always create correct calls.
Cu Selur
RE: Deoldify Vapoursynth filter - Selur - 22.09.2024
Uploaded new test version, where 'Reference frames only' is enabled by default.
using:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/BestSource/BestSource.dll")
# Import scripts
import adjust
import validate
# Source: 'G:\TestClips&Co\files\test.avi'
# Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual
# Loading G:\TestClips&Co\files\test.avi using BestSource)
clip = core.bs.VideoSource(source="G:/TestClips&Co/files/test.avi", cachepath="J:/tmp/test.json", track=0, hwdevice="cuda")
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting color using Tweak
clip = adjust.Tweak(clip=clip, hue=0.00, sat=0.00, cont=1.00, coring=True)
# changing color matrix from '470bg' to '709' for vsDeOldify
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# changing range from limited to full range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using DeOldify
from vsdeoldify import HAVC_main
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0, DeepExOnlyRefFrames=True, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
# adjusting output color from: RGB24 to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# output
clip.set_output()Cu Selur
RE: Deoldify Vapoursynth filter - Selur - 22.09.2024
Quote: I adjusted the code to always allow changing the "Ref FrameDir". (If nothing is set, Hybrid will use the temp folder)
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExOnlyRefFrames=True, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)2024-09-22 12:54:46.928
Failed to evaluate the script:
Python exception: HAVC_deepex: only_ref_frames is enabled but method != 0 (HAVC)
Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate
File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate
File "J:\tmp\tempPreviewVapoursynthFile12_54_29_135.vpy", line 48, in
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExOnlyRefFrames=True, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 282, in HAVC_main
clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 427, in HAVC_deepex
raise vs.Error("HAVC_deepex: only_ref_frames is enabled but method != 0 (HAVC)")
vapoursynth.Error: HAVC_deepex: only_ref_frames is enabled but method != 0 (HAVC)RE: Deoldify Vapoursynth filter - Dan64 - 22.09.2024
If ScFrameDir is not set need to be passed ScFrameDir=None.
If (depending on the method selected) will be issued an error, the user have to fix the input.
Just to clarify further, DeepExOnlyRefFrames=True can be used only with DeepExMethod=0.
Then is necessary to run the encode (the encode video is not usable) or the Benchmark (non encode video is generated).
After having created the ref frames and eventually fixed the colors of wrong frames, you can put "only" the fixed frames in another directory, for example "temp_fix"
The you can colorize the video using both the ref frames automatically generated by HAVC than the fixed ref frames with the following command
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExOnlyRefFrames=False, ScFrameDir="J:/tmp_fix", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)your command raised the error because you left DeepExOnlyRefFrames=True, it need to be set to False for the "True" encoding.
Dan
RE: Deoldify Vapoursynth filter - Selur - 22.09.2024
Quote:Just to clarify further, DeepExOnlyRefFrames=True can be used only with DeepExMethod=0.So DeepExOnlyRefFrames is a two pass process which is not supported by Hybrid, correct?
Then is necessary to run the encode (the encode video is not usable) or the Benchmark (non encode video is generated).
=> So atm. the correct way to handle it, is to remove that option from Hybrid.
Uploaded a new test version which:
1. removes DeepExOnlyRefFrames (might come back in case Hybrid supports 2pass handling)
2. uses 'None' if Ref. FrameDir isn't set by the user.
Now, enabling DeOldify with default values in Hybrid creates:
clip = HAVC_main(clip=clip)clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0, ScFrameDir=None, DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)So if any method other than 'HAVC' is used, the user must set 'Ref. FrameDir' to a folder with reference frames.
Cu Selur