Selur's Little Message Board - Deoldify Vapoursynth filter

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143

@Dan64: using:

Code:
# Imports
import sys
import os
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Limit frame cache to 48449MB
core.max_cache_size = 48449
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/BestSource/BestSource.dll")
# Import scripts
import vsdeoldify as havc
import validate
# Source: 'G:\TestClips&Co\files\test.avi'
# Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual
# Loading 'G:\TestClips&Co\files\test.avi' using BestSource
clip = core.bs.VideoSource(source="G:/TestClips&Co/files/test.avi", cachepath="J:/tmp/test_bestSource", track=0, hwdevice="opencl")
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_LIMITED})
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # scan type: progressive
original = clip
# changing color matrix from '470bg' to '709' for vsHAVC
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# adjusting color space from YUV420P8 to RGB24 for vsHAVC
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_in_s="limited", range_s="full")
# adding colors using HAVC
clip = havc.HAVC_main(clip, EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0, ScFrameDir=None, DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsHAVC
# adjusting output color from YUV420P8 to YUV420P10 for NVEncModel
original = core.resize.Bicubic(clip=original, format=vs.YUV420P10)
# adjusting output color from RGB24 to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_in_s="full", range_s="limited") # additional resize to match target color sampling
original = core.text.Text(clip=original,text="Original",scale=1,alignment=7)
clip = core.text.Text(clip=clip,text="Filtered",scale=1,alignment=7)
stacked = core.std.StackHorizontal([original,clip])
# set output frame rate to 25fps (progressive)
stacked = core.std.AssumeFPS(clip=stacked, fpsnum=25, fpsden=1)
# output
stacked.set_output()

the HAVC call is:

Code:
clip = havc.HAVC_main(clip, EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0, ScFrameDir=None, DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

I get:

Code:
2026-04-25 20:03:04.209
Failed to evaluate the script:
Python exception: CMNET2: number of reference frames must be at least 2, found 2

Traceback (most recent call last):
File "vapoursynth.pyx", line 3524, in vapoursynth._vpy_evaluate
File "vapoursynth.pyx", line 3525, in vapoursynth._vpy_evaluate
File "J:\tmp\tempPreviewVapoursynthFile20_01_43_472.vpy", line 44, in 
clip = havc.HAVC_main(clip, EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0, ScFrameDir=None, DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 321, in HAVC_main
return HAVC_main_presets(clip, Preset, FrameInterp, ColorModel, CombMethod, VideoTune, ColorFix,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 503, in HAVC_main_presets
clip_colored = HAVC_main_colorizer(clip, Preset, ColorModel, CombMethod, VideoTune, ColorFix, ColorTemp,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 840, in HAVC_main_colorizer
clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 1728, in HAVC_deepex
clip_colored = vs_colormnet2(clip, clip_ref, clip_sc, image_size=-1, enable_resize=enable_resize,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\vsslib\vsmodels.py", line 70, in vs_colormnet2
return vs_colormnet2_remote(clip, clip_ref, clip_sc, image_size, enable_resize, frame_propagate,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\colormnet2\__init__.py", line 138, in vs_colormnet2_remote
clip_colored = _colormnet2_client(colorizer, clip, clip_ref, clip_sc, frame_propagate, ref_weight,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\colormnet2\__init__.py", line 155, in _colormnet2_client
reader.load_clip_ref(clip_ref, clip_sc, window_size=max_memory_frames)
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\colormnet2\colormnet2_utils.py", line 105, in load_clip_ref
self.get_clip_ref_list(sc, start_frame=start_frame, window_size=window_size)
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\colormnet2\colormnet2_utils.py", line 92, in get_clip_ref_list
HAVC_LogMessage(MessageType.EXCEPTION,
File "f:\hybrid\64bit\vapoursynth\Lib\site-packages\vsdeoldify\vsslib\vsutils.py", line 45, in HAVC_LogMessage
raise vs.Error(message_text)
vapoursynth.Error: CMNET2: number of reference frames must be at least 2, found 2

Cu Selur

Ps.: also uploaded a new Hybrid_havc_test

Please set ScMinFreq=15, I left some of defaults value of the previous release, CMNET2 need a stable source of reference frames and with the default value of threshold=0.10, it seems that was able to find only 2 reference frames.
I will update the RC.

Dan

But "number of reference frames must be at least 2, found 2" => shouldn't two be enough? Or did you mean that it need '3 or more' `?

Quote:Please set ScMinFreq=15, I left some of defaults value of the previous release,

Don't really get that sentence, but using 'ScMinFreq=15':

Code:
clip = havc.HAVC_main(clip, ColorFix="none", BlackWhiteTune="none", EnableDeepEx=True, DeepExMethod=0, DeepExPreset="medium", DeepExRefMerge=0, ScFrameDir=None, ScMinFreq=15, ScNormalize=True, DeepExModel=3, DeepExEncMode=0)

doesn't crash.

Let me know when defaults change, so that I can adjust the gui accordingly.

Cu Selur

Tomorrow I will release a new RC with the updated defaults.

Dan

Okay.

When is selected the model CMNET2, must be available the same options available for ColorMNet: Mode (remote/local), Frames (number of memory frames).
The Mode="remote all-ref" has been removed in 5.8.0. It was my unsuccessful attempt to implement a sliding memory window.
This approach has been implemented in CMNET2 and thanks to the permanent memory of Xmem++, now is working very well.

Dan

Uploaded an updated Hybrid_havc_test, let me know how that works. Smile

Cu Selur

When I select CMNET2 is displayed this panel

[Image: attachment.php?aid=3568]

The option Mode and Frames are missing, in this case should be displayed the same options of ColorMNet as shown below

[Image: attachment.php?aid=3569]

Dan

Did you use the today updated Hybrid_havc_test?
Because for me it shows:

Cu Selur

Now it works (probably I unzipped the old version)

I attached the new RC5.

Please check your defaults, the current ones are:

Code:
HAVC_main(clip: vs.VideoNode, 
Preset: str = 'Medium', 
FrameInterp: int = 0,  
ColorModel: str = 'Video+ModelScope',
CombMethod: str = 'Simple',  
VideoTune: str = 'VeryVivid', 
ColorFix: str = 'Retinex/Red',
ColorTune: str = 'Light', 
ColorMap: str = 'Red->Brown', 
ColorTemp: str = "None", 
BlackWhiteTune: str = 'Light',
BlackWhiteMode: int = 0, 
BlackWhiteBlend: bool = True, 
EnableDeepEx: bool = True, 
DeepExMethod: int = 0,
DeepExPreset: str = 'Auto', 
DeepExRefMerge: int = 0, 
DeepExOnlyRefFrames: bool = False,
ScFrameDir: str = None, 
ScThreshold: float = 0.10, 
ScThtOffset: int = 1, 
ScMinFreq: int = 15,
ScMinInt: int = 1, 
ScThtSSIM: float = 0.0, 
ScNormalize: bool = False, 
DeepExModel: int = 0,
DeepExVivid: bool = False, 
DeepExEncMode: int = 0, 
DeepExMaxMemFrames=20, 
RefRange: tuple[int, int] = (0, 0),
enable_fp16: bool = True, debug_level: int = 0)

The main change is that now the section "Exemplar Models" is enabled by default and configured to use CMNET2.

Dan