Deoldify Vapoursynth filter

[Selur]

Okay.
Anything that needs to be downloaded once to setup vs-dedoldify should end somewhere below the vsdeoldify folder.
This way, I can make a torchAddon with all the stuff in it.
Anything that is generated during runtime should be in a configurable folder.
So that if Hybrid is normally installed, this can be by default set to the temp folder (or in newer versions the engine folder) and changed by the user.
vsdeoldify should throw an error is it can't write to somewhere where is need to write and not silently quit, so that Hybrid and the user can know whats happening.

Cu Selur

[Selur]

Updated the download to make sure ScFrameDir/sc_framedir is only set when set by the user, otherwise it stays at None.
This causes:

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExModel=1)

to fail with:

HAVC_deepex: method != 0 but sc_framedir is unset

=> I'll adjust Hybrid to set ScFrameDir if it's empty.
=> Updated the download
Cu Selur

[Dan64]

In the GUI when Ref merge <> "no", threshold is receiving the value of weight. 
If not changed the default value of threshold is 0.10.

Dan

P.S.
ColorMNet is working on your side ?

[Selur]

Hybrid, like the code:

if enable_refmerge:
        if ref_weight is None:
            ref_weight = refmerge_weight[ref_merge]
        if ref_thresh is None:
            ref_thresh = 0.1
        clip_sc = SceneDetect(clip, threshold=ref_thresh)
        if method in (1, 2):
            clip_sc = SceneDetectFromDir(clip_sc, sc_framedir=sc_framedir, merge_ref_frame=True,
                                         ref_frame_ext=(method == 2))
    else:
        ref_weight = 1.0
        clip_sc = None

when 'Ref merge' switches to 'no' Hybrid sets 'Threshold' to 1.
Hybrid starts out with 'threshold 0.1' due to:

def HAVC_main(clip: vs.VideoNode, Preset: str = 'Fast', VideoTune: str = 'Stable', ColorFix: str = 'Violet/Red',
              ColorTune: str = 'Light', ColorMap: str = 'None', EnableDeepEx: bool = False, DeepExMethod: int = 0,
              DeepExPreset: str = 'Fast', DeepExRefMerge: int = 0, ScFrameDir: str = None, ScThreshold: float = 0.1,
              ScMinFreq: int = 0, DeepExModel: int = 0,
              enable_fp16: bool = True)

[Dan64]

The box "Weight" should contain the value of parameter "ref_weight".
As shown in the code "ref_weight"  can have the value "refmerge_weight[ref_merge]" or 1.0 if ref_merge = 0.
The problem is that in the GUI is see these values in the box "Threshold" not in the box "Weight".



Moreover "ref_merge" should be enabled only when method is in: 0, 1, 2

In the code there is this check:

    if ref_merge > 0 and method > 2:
        raise vs.Error("HAVC_deepex: method must be in range [0-2] to be used with ref_merge > 0")

    if method in (0, 1, 2):
        sc_threshold, sc_frequency = get_sc_props(clip_ref)
        if sc_threshold == 0 and sc_frequency == 0:
            raise vs.Error("HAVC_deepex: method in (0, 1, 2) but sc_threshold and sc_frequency are not set")
        if ref_merge > 0 and sc_frequency != 1:
            raise vs.Error("HAVC_deepex: method in (0, 1, 2) and ref_merge > 0 but sc_frequency != 1")

 

Dan

[Selur]

The problem is that in the GUI is see these values in the box "Threshold" not in the box "Weight".

should be fixed now
Not sure whether I got the rest, correctly.
(Updated test version)
---
Okay, got the models&co under Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\models expexted them under colormnet somewhere.


Cu Selur

[Selur]

using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import validate
# Source: 'G:\TestClips&Co\files\test.avi'
# Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual
# Loading G:\TestClips&Co\files\test.avi using LWLibavSource
clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# changing color matrix from '470bg' to '709' for vsDeOldify
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# changing range from limited to full range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using DeOldify
from vsdeoldify import HAVC_main
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=1, ScFrameDir="J:/tmp", ScMinFreq=1, DeepExModel=0)
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
# adjusting output color from: RGB24 to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# output
clip.set_output()

I get:

Error on frame 0 request:

Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3194, in vapoursynth.publicFunction
File "src\\cython\\vapoursynth.pyx", line 3196, in vapoursynth.publicFunction
File "src\\cython\\vapoursynth.pyx", line 829, in vapoursynth.FuncData.__call__
TypeError: vs_colormnet..colormnet_clip_color_merge() got an unexpected keyword argument 'propagate'

Cu Selur

[Dan64]

Under colormnet must be put only the file DINOv2FeatureV6_LocalAtten_s2_154000.pth that must be saved in "\Lib\site-packages\vsdeoldify\colormnet\weights"
The location of torch cache is the same for all the models used by vsdeoldify.

Dan

[Selur]

I had DINOv2FeatureV6_LocalAtten_s2_154000 in Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\weights so that is correct.

The location of torch cache is the same for all the models used by vsdeoldify.

Okay.
=> so if the files are all there, deoldify should not need to write anywhere under "Program Files" on normal installations.

Cu Selur

[Dan64]

=> so if the files are all there, deoldify should not need to write anywhere under "Program Files" on normal installations.

 It is called cache because is a temporary store. If a network file stored in the cache is updated on the torch repository, the first time that the network is used by torch, the cache will be updated with the new version. This is not a frequent situation but in theory it could happen.

Dan