Deoldify Vapoursynth filter - Printable Version

Deoldify Vapoursynth filter - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: Deoldify Vapoursynth filter (/thread-3595.html)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Okay.
Anything that needs to be downloaded once to setup vs-dedoldify should end somewhere below the vsdeoldify folder.
This way, I can make a torchAddon with all the stuff in it.
Anything that is generated during runtime should be in a configurable folder.
So that if Hybrid is normally installed, this can be by default set to the temp folder (or in newer versions the engine folder) and changed by the user.
vsdeoldify should throw an error is it can't write to somewhere where is need to write and not silently quit, so that Hybrid and the user can know whats happening.

Cu Selur

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Updated the download to make sure ScFrameDir/sc_framedir is only set when set by the user, otherwise it stays at None.
This causes:

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExModel=1)

to fail with:

HAVC_deepex: method != 0 but sc_framedir is unset

=> I'll adjust Hybrid to set ScFrameDir if it's empty.
=> Updated the download
Cu Selur

RE: Deoldify Vapoursynth filter - Dan64 - 15.09.2024

In the GUI when Ref merge <> "no", threshold is receiving the value of weight.
If not changed the default value of threshold is 0.10.

Dan

P.S.
ColorMNet is working on your side ?

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Hybrid, like the code:

if enable_refmerge:

        if ref_weight is None:

            ref_weight = refmerge_weight[ref_merge]

        if ref_thresh is None:

            ref_thresh = 0.1

        clip_sc = SceneDetect(clip, threshold=ref_thresh)

        if method in (1, 2):

            clip_sc = SceneDetectFromDir(clip_sc, sc_framedir=sc_framedir, merge_ref_frame=True,

                                         ref_frame_ext=(method == 2))

    else:

        ref_weight = 1.0

        clip_sc = None

when 'Ref merge' switches to 'no' Hybrid sets 'Threshold' to 1.
Hybrid starts out with 'threshold 0.1' due to:

def HAVC_main(clip: vs.VideoNode, Preset: str = 'Fast', VideoTune: str = 'Stable', ColorFix: str = 'Violet/Red',

              ColorTune: str = 'Light', ColorMap: str = 'None', EnableDeepEx: bool = False, DeepExMethod: int = 0,

              DeepExPreset: str = 'Fast', DeepExRefMerge: int = 0, ScFrameDir: str = None, ScThreshold: float = 0.1,

              ScMinFreq: int = 0, DeepExModel: int = 0,

              enable_fp16: bool = True)

RE: Deoldify Vapoursynth filter - Dan64 - 15.09.2024

The box "Weight" should contain the value of parameter "ref_weight".
As shown in the code "ref_weight" can have the value "refmerge_weight[ref_merge]" or 1.0 if ref_merge = 0.
The problem is that in the GUI is see these values in the box "Threshold" not in the box "Weight".

[Image: attachment.php?aid=2721]

Moreover "ref_merge" should be enabled only when method is in: 0, 1, 2

In the code there is this check:

    if ref_merge > 0 and method > 2:

        raise vs.Error("HAVC_deepex: method must be in range [0-2] to be used with ref_merge > 0")

    if method in (0, 1, 2):

        sc_threshold, sc_frequency = get_sc_props(clip_ref)

        if sc_threshold == 0 and sc_frequency == 0:

            raise vs.Error("HAVC_deepex: method in (0, 1, 2) but sc_threshold and sc_frequency are not set")

        if ref_merge > 0 and sc_frequency != 1:

            raise vs.Error("HAVC_deepex: method in (0, 1, 2) and ref_merge > 0 but sc_frequency != 1")

Dan

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Quote:The problem is that in the GUI is see these values in the box "Threshold" not in the box "Weight".

should be fixed now
Not sure whether I got the rest, correctly.
(Updated test version)
---
Okay, got the models&co under Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\models expexted them under colormnet somewhere. Smile

Cu Selur

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

using:

# Imports

import vapoursynth as vs

# getting Vapoursynth core

import sys

import os

core = vs.core

# Import scripts folder

scriptPath = 'F:/Hybrid/64bit/vsscripts'

sys.path.insert(0, os.path.abspath(scriptPath))

# loading plugins

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")

# Import scripts

import validate

# Source: 'G:\TestClips&Co\files\test.avi'

# Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual

# Loading G:\TestClips&Co\files\test.avi using LWLibavSource

clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)

frame = clip.get_frame(0)

# setting color matrix to 470bg.

clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)

# setting color transfer (vs.TRANSFER_BT601), if it is not set.

if validate.transferIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)

# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.

if validate.primariesIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)

# setting color range to TV (limited) range.

clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)

# making sure frame rate is set to 25fps

clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)

# making sure the detected scan type is set (detected: progressive)

clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive

# changing color matrix from '470bg' to '709' for vsDeOldify

clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")

# changing range from limited to full range for vsDeOldify

clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")

# setting color range to PC (full) range.

clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)

# adjusting color space from YUV420P8 to RGB24 for vsDeOldify

clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")

# adding colors using DeOldify

from vsdeoldify import HAVC_main

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=1, ScFrameDir="J:/tmp", ScMinFreq=1, DeepExModel=0)

# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify

# changing range from full to limited range for vsDeOldify

clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

# adjusting output color from: RGB24 to YUV420P10 for NVEncModel

clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")

# set output frame rate to 25fps (progressive)

clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)

# output

clip.set_output()

I get:

Error on frame 0 request:

Traceback (most recent call last):

File "src\\cython\\vapoursynth.pyx", line 3194, in vapoursynth.publicFunction

File "src\\cython\\vapoursynth.pyx", line 3196, in vapoursynth.publicFunction

File "src\\cython\\vapoursynth.pyx", line 829, in vapoursynth.FuncData.__call__

TypeError: vs_colormnet..colormnet_clip_color_merge() got an unexpected keyword argument 'propagate'

Cu Selur

RE: Deoldify Vapoursynth filter - Dan64 - 15.09.2024

Under colormnet must be put only the file DINOv2FeatureV6_LocalAtten_s2_154000.pth that must be saved in "\Lib\site-packages\vsdeoldify\colormnet\weights"
The location of torch cache is the same for all the models used by vsdeoldify.

Dan

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

I had DINOv2FeatureV6_LocalAtten_s2_154000 in Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\weights so that is correct. Smile

Quote:The location of torch cache is the same for all the models used by vsdeoldify.

Okay.
=> so if the files are all there, deoldify should not need to write anywhere under "Program Files" on normal installations. Smile

Cu Selur

RE: Deoldify Vapoursynth filter - Dan64 - 15.09.2024

(15.09.2024, 19:30)Selur Wrote: => so if the files are all there, deoldify should not need to write anywhere under "Program Files" on normal installations.

It is called cache because is a temporary store. If a network file stored in the cache is updated on the torch repository, the first time that the network is used by torch, the cache will be updated with the new version. This is not a frequent situation but in theory it could happen.

Dan