Deoldify Vapoursynth filter - Printable Version

Deoldify Vapoursynth filter - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: Deoldify Vapoursynth filter (/thread-3595.html)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81

RE: Deoldify Vapoursynth filter - Dan64 - 15.09.2024

Hello Selur,

   you are right, I added these parameters in the past but I never released a new version.
   My intention is to rework completely the scene change methodology.
   I was thinking to use PySceneDetect , but I had no time to test it till now.

   I released a new RC2 to fix some problems related to the max_memory_frames.
   For unknown reasons the memory usage of ColorMNet in Vapoursynth increase exponentially, while if the same code is called in a python script (using frames saved on the disk) the memory usage is very low as described in the paper. But in Vapoursynth it is a nightmare, because this filter is taking advantage for using the frames memory and I had to adapt the code to force the filter to use only a limited number of frames.
   I don't have understood the reason. There is nothing on internet about this problem and it is not possible to fully debug Vapoursynth scripts.
   For the moment I found this solution.
   My be I will be able to improve the memory management, but for the moment this is the only solution that I found (try to set max_memory_frames = 0 to see the problem).

Thanks,
Dan

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Okay, will start again reading the function, or should I better wait a week or so before trying to adjust Hybrid to this?

Cu Selur

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

btw.:

:param ref_weight:          If enable_refmerge = True, represent the weight used to merge the reference frames.

                                If is not set, is assigned automatically a value of 0.5

the description, does not match the code:

if enable_refmerge:

        if ref_weight is None:

            ref_weight = refmerge_weight[ref_merge]

        if ref_thresh is None:

            ref_thresh = 0.1

        clip_sc = SceneDetect(clip, threshold=ref_thresh)

        if method in (1, 2):

            clip_sc = SceneDetectFromDir(clip_sc, sc_framedir=sc_framedir, merge_ref_frame=True,

                                         ref_frame_ext=(method == 2))

    else:

        ref_weight = 1.0

        clip_sc = None

Cu Selur

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Send you a link to a not properly working dev version for testing, totally unsure whether everything is en-/disabled as it should.
Since some other stuff seems to have changed too.
I thought that:

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0)

should use ColorMNet (since it's the default in HAVC_deepex), but it does not seem like anything got downloaded,...

Cu Selur

RE: Deoldify Vapoursynth filter - Dan64 - 15.09.2024

I like the new GUI Smile

I will test it to see if everything is Ok.

Regarding the parameters ref_weight, ref_thresh they were already available in the release 4.0.0: __init__.py
But has been added to handle very specific situations and it is right that are not included in the GUI.

I changed HAVC_main() to include also the parameter DeepExModel (see attached RC3) so that it will be possible to select the Exemplar based model to use for colors propagation.

Thanks,
Dan

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Quote:But has been added to handle very specific situations and it is right that are not included in the GUI.

I already did add them to the gui,...
=> does not seem like ColorMNet is used when calling:

from vsdeoldify import HAVC_main

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, DeepExModel=0)

There was no download,... => What should be downloaded to where when DeepExModel=0 <> ColorMNet is used?

Updated the test download (same link).

Cu Selur

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Also, I'm totally unsure when Ref merge, Ref FrameDir and Reference Frames only should be enabled and set and then not.
=> need some details

RE: Deoldify Vapoursynth filter - Dan64 - 15.09.2024

(15.09.2024, 14:26)Selur Wrote: => does not seem like ColorMNet is used when calling:

from vsdeoldify import HAVC_main clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, DeepExModel=0)
There was no download,... => What should be downloaded to where when DeepExModel=0 <> ColorMNet is used?

I the torch cache dir are downloaded:

the folder repo: facebookresearch_dinov2_main (it seems strange but torch is capable to do it)
under the folder chekpoints are downloaded the following networks:

resnet18-5c106cde.pth
dinov2_vits14_pretrain.pth
resnet50-19c8e357.pth

having decided to set the the torch cache under the model filter's folder, implies that Hybrid need to be installed in a writeable directory.

Dan

RE: Deoldify Vapoursynth filter - Selur - 15.09.2024

Quote: having decided to set the the torch cache under the model filter's folder, implies that Hybrid need to be installed in a writeable directory.

ARGH,...
No! The torchAddon needs to include the files.
Hybrid needing to be installed into a writeable directory is a NO GO.
Anything that needs to be done during runtime should not end in a program folder.

Cu Selur

RE: Deoldify Vapoursynth filter - Dan64 - 15.09.2024

It was a your requirement in this post: #7 and you confirmed the approach in this post: #12
I can change this behaviour, just let me know what are your requirements.

Dan