Deoldify Vapoursynth filter

[Dan64]

It's Ok

Dan

[Selur]

Would be good if you could do checks like "HAVC_deepex: method != 0 but sc_framedir is unset" also in HAVC_main before calling HAVC_deepex since atm. one has to wait unnecessarily to get that message.

Updated test version, to include two additional checks that will throw an error during script creation:
a. DeepExMethod != 0 requires vsDeOldifySceneFramedir to be set
b. Preview will not be started if ColorMNet + sync + frame=0 is used.

Cu Selur

[Dan64]

New RC13 release, where I added some controls (including the the one on DeepExMethod).
I added 2 additional selections for the parameter VideoTune in HAVC_main: deoldify (the more stable) and ddcolor (the more vivid and less stable).

Dan

[Selur]

Will test tomorrow and report back.
away from my pc for today

[Dan64]

I tested the sync mode and is working.
Since the pre-process could take hours to complete (depending on the number of frames).
There is something that I can print in output to help you to take track of the progress of the pre-process ?

Dan

[Selur]

In theory, vsViewer should see Vapoursynth info messages and pass them through to Hybrid.
Those should be visible in Hybrids debug output iirc. and if they are there, I could write some code to turn the outputs into some sort of progress indication,...

[Dan64]

This is a sample of current output

D:\Programs\Hybrid\64bit\Vapoursynth\vspipe.exe "sample_720p_2m_colormnet.vpy" - -c y4m -p   | D:\Programs\Hybrid\64bit\x265.exe --preset fast --input - --fps 25/1 --output-depth 10 --y4m --profile main10 --b-adapt 2 --crf 20.00 --psy-rd 2.00 --deblock=-1:-1 --no-mcstf --psnr --ssim --range limited --colormatrix bt470bg --sar 1:1 --output "sample_720p_2m.265"
Warning: D:\Programs\Hybrid\64bit\Vapoursynth\Lib\site-packages\kornia\feature\lightglue.py:44: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd(cast_inputs=torch.float32)
Warning: D:\Programs\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\deepex\models\vgg19_gray.py:130: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  model.load_state_dict(torch.load(vgg19_gray_path))
Warning: HAVC_deepex:  gpu_mem_total = 12.0 GB, max_memory_frames = 10000
Warning: Start coloring 3021 frames...
Warning: Start coloring from 0 to 1000
Warning: Colored Frame: 0
Warning: Colored Frame: 50
Warning: Colored Frame: 100
Warning: Colored Frame: 150
Warning: Colored Frame: 200
Warning: Colored Frame: 250
Warning: Colored Frame: 300
Warning: Colored Frame: 350
Warning: Colored Frame: 400
Warning: Colored Frame: 450
Warning: Colored Frame: 500
Warning: Colored Frame: 550
Warning: Colored Frame: 600
Warning: Colored Frame: 650
Warning: Colored Frame: 700
Warning: Colored Frame: 750
Warning: Colored Frame: 800
Warning: Colored Frame: 850
Warning: Colored Frame: 900
Warning: Colored Frame: 950
Warning: Start coloring from 1000 to 2000
Warning: Colored Frame: 1000
Warning: Colored Frame: 1050
Warning: Colored Frame: 1100
Warning: Colored Frame: 1150
Warning: Colored Frame: 1200
Warning: Colored Frame: 1250
Warning: Colored Frame: 1300
Warning: Colored Frame: 1350


Warning: Colored Frame: 2950
Warning: Start coloring from 3000 to 3021
Warning: Colored Frame: 3000
Script evaluation done in 411.98 seconds
y4m  [info]: 1280x920 fps 25/1 i420p10 sar 1:1 unknown frame count
raw  [info]: output file: sample_720p_2m.265
x265 [info]: HEVC encoder version 3.6+36-4f8996985
x265 [info]: build info [Windows][GCC 14.1.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [warning]: --psnr used with psy on: results will be invalid!
x265 [warning]: --tune psnr should be used if attempting to benchmark psnr!
x265 [info]: Main 10 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 20 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 4 / wpp(15 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
x265 [info]: Keyframe min / max / scenecut / bias  : 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 15 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-20.0 / 0.60
x265 [info]: tools: rd=2 psy-rd=2.00 rskip mode=1 signhide tmvp fast-intra
x265 [info]: tools: strong-intra-smoothing lslices=5 deblock(tC=-1:B=-1) sao
1086 frames: 2.30 fps, 2164.02 kb/s

I'm using the function

vs.core.log_message(2, "Colored Frame: " + str(i))

To write in output the messages.

Let me know if I have to change the output to help you to report the progress.

Thanks,
Dan

[Selur]

If you know the overall frames that get colored, output: "Colored Frame: current/all" this way reporting progress will be easier.

[Selur]

Also, does that output appear in the Hybrids debug output somewhere?

[Dan64]

Updated test version, to include two additional checks that will throw an error during script creation:
a. DeepExMethod != 0 requires vsDeOldifySceneFramedir to be set

Please remove them because I added the controls in HAVC_main.

Thanks,

---

Also, does that output appear in the Hybrids debug output somewhere?

I attached my debug output, but to me seems that is not shown the vspipe output.

In meanwhile I discovered a bug regarding the debug output.

In my configuration I set to adjust the path with output folder, but this adjustment do not apply any more on debug output.



As you can see I switched to path sample2 (I provided in input a clip stored in sample2), but the debug output was not updated (still the old path sample3).
Even worse, if I unset the debug output, the output file is not saved.
Please fix.

Dan