Deoldify Vapoursynth filter

[Selur]

side note: you should make sure to crop away all black bars before applying retinex

No, there is no way to reverse the effect of applying Retinex.
You could apply it after the colorization, by adjusting the FilterOrder under 'Filtering->Vapoursynth->Filter Order/Queue'.
(read: [INFO] About Vapoursynth Filter Order/Queue)

With a custom script one could:
a. copy the clip before applying Retinex
a. apply Retinex on the clip
b. apply DeOldify on the clip
c. take the chroma from the filtered clip to the previously copied clip

!Untested! (just an idea, some small adjustements might be needed)
In Hybrid this could probably be done by:
1. creating a custom section before applying Retinex

global before_retinex = clip

2. creating a custom section after applying DeOldify

chroma = core.std.ShufflePlanes(clip, planes=[1, 2], colorfamily=vs.YUV)
clip = core.std.ShufflePlanes(clips=[before_retinex, chroma], planes=[0, 1, 2], colorfamily=vs.YUV)

Cu Selur

[zspeciman]

Thank you Selur, the cropping out the black border works really well.  There was very little cropping but the difference is significant.
The reason I wanted to reverse Retinex is these blocks that are formed. See attached pics.  Any suggestions how to reduce that?  I thought restoring the video to its original darkness would do the trick.   Perhaps there is a dithering method to apply here, if such a tool exist.

[Selur]

You could apply strong blocking with a LimitMask before Retinex, that should at least reduce the blocking.

[hallomanbh]

HI

ImportError: DLL load failed while importing spatial_correlation_sampler_backend: The specified procedure could not be found.


2024-12-10 22:05:35.422
Script was successfully evaluated. Output video info:
Frames: 3918 | Time: 0:02:10.731 | Size: 2560x720 | FPS: 30000/1001 = 29.97 | Format: YUV420P8
2024-12-10 22:05:37.857
[VSE Server]: incoming connection
[VSE Server]: ConnectedState
[VSE Server]: socket is ready to be read
[VSE Server]: connection open: true
[VSE Server]: connection readable: true
[VSE Server] - Message received: changeTo ### C:\Users\SOS\AppData\Local\Temp\tempPreviewVapoursynthFile22_05_37_845.vpy ###
2024-12-10 22:06:05.933
Failed to evaluate the script:
Python exception: DLL load failed while importing spatial_correlation_sampler_backend: The specified procedure could not be found.

Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate
File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate
File "C:\Users\SOS\AppData\Local\Temp\tempPreviewVapoursynthFile22_05_37_845.vpy", line 45, in
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 317, in HAVC_main
clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 615, in HAVC_deepex
clip_colored = vs_colormnet(clip, clip_ref, clip_sc, image_size=-1, enable_resize=enable_resize,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\vsslib\vsmodels.py", line 48, in vs_colormnet
return vs_colormnet_local(clip, clip_ref, clip_sc, image_size, enable_resize, frame_propagate,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\__init__.py", line 52, in vs_colormnet_local
colorizer = ColorMNetRender(image_size=image_size, vid_length=vid_length, enable_resize=enable_resize,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\colormnet_render.py", line 90, in __init__
self._colorize_init(image_size, vid_length, propagate)
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\colormnet_render.py", line 144, in _colorize_init
self.network = ColorMNet(self.config, self.config['model']).cuda().eval()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\model\network.py", line 37, in __init__
self.short_term_attn = LocalGatedPropagation(d_qk=64, # 256
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\model\attention.py", line 763, in __init__
from spatial_correlation_sampler import SpatialCorrelationSampler
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\spatial_correlation_sampler\__init__.py", line 1, in
from .spatial_correlation_sampler import SpatialCorrelationSampler, spatial_correlation_sample
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\spatial_correlation_sampler\spatial_correlation_sampler.py", line 6, in
import spatial_correlation_sampler_backend as correlation
ImportError: DLL load failed while importing spatial_correlation_sampler_backend: The specified procedure could not be found.
2024-12-10 22:06:06.168
[VSE Server] - Message received: changeTo ### C:\Users\SOS\AppData\Local\Temp\tempPreviewVapoursynthFile22_05_42_532.vpy ###
2024-12-10 22:06:13.389
Failed to evaluate the script:
Python exception: DLL load failed while importing spatial_correlation_sampler_backend: The specified procedure could not be found.
Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate
File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate
File "C:\Users\SOS\AppData\Local\Temp\tempPreviewVapoursynthFile22_05_42_532.vpy", line 45, in
clip = HAVC_main(clip=clip, Preset="medium", EnableDeepEx=True, DeepExMethod=0, DeepExPreset="slow", DeepExRefMerge=3, ScFrameDir="C:/Users/SOS/Videos/NTR/new ref", ScMinFreq=1, ScNormalize=True, DeepExModel=0, DeepExEncMode=1, DeepExMaxMemFrames=45)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 317, in HAVC_main
clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 615, in HAVC_deepex
clip_colored = vs_colormnet(clip, clip_ref, clip_sc, image_size=-1, enable_resize=enable_resize,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\vsslib\vsmodels.py", line 48, in vs_colormnet
return vs_colormnet_local(clip, clip_ref, clip_sc, image_size, enable_resize, frame_propagate,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\__init__.py", line 52, in vs_colormnet_local
colorizer = ColorMNetRender(image_size=image_size, vid_length=vid_length, enable_resize=enable_resize,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\colormnet_render.py", line 90, in __init__
self._colorize_init(image_size, vid_length, propagate)
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\colormnet_render.py", line 144, in _colorize_init
self.network = ColorMNet(self.config, self.config['model']).cuda().eval()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\model\network.py", line 37, in __init__
self.short_term_attn = LocalGatedPropagation(d_qk=64, # 256
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\model\attention.py", line 763, in __init__
from spatial_correlation_sampler import SpatialCorrelationSampler
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\spatial_correlation_sampler\__init__.py", line 1, in
from .spatial_correlation_sampler import SpatialCorrelationSampler, spatial_correlation_sample
File "C:\Program Files\Hybrid\64bit\Vapoursynth\Lib\site-packages\spatial_correlation_sampler\spatial_correlation_sampler.py", line 6, in
import spatial_correlation_sampler_backend as correlation
ImportError: DLL load failed while importing spatial_correlation_sampler_backend: The specified procedure could not be found.

[Selur]

No debug output, no info on the settings and versions used.
=> at least to me, that is useless.
Seeing the resolution "2560x720" I suspect it might be simply an issue of not enough vram.

[Selur]

If you can delay the release is better, I'm still working on improving the scene change detection algorithm.

Any update on that?

Cu Selur

[Dan64]

I just released the version: v4.5.1

I Improved Scene detection algorithm (implemented in the class SceneDetection).

In HAVC_main() have been added the following parameters:

• ScThtOffset: Offset index used for the Scene change detection. The comparison will be performed, between frame[n] and frame[n-offset]. An offset > 1 is useful to detect blended scene change, range[1, 25]. Default = 1.
  
• ScMinInt: Minimum number of frame interval between scene changes, range[1, 25]. Default = 1.
  

In HAVC_ddeoldify() have been added the following parameters:

• sc_tht_offset: Offset index used for the Scene change detection. The comparison will be performed, between frame[n] and frame[n-offset]. An offset > 1 is useful to detect blended scene change, range[1, 25]. Default = 1.
  
• sc_min_int: Minimum number of frame interval between scene changes, range[1, 25]. Default = 1.
  

In the class SceneDetection I reimplemented the Vapoursynth function misc.SCDetect using the Vapoursynth function std.PlaneStats.

The new implementation is activated if the following conditions are verified: sc_tht_filter > 0.0 or threshold < 0.05 or tht_offset > 1.

The suggested parameters for scene detection are:

• ScThreshold=0.10 
  
• ScThtOffset=10 
  
• ScMinFreq=15
  
• ScMinInt=5 
  
• ScThtSSIM=0.65
  
• ScNormalize=True
  

Dan

[Selur]

Nice, will adjust the test version tomorrow and send you a link.

[Selur]

Updated the test download, let me know whether it works as expected.

Cu Selur

[Dan64]

I installed the dev version: Hybrid_dev_2024.12.14-162619.exe

But in this version are missing the new parameters added in v4.5.1




Could you check ?

Dan