Deoldify Vapoursynth filter

[Dan64]

Here a simplified version, I added also the method=5.

[djilayeden]

Dear Dan,

Thank you for sharing your insights and results with DeepRemaster on Casablanca (1942). Your explanation of the challenges with ColorMNet and how DeepRemaster effectively handles desynchronized reference frames is truly fascinating. The approach of balancing past and future reference images makes a lot of sense, and the improvement is clearly visible in the blue-parrot scene, what a great example!

It’s exciting to see how DeepRemaster is pushing the boundaries of automated colorization, especially with its ability to store full frames in a tensor array for more consistent results. Adding it to the HAVC filter is a fantastic step forward, and I can’t wait to see what further refinements and applications will come next.

Thanks again for your dedication and for sharing your work. Bravo!

Best regards,

[Selur]

Too late for me today, but I'll try to adjust Hybrid to that tomorrow after work.

[djilayeden]

Dear Selur,

Good luck with the adjustments to Hybrid! I’m sure your implementation will make a big difference. Bravo for your dedication and hard work, it's truly appreciated!

Looking forward to seeing the results.

Best regards

[Selur]

@Dan64: A few questions:
a. HAVC_restore_video is not used, is that correct?
b. is it correct that HAVC_stabilize only gets used with deepEx?
Please update the graph accordingly.

How should ScFrameDir be set with method 5 (for a non-custom preset/speed)?

-----------------
The following code would be generated atm:

Default for a 470gb YUV source:

# changing color matrix from '470bg' to '709' for vsDeOldify
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# changing range from limited to full range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using DeOldify
from vsdeoldify import HAVC_main
clip = HAVC_main(clip=clip)
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

Switching B&W Tune to 'light':

# changing color matrix from '470bg' to '709' for vsDeOldify
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# changing range from limited to full range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using DeOldify
clip = HAVC_bw_tune(clip=clip, bw_tune="light", action="ON")
from vsdeoldify import HAVC_main
clip = HAVC_main(clip=clip, BlackWhiteTune="light")
clip = HAVC_bw_tune(clip=clip, bw_tune="light", action="OFF")
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

Enabling DeepEx:

# changing color matrix from '470bg' to '709' for vsDeOldify
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# changing range from limited to full range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using DeOldify
clip = HAVC_bw_tune(clip=clip, bw_tune="light", action="ON")
from vsdeoldify import HAVC_main
clipRef = HAVC_ddeoldify(clip=clip, deoldify_p=[0, 24, 1, 0], ddcolor_p=[1, 24, 1, 0, True])
clip = HAVC_main(clip=clip, BlackWhiteTune="light", EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0, DeepExOnlyRefFrames=False, clip_ref=clipRef, DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
clip = HAVC_bw_tune(clip=clip, bw_tune="light", action="OFF")
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

Setting DeepEx Method to 5 and setting test_ref.avi as reference => no clue what to set


Switching to Custom:
(So Speed: Custom, B&W Tune: light, DeepEx enabled, DeepExMethod: 5, test_ref.avi as reference)

# changing color matrix from '470bg' to '709' for vsDeOldify
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# changing range from limited to full range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using DeOldify
clip = HAVC_bw_tune(clip=clip, bw_tune="light", action="ON")
# loading reference clip for Remaster
clipRef = core.bs.VideoSource(source="G:/TestClips&Co/files/test_ref.avi")
if clipRef.num_frames != clip.num_frames:
  raise vs.Error(f"HAVC: Reference clip ({clipRef.num_frames}) and source clip frame count ({clip.num_frames}) differ!")
if clipRef.format.id != vs.RGB24:
  refframe = clipRef.get_frame(0)
  if refframe.props.get('_ColorRange', None) != vs.RANGE_FULL :
    clipRef = core.resize.Bicubic(clipRef, range_in_s="limited", range_s="full")
  if frame.props.get('_Matrix', None) != vs.MATRIX_BT709 :
    clipRef = core.resize.Bicubic(clipRef, matrix_in_s="470bg", matrix_s="709")
  clipRef = core.resize.Bicubic(clip=clipRef, width=clip.width, height= clip.height, format=vs.RGB24, matrix_in_s="709", range_s="full")
from vsdeoldify import HAVC_deepex
clip = HAVC_deepex(clip=clip, clip_ref=clipRef, method=5, ref_merge=0, sc_framedir=None, ex_model=0, max_memory_frames=0, encode_mode=0)
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

Enabling Chroma smoothing, Chroma Stabilizer, Darkness:

# changing color matrix from '470bg' to '709' for vsDeOldify
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# changing range from limited to full range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using DeOldify
clip = HAVC_bw_tune(clip=clip, bw_tune="light", action="ON")
# loading reference clip for Remaster
clipRef = core.bs.VideoSource(source="G:/TestClips&Co/files/test_ref.avi")
if clipRef.num_frames != clip.num_frames:
  raise vs.Error(f"HAVC: Reference clip ({clipRef.num_frames}) and source clip frame count ({clip.num_frames}) differ!")
if clipRef.format.id != vs.RGB24:
  refframe = clipRef.get_frame(0)
  if refframe.props.get('_ColorRange', None) != vs.RANGE_FULL :
    clipRef = core.resize.Bicubic(clipRef, range_in_s="limited", range_s="full")
  if frame.props.get('_Matrix', None) != vs.MATRIX_BT709 :
    clipRef = core.resize.Bicubic(clipRef, matrix_in_s="470bg", matrix_s="709")
  clipRef = core.resize.Bicubic(clip=clipRef, width=clip.width, height= clip.height, format=vs.RGB24, matrix_in_s="709", range_s="full")
from vsdeoldify import HAVC_deepex
clip = HAVC_deepex(clip=clip, clip_ref=clipRef, method=5, ref_merge=0, dark=True, smooth=True, sc_framedir=None, ex_model=0, max_memory_frames=0, encode_mode=0)
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

Cu Selur

[Dan64]

Hello Selur,

   I reworked the code, to simplify the reading and released a new RC3 (attached).
   Now the file _init_.py contains only the function HAVC_main()

   The function HAVC_main is important because it implement all the HAVC filter combinations.

   I added also the function HAVC_read_video, that mimic what usually Hybrid does when must read a clip.
   It was necessary because HAVC_main must be able to use all the filters without additional input at the exception of the clip to colorize.  
  
   The implementation of methods 0-4 in Hybrid was already Ok.
   For the method=5 it necessary to manage in special way the field RefDir/File.
   I included in the attachement 2 examples of method=5 usage, so that should be easy to understand the implementation in the GUI.
   In the attachment there is also an updated version of the HAVC_diagram.

Thanks,
Dan

[Selur]

The implementation of methods 0-4 in Hybrid was already Ok.

Since the did not match what was shown in the diagram, I threw that code out.

Now the file _init_.py contains only the function HAVC_main()

Nice, so Hybrid doesn't need to call anything aside from HAVC_main.
That would simplify the script creation a lot.
But looking at your example, HAVC_main isn't called at all.
You call 'HAVC_read_video' and 'HAVC_restore_video'.

=> no clue how HAVC should be used at all now.
First you write that only HAVC_main needs to be used from now on since it now does everything, then your example doesn't use it,..


Cu Selur

[Selur]

Uploaded a new Hybrid_deolify, this version does not support 'custom', please check if this version works as you want.
If it does not, I will try to fix it until it works.
Then I will look at the custom stuff, one method at a time.

Cu Selur

[Selur]

Tried calling:

# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using DeOldify
from vsdeoldify import HAVC_main
clip = havc.HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=5, DeepExRefMerge=0, DeepExOnlyRefFrames=False, ScFrameDir="G:/TestClips&Co/files/test_ref.avi", DeepExModel=1, DeepExEncMode=0, DeepExMaxMemFrames=0)
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

and got:

Error on frame 0 request:

Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3216, in vapoursynth.publicFunction
File "src\\cython\\vapoursynth.pyx", line 3218, in vapoursynth.publicFunction
File "src\\cython\\vapoursynth.pyx", line 834, in vapoursynth.FuncData.__call__
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\vsslib\vsmodels.py", line 108, in deepex_clip_color
img_color = colorizer.colorize_frame(img_orig, wls_filter_on=wls_on, render_vivid=render_vivid)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\deepex\__init__.py", line 200, in colorize_frame
wls_filter = cv2.ximgproc.createFastGlobalSmootherFilter(
^^^^^^^^^^^^
AttributeError: module 'cv2' has no attribute 'ximgproc'

seems like copying RC3 into my site-packages folder broke something or some stuff is missing.

Cu Selur

[Selur]

Okay, aside from Deep-Remaster all other DeepEx Models fail in my setup now.
ColorMNet: Failed to initialize ColorMNet[remote] try ColorMNet[local] (reinstalled the spatial_correlation_sampler_backend I attached before, but that doesn't work either)
Deep-Exemplar: AttributeError: module 'cv2' has no attribute 'ximgproc'
opencv-python is already installed

python -m pip install opencv-python
Requirement already satisfied: opencv-python in f:\hybrid\64bit\vapoursynth\lib\site-packages (4.11.0.86)
Requirement already satisfied: numpy>=1.21.2 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from opencv-python) (1.26.4)

Okay, uinstalling it and installing opencv-contrib-python:

F:\Hybrid\64bit\Vapoursynth>python -m pip uninstall opencv-contrib-python
WARNING: Skipping opencv-contrib-python as it is not installed.

F:\Hybrid\64bit\Vapoursynth>python -m pip uninstall opencv-python
Found existing installation: opencv-python 4.11.0.86
Uninstalling opencv-python-4.11.0.86:
  Would remove:
    f:\hybrid\64bit\vapoursynth\lib\site-packages\cv2\*
    f:\hybrid\64bit\vapoursynth\lib\site-packages\opencv_python-4.11.0.86.dist-info\*
Proceed (Y/n)?
  Successfully uninstalled opencv-python-4.11.0.86
WARNING: Failed to remove contents in a temporary directory 'F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\~v2'.
You can safely remove it manually.

F:\Hybrid\64bit\Vapoursynth>python -m pip install opencv-contrib-python
Collecting opencv-contrib-python
  Using cached opencv_contrib_python-4.11.0.86-cp37-abi3-win_amd64.whl.metadata (20 kB)
Requirement already satisfied: numpy>=1.21.2 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from opencv-contrib-python) (1.26.4)
Using cached opencv_contrib_python-4.11.0.86-cp37-abi3-win_amd64.whl (46.2 MB)
Installing collected packages: opencv-contrib-python
Successfully installed opencv-contrib-python-4.11.0.86

Seems to have fixed it. (no clue whether that now broke something else.)

So only ColorMNet is broken.