Deoldify Vapoursynth filter

[Dan64]

With the last version the problem has been solved.

I noted that when is selected a Preset the parameters are propagated also in the custom view.
This is not necessary since I think is adding an additional layer of complexity in the development.

I guess that you are replicating the code of function ddeoldify_main().

In your version the value "none" for ColorFix is not managed correctly, in my code "none" is managed as a special case in this way

 

if co_id == 0:
        hue_range = "none"
        hue_range2 = "none"
else:   
        hue_range = hue_fix[co_id] + "|0.3,0.1"
        hue_range2 = hue_fix[co_id] + "|0.8,0"

In your version when "none" is selected the hue_range becomes "none|0.3,0.1"

I attached an updated version where I changed the code of  ddeoldify_main(). 

I think that the link with the custom parameters is not necessary, but is up to you to decide this implementation.

In the case you want to keep the link, please update your code with the version attached.

Thanks,
Dan

[Selur]

This is not necessary since I think is adding an additional layer of complexity in the development.

It sure does, but it makes it easier to use a preset as base and then adjust it.

Will look at it tomorrow after work and fix it.

[Dan64]

Hello Selur,

  I added a new parameter "ColorTune" to function ddeoldify_main() this parameter allows to define the intensity of noise reduction applied by ColorFix.
  Please update your code with the attached version.

Thanks,
Dan

[Selur]

will look at it later,..

[Selur]

Updated the test download, hopefully I didn't miss anything.

Cu Selur

[zspeciman]

Dan, is the video output from the original DeOldify RenderFactor=21 identical to Hybrid DeOldify only Model Video RenderFactor=21 (all other settings off)  ?

yes it is the same, but in my tests the optimal render factor for DeOldify is RF=24, this is also the value used by Preset "Fast".

I noticed a difference between the Original DeOldify and the Hybrid-DeOldify.  I've attached clips and pics.  I'm using Hybrid 2024.04.06.1 
If possible run it on your computer to see if you reach the same results as I, maybe I didn't install something right.

TestClipSingerBW.mp4 = source video black & white
TestClipSingerResultsDeOldifyOriginalRF24.mp4 = Original DeOldify video at Render Factor 24
TestClipSingerResultsDeOldifyHybridRF24.mp4 = Hybrid-DeOldify video at Render Factor 24 (DeOldify only; Model Video, RF 24, everything else disabled)

Compare frames around 185, that is the biggest difference.  The original DeOldify appears a bit more stable.

Then I wanted to see what happened if these two DeOldify produced clips were used as source for DDcolor: Artistic, Chroma Stabilizer enabled [5, 'A', 1, 60, 0.5, 0.8], RF 24.  Perhaps that extra stability is passed forward.
TestClipSingerResultsDeOldifyHybridRF24-DD24-CS.mp4 & TestClipSingerResultsDeOldifyOriginalRF24-DD24-CS.mp4

It looks, at least to my eyes, that the DDcolor of the Original DeOldify clip as source is a bit less colorful and more stable.  Also there were no difference in using Hybrid-DeOldify or the black&white video as source when using DDcolor, they looked the same.

[Dan64]

Dan, is the video output from the original DeOldify RenderFactor=21 identical to Hybrid DeOldify only Model Video RenderFactor=21 (all other settings off)  ?

yes it is the same, but in my tests the optimal render factor for DeOldify is RF=24, this is also the value used by Preset "Fast".

I noticed a difference between the Original DeOldify and the Hybrid-DeOldify.  I've attached clips and pics.  I'm using Hybrid 2024.04.06.1 
If possible run it on your computer to see if you reach the same results as I, maybe I didn't install something right.

Could you provide also the scripts used to generate the clips ?

Dan

[Dan64]

Hello Selur,

  I need your support to answer to zspeciman's question.

  This is the frame 185 taken from the video "TestClipSingerBW.mp4"

 
  
  If I use the following script

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'D:/Programs/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="D:/Programs/Hybrid/64bit/vsfilters/Support/libimwri.dll")
# Import scripts
import validate
# Source: 'TestClipSingerBW_f185.png'
# Current color space: RGB24, bit depth: 8, resolution: 480x360, frame rate: 25fps, scanorder: progressive, yuv luminance scale: full, matrix: 709
# Loading TestClipSingerBW_f185.png using vsImageReader
clip = core.imwri.Read(["TestClipSingerBW_f185.png"])
clip = core.std.Loop(clip=clip, times=100)
frame = clip.get_frame(0)
# setting color transfer (170), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip, _Transfer=6)
# setting color primaries info (to 470), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip, _Primaries=5)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=0) # progressive
# adding colors using DeOldify
from vsdeoldify import ddeoldify
clip = ddeoldify(clip=clip, method=0, deoldify_p=[0, 24, 1, 0])
# adjusting output color from: RGB24 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="709", range_s="full")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# output
clip.set_output()

 
I obtain the following picture

 
 
that is the same obtained using the original DeOldify

 But if I use the following script

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'D:/Programs/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="D:/Programs/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import validate
# Source: 'TestClipSingerBW.mp4'
# Current color space: YUV420P8, bit depth: 8, resolution: 480x360, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg
# Loading TestClipSingerBW.mp4 using LWLibavSource
clip = core.lsmas.LWLibavSource(source="TestClipSingerBW.mp4", format="YUV420P8", stream_index=0, cache=0, fpsnum=25, prefer_hw=0)
clip = core.std.Loop(clip=clip[185], times=257)
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# setting color transfer (170), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip, _Transfer=6)
# setting color primaries info (to 470), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip, _Primaries=5)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=1)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=0) # progressive
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
# adding colors using DeOldify
from vsdeoldify import ddeoldify
clip = ddeoldify(clip=clip, method=0, deoldify_p=[0, 24, 1, 0])
# adjusting output color from: RGB24 to YUV420P10 for x265Model
#clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# output
clip.set_output()

 I obtain the following picture

 

  I think that is an issue related to the color space conversion, both my first script and Deoldify original are working on "color space: RGB24, bit depth: 8, yuv luminance scale: full, matrix: 709"
  While the second script is using in input "color space: YUV420P8, bit depth: 8, yuv luminance scale: limited, matrix: 470bg".

   Fortunately is not a big issue because using the following script

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'D:/Programs/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="D:/Programs/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import validate
# Source: 'TestClipSingerBW.mp4'
# Current color space: YUV420P8, bit depth: 8, resolution: 480x360, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg
# Loading TestClipSingerBW.mp4 using LWLibavSource
clip = core.lsmas.LWLibavSource(source="TestClipSingerBW.mp4", format="YUV420P8", stream_index=0, cache=0, fpsnum=25, prefer_hw=0)
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# setting color transfer (170), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip, _Transfer=6)
# setting color primaries info (to 470), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip, _Primaries=5)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=1)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=0) # progressive
# adjusting color space from YUV420P8 to RGB24 for vsDeOldify
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
# adding colors using DeOldify
from vsdeoldify import ddeoldify_main
clip = ddeoldify_main(clip=clip)
# adjusting output color from: RGB24 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# output
clip.set_output()

 I obtain this picture that is much better

 

Dan

[zspeciman]

See attached, I hope I've done this right, I'm still learning.

For TestClipSingerResultsDeOldifyOriginalRF24.mp4, that is straight from DeOldify colab

[Dan64]

See attached, I hope I've done this right, I'm still learning.

For TestClipSingerResultsDeOldifyOriginalRF24.mp4, that is straight from DeOldify colab

  It is not more necessary I was able to reproduce the problem.

  When will be released the new version of vsdeoldify, it will be available Presets that will simplify the usage of this filter.
  This is the picture of frame 185 obtained using the new version with RF=24 (Preset="fast").

   

Dan