Deoldify Vapoursynth filter

[Dan64]

I noted that you used DeepExMethod=3 -> external RF same as video. This imply that the reference frames are exactly equals in size to the source. 
If there is even a little difference of few pixels you can see the gray area on the bottom. 
Try using  DeepExMethod=4 and DeepExVivid=False.

Dan

[zspeciman]

I have observed that on fast motion clips, on some settings the image looks distorted.

Here is an example:
Charlie Chaplin - The Kid - Fight Scene

Attached are frames 6883 and 6900 along with their counterparts with the distortions.

Using speed fast, Video+Artistic, Simple and Vivid:  Image looks normal


The distortion happens when I add this
Denoise is enabled with Color tweaks retinex/red (other color tweaks are fine)
or 
Stabilize Temp is enabled, any settings
or
Exemplar Models enabled/HAVC or Deep_exemplar/ColorMNet any Ref merge including "no"

Is this the normal limitation of these settings for fast moving objects?

[Selur]

Is this the normal limitation of these settings for fast moving objects?

Seems like the (temporal) denoising HAVC does is causing problems.
=> You will have to wait for Dan64. (Without proper details and source, I can't even say whether this could be caused by a problem with the source filter.)

Cu Selur

[zspeciman]

here is the source

https://gofile.io/d/GxUtxn

[Dan64]

I expect that this problem could happen using  "Stabilize Temp".
In the last version introduced 2 new options:

1)  "Stabilize Temp"-> It is ColorMnet with Ref merge ranging from verylow (-> very high) and veryhigh (-> very low)
2)  "Interpolation" -> It is Deep-Exemplar with SC min freq set to 1 to 10

These are convenience settings for using Exemplar Models without the need to set all the parameters.
They are using the the Examplar Models in 2 different ways:

1)  "Stabilize Temp": ColorMNet with SC thresh = 0.10 is used to get frames with more stable colors, which are merged with a variable weight with HAVC frame, this will improve the color stability at a cost of less colored frames.

2)  "Interpolation" -> in this case is used Deep-Exemplar to propagate the colors in the reference frames to the next frames (in the new coming version this behavior will change) 

I already noticed this problem when using "Stabilize Temp", probably due to the merge, in the other cases it is unexpected. 
I will do some test, but usually in a clip with speed of 24-25 fps this problem is not noticeable, so I don't consider this a relevant issue.

Dan

[Dan64]

I performed some tests, but I was unable to reproduce your problem.

I attached the results with the scripts used to obtain the reference pictures.

Try to upgrade your NVIDIA windows drivers to the latest version in some cases it can help.

Dan

[zspeciman]

Thank you Dan, I appreciate your help and your time.  I did a fresh Hybrid install, reinstalled nvidia drivers.  Run your scripts, but still the same distortions.  
I hope others can run these tests to verify if they have the same issues or its just my computer acting up.

[Selur]

Seems like the temporal stabilization is the problem.
Denoise 'none':

clip = havc.HAVC_main(clip, VideoTune="vivid", ColorFix="none", ColorTune="none")

Denoise 'light':

clip = havc.HAVC_main(clip, VideoTune="vivid", ColorFix="none")

Denoise 'light' + Color Tweaks 'Retinex/Red':

clip = havc.HAVC_main(clip, VideoTune="vivid", ColorFix="retinex/red")

Denoise 'light' + Color Tweaks 'none' + Stabilize (temp) 'very low':

clip = havc.HAVC_main(clip, VideoTune="vivid", ColorTemp="very low", ColorFix="none")

Happens with when 'Stabilze (temp)' is set to anything but 'none'.

=> Seems like the stabilization is not only done on the chroma, but also on the luma plain. (not sure if this is as intended)

Cu Selur

[Dan64]

Unfortunately I can't reproduce the problem using my development version.

What I get using this code:

clip = havc.HAVC_main(clip, Preset="fast", ColorModel="Video+Artistic", CombMethod="Simple", VideoTune="vivid", ColorTemp="medium", ColorFix="violet/red", FrameInterp=0, ColorMap="none", ColorTune="medium", BlackWhiteTune="none", EnableDeepEx=False, enable_fp16=True)

Is this picture



I attached my development version 5.6.5 RC1, try it to see if the problem is still present.

Dan

P.S.
With this version is strongly suggested to set Interpolation equal to 4 or 5.

[Selur]

5.6.5 RC1 does seem to help here.
btw. adding TimeCube with "AGF -Optima + film" helps with the colors. (https://imgsli.com/NDM1NzQx)
so fingers crossed, that it also helps zspeciman.

Cu Selur