Deoldify Vapoursynth filter

[Dan64]

If you want to use DeepEx you need to provide the reference frames.
With HAVC the reference frames can be generated using DDcolor and/or DeOldify.
There are several options to control the reference frame generation: 

1) scene change threshold: if you are not happy with the reference frames, you can lower its value till 0.01
2) reference frame frequency: you can generate reference frames with a given frequency, if you are not happy with the reference frame you can increase the frequency.
3) external reference frames: this option allows to apply the reference frames exactly where the user believes they are needed.

These options allow to solve your problem, is up to you decide what option fit well with your needs.

Dan

---

when would it be available to the general public?

Yes it is available to public. You need:

1) install the last Hybrid version from the page download
2) install the torch package VapoursynthR68_torch_2024.06.15.7z from Hybrid Windows Addons

In the Google folder there is a file README with instructions.

Dan

[zspeciman]

Thank you Dan for your help, I'll keep on playing with this.  
For the tough scenes where for a few seconds a car slowly changes to a different color from silver to gold or when a person shirt starts blue, goes to pink and back to blue.  Your suggestion is to put a reference frame of the blue shirt, or the car as gold at the beginning of that scene and that will negate / minimize the pink shirt or the silver car from appearing.

[Dan64]

Thank you Dan for your help, I'll keep on playing with this.  
For the tough scenes where for a few seconds a car slowly changes to a different color from silver to gold or when a person shirt starts blue, goes to pink and back to blue.  Your suggestion is to put a reference frame of the blue shirt, or the car as gold at the beginning of that scene and that will negate / minimize the pink shirt or the silver car from appearing.

yes you can try to enforce a given color if the scene don't change too much.

Dan

[hallomanbh]

I think this is better

https://github.com/satoshiiizuka/siggrap...emastering

[Dan64]

I thin this better

https://github.com/satoshiiizuka/siggrap...emastering

Using the same input and reference frames used in the post: #500

With Deep Remaster I obtained the following frame



While with DeepEx I obtained this frame



The hands are colored better, but the color of wool sweater is almost lost.

With this model it will be necessary to provide a reference image every 15 frames to keep the colors.

Not a big improvement.

I will think about it.

Dan

[zspeciman]

also the green sweater color bleed into her neck is gone.  Would be nice to see a side by side video comparisons of the two methods.

[Dan64]

For those interested in evaluating Deep Remaster I attached the filter (the file vsremaster.zip must be unzipped in Hybrid\64bit\Vapoursynth\Lib\site-packages).

It is implemented only the coloring part, the "remaster" code is embarrassing , it doesn't work even on the examples provided by the author.

But even the coloring part is not working well, the implementation is years light far from what is described in the paper.

I attached also an example, with: script, clip and reference images.

Unfortunately I spent time to implement it, so I decided to provide it to people who might be interested.

Dan

EDIT:
to use the filter is necessary to download the file remasternet.pth.tar (is not a tar, just a "pth" renamed as "pth.tar") and copy it in: "Hybrid\64bit\Vapoursynth\Lib\site-packages\vsremaster\model"

P.S.
I will not provide any support for this filter.

[Selur]

I'll probably not test it, but thanks anyway, there might be others interested in it.

Cu Selur

[Dan64]

Despite the unfortunate results obtained with Deep Remaster, I'm very happy for the results obtained with DeepEx.

In one of my first attempt to colorize a movie I tried to colorize the movie "The Thing (1951)", but I was not happy with the result obtained.

For this movie was available also a colored VHS version, but was a very low resolution version and was about 6min shorter.

Comparing the colored version with the version obtained using (at that time) DeOldify I decided to keep only the B&W remastered version in HD.

But now thanks to DeepEx I was able to colorize perfectly the HD remastered copy, obtaining the same colors.

To do that, I exported the (synced) colored version using a frequency of 2 frames, and then I used DeepEx (with external RF same as video) to color the HD version.
DeepEx was able to perfectly apply the colors despite the difference is resolution and that the frames were not always perfectly in sync.

Here an example:

Low resolution colored frame



And here the HD frame colored with DeepEx.



Unfortunately is very uncommon that for a old movie are available both a low resolution colored version and a HD remastered version in B&W.

But in this case DeepEx is the only filter being able to use successfully this combination.

Dan

[Selur]

The result does look nice.