ProPainter Vapoursynth filter

[Dan64]

That is an interesting project, lets see what can be done with this clip

ProPainter filter worked quite well.

You can see a glitch at the 8 second position.

Dan

[zspeciman]

That is AMAZING!!!  I had no idea what to expect, but nothing this good. How long does it take to remove the logo?

[Dan64]

I included the encoding report, on my RTX 3060, the avg inference speed was about 3.11 fps.
The trick is to use the mask region, to lower the size of the frame. In this case I used a mask 420x300.
Without the mask removing a mask from a 1080p movie will be almost impossible.
 
Maybe by increasing the Mask Dilation to 16 could remove the glitch. 
I also attached the script used.

On Selur's GoogleDrive there is a folder called experimental that can be used to test ProPainter.

To use it:

1) Copy your Hybrid installation in another folder, for example Hybrid.R68
2) Delete the folder Vapoursynth under Hybrid.R68\64bit
3) Extract the content of the archive: VapoursynthR68_propainter_test_setup.7z in Hybrid.R68\64bit
4) Extract the content of archive: Hybrid_propainter_test.zip in Hybrid.R68 and override the file Hybrid.exe

Dan

P.S.
You can find the ProPainter filter under Filtering->Vapoursynth->Other (the last filer of panel)
You can find some explanation on the parameters at: https://github.com/dan64/vs-propainter

[Selur]

To test whether this is a scene change problem (which I suspected), I encoded 0-173 and 174-522. (522 should be 521)
I then noticed I forgot to add ProPainter to ApplyOnlyto,..
Will add it an test, whether this could be done with it in one go.
In general adding an option to check for scene changes and not go over scene changes might be a good addition to the filter. (I could also add this in Hybrid, but I think this might be something useful to the filter itself.)

Cu Selur

Ps.: no ApplyOnlyTo, is not solution, it does not 'cut' internally. (thus the result is the same as when the filter is applied to the whole clip, which would have been clear if I just had read https://github.com/Irrational-Encoding-W...emapFrames again )

[Dan64]

Nice discovery! 
I will add the scene change detection in the next release. 
Near the end of the clip is (a little) visible the logo mask (almost transparent), maybe it will be necessary also increase the Mask dilation.

Dan

[Selur]

I will add the scene change detection in the next release.

Nice! Looking forward to it.
For the scene detection (https://github.com/dubhater/vapoursynth-scxvid or https://github.com/dubhater/vapoursynth-wwxd could be used).

Near the end of the clip is (a little) visible the logo mask (almost transparent), maybe it will be necessary also increase the Mask dilation.

Yeah, I think so too.

Cu Selur

[Dan64]

I will add the scene change detection in the next release.

Nice! Looking forward to it.
For the scene detection  (https://github.com/dubhater/vapoursynth-scxvid or https://github.com/dubhater/vapoursynth-wwxd could be used).

The code is 10 year old. I don't expect anything "revolutionary".

I will use the good old one MiscFilters.dll included in Vapoursynth.
I had the possibility to deeply test it for the HAVC development and I'm quite satisfied.

Dan

[Selur]

I had the possibility to deeply test it for the HAVC development and I'm quite satisfied.

👍

Cu Selur

[Dan64]

Hello Selur

   I released version 1.1.0 with scene detection: https://github.com/dan64/vs-propainter/r...tag/v1.1.0

  Let me know if is working for you.

  I noted that the capability of the filter of reconstructing the portion of image hidden by the logo depends also on the length of the batch of frames used for the inference.

  Here a comparison on frame 174: https://imgsli.com/MjcwMTMy

It seems that a length between 80 and 120 provides satisfactory results

Dan

[Selur]

Seems to work fine.
Updated the test download.

Cu Selur