Selur's Little Message Board - ProPainter Vapoursynth filter

Pages: 1 2 3 4 5 6 7 8 9 10 11 12

I realise now that you have it actively working in Hybrid, is there any chance I could have access to a dev where it's operating? I tried the latest dev in your experimental folder but cannot find propainter in the 'other' tab. Sorry if I've missed something obvious.

You, just need the normal version with the torch-addon. (linked on the download page)

Cu Selur

(25.10.2024, 11:38)jamesharding26 Wrote: [ -> ]Hi all,

I am trying to run the script and running into an error that I don't understand, if anyone can help that would be much appreciated

Code:
Failed to evaluate the script: Python exception: list index out of range Traceback (most recent call last): File "src\\cython\\vapoursynth.pyx", line 3233, in vapoursynth._vpy_evaluate File "src\\cython\\vapoursynth.pyx", line 3234, in vapoursynth._vpy_evaluate File "", line 11, in File "C:\Users\James\AppData\Local\Programs\Python\Python312\Lib\site-packages\vspropainter\__init__.py", line 24, in from vspropainter.propainter_render import ModelProPainterIn, ModelProPainterOut File "C:\Users\James\AppData\Local\Programs\Python\Python312\Lib\site-packages\vspropainter\propainter_render.py", line 22, in from vspropainter.model.recurrent_flow_completion import RecurrentFlowCompleteNet File "C:\Users\James\AppData\Local\Programs\Python\Python312\Lib\site-packages\vspropainter\model\recurrent_flow_completion.py", line 7, in from vspropainter.model.misc import constant_init File "C:\Users\James\AppData\Local\Programs\Python\Python312\Lib\site-packages\vspropainter\model\misc.py", line 57, in IS_HIGH_VERSION = [int(m) for m in list(re.findall(r"^(\d+)\.(\d+)\.(\d+)([\w\d\.].*)?$",\ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ IndexError: list index out of range

Thanks

James

This was a change applied by Selur in this post: #22
I think that Selur can suggest a fix for this issue.

Dan

Ahh,... that was needed since the way pytorch changed how the version was encoded.
This should work with new pytorch versions 2.6.* for older version your probably need to change the code, see: https://github.com/sczhou/CodeFormer/pul...2bb40536dc

Cu Selur

If you are using Windows, it is safe to edit the file:

Code:
"C:\Users\James\AppData\Local\Programs\Python\Python312\Lib\site-packages\vspropainter\model\misc.py", line 57

and set

Code:
IS_HIGH_VERSION = False

When IS_HIGH_VERSION is True the filter check if is available the mps backend, but this backend is useful only on MacOS, so if you are using Windows you can skip this check.

Dan

Released a new version that should fix this issue: https://github.com/dan64/vs-propainter/r...ag/v1.12.1

Dan

@Dan64: I'm trying to remove the logo mentioned here of this clip.
I created a video mask
Code I used to create the logo:

Code:
clip = clip.resize.Bicubic(format=vs.RGBS)

# Extract R, G, B planes

r = clip.std.ShufflePlanes(0, vs.GRAY)

g = clip.std.ShufflePlanes(1, vs.GRAY)

b = clip.std.ShufflePlanes(2, vs.GRAY)

# Build an expression that detects yellow-ish pixels:

# - R and G are both high (>= 0.6)

# - B is relatively low (<= 0.5)

expr = "x 0.6 > y 0.6 > and z 0.5 < and x y - abs 0.1 < and 1 0 ?"

mask = core.std.Expr([r, g, b], expr=[expr])

# Clean up the mask

clip = mask.std.Inflate().std.Minimum().std.Minimum().std.Convolution(matrix=[1]*9).std.Maximum().std.Maximum()

clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)

# colorformat GRAYS

which seems fine covering the logo for most of the clip.
Problem is when using:

Code:
# Inpainting using ProPainter

from vspropainter import propainter as ProPainter

clipMask = core.lsmas.LWLibavSource(source="G:/Output/logo_mask2.mp4", format="RGB24", cache=0)

clip = ProPainter(clip=clip, clip_mask=clipMask, length=500, neighbor_length=20, raft_iter=10, mask_region=(500,256,390,240))

starting with frame 149 the logo reappears (isn't properly removed).

the mask clip properly covers the logo:

If I reduce the length to 20, logo is gone until 160,...
I first thought this could this be a bug in the length&co parameter handling, for the edge case where the end of the clip is approached, in the wrapper.
To test my theory, I added clip + core.std.Reverse(clip).std.Trim(1), created a new mask clip and used that to look at frame 149+.
That didn't help, it even seemed to make it worse. Then I thought maybe the scene threshold was the issue, so I set that to 0, but that also didn't help.
Last thing I tried was disabling the mask region, but that also didn't really help either.
Do you think this is a problem of:
a. finding the right parameters
b. a bug in the wrapper
c. ProPainter just strangely failing
?

Cu Selur

Hello Selur,

theoretically clip mask should work (see attached sample), but I never used it on long clips.
It seems that ProPainter is not able to manage properly the clip mask when it becomes almost static.
You can try to split the movie in 2 parts, the moving part (till frame 99) and the static part (from 100 till 202).
It could be also possible that ProPainter is not able to properly manage moving masks when are not available enough frames to perform the inference.
In effect in the attached sample the player is simply going from right to left and then form left to right and in this case the inference is very simple.

Dan

It seems that the problem is on the clip mask.

If you extract the frame 165 from the mask and copy it as transparent selection over the frame 165 rendered by ProPainter.

You obtain the following frame.

[Image: attachment.php?aid=3227]

As you can see the letter I of the mask is over the letter F the rendered image. Try to check the coordinates they are different.

For the tests is better to don't use the crop region and use the full frame, but with half size, something like this

Code:
clipMask = clipMask.resize.Spline36(width=640, height=360)

clip = clip.resize.Spline36(width=640, height=360)

clip = ProPainter(clip=clip, clip_mask=clipMask, length=202)

Dan

That makes sense, I did not take the future masking into account => so I will have to look into a better method to create the masking clip. Thanks!

Cu Selur

Pages: 1 2 3 4 5 6 7 8 9 10 11 12