Selur's Little Message Board
ProPainter Vapoursynth filter - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: ProPainter Vapoursynth filter (/thread-3720.html)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12

RE: ProPainter Vapoursynth filter - Selur - 28.05.2024

Quote:I will introduce the parameter frame width/height because the speed will decrease with the frame size and review the mask(s) management.
What should that parameter do?
Maybe, optionally to speed things up, something like Hybrids 'Only on Region'-filter option could be used (which uses animate) to only apply the filter to a specific region.
# adjusting color using Levels on YUV420P8 (8 bit)
def vsLevelsRegion1(clip,*args):
  return core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=16, max_out=235, gamma=2.00, planes=[0])
end = clip.num_frames-1
mapping = [
  (0, end), [vsLevelsRegion1],
]
clip = animate.run(clip, mapping, selection=(50,60,96,58))
[Image: grafik.png]
The selection parameter of animate, basically crops out a region and later puts it back in, so the region the filter gets applied to gets smaller.

Cu Selur

RE: ProPainter Vapoursynth filter - Dan64 - 28.05.2024

This is a good idea, but it is necessary to find a way to insert propainter in the rendering chain.

clip -> crop_frame -> propaint(crop_frame) -> full_frame -> clip

you are suggesting to insert propainter in the place of vsLevelsRegion1 ?

Dan

RE: ProPainter Vapoursynth filter - Selur - 28.05.2024

That is what animate does. Smile
Quote:you are suggesting to insert propainter in the place of vsLevelsRegion1 ?
yes, something like:
import animate
def vsAnimateRegion1(clip,*args):
  return propainter(clip, length=25, mask_path="running_car_mask.png", device_index=0, enable_fp16=True)
end = clip.num_frames-1
mapping = [
  (0, end), [vsAnimateRegion1],
]
clip = animate.run(clip, mapping, selection=(400,80,100,180))
should work. (This does not work, since the png is not 400x80.)

Hmm,... using the above with
return propainter(clip, length=25, mask_path="running_car_mask_cropped_400x80.png", device_index=0, enable_fp16=True) # image attached
Does not work. Not sure why.
[Image: grafik.png]

Cu Selur

RE: ProPainter Vapoursynth filter - Dan64 - 28.05.2024

I tested this script

from vspropainter import propainter
def vsPropainter(clip,*args):
  return propainter(clip, length=25, mask_path="running_car_mask_small.png", device_index=0, enable_fp16=True)
end = clip.num_frames-1
mapping = [
  (0, end), [vsPropainter],
]
clip = animate.run(clip, mapping, selection=(344,72,123,182))

with the attached mask but is not working Sad

Dan

RE: ProPainter Vapoursynth filter - Selur - 28.05.2024

Yeah, I did the same. (see: above Wink)

No clue either why it's not working. (I made sure to use a resolution that is mod16.)

Cu Selur

RE: ProPainter Vapoursynth filter - Dan64 - 28.05.2024

I think that the filter to be able to remove the logo need a large context.
In the selected sample the logo is at the center of image, that is an optimal position for the filter.
Not sure if the filter will be able to remove the logo in the case is located for example at the bottom right of the frame.
I will test it.

Dan

RE: ProPainter Vapoursynth filter - Selur - 28.05.2024

Might be that there is a restriction between the resolution of the video and the amount of pixels that are marked for replacement.
i.e. replacement region (white part of the mask) needs to be less than 50% of the image or something like that.

RE: ProPainter Vapoursynth filter - Dan64 - 28.05.2024

Try this example using the following command

clip = propainter(clip, length=250, mask_path="sample2_mask.png", enable_fp16=True, mask_dilation=16)

This time logo removal is good but not perfect.

Also the speed decreased to 0.40 fps, with the GPU memory peaked to 12145mb vs max memory of 12288mb.

Dan

RE: ProPainter Vapoursynth filter - Selur - 28.05.2024

Don't really see a difference with the mask dilation https://imgsli.com/MjY3ODgy, but the result looks fine in my book, given the (bad) quality of the source.
(The mask dilation should potentially increase the edge of the mask.)
Speed-wise with mask_dilation=16, I get:
encoded 235 frames in 107.66s (2.18 fps), 1811.52 kb/s, Avg QP:22.3

Cu Selur

Ps.: Wohaa,.. using a HD source eats ~32GB RAM and 15.2GB VRAM.
[Image: grafik.png]

RE: ProPainter Vapoursynth filter - Selur - 28.05.2024

Okay, 1080p isn't usable atm. without something to limit the filtered size.
(45min and preview is still not visible)
Big Grin

Going to bed now.

Cu Selur