ProPainter Vapoursynth filter

[Selur]

Got the GPU working in my current R68 test setup, using:

# Imports import vapoursynth as vs # getting Vapoursynth core import site import sys import os core = vs.core # Import scripts folder scriptPath = 'F:/Hybrid/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) os.environ["CUDA_MODULE_LOADING"] = "LAZY" #os.environ["CUDA_LAUNCH_BLOCKING"] = "1" # loading plugins core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll") # Import scripts import validate clip = core.lsmas.LWLibavSource(source=r"running_car.mp4", format="YUV420P8", stream_index=0, cache=0, fpsnum=25,repeat=True, prefer_hw=0) frame = clip.get_frame(0) # Setting detected color matrix (709). clip = core.std.SetFrameProps(clip=clip, _Matrix=1) # setting color transfer (709), if it is not set. if validate.transferIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Transfer=1) # setting color primaries info (to 709), if it is not set. if validate.primariesIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Primaries=1) # setting color range to TV (limited) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=1) # making sure frame rate is set to 25fps clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # making sure the detected scan type is set (detected: progressive) clip = core.std.SetFrameProps(clip=clip, _FieldBased=0) # progressive # changing range from limited to full range for propainter clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full") # setting color range to PC (full) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=0) # adjusting color space from YUV420P8 to RGB24 for propainter clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full") # adding colors using propainter from vspropainter import propainter org = clip clip = propainter(clip, length=25, mask_path="running_car_mask.png", device_index=0) clip = core.std.StackHorizontal([org.text.Text("Original"), clip.text.Text("Filtered")]) # changing range from full to limited range for propainter clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited") # adjusting output color from: RGB24 to YUV420P8 clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="709", range_s="limited") # set output frame rate to 25fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # output clip.set_output()

the color shift is present there too:

I'll package and upload my current R68 setup to my GoogleDrive. (this will take me ~1hour, 5min compression, 55min upload)

=> Package is uploaded
So once:
a. the color issue is fixed
b. instead of a single image a video/mask clip can be provided as input
the filter could be added to Hybrid in a future version. (R68 torch addon is not really ready for general usage)
@Dan64:could it be a RGB vs. BGR issue?
Yes, adding:

# rgb to bgr clip = vs.core.std.ShufflePlanes(clips=clip,planes=[2,1,0], colorfamily=vs.RGB)

below:

ppaint = ModelProPainter(device, weights_dir, mask_path, mask_dilation, neighbor_length, ref_stride, raft_iter)

in the __init__.py, fixes the colors:

seems like a COLOR_BGR2RGB is missing somewhere.

Cu Selur

Ps.: works for poisondeathray over at Doom9s