ProPainter Vapoursynth filter

[Selur]

Replacing:

if clip.format.bits_per_sample != 8: clip = vs.core.resize.Bicubic(clip, format=clip.format.replace(bits_per_sample=8))

with:

if clip.format.bits_per_sample != 8: if clip.format.sample_type == vs.FLOAT: clip = vs.core.resize.Bicubic(clip, format=vs.RGB24, range_s="full") else: clip = vs.core.resize.Bicubic(clip, format=clip.format.replace(bits_per_sample=8))

does seem to fix the problem. (in propainter_utils.py)

[Dan64]

Hi Selur,

  could you clarify better the problem ?

Dan

[Selur]

When using the attached mask, it crashes with the above error message unless I change the code as I did.

Cu Selur

[Dan64]

But the 1-bit mask is an image, while you are getting error on the clip.
Please provide a complete sample with: clip + mask + script

Dan

[Selur]

Doh, you are right.
Here's a link to the source: https://www.mediafire.com/file/nxspwubdh...g.mkv/file

[Dan64]

I tested on my side on R74 and it is working.
Please try to test on R74 to see if it works also on your side. 

If, as I suspect, is a problem introduced in R76. 
You should open an issue to myrsloik on Vapoursynth.

Dan

[Selur]

Will do some testing tomorrow, got my gpu busy till tomorrow.

[Dan64]

I tested it with the last dev version  with R77 released today, and it works with no issues.
Given that you are using 1-bit mask you should use: mask_enhance=True, but in any case it works also with  mask_enhance=False.

I noted that in the R77 bundle there is the folder vsdeoldify with about 4GB of models that can be removed, because is not used any more.

Dan

P.S.
to read the source I used: core.lsmas.LWLibavSource()

[Selur]

I noted that in the R77 bundle there is the folder vsdeoldify with about 4GB of models that can be removed, because is not used any more.

Can you be more specific which pth files are no longer needed?


----
Hmm,.. using:

# Imports import sys import os import vapoursynth as vs # getting Vapoursynth core core = vs.core # Limit frame cache to 48449MB core.max_cache_size = 48449 # Import scripts folder scriptPath = 'F:/Hybrid/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) # loading plugins core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll") # Import scripts import validate # Source: 'C:\Users\Selur\Desktop\1 re-encode, no filtering.mkv' # clip current meta; color space: YUV420P8, bit depth: 8, resolution: 640x480, fps: 25, color matrix: 470bg, color primaries: Unspecific, color transfer: Unspecified, yuv luminance scale: limited, scanorder: progressive, full height: true (Source) # Loading 'C:\Users\Selur\Desktop\1 re-encode, no filtering.mkvÄ using LWLibavSource clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/1 re-encode, no filtering.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0) frame = clip.get_frame(0) # setting color matrix to 470bg. clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG) # setting color transfer (vs.TRANSFER_BT601), if it is not set. if validate.transferIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601) # setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set. if validate.primariesIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG) # setting color range to TV (limited) range. prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange' clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_LIMITED}) # making sure frame rate is set to 25fps clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # making sure the detected scan type is set (detected: progressive) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # scan type: progressive # changing color matrix from '470bg' to '709' for vsProPainter clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709") # changing range from limited to full range for vsProPainter clip = core.resize.Bicubic(clip, format=vs.YUV420P8, range_in_s="limited", range_s="full") # setting color range to PC (full) range. prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange' clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_FULL}) # Inpainting using ProPainter from vspropainter import propainter as ProPainter clipMask = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/1 re-encode, no filtering.mkv", format="RGB24", cache=0) clip = ProPainter(clip, clip_mask=clipMask, mask_dilation=8, mask_region=(640,256,0,200)) # changing color matrix from '709' to '470bg' undoing color matrix change for vsProPainter clip = core.resize.Bicubic(clip, matrix_in_s="709", matrix_s="470bg") # changing range from full to limited range for vsProPainter clip = core.resize.Bicubic(clip, format=vs.YUV420P8,range_in_s="full", range_s="limited") # setting color range to TV (limited) range. prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange' clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_LIMITED}) # adjusting output color from YUV420P8 to YUV420P10 for NVEncModel clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10) # set output frame rate to 25fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # output clip.set_output() # clip current meta; color space: YUV420P10, bit depth: 10, resolution: 640x480, fps: 25, color matrix: 470bg, color primaries: Unspecific, color transfer: Unspecified, yuv luminance scale: limited, scanorder: progressive, full height: true (Meta)

I get:

Not sure what is broken now.

Cu Selur