RE: ProPainter Vapoursynth filter - Dan64 - 28.06.2024
I was not motivated in adding outpainting mode because on github page all the examples were on inpainting while on the project page on 16 clips only 1 was on outpainting.
So I removed the outpainting mode in the Vapoursynth version.
If you are interested in having it, I can try to add it, let me know.
Dan
RE: ProPainter Vapoursynth filter - Selur - 28.06.2024
Yeah, if you find the time and motivation, outpainting would be nice.
Main usages for it that I see:
a. outpainting to do 4:3 <> 16:9 conversions (but that is probably too much to ask)
b. outpainting the edges of video content with distortions&co
Cu Selur
RE: ProPainter Vapoursynth filter - Dan64 - 28.06.2024
(27.06.2024, 20:25)Selur Wrote: Ps.: just updated the deoldify test download, since I noticed the 'Interference' didn't do a thing.
Not clear to what parameter you are referring to. This issue was already fixed in version "2024.06.28.1" ?
Dan
RE: ProPainter Vapoursynth filter - Selur - 28.06.2024
DOH,... mixed it up and meant ProPainter.
(wrote it correctly into the changelog)
RE: ProPainter Vapoursynth filter - Dan64 - 28.06.2024
Anyway given the speed of ProPainter when is applied on small masks, probably the "outpainting" will be very slow...
I will try to add it.
Dan
RE: ProPainter Vapoursynth filter - Selur - 28.06.2024
Thanks for looking into it.
Cu Selur
RE: ProPainter Vapoursynth filter - Dan64 - 28.06.2024
I attached the filter version with the support of outpainting and a demo for test.
As expected is not working, since cannot invent what has never seen.
Dan
RE: ProPainter Vapoursynth filter - Selur - 28.06.2024
Thanks, I'll give it a spin.
RE: ProPainter Vapoursynth filter - Selur - 28.06.2024
Using:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("F:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/EEDI3m_opencl.dll")# vsQTGMC
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/NNEDI3CL.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")
# Import scripts
import havsfunc
import validate
# Source: 'G:\TestClips&Co\files\MPEG-2\Mächte des Wahnsinns.m2v'
# Current color space: YUV420P8, bit depth: 8, resolution: 720x576, frame rate: 25fps, scanorder: top field first, yuv luminance scale: limited, matrix: 470bg, transfer: bt.470 system b/g, primaries: bt.601 pal
# Loading G:\TestClips&Co\files\MPEG-2\Mächte des Wahnsinns.m2v using DGSource
clip = core.dgdecodenv.DGSource("J:/tmp/m2v_10f10703815915d676a763d34caf8ab1_853323747.dgi",fieldop=0)# 25 fps, scanorder: top field first
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip=clip, _Matrix=5)
# setting color transfer (601), if it is not set.
if validate.transferIsInvalid(clip):
clip = core.std.SetFrameProps(clip=clip, _Transfer=5)
# setting color primaries info (to 470), if it is not set.
if validate.primariesIsInvalid(clip):
clip = core.std.SetFrameProps(clip=clip, _Primaries=5)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=1)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: top field first)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=2) # tff
original = clip
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=True, opencl=True) # new fps: 50
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=0) # progressive
# Deinterlacing using QTGMC
original = havsfunc.QTGMC(Input=original, Preset="Fast", TFF=True, opencl=True) # new fps: 50
# Making sure content is preceived as frame based
original = core.std.SetFrameProps(clip=original, _FieldBased=0) # progressive
clip = core.std.CropRel(clip=clip, left=2, right=0, top=68, bottom=70)# cropping to 718x438
original = core.std.CropRel(clip=original, left=2, right=0, top=68, bottom=70)# cropping to 718x438
# Resizing using 10 - bicubic spline
clip = core.fmtc.resample(clip=clip, kernel="spline16", w=720, h=310, interlaced=False, interlacedd=False) # resolution 720x310 before YUV420P8 after YUV420P16
# changing range from limited to full range for vsProPainter
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=0)
# adjusting color space from YUV420P8 to RGB24 for vsProPainter
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# Inpainting using ProPainter
from vspropainter import propainter_outpaint as ProPainterOutpaint
clip = ProPainterOutpaint(clip=clip, length=80, outpaint_size=(720, 322)) # outpainting to 720x322
# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsProPainter
# changing range from full to limited range for vsProPainter
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
clip.set_output()
the output is 720x310 not 720x322.
Looking at the example:
clip = core.std.CropRel(clip=clip, left=0, right=0, top=52, bottom=52)# cropping to 720x300
## changing color matrix from '470bg' to '709' for vsProPainter
clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")
# changing range from limited to full range for vsProPainter
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=0)
# adjusting color space from YUV420P8 to RGB24 for vsProPainter
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# Inpainting using ProPainter
import vspropainter
clip = vspropainter.propainter_outpaint(clip=clip, length=80, outpaint_size=(720, 340))
the output is 720x300. Same for the second exsample.
=> This does not seem correct.
Cu Selur
RE: ProPainter Vapoursynth filter - Dan64 - 28.06.2024
attached version with correct resize.
Dan
The outpaint mode is based on the same code used for the inpainting.
In outpaint mode ProPainter builds automatically a mask on the edges of the clip and then try to remove it as in inpainting mode.
As example I atteched the runnicar demo, where there are 2 scripts:
- runcar_outpaint.vpy : script using the new outpainting function
- runcar_inpaint.vpy : script using the old inpainting function with a mask on the edges
both the scripts are providing the same results.
I hope that this can help in understanding better how ProPainter works.
Dan
|