scratch removal

[mohanohi]

Hi,
Any idea to remove the bottom scratch? Each clip at the end of the frame scratch exists.

https://pasteboard.co/nR0xQ2PL3Utd.png

[Selur]

Since it's about an image: use an image editor and fill the line.

If it's a video, you should remove the problem before doing upscaling 2048x1556 seems like an upscaled resolution.

Looking at the separated YUV channels the scratch is only in the Y channel.
Share a sample and I can suggest something depending on the nature of the problem (depending on whether the line is static or not different approaches are possible)
Other than that:
a. you could write a custom script in which you:
1. downscale the source to a resolution where the line isn't visible
2. upscale to the original resolution
3. replace the line with the down- and upscaled version by masking it
b. with some masking something like Balanced Border, EdgeFixer or Bore might work.
c. if it's static: create a mask and use proPainter (or other inpainting methods), this will likely result in the best output, but is probably overkill at the current resolution.
Got a few ideas, but without an actual video they are probably not really helpful.
=> should be fixable, but will require some tweaking

Cu Selur

[mohanohi]

Thanks for the reply.
This is a film scan sequence file. Not an image. These line occurs when a negative cutter make a bad tape joint 😄. This line is consistent with each clip end frame only. No entire clip. I am doing a feature film Restoration right now, and because of budget problem they are not going for Damiant / phoenix Restoration. I am doing color grading and some cleanup in Davinci. Davinci has revival tool to manually do inpaint where in it looks into previous frame and does inpaint. But the result is not that great.

Can't do upscale and loose information. In feature film Restoration every frame is a diamond!!!.

Was able to remove hairs and spots with spotless. If you could implement Deltarestore it would be great. Any other tips for this situation?

[Selur]

Detarestore will not help with that scratch, it's basically a wrapper around Spotless to make it less destructive. (restore some moving parts)
Haven't seen anyone do a port for Vapoursynth or make it a proper script, even for Avisynth, so I wouldn't hold my breath for it to appear in Hybrid any time soon.
Got just the last frame in the native resolution? (normal Inpainting should work fine on this, but applying it on high resolution content is a pain, since you easily run out of memory.)

Cu Selur

Ps.: https://forum.doom9.org/showthread.php?t=184550 might be interessting

[Selur]

Quick test with inpainting through VSGAN, proPainter should be better, but for that resolution is too high:

# Imports
import vapoursynth as vs
import os
import sys
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libimwri.dll")
# Import scripts
import mvsfunc

# Loading C:\Users\Selur\Desktop\photo_1.png using vsImageReader
clip = core.imwri.Read(["C:/Users/Selur/Desktop/nR0xQ2PL3Utd.png"])
clip = core.std.Loop(clip=clip, times=100)
# Input color space is assumed to be RGB24

# resizing using VSGAN
from vsgan import ESRGAN
mask = core.imwri.Read(["C:/Users/Selur/Desktop/mask.png"])
mask = core.std.BinarizeMask(mask,threshold=16)
mask = core.std.Maximum(mask)
mask = core.std.Maximum(mask)
green = core.std.BlankClip(clip, color=[0,255,0])
clip = core.std.MaskedMerge(clip, green , mask) # masked clip

org = clip
vsgan = ESRGAN(clip=clip,device="cuda")
model = "F:/Hybrid/64bit/vsgan_models/1x_NMKD-YandereInpaint_375000_G.pth"
vsgan.load(model)
vsgan.apply()
clip = vsgan.clip

clip = core.std.StackHorizontal([org, clip])

# Output
clip.set_output()

Not that good, but tunning multiple Inpainting calls with multiple masks probably would work even when using this inpainting model.

Another 1pass cv based inpainting examples using https://github.com/dnjulek/VapourSynth-cv_inpaint:
InpaintFSR:

InpaintNS:

InpaintShiftMap:

InpaintTelea:

like I wrote multiple inpainting passes with different masks will probably help more.

Cu Selur
Ps.: https://forum.doom9.org/showthread.php?t=184485 might also be interessting.

[mohanohi]

Does this script help to remove the line?

https://silentaperture.gitlab.io/mdbook-...html#bbmod

[Selur]

I doubt it, but bbmod=BalancedBorders in Hybrid, so you can try it yourself.

[mohanohi]

Thanks.