How is called this kind of "defect" and how to mitigate? (if possible)

[Bartoloni]

this is what happens on final result using MCdegrainsharp and QTGCM deinterlace.. on the original footage is not too much visible but after the degrain is very very evident.



original footage: https://www.wcn.it/defect.zip

the problem seems related to some sort of fast interference (generated on the TBC) and red seems more influenced from it. the strange thing is that just on this tape is very very evident.

very thnak you for any help!

[Selur]

Looks like some kind of aliasing.
1rst idea try:
1. denoising the separate fields before deinterlacing/ivtc.
2. applying Santiag with increased strength afterward.
will look at it after work.

Cu Selur

[Selur]

Here's what that first idea does:

# converting interlaced to half-height progressive for filtering (vsMCTemporalDenoise) (separate fields) clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited") clip = core.std.SeparateFields(clip, tff=False) clipEven = clip[::2] # resolution 720x288 clipEven = core.std.SetFrameProps(clip=clipEven, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive clipOdd = clip[1::2] # resolution 720x288 clipOdd = core.std.SetFrameProps(clip=clipOdd, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive # denoising using MCTemporalDenoise clipEven = denoise.MCTemporalDenoise(i=clipEven, settings="very high", ncpu=10) # denoising using MCTemporalDenoise clipOdd = denoise.MCTemporalDenoise(i=clipOdd, settings="very high", ncpu=10) # converting half-height progressive to interlacedbefore deinterlacing clip = core.std.Interleave([clipOdd, clipEven]) clip = core.std.DoubleWeave(clip=clip, tff=False) # resolution 720x576 clip = core.std.SelectEvery(clip, 2, 0) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_TOP) # tff # Deinterlacing using TIVTC clip = core.tivtc.TFM(clip=clip, chroma=True) clip = core.tivtc.TDecimate(clip=clip, mode=7, rate=23.9760, dupThresh=0.04, vidThresh=3.50, sceneThresh=15.00)# new fps: 23.976 # Making sure content is preceived as frame based clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive # applying anti aliasing using santiag clip = antiAliasing.santiag(c=clip, strh=3, strv=3, nns=2, qual=2, pscrn=2, opencl=True)

clip

2nd idea: using SpotLess before the deinterlacing and BasicVSR++ on the edges after Santiag. (script,clip)
3rd idea: adding CodeFormer to before BasicVSR++ (script,clip)

Hope these help to get it started.

Cu Selur

[Bartoloni]

impressive result!, very very thank you for your time and your help Selur!

just tried the first script:

Failed to evaluate the script: Python exception: invalid syntax (tempPreviewVapoursynthFile13_11_54_548.vpy, line 65) Traceback (most recent call last): File "src\\cython\\vapoursynth.pyx", line 3351, in vapoursynth._vpy_evaluate File "C:\Users\Bfield\AppData\Local\Temp\tempPreviewVapoursynthFile13_11_54_548.vpy", line 65 clip = antiAliasing.santiag(c=clip, strh=3, strv=3, nns=2, qual=2, pscrn=2, opencl=True)6 ^ SyntaxError: invalid syntax

and i think the "6" need to be removed

but after:

Failed to evaluate the script: Python exception: name 'denoise' is not defined Traceback (most recent call last): File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate File "C:\Users\Bfield\AppData\Local\Temp\tempPreviewVapoursynthFile13_12_59_789.vpy", line 90, in clip = beforeDeCross(clip) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\Bfield\AppData\Local\Temp\tempPreviewVapoursynthFile13_12_59_789.vpy", line 23, in beforeDeCross clipEven = denoise.MCTemporalDenoise(i=clipEven, settings="very high", ncpu=10) ^^^^^^^ NameError: name 'denoise' is not defined

[Selur]

Sorry, totally missed your post.
Uploaded a new dev to be sure we use the same versions.

Cu Selur

[Bartoloni]

just downloaded but it's doing the same

Failed to evaluate the script: Python exception: name 'denoise' is not defined Traceback (most recent call last): File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate File "C:\Users\Bfield\AppData\Local\Temp\tempPreviewVapoursynthFile23_05_49_117.vpy", line 61, in clip = beforeEnd(clip) ^^^^^^^^^^^^^^^ File "C:\Users\Bfield\AppData\Local\Temp\tempPreviewVapoursynthFile23_05_49_117.vpy", line 22, in beforeEnd clipEven = denoise.MCTemporalDenoise(i=clipEven, settings="very high", ncpu=10) ^^^^^^^ NameError: name 'denoise' is not defined

[Selur]

Are you sure you are using the latest dev?
=> please create a debug output, since I can't reproduce this here.
for me scripts like:

# Imports import vapoursynth as vs # getting Vapoursynth core import ctypes import sys import os core = vs.core # Import scripts folder scriptPath = 'F:/Hybrid/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) # Loading Support Files Dllref = ctypes.windll.LoadLibrary("F:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll") # loading plugins core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/DCTFilter.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DeblockFilter/Deblock/Deblock.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libmvtools.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll") # Import scripts import denoise import validate # Source: 'G:\TestClips&Co\files\test.avi' # Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual # Loading G:\TestClips&Co\files\test.avi using LWLibavSource clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0) frame = clip.get_frame(0) # setting color matrix to 470bg. clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG) # setting color transfer (vs.TRANSFER_BT601), if it is not set. if validate.transferIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601) # setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set. if validate.primariesIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG) # setting color range to TV (limited) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED) # making sure frame rate is set to 25fps clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # making sure the detected scan type is set (detected: progressive) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive # denoising using MCTemporalDenoise clip = denoise.MCTemporalDenoise(i=clip, settings="very high", ncpu=10) # adjusting output color from: YUV420P8 to YUV420P10 for NVEncModel clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited") # set output frame rate to 25fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # output clip.set_output()

get generated and work fine.

Cu Selur

[Bartoloni]

sorry for the late reply Selur
here the debug log .. i just installed the latest dev version deleting any folder present before

http://www.wcn.it/debug2025.zip

the preview is showing the same thing.

maybe i'm doing something wrong..

i place the clip.. select the output as MKV, add the audio track... set the quantization to 12 (constant 1 pass)

open CUSTOM tab (vapoursynth)

select "INSERT BEFORE END"

added the code.

[Selur]

Okay, no wonder that isn't working!
Your script is:

# Imports import vapoursynth as vs # getting Vapoursynth core import sys import os core = vs.core # Import scripts folder scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) # loading plugins core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll") # defining beforeEnd-function - START def beforeEnd(clip): # converting interlaced to half-height progressive for filtering (vsMCTemporalDenoise) (separate fields) clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited") clip = core.std.SeparateFields(clip, tff=False) clipEven = clip[::2] # resolution 720x288 clipEven = core.std.SetFrameProps(clip=clipEven, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive clipOdd = clip[1::2] # resolution 720x288 clipOdd = core.std.SetFrameProps(clip=clipOdd, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive # denoising using MCTemporalDenoise clipEven = denoise.MCTemporalDenoise(i=clipEven, settings="very high", ncpu=10) # denoising using MCTemporalDenoise clipOdd = denoise.MCTemporalDenoise(i=clipOdd, settings="very high", ncpu=10) # converting half-height progressive to interlacedbefore deinterlacing clip = core.std.Interleave([clipOdd, clipEven]) clip = core.std.DoubleWeave(clip=clip, tff=False) # resolution 720x576 clip = core.std.SelectEvery(clip, 2, 0) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_TOP) # tff # Deinterlacing using TIVTC clip = core.tivtc.TFM(clip=clip, chroma=True) clip = core.tivtc.TDecimate(clip=clip, mode=7, rate=23.9760, dupThresh=0.04, vidThresh=3.50, sceneThresh=15.00)# new fps: 23.976 # Making sure content is preceived as frame based clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive # applying anti aliasing using santiag clip = antiAliasing.santiag(c=clip, strh=3, strv=3, nns=2, qual=2, pscrn=2, opencl=True) return clip # defining beforeEnd-function - END # Import scripts import validate # Source: 'C:\Users\Bfield\Desktop\defect.avi' # Current color space: YUV420P8, bit depth: 8, resolution: 720x576, frame rate: 30fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: ULH0 # Loading C:\Users\Bfield\Desktop\defect.avi using LWLibavSource clip = core.lsmas.LWLibavSource(source="C:/Users/Bfield/Desktop/defect.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0) frame = clip.get_frame(0) # setting color matrix to 470bg. clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG) # setting color transfer (vs.TRANSFER_BT601), if it is not set. if validate.transferIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601) # setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set. if validate.primariesIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG) # setting color range to TV (limited) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED) # making sure frame rate is set to 30fps clip = core.std.AssumeFPS(clip=clip, fpsnum=30, fpsden=1) # making sure the detected scan type is set (detected: progressive) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive clip = beforeEnd(clip) # set output frame rate to 30fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=30, fpsden=1) # output clip.set_output()

In your custom section you use:
'denoise','tivtc','antiAliasing' but you don't import the used scripts, or load the required plugins.

No clue why you are using a custom section, to begin with.
The stuff I did in post #3, you can archive by:

• loading your source
  
• enabling and settings 'Filtering->(De-)Interlace/Telecine->Overwrite input scan type to' to 'telecine'
  
• setting 'Filtering->(De-)Interlace/Telecine->TIVTC Settings->TDecimate->Mode' to '7'
  
• setting 'Filtering->(De-)Interlace/Telecine->TIVTC Settings->TDecimate->Rate' to '23.976'
  
• setting 'Filtering->Vapoursynth->Misc->Script->Filter interlace handling' to 'separate fields'
  
• enabling 'Filtering->Vapoursynth->DeNoise->MC Temporal Denoise'
  
• setting 'Filtering->Vapoursynth->DeNoise->MC Temporal Denoise->Settings' to 'very high' (optionall increasing ncpu to 10)
  
• switching to 'Filtering->Vapoursynth->Misc->Filter Order/Queue' and moving 'MCTemporalDenois' above 'Deinterlace'
  
• enabling 'Filtering->Vapoursynth->Line->AntiAliasing->Santiag'
  
• setting 'Filtering->Vapoursynth->Line->AntiAliasing->Santiag->strength->h&v' both to '3 (optionally enabling 'OpenCL')
  

For me, the Vapoursynth Script Preview then shows:

# Imports import vapoursynth as vs # getting Vapoursynth core import ctypes import sys import os core = vs.core # Import scripts folder scriptPath = 'F:/Hybrid/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) # Loading Support Files Dllref = ctypes.windll.LoadLibrary("F:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll") # loading plugins core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libsangnom.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/EEDI2.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/EEDI3m_opencl.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/NNEDI3CL.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/DCTFilter.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DeblockFilter/Deblock/Deblock.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libmvtools.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll") # Import scripts import antiAliasing import denoise import validate # Source: 'C:\Users\Selur\Desktop\defect.avi' # Current color space: YUV420P8, bit depth: 8, resolution: 720x576, frame rate: 30fps, scanorder: telecine, yuv luminance scale: limited, matrix: 470bg, format: ULH0 # Loading C:\Users\Selur\Desktop\defect.avi using LWLibavSource clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/defect.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0) frame = clip.get_frame(0) # setting color matrix to 470bg. clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG) # setting color transfer (vs.TRANSFER_BT601), if it is not set. if validate.transferIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601) # setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set. if validate.primariesIsInvalid(clip): clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG) # setting color range to TV (limited) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED) # making sure frame rate is set to 30fps clip = core.std.AssumeFPS(clip=clip, fpsnum=30, fpsden=1) # making sure the detected scan type is set (detected: telecine) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_TOP) # tff # converting interlaced to half-height progressive for filtering (vsMCTemporalDenoise) (separate fields) clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited") clip = core.std.SeparateFields(clip, tff=False) clipEven = clip[::2] # resolution 720x288 clipEven = core.std.SetFrameProps(clip=clipEven, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive clipOdd = clip[1::2] # resolution 720x288 clipOdd = core.std.SetFrameProps(clip=clipOdd, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive # denoising using MCTemporalDenoise clipEven = denoise.MCTemporalDenoise(i=clipEven, settings="very high", ncpu=10) # denoising using MCTemporalDenoise clipOdd = denoise.MCTemporalDenoise(i=clipOdd, settings="very high", ncpu=10) # converting half-height progressive to interlacedbefore deinterlacing clip = core.std.Interleave([clipOdd, clipEven]) clip = core.std.DoubleWeave(clip=clip, tff=False) # resolution 720x576 clip = core.std.SelectEvery(clip, 2, 0) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_TOP) # tff # Deinterlacing using TIVTC clip = core.tivtc.TFM(clip=clip) clip = core.tivtc.TDecimate(clip=clip, mode=7, rate=23.9760, dupThresh=0.04, vidThresh=3.50, sceneThresh=15.00)# new fps: 23.976 # Making sure content is preceived as frame based clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive # applying anti aliasing using santiag clip = antiAliasing.santiag(c=clip, strh=3, strv=3, nns=2, qual=2, pscrn=2, opencl=True) # adjusting output color from: YUV444P8 to YUV420P10 for NVEncModel clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited") # set output frame rate to 23.976fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001) # output clip.set_output()

In theory, you could use a custom section, but when using a custom section you are responsible to load all dependencies&co.

Cu Selur

[Bartoloni]

very thank you for the help!