![]() |
|
VS preview error with smdegrain refine motion, block & overlap size, and color format - Printable Version +- Selur's Little Message Board (https://forum.selur.net) +-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html) +--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html) +--- Thread: VS preview error with smdegrain refine motion, block & overlap size, and color format (/thread-4502.html) |
VS preview error with smdegrain refine motion, block & overlap size, and color format - andrewschen - 03.10.2026 In SMDegrain, there are certain combinations of refine motion, block & overlap size, and color format which cause error in VS preview. Are they bugs, or supposed so? In Hybrid_dev_2026.06.25-152245 (old version of SMDegrain): YUV410p8, YUV411p8: always error refine motion on + overlap size 2 + YUV ( != 444 ) : error Not happening in refine motion off / overlap size != 2 / YUV444&GRAY In Hybrid_dev_2026.10.02-19438: YUV410p8, YUV411p8: always error and in other situation, if YUV ( != 444 ), these combination will cause error: refine motion 1/2/3 pass + size(block,overlap) = (32,2) refine motion 2/3/4 pass + size(block,overlap) = (64,4) sample code & error output: refine motion 1 pass + size(block,overlap) = (32,2) # Imports
import sys
import os
import vapoursynth as vs
# using Vapoursynth R79
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'D:/APPD/Hybrid_dev_20261002-19438/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="D:/APPD/Hybrid_dev_20261002-19438/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/akarin/libakarin.dll")
core.std.LoadPlugin(path="D:/APPD/Hybrid_dev_20261002-19438/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/mvutensils/mvutensils.dll")
core.std.LoadPlugin(path="D:/APPD/Hybrid_dev_20261002-19438/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/fmtconv.dll")
core.std.LoadPlugin(path="D:/APPD/Hybrid_dev_20261002-19438/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import nnedi3_resample
import smdegrain
import validate
# Source: 'C:\Users\mmddffkk\Desktop\tmp\SAMPLE.mkv'
# clip current meta; color space: YUV420P8, bit depth: 8, resolution: 1920x1080, fps: 23.976, color matrix: 709, color primaries: Unspecific, color transfer: Unspecified, yuv luminance scale: limited, scanorder: progressive, full height: true (Source)
# Loading 'C:\Users\mmddffkk\Desktop\tmp\SAMPLE.mkv' using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/mmddffkk/Desktop/tmp/SAMPLE.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 709.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)
# setting color transfer (vs.TRANSFER_BT709), if it is not set.
if validate.transferIsInvalid(clip):
clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709)
# setting color primaries info (to vs.PRIMARIES_BT709), if it is not set.
if validate.primariesIsInvalid(clip):
clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709)
# setting color range to TV (limited) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_LIMITED})
# making sure frame rate is set to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # scan type: progressive
# removing grain using SMDegrain
clip = smdegrain.SMDegrain(clip, RefineMotion=1, interlaced=False, overlap=2, blksize=32, thSCD1=1600, tools={'mv': 'mvutensils', 'expr': 'akarin', 'nnedi3': 'znedi3', 'dfttest': 'dfttest', 'nlmeans': 'nlm_ispc', 'bm3d': 'bm3d', 'rg': 'zsmooth', 'median': 'zsmooth', 'boxblur': 'std', 'scd': 'std'})
# adjusting output color from YUV420P8 to YUV420P10 for SvtAv1Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10)
# set output frame rate to 23.976fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# output
clip.set_output()
# clip current meta; color space: YUV420P10, bit depth: 10, resolution: 1920x1080, fps: 23.976, color matrix: 709, color primaries: Unspecific, color transfer: Unspecified, yuv luminance scale: limited, scanorder: progressive, full height: true (Meta)
# script was created by Hybrid 2026.10.02.1Failed to evaluate the script:
Python exception: Degrain: Degrain2: Failed to retrieve first frame from super clip. Error message: Recalculate: The chosen block size has no multiple that will process the entire frame without exceeding the super clip padding, derive a new suitable super clip and try again
Traceback (most recent call last):
File "vapoursynth.pyx", line 3832, in vapoursynth._vpy_evaluate
File "vapoursynth.pyx", line 3833, in vapoursynth._vpy_evaluate
File "\\fx705d\ENCTMP\tempPreviewVapoursynthFile02_48_22_671.vpy", line 41, in
clip = smdegrain.SMDegrain(clip, RefineMotion=1, interlaced=False, overlap=2, blksize=32, thSCD1=1600, tools={'mv': 'mvutensils', 'expr': 'akarin', 'nnedi3': 'znedi3', 'dfttest': 'dfttest', 'nlmeans': 'nlm_ispc', 'bm3d': 'bm3d', 'rg': 'zsmooth', 'median': 'zsmooth', 'boxblur': 'std', 'scd': 'std'})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\APPD\Hybrid_dev_20261002-19438\64bit\vsscripts\smdegrain.py", line 259, in SMDegrain
output = MV.Degrain(mfilter, super_render, *vectors, thsad=thSAD, thsadc=thSADC, plane=plane, limit=limit, limitc=limitc, thscd1=thSCD1, thscd2=thSCD2,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\APPD\Hybrid_dev_20261002-19438\64bit\vsscripts\misc.py", line 1410, in Degrain
return self._degrain(clip, super, *vectors, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\APPD\Hybrid_dev_20261002-19438\64bit\vsscripts\misc.py", line 1352, in _degrain
return core.mvu.Degrain(clip, super, vec_list, limit=mvu_limit, **weight_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "vapoursynth.pyx", line 3559, in vapoursynth.Function.__call__
vapoursynth.Error: Degrain: Degrain2: Failed to retrieve first frame from super clip. Error message: Recalculate: The chosen block size has no multiple that will process the entire frame without exceeding the super clip padding, derive a new suitable super clip and try againRE: VS preview error with smdegrain refine motion, block & overlap size, and color format - Selur - 03.10.2026 YUV410p8, YUV411p8: always errorSeems like I need to add some padding when mvutensils is used. As a workaround you can set SMDegrain->Tools->MV to MV. From what I can see, this is not a bug, but a limitation of mvutensils which I was not aware of. => I'll look at it tomorrow and think of a general workaround. (my guess is that this might also happen in some other filters) Cu Selur RE: VS preview error with smdegrain refine motion, block & overlap size, and color format - Selur - 03.10.2026 Replacing the files in Hybrid\64bit\vsscripts with the latest from https://github.com/Selur/VapoursynthScriptsInHybrid should fix the problem. (will release new dev version with this tomorrow) Cu Selur RE: VS preview error with smdegrain refine motion, block & overlap size, and color format - andrewschen - 05.10.2026 Hybrid_dev_2026.10.04-204526 fixed the issue mentioned above. There's another thing about thSCD1, by design Hybrid will adjust the value of thSCD1 according to the fixed block size I choose. and I think there are some bugs during adjusting thSCD1. case 1: choose block size 4, and thSCD1 will be set to 100, and 8->400 & 16->1600, now dial back to block size 8 or 4, thSCD1 won't dial back to 400/100, it stay at 1600. case 2. Hybrid do not adjust thSCD1 if block size is 32 or above. In this case thSCD1 might be 0 or any value previously adjusted and left in the field of thSCD1. I don't know what value thSCD1 should be when block size >=32, but if the value is 0, the output video look like that it's never filtered by smdegrain, all noise are still there. RE: VS preview error with smdegrain refine motion, block & overlap size, and color format - Selur - 05.10.2026 Will look at it when I'm back home,... RE: VS preview error with smdegrain refine motion, block & overlap size, and color format - Selur - 05.10.2026 It's probably not a bug. The assumtion that a specific blacksize enforces a specific threshold is wrong. Changing the block size will result in a change of only the minimum threshold. So unless the value is not below that minimum it will not change. => will look at it and do some testing. Cu Selur RE: VS preview error with smdegrain refine motion, block & overlap size, and color format - Selur - 05.10.2026 Okay, there were bugs. Thanks for the report, both cases were real bugs. What happened: Hybrid didn't actually set thSCD1 from the block size, it only raised the lower limit of the field (4: 100, 8: 400, 16: 1600). So the value went up, but never back down, and for block sizes 32 and above nothing was adjusted at all. With thSCD1=0 every block counts as "changed", so every frame is treated as a scene change and SMDegrain filters nothing - that's why your output looked untouched. While looking into it I also measured that the coupling isn't needed at all: mvtools and mvutensils both normalize thSCD1 to an 8x8 block internally, so 400 means the same for every block size. Changed for the next version: - thSCD1 is no longer tied to the block size and is available with any block size (including "auto"). - 0 now means "auto" (400, or 0.35*thSAD+260 with "4.x formulas") and is never passed on to the script; otherwise the range is 1-16320. Until then: with a fixed block size simply enter thSCD1 by hand (400 is the normal default), and don't leave it at 0. Cu Selur RE: VS preview error with smdegrain refine motion, block & overlap size, and color format - Selur - 05.10.2026 Uploaded a new dev with adjusted smdegrain script and ui. (and tons of other small bugs, I ran into while preparing for Vapoursynth R80+) Cu Selur RE: VS preview error with smdegrain refine motion, block & overlap size, and color format - andrewschen - 06.10.2026 Built-in SMDegrain in Hybrid_dev_2026.10.05-203137 works ok by itself. But if I install the whole addon (vsmlrt, vsgan, onnx model, and both VapoursynthR79_torch_2026.09.26 & VapoursynthR79_torch_2026.10.03 were tested), SMDegrain will not work at all, job crashed immediately. guess the vstorch addon needs adjusted too? RE: VS preview error with smdegrain refine motion, block & overlap size, and color format - Selur - 06.10.2026 Hmm, I expected VapoursynthR79_torch_2026 to work, but will look at it when I'm back home. Does Vapoursynth Preview show an error? Cu Selur |