VS preview error with smdegrain refine motion, block & overlap size, and color format

[andrewschen]

In SMDegrain, there are certain combinations of refine motion, block & overlap size, and color format which cause error in VS preview. Are they bugs, or supposed so?

In Hybrid_dev_2026.06.25-152245 (old version of SMDegrain):

YUV410p8,  YUV411p8: always error

refine motion on + overlap size 2 + YUV ( != 444 ) : error
Not happening in refine motion off / overlap size != 2 /  YUV444&GRAY


In Hybrid_dev_2026.10.02-19438:

YUV410p8,  YUV411p8: always error

and in other situation, if YUV ( != 444 ), these combination will cause error:

refine motion 1/2/3 pass + size(block,overlap) = (32,2) 
refine motion 2/3/4 pass + size(block,overlap) = (64,4)

sample code & error output:  refine motion 1 pass + size(block,overlap) = (32,2)

# Imports import sys import os import vapoursynth as vs # using Vapoursynth R79 # getting Vapoursynth core core = vs.core # Import scripts folder scriptPath = 'D:/APPD/Hybrid_dev_20261002-19438/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) # loading plugins core.std.LoadPlugin(path="D:/APPD/Hybrid_dev_20261002-19438/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/akarin/libakarin.dll") core.std.LoadPlugin(path="D:/APPD/Hybrid_dev_20261002-19438/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/mvutensils/mvutensils.dll") core.std.LoadPlugin(path="D:/APPD/Hybrid_dev_20261002-19438/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/fmtconv.dll") core.std.LoadPlugin(path="D:/APPD/Hybrid_dev_20261002-19438/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll") # Import scripts import nnedi3_resample import smdegrain import validate # Source: 'C:\Users\mmddffkk\Desktop\tmp\SAMPLE.mkv' # clip current meta; color space: YUV420P8, bit depth: 8, resolution: 1920x1080, fps: 23.976, color matrix: 709, color primaries: Unspecific, color transfer: Unspecified, yuv luminance scale: limited, scanorder: progressive, full height: true (Source) # Loading 'C:\Users\mmddffkk\Desktop\tmp\SAMPLE.mkv' using LWLibavSource clip = core.lsmas.LWLibavSource(source="C:/Users/mmddffkk/Desktop/tmp/SAMPLE.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0) frame = clip.get_frame(0) # setting color matrix to 709. clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709) # setting color transfer (vs.TRANSFER_BT709), if it is not set. if validate.transferIsInvalid(clip):   clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709) # setting color primaries info (to vs.PRIMARIES_BT709), if it is not set. if validate.primariesIsInvalid(clip):   clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709) # setting color range to TV (limited) range. prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange' clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_LIMITED}) # making sure frame rate is set to 23.976fps clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001) # making sure the detected scan type is set (detected: progressive) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # scan type: progressive # removing grain using SMDegrain clip = smdegrain.SMDegrain(clip, RefineMotion=1, interlaced=False, overlap=2, blksize=32, thSCD1=1600, tools={'mv': 'mvutensils', 'expr': 'akarin', 'nnedi3': 'znedi3', 'dfttest': 'dfttest', 'nlmeans': 'nlm_ispc', 'bm3d': 'bm3d', 'rg': 'zsmooth', 'median': 'zsmooth', 'boxblur': 'std', 'scd': 'std'}) # adjusting output color from YUV420P8 to YUV420P10 for SvtAv1Model clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10) # set output frame rate to 23.976fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001) # output clip.set_output() # clip current meta; color space: YUV420P10, bit depth: 10, resolution: 1920x1080, fps: 23.976, color matrix: 709, color primaries: Unspecific, color transfer: Unspecified, yuv luminance scale: limited, scanorder: progressive, full height: true (Meta) # script was created by Hybrid 2026.10.02.1

Failed to evaluate the script: Python exception: Degrain: Degrain2: Failed to retrieve first frame from super clip. Error message: Recalculate: The chosen block size has no multiple that will process the entire frame without exceeding the super clip padding, derive a new suitable super clip and try again Traceback (most recent call last): File "vapoursynth.pyx", line 3832, in vapoursynth._vpy_evaluate File "vapoursynth.pyx", line 3833, in vapoursynth._vpy_evaluate File "\\fx705d\ENCTMP\tempPreviewVapoursynthFile02_48_22_671.vpy", line 41, in clip = smdegrain.SMDegrain(clip, RefineMotion=1, interlaced=False, overlap=2, blksize=32, thSCD1=1600, tools={'mv': 'mvutensils', 'expr': 'akarin', 'nnedi3': 'znedi3', 'dfttest': 'dfttest', 'nlmeans': 'nlm_ispc', 'bm3d': 'bm3d', 'rg': 'zsmooth', 'median': 'zsmooth', 'boxblur': 'std', 'scd': 'std'}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\APPD\Hybrid_dev_20261002-19438\64bit\vsscripts\smdegrain.py", line 259, in SMDegrain output = MV.Degrain(mfilter, super_render, *vectors, thsad=thSAD, thsadc=thSADC, plane=plane, limit=limit, limitc=limitc, thscd1=thSCD1, thscd2=thSCD2, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\APPD\Hybrid_dev_20261002-19438\64bit\vsscripts\misc.py", line 1410, in Degrain return self._degrain(clip, super, *vectors, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\APPD\Hybrid_dev_20261002-19438\64bit\vsscripts\misc.py", line 1352, in _degrain return core.mvu.Degrain(clip, super, vec_list, limit=mvu_limit, **weight_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "vapoursynth.pyx", line 3559, in vapoursynth.Function.__call__ vapoursynth.Error: Degrain: Degrain2: Failed to retrieve first frame from super clip. Error message: Recalculate: The chosen block size has no multiple that will process the entire frame without exceeding the super clip padding, derive a new suitable super clip and try again

[Selur]

YUV410p8, YUV411p8: always error

I assume you mean: YUV420P8, YUV422P8, since that is what your script uses.
Seems like I need to add some padding when mvutensils is used.
As a workaround you can set SMDegrain->Tools->MV to MV.
From what I can see, this is not a bug, but a limitation of mvutensils which I was not aware of.

=> I'll look at it tomorrow and think of a general workaround. (my guess is that this might also happen in some other filters)

Cu Selur

[Selur]

Replacing the files in Hybrid\64bit\vsscripts with the latest from https://github.com/Selur/VapoursynthScriptsInHybrid should fix the problem.
(will release new dev version with this tomorrow)

Cu Selur

[andrewschen]

Hybrid_dev_2026.10.04-204526 fixed the issue mentioned above.

There's another thing about thSCD1, by design Hybrid will adjust the value of thSCD1 according to the fixed block size I choose. and I think there are some bugs during adjusting thSCD1.

case 1: choose block size 4, and thSCD1 will be set to 100, and 8->400 & 16->1600, now dial back to block size 8 or 4, thSCD1 won't dial back to 400/100, it stay at 1600.

case 2. Hybrid do not adjust thSCD1 if block size is 32 or above. In this case thSCD1 might be 0 or any value previously adjusted and left in the field of thSCD1. I don't know what value thSCD1 should be when block size >=32, but if the value is 0, the output video look like that it's never filtered by smdegrain, all noise are still there.

[Selur]

Will look at it when I'm back home,...

[Selur]

It's probably not a bug.
The assumtion that a specific blacksize enforces a specific threshold is wrong.
Changing the block size will result in a change of only the minimum threshold.
So unless the value is not below that minimum it will not change.
=> will look at it and do some testing.

Cu Selur

[Selur]

Okay, there were bugs.
Thanks for the report, both cases were real bugs.

What happened: Hybrid didn't actually set thSCD1 from the block size, it only raised the lower limit of the field (4: 100, 8: 400, 16: 1600). So the value went up, but
never back down, and for block sizes 32 and above nothing was adjusted at all.
With thSCD1=0 every block counts as "changed", so every frame is treated as a scene change and SMDegrain filters nothing - that's why your output looked untouched.

While looking into it I also measured that the coupling isn't needed at all: mvtools and mvutensils both normalize thSCD1 to an 8x8 block internally, so 400 means the same for every block size.

Changed for the next version:
- thSCD1 is no longer tied to the block size and is available with any block size (including "auto").
- 0 now means "auto" (400, or 0.35*thSAD+260 with "4.x formulas") and is never passed on to the script; otherwise the range is 1-16320.

Until then: with a fixed block size simply enter thSCD1 by hand (400 is the normal default), and don't leave it at 0.

Cu Selur

[Selur]

Uploaded a new dev with adjusted smdegrain script and ui.
(and tons of other small bugs, I ran into while preparing for Vapoursynth R80+)

Cu Selur

[andrewschen]

Built-in SMDegrain in Hybrid_dev_2026.10.05-203137 works ok by itself.

But if I install the whole addon (vsmlrt, vsgan, onnx model, and both VapoursynthR79_torch_2026.09.26 & VapoursynthR79_torch_2026.10.03 were tested), SMDegrain will not work at all, job crashed immediately. guess the vstorch addon needs adjusted too?

[Selur]

Hmm, I expected VapoursynthR79_torch_2026 to work, but will look at it when  I'm back home.
Does Vapoursynth Preview show an error?

Cu Selur