[HELP] Resizing Question

[Selur]

encoding call looks correct:

"C:\Program Files\Hybrid\64bit\Vapoursynth\vspipe.exe" "C:\Users\Administrator\AppData\Local\Temp\encodingTempSynthSkript_2021-08-07@01_38_01_7410.vpy" - --y4m |  "C:\Program Files\Hybrid\64bit\ffmpeg.exe" -y -noautorotate -nostdin -threads 4 -f yuv4mpegpipe -i - -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -vcodec ffv1 -coder 0 -context 0 -g 1 -level 1 -metadata encoding_tool="Hybrid 2021.08.06.1" "C:\Users\Administrator\AppData\Local\Temp\1917 (2019)_new_2021-08-07@01_38_01_7410_02.mkv"

but aborts with:

[yuv4mpegpipe @ 00000000004b9180] Header too large.

Vapoursynth script looks correct to me:

# Imports
import os
import sys
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ColorFilter/ToneMap/tonemap.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import muvsfunc
import adjust
# source: 'D:\Encoding Experiments\Testing\1917\1917 (2019).mkv'
# current color space: YUV420P10, bit depth: 10, resolution: 3840x2160, fps: 23.976, color matrix: 2020ncl, yuv luminance scale: limited, scanorder: progressive
# Loading D:\Encoding Experiments\Testing\1917\1917 (2019).mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="D:/Encoding Experiments/Testing/1917/1917 (2019).mkv", format="YUV420P10", cache=0, prefer_hw=0)
# making sure input color matrix is set as 2020ncl
clip = core.resize.Bicubic(clip, matrix_in_s="2020ncl",range_s="limited")
# making sure frame rate is set to 23.976
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# Color Adjustment
clip = adjust.Tweak(clip=clip, hue=0.00, sat=1.16, cont=1.18, coring=True)
# color adjustment using ToneMap
clip = core.resize.Bilinear(clip=clip, format=vs.RGBS, range_in_s="limited", matrix_in_s="2020ncl", primaries_in_s="2020", primaries_s="2020", transfer_in_s="st2084", transfer_s="linear",dither_type="none", nominal_luminance=1000)
clip = core.tonemap.Mobius(clip=clip, exposure=1.850, transition=0.100)
clip = core.resize.Bilinear(clip=clip, format=vs.YUV420P8, matrix_s="709", primaries_s="709",range_s="limited", transfer_s="709", transfer_in_s="linear", range_in_s="limited", dither_type="ordered")
# denoising using TTempSmooth
clip =  core.ttmpsm.TTempSmooth(clip=clip, maxr=2, strength=1, planes=[0])
# Resizing: 3840x2160 -> 1920x1080
clip = muvsfunc.SSIM_downsample(clip=clip, w=1920, h=1080, smooth=1, use_fmtc=True, gamma=False, fulls=False, fulld=False)
clip= core.resize.Bicubic(clip=clip, format=vs.YUV420P8)
# set output frame rate to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()

Does the Vapoursynth Preview work correctly?

I did try to reproduce the issue here using:

# Imports
import os
import sys
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'I:/workspace/Hybrid/debug/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/workspace/Hybrid/debug/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll")
core.std.LoadPlugin(path="I:/workspace/Hybrid/debug/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/workspace/Hybrid/debug/64bit/vsfilters/ColorFilter/ToneMap/tonemap.dll")
core.std.LoadPlugin(path="I:/workspace/Hybrid/debug/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import muvsfunc
# source: 'G:\TestClips&Co\files\HDR\HDR10\4K sun HDR test.mp4'
# current color space: YUV420P10, bit depth: 10, resolution: 3840x2160, fps: 25, color matrix: 2020ncl, yuv luminance scale: limited, scanorder: progressive
# Loading G:\TestClips&Co\files\HDR\HDR10\4K sun HDR test.mp4 using LibavSMASHSource
clip = core.lsmas.LibavSMASHSource(source="G:/TestClips&Co/files/HDR/HDR10/4K sun HDR test.mp4")
# making sure input color matrix is set as 2020ncl
clip = core.resize.Bicubic(clip, matrix_in_s="2020ncl",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# color adjustment using ToneMap
clip = core.resize.Bilinear(clip=clip, format=vs.RGBS, range_in_s="limited", matrix_in_s="2020ncl", primaries_in_s="2020", primaries_s="2020", transfer_in_s="st2084", transfer_s="linear",dither_type="none", nominal_luminance=1000)
clip = core.tonemap.Mobius(clip=clip)
clip = core.resize.Bilinear(clip=clip, format=vs.YUV420P8, matrix_s="709", primaries_s="709",range_s="limited", transfer_s="709", transfer_in_s="linear", range_in_s="limited", dither_type="ordered")
# denoising using TTempSmooth
clip =  core.ttmpsm.TTempSmooth(clip=clip)
# Resizing: 3840x2160 -> 1920x1080
clip = muvsfunc.SSIM_downsample(clip=clip, w=1920, h=1080, smooth=1, use_fmtc=True, gamma=False, fulls=False, fulld=False)
clip= core.resize.Bicubic(clip=clip, format=vs.YUV420P8)
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 25.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

(should be similar encough to spot errors)
-> couldn't reproduce the problem.

Okay, one issue I spotted, the output of your vapoursynth script isn't YUV420P10 but YUV420P8, but that's not causing the crash.
-> looking into it

Cu Selur

Ps.: As a side note: I would usually move Tweak below ToneMap.

[Selur]

Found and fixed issue of the FFV1 bitdepth mixup.
Will send you a link to a new dev version in ~15min via pm.
But that probably didn't fix the Vapoursynth crash.
Since I can't reproduce the crash here, please check the Vapoursynth Preview and report any errors.

Cu Selur

[Lyra]

Found and fixed issue of the FFV1 bitdepth mixup.
Will send you a link to a new dev version in ~15min via pm.
But that probably didn't fix the Vapoursynth crash.
Since I can't reproduce the crash here, please check the Vapoursynth Preview and report any errors.

Cu Selur

Here's the debug log from the vapoursynth preview. As before, it works with everything else but crashes with SSIMD. In case it makes your life easier, I'm the win7 user that's replacing the vapoursynth folder with the one from 2021.04.12.1.

As for putting the tweak after tonemapping, I was actually wondering about that myself, every time I would adjust contrast the exposure would be off a bit and vice versa so I had to mess with it a lot to get it just right, I suppose all that effort was in vain lol.

[Selur]

You can change the FilterOrder under "Filtering->Vapoursynth->Misc->Filter Order/Queue".

Cu Selur

Ps.: I fixed SSMID, a while ago, but broke something else on which I'm working atm.

[Lyra]

You can change the FilterOrder under "Filtering->Vapoursynth->Misc->Filter Order/Queue".

Cu Selur

Ps.: I fixed SSMID, a while ago, but broke something else on which I'm working atm.

Success! Both vapourynth preview and encoding work, but I noticed it doesn't produce nearly as detailed of a result as the MadVR implementation of SSIM downsampling. I'm not sure why yet, but I'm wondering if it might be the default resample kernel being bicubic? Could you expose the kernel and use_fmt arguments?

[Selur]

I'll add the kernel after some sleep,...24 hours without sleep atm.

[Lyra]

I'll add the kernel after some sleep,...24 hours without sleep atm.

Oh, jeez, sleep well!

Edit:
So after spending the night looking into SSIMD, it looks like the muvsfunc implementation of SSIMD isn't actually SSIMD at all! Looking at the script, it doesn't even look close, and the author themselves say it's a "pseudo-implementation". Anyway, I tracked down what MPV uses for SSIMD and I found a GLSL script, linked below

https://gist.github.com/igv/36508af3ffc8...1d6969be10

As you can probably tell by comparing the two scripts they're nothing alike at all, and given how on MPV it looks wayyy better, I'd bet money this is the right way to implement it.

[Selur]

No problem, will do.

Cu Selur

[Selur]

Send you a link for testing.

Cu Selur

[Lyra]

Send you a link for testing.

Cu Selur

After testing with box set to zero and lanczos as the kernel, I can confirm it acts exactly identical to lanczos. Interestingly, when using it with tcanny of .5 it adds in just the right amount of edge detection so I can now see why the author calls it a pseudo implementation. It could be a nice minor improvement to lanczos if tcanny .5 was made the default setting.

That said I definitely prefer real SSIMD, so I'm going to mess around with the GLSL filter.