Possible typo Crashing Temporal filter.

[Wisperer]

Hm. So I have encountered an issue with the MCTemporalDenoiser. Basically it will crash. Every time. I've tracked down the issue to an unloaded plugin/filter.
It's asking for libneofft3d.so. However libneo-fft3d.so is loaded, so I think this was a typo Oversight.

Here is the .vpy script generated by hybrid. Note in this particular set up libneo-fft3d.so is loaded for a different filter/plugin. but omitting the libneofft3d.so line will crash it even thought the correct plugin is loaded it's misnamed, and leaving it in will crash it because what it's looking for isn't there once again due to the typo. Loading a second renamed copy crashes things as well

# Imports

import os

import sys

import functools

import vapoursynth as vs

core = vs.get_core()

# Import scripts folder

scriptPath = '/opt/hybrid/vsscripts'

sys.path.append(os.path.abspath(scriptPath))

# Loading Plugins

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libttempsmooth.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libneofft3d.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libdctfilter.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libdeblock.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libtdeintmod.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libaddgrain.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libneo-fft3d.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libdfttest.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libeedi3m.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/vsznedi3.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libmvtools.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libtemporalsoften.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libscenechange.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libfmtconv.so")

core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libvslsmashsource.so")

# Import scripts

import havsfunc

# source: '/home/fletcher/Documents/Kanokon BD 1/Kanokon 2.mkv'

# current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, fps: 29.97, color matrix: 709, yuv luminance scale: limited, scanorder: top field first

# Loading /home/fletcher/Documents/Kanokon BD 1/Kanokon 2.mkv using LWLibavSource

clip = core.lsmas.LWLibavSource(source="/home/fletcher/Documents/Kanokon BD 1/Kanokon 2.mkv", format="YUV420P8", cache=0, prefer_hw=0)

# making sure input color matrix is set as 709

clip = core.resize.Point(clip, matrix_in_s="709",range_s="limited")

# making sure frame rate is set to 29.970

clip = core.std.AssumeFPS(clip, fpsnum=30000, fpsden=1001)

# Setting color range to TV (limited) range.

clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

clip2clip=clip

clip2clip = havsfunc.QTGMC(Input=clip2clip, Preset="slow", TFF=False,FPSDivisor=2)

clip = core.vivtc.VFM(clip=clip, clip2=clip2clip, order=0, mode=3, blockx=16, blocky=16, scthresh=10)

clip = core.vivtc.VDecimate(clip=clip, clip2=clip2clip, scthresh=10, blockx=16, blocky=16)# new fps: 23.976

# Fix combed frames

# adding helper function

def conditionalDeint(n, f, orig, deint):

    if f.props._Combed:

        return deint

    else:

        return orig

clipDeint = havsfunc.QTGMC(Input=clip, Preset="slow", TFF=True, TR2=1, SourceMatch=3, FPSDivisor=2)

clipCombProps = core.tdm.IsCombed(clip=clip, cthresh=8, blockx=8, blocky=8)

clip = core.std.FrameEval(clip, functools.partial(conditionalDeint, orig=clip, deint=clipDeint), clipCombProps)

clip = havsfunc.Vinverse(clp=clip, sstr=2.80)

# denoising using MCTemporalDenoise

clip = havsfunc.MCTemporalDenoise(i=clip, settings="medium")

# adjusting output color from: YUV420P8 to YUV420P10 for x264Model (i420@10-bit)

clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")

# Output
[color=#333333][font=Tahoma, Verdana, Arial, sans-serif][size=small][color=#333333][size=small][font=Tahoma, Verdana, Arial, sans-serif]clip.set_output()[/font][/size][/color][/size][/font][/color]

[Selur]

Will look at it when I get home tonight.

Cu Selur

[Selur]

I compiled a dev version in which I changed 'libneo-fft3d' to 'libneofft3d', which will hopefully fix the issue.
-> send you a link for testing via PM (haven't tested it myself)

Cu Selur

[Wisperer]

I compiled a dev version in which I changed 'libneo-fft3d' to 'libneofft3d', which will hopefully fix the issue.
-> send you a link for testing via PM (haven't tested it myself)

Cu Selur

The Download link just Gave me a 404 error.

Also I'm going to have to look, but I recall encountering the same issue with a different filter as well.

[Selur]

link should work now (accidentally uploaded the file to a wrong folder )

Cu Selur

[Wisperer]

link should work now (accidentally uploaded the file to a wrong folder )

Cu Selur

Good news. Seems to have fixed what ever that was, bad news though. Different bug!

The Bug:

2020-12-25 17:14:34.258
Failed to evaluate the script:
Python exception: DFTTest: Function does not take argument(s) named sstring

Traceback (most recent call last):
File "src/cython/vapoursynth.pyx", line 2244, in vapoursynth.vpy_evaluateScript
File "src/cython/vapoursynth.pyx", line 2245, in vapoursynth.vpy_evaluateScript
File "/tmp/encodingTempSynthSkript_2020-12-25@17_14_05_6010.vpy", line 49, in
clip = havsfunc.MCTemporalDenoise(i=clip, settings="low")
File "/usr/lib/python3.9/site-packages/havsfunc.py", line 2781, in MCTemporalDenoise
p = core.dfttest.DFTTest(i, tbsize=1, sstring='0.0:4.0 0.2:9.0 1.0:15.0', planes=planes)
File "src/cython/vapoursynth.pyx", line 2042, in vapoursynth.Function.__call__
vapoursynth.Error: DFTTest: Function does not take argument(s) named sstring

The VPY:

# Imports
import os
import sys
import functools
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = '/home/fletcher/Downloads/hybrid/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libttempsmooth.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libdctfilter.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libdeblock.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libaddgrain.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libneofft3d.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libdfttest.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libeedi3m.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/vsznedi3.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libmvtools.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libtemporalsoften.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libscenechange.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libfmtconv.so")
core.std.LoadPlugin(path="/home/fletcher/.hybrid/vsplugins/libvslsmashsource.so")
# Import scripts
import havsfunc
# source: '/home/fletcher/Documents/Kanokon BD 1/Kanokon 1.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, fps: 29.97, color matrix: 709, yuv luminance scale: limited, scanorder: top field first
# Loading /home/fletcher/Documents/Kanokon BD 1/Kanokon 1.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="/home/fletcher/Documents/Kanokon BD 1/Kanokon 1.mkv", format="YUV420P8", cache=0, prefer_hw=0)
# making sure input color matrix is set as 709
clip = core.resize.Point(clip, matrix_in_s="709",range_s="limited")
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
clip2clip = clip
clip2clip = havsfunc.QTGMC(Input=clip2clip, Preset="fast", TFF=True,FPSDivisor=2)
clip = core.vivtc.VFM(clip=clip, order=1, mode=1)
# VDecimate helper function
def postprocess(n, f, clip, deinterlaced):
  if f.props['_Combed'] > 0:
    return deinterlaced
  else:
    return clip
postprocessed_clip = core.std.FrameEval(clip=clip, eval=functools.partial(postprocess, clip=clip, deinterlaced=clip2clip), prop_src=clip)
clip = core.vivtc.VDecimate(clip=clip)# new fps: 23.976
clip = havsfunc.Vinverse(clp=clip)
# denoising using MCTemporalDenoise
clip = havsfunc.MCTemporalDenoise(i=clip, settings="low")
# adjusting output color from: YUV420P8 to YUV420P10 for x264Model (i420@8)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()

[Selur]

The error says:

File "/usr/lib/python3.9/site-packages/havsfunc.py", line 2781, in MCTemporalDenoise
p = core.dfttest.DFTTest(i, tbsize=1, sstring='0.0:4.0 0.2:9.0 1.0:15.0', planes=planes)

looking at: https://github.com/Selur/VapoursynthScri...avsfunc.py
line 2781 does not call DFTTest.
In the havsfunc.py for Hybrid, line 3141 contains:

p = i.dfttest.DFTTest(tbsize=1, slocation=[0.0,4.0, 0.2,9.0, 1.0,15.0], planes=planes)

The dev version I send you (line the normal package), comes with a vsscripts folders which is a checkout of the git (https://github.com/Selur/VapoursynthScriptsInHybrid/).
The script you use seems to differ.
So my guess is that this isn't a bug in Hybrid, but you using old, new or simply other script versions than Hybrid.

I suspect that copying the content of the vsfolder (the git) into your site-packages folder will fix the issue.
Alternatively delete the script files inside your site-packages folder, since Hybrid explicitly adds it's script folder to the Vapoursynth search path.

# Import scripts folder
scriptPath = '/home/fletcher/Downloads/hybrid/vsscripts'
sys.path.append(os.path.abspath(scriptPath))

Cu Selur

Ps.: Vapoursynth atm. does not have an option to disable the auto import of the site-packages folder.

[Wisperer]

Ah I know what it is. I have been dealing with basically two side by side Vapoursynth installs. One for Hybrid and one for YOLOcr. Yolocr is a ORC set up powered by Vapoursynth. Your set up does some *VERY* specific things I have to work with in order to have BOTH working on my system. Your default set up breaks Yolocr and the default Yolorc breaks Hybrid. So I've basically Partitioned off the Hybrid stuff into it's own folders best I can, but it seems it still makes calls to the system libraries. So yeah adding an option to set import libraries would be really nice.

Yolocr for reference

https://bitbucket.org/YuriZero/yolocr/src/master/

[Wisperer]

So I think that dev build is broken some how. Whenever I go to encode anything with Vapoursynth now it crashes ate the very end of the encode right before it outputs the video and cleans up the files.
The error at crash

2020.12.26 - 00:31:44_Arch Linux (64bit)_2020.12.25.1 - level 9: x264 output: 37523 frames: 20.15 fps, 7173.09 kb/s
2020.12.26 - 00:31:44_Arch Linux (64bit)_2020.12.25.1 - level 9: x264 output: 37533 frames: 20.15 fps, 7171.70 kb/s
2020.12.26 - 00:31:45_Arch Linux (64bit)_2020.12.25.1 - level 9: x264 output: 37544 frames: 20.15 fps, 7170.50 kb/s
2020.12.26 - 00:31:45_Arch Linux (64bit)_2020.12.25.1 - level 9: x264 output: 37559 frames: 20.16 fps, 7168.36 kb/s
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: 2020-12-26@00_00_31_5710_06_video ### /home/fletcher/Documents/Kanokon BD 1/Kanokon 2_new_new_new_new_new_new.mkv ### Helper(1) finished with exitCode: 11 and exitStatus: 1
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: aborting,.. 2020-12-26@00_00_31_5710_06_video
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: set 'CRASHED' for: 2020-12-26@00_00_31_5710_06_video
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: -> 2020-12-26@00_00_31_5710_06_video crashed: Helper1, exitCode: 11
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: 2020-12-26@00_00_31_5710_06_video: Helper1, exitCode: 11
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: 2020-12-26@00_00_31_5710_06_video: Helper1, exitCode: 11
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: Kill all processes of this job!
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: Killing helper,...
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: Killing process,...
2020.12.26 - 00:31:49_Arch Linux (64bit)_2020.12.25.1 - level 9: Aborting '2020-12-26@00_00_31_5710_06_video' due to: Helper1, exitCode: 11

[Selur]

So yeah adding an option to set import libraries would be really nice.

Sadly this is not up to me, but a missing feature in Vapoursynth.
does adding:

sys.path.append(os.path.abspath("/usr/lib/python3.9/site-packages/"))

before the '# Import scripts folder' part in the script make the script workable on your setup?
(not sure whether this break other things)

So I think that dev build is broken some how.

Does the Vapoursynth Preview work? (it probably does since 37559 frames got decoded, no clue how much of the source that is)

From the crash output, it does look like the encoding works, but Hybrid aborts since the decoder aborts with a non-zero exitStatus&Code.
Can't say more from just a portion of the debug output. Could also be a problem with the decoding of a portion of the input, Vapoursynth running out of memory,...

Cu Selur