[HELP] Crash When Encoding DV AVI to x264?

[zhammerle]

Hello all, I'm still fairly new to Hybrid so please bear with me. I have successfully used Hybrid in the past to encode avi files from VirtualDub. In this case, I am trying to re-encode DV video in the form of avi files to convert them to mp4, and also to deinterlace. I have attached the debug output. Please let me know if there is anything else you need from me to help figure out this issue.

[Selur]

Okay, seeing that you are using Windows 7 a few things.
Windows 7 was dropped with 2021.01.31.1 (see: https://www.selur.de/changelog), but it might still work iirc. there are a few users that use Hybrid on Windows 7 with the latest version.

Here are a few things you can try:

• check the Vapoursynth Preview and see whether it shows an error, let me know if it works or not.
  Since x264 reports ' y4m [error]: bad sequence header magic' it probably is a decoding problem.
  In case the decoding of the file fails:
  • Since you use DV and before used VirtualDub, you could try whether it helps to enable "Filtering->Vapoursynth->Misc->Source->Prefer AviSource for .avi input". This way Hybrid will look for a 64bit vfw dv codec on your system an in case there is one use that.
    
• change 'Filtering->Support' to 'Avisynth' and try whether the preview works fine.
  • Since you use DV and before used VirtualDub, you could try whether it helps to enable "Filtering->Avisynth->Misc->Source->Prefer AviSource for .avi input". This way Hybrid will look for a 32it vfw dv codec on your system an in case there is one use that. (64bit version in case Config->Internals->Avisynth->Avisynth type" is set to '64bit'.)
    
  • Since ffms2 is used as decoder, you could try using an older FFMS2 version (https://github.com/FFMS/ffms2/releases); 32bit dll belongs into 'Hybrid\32bit\avisynthPlugins'-folder, 64bit dll belongs into 'Hybrid\64bit\Avisynth\avisynthPlugins' folder.
    
• try a later Hybrid version
  

If you can upload a small sample somewhere and send me a link to it I can try whether it's a general decoding issue with your file.
I tried some content with the same characteristics of your file (# color space: YUV411P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: bottom field first) and had no issue decoding (and encoding to 422 x264) it.

The Vapoursynth script used:

# Imports
import os
import sys
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/temporalsoften.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# Import scripts
import havsfunc
# source: 'E:\Temp Videos\Means DV Temp\Allie Bball - Emily Field Trip '05 temp.avi'
# current color space: YUV411P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: bottom field first
# Loading source using FFMS2
clip = core.ffms2.Source(source="E:/Temp Videos/Means DV Temp/Allie Bball - Emily Field Trip '05 temp.avi",cachefile="C:/Users/Zack/AppData/Local/Temp/2021-07-31@13_00_52_5310.ffindex",format=vs.YUV411P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting color space from YUV411P8 to YUV444P8 for VsQTGMC
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited")
# setting field order to what QTGMC should assume (bottom field first)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=1)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Faster", TFF=False) # new fps: 59.94
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
# adjusting output color from: YUV444P8 to YUV422P8 for x264Model (i422@8)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV422P8, range_s="limited")
# set output frame rate to 59.940fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001)
# Output
clip.set_output()

also looks fine to me.


Cu Selur

[zhammerle]

Okay, seeing that you are using Windows 7 a few things.
Windows 7 was dropped with 2021.01.31.1 (see: https://www.selur.de/changelog), but it might still work iirc. there are a few users that use Hybrid on Windows 7 with the latest version.

Here are a few things you can try:

• check the Vapoursynth Preview and see whether it shows an error, let me know if it works or not.
  Since x264 reports ' y4m [error]: bad sequence header magic' it probably is a decoding problem.
  In case the decoding of the file fails:
  • Since you use DV and before used VirtualDub, you could try whether it helps to enable "Filtering->Vapoursynth->Misc->Source->Prefer AviSource for .avi input". This way Hybrid will look for a 64bit vfw dv codec on your system an in case there is one use that.
    
• change 'Filtering->Support' to 'Avisynth' and try whether the preview works fine.
  • Since you use DV and before used VirtualDub, you could try whether it helps to enable "Filtering->Avisynth->Misc->Source->Prefer AviSource for .avi input". This way Hybrid will look for a 32it vfw dv codec on your system an in case there is one use that. (64bit version in case Config->Internals->Avisynth->Avisynth type" is set to '64bit'.)
    
  • Since ffms2 is used as decoder, you could try using an older FFMS2 version (https://github.com/FFMS/ffms2/releases); 32bit dll belongs into 'Hybrid\32bit\avisynthPlugins'-folder, 64bit dll belongs into 'Hybrid\64bit\Avisynth\avisynthPlugins' folder.
    
• try a later Hybrid version
  

If you can upload a small sample somewhere and send me a link to it I can try whether it's a general decoding issue with your file.
I tried some content with the same characteristics of your file (# color space: YUV411P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: bottom field first) and had no issue decoding (and encoding to 422 x264) it.

The Vapoursynth script used:

# Imports
import os
import sys
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/temporalsoften.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# Import scripts
import havsfunc
# source: 'E:\Temp Videos\Means DV Temp\Allie Bball - Emily Field Trip '05 temp.avi'
# current color space: YUV411P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: bottom field first
# Loading source using FFMS2
clip = core.ffms2.Source(source="E:/Temp Videos/Means DV Temp/Allie Bball - Emily Field Trip '05 temp.avi",cachefile="C:/Users/Zack/AppData/Local/Temp/2021-07-31@13_00_52_5310.ffindex",format=vs.YUV411P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting color space from YUV411P8 to YUV444P8 for VsQTGMC
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited")
# setting field order to what QTGMC should assume (bottom field first)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=1)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Faster", TFF=False) # new fps: 59.94
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
# adjusting output color from: YUV444P8 to YUV422P8 for x264Model (i422@8)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV422P8, range_s="limited")
# set output frame rate to 59.940fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001)
# Output
clip.set_output()

also looks fine to me.


Cu Selur

Thank you so much for the quick response! I checked the preview and the log said "Failed to Initialize VapourSynth." Also, I updated to the latest version of Hybrid. I tried enabling the "Prefer Avisource for .avi input," however it still crashed.

I switched over to Avisynth and preferred Avisource for that, but I got the following error when I ran the preview:

Avisynth preview error: -> MDegrain1 : no valid OVERSCHROMA function for 16x16, bitsperpixel=8, lsb_flag=0, level=1 (C:\Program Files\Hybrid\64bit\Avisynth\avisynthPlugins\QTGMC.avsi, line 759) (C:\Users\Zack\AppData\Local\Temp\tempPreviewAvisynthFile15_16_55_602_tmp.avs, line 16)

I can also confirm that it is not one specific file that I'm having this issue with, as I have tried several similar files and have had the same results.

[Selur]

Please don't blindy quote my whole posts, it really makes it harder to read your post. :/

Avisynth preview error: -> MDegrain1 : no valid OVERSCHROMA function for 16x16, bitsperpixel=8, lsb_flag=0, level=1 (C:\Program Files\Hybrid\64bit\Avisynth\avisynthPlugins\QTGMC.avsi, line 759) (C:\Users\Zack\AppData\Local\Temp\tempPreviewAvisynthFile15_16_55_602_tmp.avs, line 16)

Okay, I can reproduce that problem is that QTGMC doesn't support 411 sampling.
Try:

• enable "Filtering->Custom->Insert before"
  
• set "Filtering->Custom->Insert before" to "Deinterlace"
  
• enable "Filtering->Custom->enable" and enter
  
  # adjust color to YV12 (color matrix: Rec601)
ConvertToYV12(ChromaOutPlacement="DV")
# colorformat YV12

that hopefully fixes the issue. (will adjust Avisynth QTGMC handle 411 content in the next release) -> send you a link to a dev version which does the adjustment automatially so no custom code for Avisynth is needed.

Cu Selur