VapourSynth crashing on large files - Printable Version +- Selur's Little Message Board (https://forum.selur.net) +-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html) +--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html) +--- Thread: VapourSynth crashing on large files (/thread-4266.html)

VapourSynth crashing on large files - OhNoxius - 15.11.2025 Hi there, I'm scratching my head trying to get Vapoursynth working on some VHS captures I did (FFV1 compressed, 4:2:2 8-bit) and encoding with x264. Basically small snippets work, even with Hybrid's *experimental frame cut function. But when I try to encode the whole file, the process crashes, sometimes almost instantly, sometimes after several minutes of encoding. I somehow managed to get one 30min video through on the previous 2025.10.15 development build (juggling with deinterlacing settings, adding or removing various Vapoursynth filters (notably CCD), enabling Bestsource (Cuda), adding audio or not, framecut etc), but now I'm again stuck on another long video, while smaller segments do get encoded. p.s. I attached a debug file, and I'm not sure what's exactly recorded in there, but while debugging was enabled there was a crash of a large file while afterwards a small segment (cut out with VirtualDub from the same video) did get encoded. RE: VapourSynth crashing on large files - Selur - 15.11.2025 The used Vapoursynth script: # Imports import vapoursynth as vs # getting Vapoursynth core import ctypes import sys import os core = vs.core # Import scripts folder scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) # Loading Support Files Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll") # loading plugins core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SharpenFilter/AWarpSharp2/libawarpsharp2.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/Cnr2/libcnr2.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/fft3dfilter.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/FluxSmooth/libfluxsmooth.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeCrawlFilter/DeDot/libdedot.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")# vsQTGMC core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libllvmexpr.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/ZSmooth/zsmooth.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DerainbowFilter/DeCross/libdecross.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/BestSource/BestSource.dll") # Import scripts import dehalo import color import artifacts import qtgmc import validate # Source: 'D:\FABIO157-elvinRavenna[PANA-TBC3000-ATI600].avi' # Current color space: YUV422P8, bit depth: 8, resolution: 720x576, frame rate: 25fps, scanorder: top field first, yuv luminance scale: limited, matrix: 470bg, format: FFV1 # Loading D:\FABIO157-elvinRavenna[PANA-TBC3000-ATI600].avi using BestSource) clip = core.bs.VideoSource(source="D:/FABIO157-elvinRavenna[PANA-TBC3000-ATI600].avi", cachepath="D:/tempHybrid/FABIO157-elvinRavenna[PANA-TBC3000-ATI600]_bestSource_2025-11-15@17_48_15_4910", track=0, hwdevice="cuda") frame = clip.get_frame(0) # setting color matrix to 470bg. clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG) # setting color transfer (vs.TRANSFER_BT601), if it is not set. if validate.transferIsInvalid(clip):   clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601) # setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set. if validate.primariesIsInvalid(clip):   clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG) # setting color range to TV (limited) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED) # making sure frame rate is set to 25fps clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # making sure the detected scan type is set (detected: top field first) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_TOP) # tff # rainbow removal using DeCross clip = core.decross.DeCross(clip) # Deinterlacing using QTGMC clip = qtgmc.QTGMC(clip, Preset="Fast", TFF=True)# new fps: 50 # Making sure content is preceived as frame based clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive clip = core.dedot.Dedot(clip) # adjusting color using Tweak clip = color.Tweak(clip, hue=-0.02, sat=0.99, cont=0.99, bright=4, coring=False) clip = core.std.Crop(clip, left=18, right=2, top=0, bottom=10)# cropping to 700x566 # denoising using FluxSmooth clip = core.flux.SmoothT(clip) # denoising using FFT3DFilter clip = core.fft3dfilter.FFT3DFilter(clip) # chroma denoising using VsCnr2 clip = core.cnr2.Cnr2(clip, mode="oxx") # adjusting color space from YUV422P8 to RGB24 for vsCCD clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_in_s="limited", range_s="full") # chroma denoising using CCD clip = core.zsmooth.CCD(clip, threshold=10.00, temporal_radius=1, points=[True,True,False]) # adjusting color space from RGB24 to YUV444P16 for vsEdgeCleaner clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, matrix_s="470bg", range_in_s="full", range_s="limited") # additional resize to allow target color sampling # Dehalo using EdgeCleaner clip = dehalo.EdgeCleaner(clip) # letterboxing 700x566 to 720x576 clip = core.std.AddBorders(clip, left=10, right=10, top=5, bottom=5) # adjusting output color from: YUV444P16 to YUV422P8 for x264Model clip = core.resize.Bicubic(clip=clip, format=vs.YUV422P8, dither_type="error_diffusion") # set output frame rate to 50fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=50, fpsden=1) # output clip.set_output() seems fine, encoding does start, but crashes after 197 frames: 2025.11.15 - 17:50:09_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: 2025-11-15@17_48_15_4910_02_video ### D:\FABIO157-elvinRavenna[PANA-TBC3000-ATI600]_new_new.mkv ### x264 [info]: using SAR=16/15 2025.11.15 - 17:50:09_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: 2025-11-15@17_48_15_4910_02_video ### D:\FABIO157-elvinRavenna[PANA-TBC3000-ATI600]_new_new.mkv ### x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 2025.11.15 - 17:50:09_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: 2025-11-15@17_48_15_4910_02_video ### D:\FABIO157-elvinRavenna[PANA-TBC3000-ATI600]_new_new.mkv ### x264 [info]: profile High 4:2:2, level 5.1, 4:2:2, 8-bit 2025.11.15 - 17:50:10_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: x264 output: 6 frames: 1.82 fps, 7419.80 kb/s 2025.11.15 - 17:50:10_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: x264 output: 11 frames: 3.07 fps, 6009.38 kb/s 2025.11.15 - 17:50:10_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: x264 output: 16 frames: 4.15 fps, 5575.98 kb/s 2025.11.15 - 17:50:10_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: x264 output: 21 frames: 5.09 fps, 5362.55 kb/s 2025.11.15 - 17:50:11_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: x264 output: 26 frames: 5.92 fps, 5221.49 kb/s 2025.11.15 - 17:50:11_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: x264 output: 31 frames: 6.66 fps, 4992.08 kb/s 2025.11.15 - 17:50:11_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: x264 output: 36 frames: 7.33 fps, 4702.92 kb/s 2025.11.15 - 17:50:20_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: x264 output: 197 frames: 14.33 fps, 4092.44 kb/s 2025.11.15 - 17:50:20_Windows 10 Version 22H2 (64bit)_2025.11.09.1 - level 9: 2025-11-15@17_48_15_4910_02_video ### D:\FABIO157-elvinRavenna[PANA-TBC3000-ATI600]_new_new.mkv ### Helper(1) finished with exitCode: -1073741819 and exitStatus: Could be a memory or cpu or heat problem? Is your system over- or underclocked? Are you running out of space? Could be some other software interfering or a driver problem. Since FFV1 is not supported by cuda, try whether not using hwdevice="cuda" or using another source filter makes a difference. Cu Selur RE: VapourSynth crashing on large files - OhNoxius - 17.11.2025 Thanks for the suggestions. I'm working on a i7-4771 // 8GB DDR3 // GTX960 2GB, nothing overclocked, C: has 34GB and D: has 50GB free space. Setting bestsource=cpu didn't help, nor using FFMpegSource2 or LWLibAVSource. I remember setting bestsource=cuda because that somehow caused some succesfull encodes (although that shouldn't make any sense...) Maybe this is relevant: I remember cropping the previous 30min beforehand and applying the CCD filter in VirtualDub2, before applying VapourSynth filters and encoding with Hybrid (cropping & letterboxing disabled). Went to an i5-9600KF // 16GB // GTX1070ti 8GB and tried encoding the same video with same VapourSynth filters on. Crashed again, almost instantly. Then set bestsource=cuda & disabled VapourSynth CCD, did a VapourSynth preview (compare view) do be sure the VapourSynth was working, and managed to get it encoded untill around 50%... then crash. Then set bestsource=cpu and x264 profile=High (instead of High4:2:2 with 8-bit precision), and got it to 100%! -> enabled audio encoding to FLAC: success -> enabled CCD: crash -> disabled CCD, enabled cut support (frame selection): crash Went back to my other machine and tried to repeat the same actions... and got to a 100%! (no audio or CCD enabled. Tried enabling cut support and encoding did start but crashed underway) All this actually feels so random... I could swear I had tried the exact same settings on my i7 before, and now it suddenly works? Is there any way to know where in the process the crash occurs? Is it a x264 thing, or is Vapoursynth crashing? Thanks Karel p.s.  I have another i9-13900H laptop (with no dedicated graphics card) I can test on... RE: VapourSynth crashing on large files - Selur - 18.11.2025 Based in the debug output the Decoder crashes. So it is probably a filter, the source or something in the system and not the encoder.