[MAJOR BUG] Encoding finishes prematurely

[Lipomo]

Hi have an annoying bug for days now, where I run an encoding, and it randomly finished before it's supposed to, and like nothing happened.
Sometimes it finishes after 6min, sometimes 40 min, I even had an encoding that only lacked a few seconds, it seems really random.




I don't know what to do, when I run an encoding before going to bed, it's always a suspense whether the video will be complete or no  

Here are both logs

Report log

HybridDebugOutput

[Selur]

(calling it 'MAJOR BUG' just triggers me to not take you seriously, especially calling it a major bug while not using the latest dev)

Vapoursynth script:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# loading plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libvs_placebo.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/CTMF/CTMF.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/DCTFilter.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeblockFilter/Deblock/Deblock.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import havsfunc
import validate
# Source: 'C:\Users\Admin\Desktop\FILM 1 - ETAPE 3 - HUE FIX DAVINCI00000000.mkv'
# Current color space: YUV422P8, bit depth: 8, resolution: 1920x1080, frame rate: 23.976fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709, transfer: bt.709, primaries: bt.709
# Loading C:\Users\Admin\Desktop\FILM 1 - ETAPE 3 - HUE FIX DAVINCI00000000.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Admin/Desktop/FILM 1 - ETAPE 3 - HUE FIX DAVINCI00000000.mkv", format="YUV422P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 709.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)
# setting color transfer (vs.TRANSFER_BT709), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709)
# setting color primaries info (to vs.PRIMARIES_BT709), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
clip = core.std.CropRel(clip=clip, left=240, right=240, top=0, bottom=0)# cropping to 1440x1080
# denoising using MCTemporalDenoise
clip = havsfunc.MCTemporalDenoise(i=clip, settings="high", thSAD=600, thSAD2=600, thSCD1=600, thSCD2=130, truemotion=False, MVglobal=True, pel=4, pelsearch=5, search=5, searchparam=5, MVsharp=2, DCT=0, ncpu=1)
# applying FineDeHalo to remove halos
clip = havsfunc.FineDehalo(clip, rx=3.50, ry=2.80, darkstr=0.00, edgeproc=1.00)
# adjusting color space from YUV422P8 to YUV444P16 for vsGLSLDarken
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, range_s="limited")
# Using Anime4k Darken GLSL filter for line darkening
with open("C:/Program Files/Hybrid/64bit/vsfilters/GLSL/parameterized/Anime4K_Darken_HQ.glsl") as glslf:
  glsl = glslf.read()
glsl = glsl.replace('#define STRENGTH 1.5', '#define STRENGTH 0.3')
clip = core.placebo.Shader(clip=clip, shader_s=glsl, width=clip.width, height=clip.height)
# adjusting output color from: YUV444P16 to YUV420P8 for FFV1Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited", dither_type="none")
# set output frame rate to 23.976fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# output
clip.set_output()

looks fine, also includes no filter that should change the frame count.
=> Does the Vapoursynth Preview show all the frames?
If not, you could try another source filter. (Bestsource or FFmpegSource2; BestSource is the default decoder in newer dev versions)

Hybrid expected 72647@23.976, but the encoding stops at 14367 frames without any warnings or erros.

My guess is that:
a. either the source filter has a problem with the source, then the Vapoursynth preview should show only a fraction of the frames
b. some energy saving function of Windows or some antivirus software triggers ffmpeg to stop processing.

=> don't see any bug in the processing of Hybrid.

Cu Selur

[Lipomo]

(calling it 'MAJOR BUG' just triggers me to not take you seriously)

Vapoursynth script:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# loading plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libvs_placebo.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/CTMF/CTMF.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/DCTFilter.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeblockFilter/Deblock/Deblock.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import havsfunc
import validate
# Source: 'C:\Users\Admin\Desktop\FILM 1 - ETAPE 3 - HUE FIX DAVINCI00000000.mkv'
# Current color space: YUV422P8, bit depth: 8, resolution: 1920x1080, frame rate: 23.976fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709, transfer: bt.709, primaries: bt.709
# Loading C:\Users\Admin\Desktop\FILM 1 - ETAPE 3 - HUE FIX DAVINCI00000000.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Admin/Desktop/FILM 1 - ETAPE 3 - HUE FIX DAVINCI00000000.mkv", format="YUV422P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 709.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)
# setting color transfer (vs.TRANSFER_BT709), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709)
# setting color primaries info (to vs.PRIMARIES_BT709), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
clip = core.std.CropRel(clip=clip, left=240, right=240, top=0, bottom=0)# cropping to 1440x1080
# denoising using MCTemporalDenoise
clip = havsfunc.MCTemporalDenoise(i=clip, settings="high", thSAD=600, thSAD2=600, thSCD1=600, thSCD2=130, truemotion=False, MVglobal=True, pel=4, pelsearch=5, search=5, searchparam=5, MVsharp=2, DCT=0, ncpu=1)
# applying FineDeHalo to remove halos
clip = havsfunc.FineDehalo(clip, rx=3.50, ry=2.80, darkstr=0.00, edgeproc=1.00)
# adjusting color space from YUV422P8 to YUV444P16 for vsGLSLDarken
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, range_s="limited")
# Using Anime4k Darken GLSL filter for line darkening
with open("C:/Program Files/Hybrid/64bit/vsfilters/GLSL/parameterized/Anime4K_Darken_HQ.glsl") as glslf:
  glsl = glslf.read()
glsl = glsl.replace('#define STRENGTH 1.5', '#define STRENGTH 0.3')
clip = core.placebo.Shader(clip=clip, shader_s=glsl, width=clip.width, height=clip.height)
# adjusting output color from: YUV444P16 to YUV420P8 for FFV1Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited", dither_type="none")
# set output frame rate to 23.976fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# output
clip.set_output()

looks fine, also includes no filter that should change the frame count.
=> Does the Vapoursynth Preview show all the frames?
If not, you could try another source filter. (Bestsource or FFmpegSource2)

Hybrid expected 72647@23.976, but the encoding stops at 14367 frames without any warnings or erros.

My guess is that:
a. either the source filter has a problem with the source, then the Vapoursynth preview should show only a fraction of the frames
b. some energy saving function of Windows or some antivirus software triggers ffmpeg to stop processing.

=> don't see any bug in the processing of Hybrid.

Cu Selur

Sorry then, but to me when you encode a video and it's the lottery whether it will encode or not, it's quite a major bug.


a. Yes the frame are good

 

b. Hmm that's not impossible. i didn't know that an antivirus could kill ffmpeg. 

OK so what I'll do, is fresh reinstall Hybrid to the last dev version and encode with AVAST disabled.

But I while ago I put "C:\Program Files\Hybrid\Hybrid.exe" and "C:\Program Files\Hybrid\" so I guess ffmpeg.exe run from Hybrid folder should be ignored from AVAST.

[Selur]

Also check your energy saving settings.
Since the Vapoursynth Preview shows all the frames, something must be messing with the encoding call:

vspipe "C:\Users\Admin\AppData\Roaming\hybrid\encodingTempSynthSkript_2024-09-05@08_25_00_6510_0.vpy" - -c y4m |  ffmpeg -y -noautorotate -nostdin -threads 8 -f yuv4mpegpipe -i - -an -sn -color_primaries bt709 -color_trc bt709 -colorspace bt709 -color_range tv -pix_fmt yuv420p -vsync 0 -vcodec ffv1 -coder 0 -context 0 -g 1 -level 3 -slices 4 -slicecrc 0 -metadata encoding_tool="Hybrid 2024.08.18.1" "C:\Users\Admin\AppData\Roaming\hybrid\FILM 1 - ETAPE 3 - HUE FIX DAVINCI00000000_new_1_2024-09-05@08_25_00_6510_02.mkv"

As a side note, I would also advise putting the temp folder (not in the AppData folder) to a separate dedicated folder and exclude it from the virus scan.

Cu Selur

[Lipomo]

Also check your energy saving settings.
Since the Vapoursynth Preview shows all the frames, something must be messing with the encoding call:

vspipe "C:\Users\Admin\AppData\Roaming\hybrid\encodingTempSynthSkript_2024-09-05@08_25_00_6510_0.vpy" - -c y4m |  ffmpeg -y -noautorotate -nostdin -threads 8 -f yuv4mpegpipe -i - -an -sn -color_primaries bt709 -color_trc bt709 -colorspace bt709 -color_range tv -pix_fmt yuv420p -vsync 0 -vcodec ffv1 -coder 0 -context 0 -g 1 -level 3 -slices 4 -slicecrc 0 -metadata encoding_tool="Hybrid 2024.08.18.1" "C:\Users\Admin\AppData\Roaming\hybrid\FILM 1 - ETAPE 3 - HUE FIX DAVINCI00000000_new_1_2024-09-05@08_25_00_6510_02.mkv"

As a side note, I would also advise putting the temp folder (not in the AppData folder) to a separate dedicated folder and exclude it from the virus scan.

Cu Selur

OK  I'll try :

- Encoding without AVAST for a while
- If there's no issue, I'll try with AVAST again, but following your advice about moving the temp folder and excluding it from AVAST scan
- If there's still no issue I'll try with temp folder not excluded like default and see if the issue comes back

I guess it'll take some time, but I'll come back to you when I'll figure this out.

Thank you

[Selur]

Fingers crossed
btw. depending on what your source format it, using gpu decoding might speed things up.

[Lipomo]

Fingers crossed
btw. depending on what your source format it, using gpu decoding might speed things up.

I would really like that since I encoded at about 3 fps with filters..
So you mean gpu encoding isn't enabled by default ? How can I enable it and would it work with FFV1 source ?

[Selur]

1. Neither gpu encoding nor decoding is enabled by default.
2. afaik there is no decoder or encoder chip out which supports FFV1
3. so if your source is FFV1 then gpu decoding won't work. (since your target is FFV1 gpu encoding also is not an option)

Cu Selur

[Lipomo]

1. Neither gpu encoding nor decoding is enabled by default.
2. afaik there is no decoder or encoder chip out which supports FFV1
3. so if your source is FFV1 then gpu decoding won't work. (since your target is FFV1 gpu encoding also is not an option)

Cu Selur

Got it.