x264 encoding error.

[hjatsu]

Ah, sorry for too broad of a test case. And too bad they don't crash on your end, definitely something funky about my hardware or Windows then. Maybe tomorrow I can try the jobs on a completely different machine, or on this same machine using the Linux version..

After trying those test jobs a few times, this was my best result when I tried running them all in parallel instead of sequentially. Sequentially they usually all failed: 

=> Does the Vapoursynth Preview work fine?

Yes, I have not come across any issues there. Scrolling and even playback work just fine through all parts of the source. Checked the debug log too and only thing resembling an error was this and it appears before any scrolling has been done in the window:

2024.08.18 - 20:51:57_Windows 11 Version 23H2 (64bit)_2024.08.14.1 - level 9: Preview::indexingIsFinished 'VAPOURSYNTHPREVIEW' ignored,..
2024.08.18 - 20:51:59_Windows 11 Version 23H2 (64bit)_2024.08.14.1 - level 9: Got unhandled IPC message:  Q t  w a r n i n g :  Q P i x m a p : : s c a l e d :  P i x m a p  i s  a  n u l l  p i x m a p

(horribly slow due to QTGMC(Input=clip, Preset="Very Slow"), but it worked fine. No clue why you would use 'Very Slow' on such a source)

Still getting to know QTGMC I quess, just thought "Very Slow" would give the best quality deint result. (Or if you mean wasting all that processing to such a TV program I understand, but this was just a sample I could share publicly, as other are family videos). 

QTGMC was what led me to Hybrid really, as my initial plan was to simply ffmpeg the VOBs using its simple deint filters but unfortunately I learned there are more advanced ways...and down the rabbithole I am now!

Out of curiosity (I doubt this has anything to do with the crash):
What settings do you enable aside from:

• "Crop/Resize->Base->Picture Crop" (left=12, right=12, top=2, bottom=10)
  
• "Filtering->(De-)Interlace/Telecine->QTGMC Vapoursynth->Preset->Very Slow"
  
• "Filtering->Vapoursynth->DeNoise->HQDN3D"
  "Spatial luma strength" 2.00
  "Temporal luma strength" 4.00
  "Spatial chromastrength" 2.00
  "Temporal chroma strength" 6.00
  

That's about it really. For a couple clips I have used Color > Tweak too. In x264 encoding settings I've limited the thread count (avoiding heat issues) and extended the lookaheads to 120 frames. (Attempted to use EZDenoise, but after some tests found that HQDN3D looked the best.)

are you using for Hybrid to think the output should be 720x576?
For me the:

# fallback - adjusting from 696x576 to output resolution 720x576
clip = core.resize.Spline64(clip=clip, width=720, height=576) # resolution 720x576)

For some reason Hybrid on your end does seem to think the output should be 720x576, but I don't see why.

Ah, that is one thing I forgot in these test jobs: I noticed that after I add a source to Hybrid, I always need to go and touch the Crop setting (just click any of the arrows) even though they are remembered there from previous job, or otherwise the x264 tab (and I guess now Vapoursynt too) has the source resolution instead of the correct, cropped resolution. So in all of my actual jobs I have not had that fallback/resize line, just checked a previous job file.

Ps.: side note about the source filtering: ChromaShiftSP with x=2 and SpotLess with temporal radius 3 and some deblocking and dehaloing should be applied.
Ps.: Really try to disable / uninstall that antivirus software.

Thanks! Will for sure test those filters out too.
And AV software has been disabled for all my tests lately.


But yeah, I guess I'll continue testing on a totally different platform instead of this machine to try and isolate the issue better. Will keep this thread updated. Thanks again for all the effort, really sorry to keep bothering you! 

[hjatsu]

Good news, sort of.. 

I did not have time to test on another hardware yet but I did some more tests with the current one, based on the hypothesis of crash happening at around the x264 lookahead value, so when input material is ending. Here are my results, all repeatable at least 3/3 times:

--sync-lookahead 200 --rc-lookahead 200

Crash at 456/858 (402 frames left) --> About double of the lookahead values. Note bobbed deint.

--sync-lookahead 10 --rc-lookahead 10

Crash at 814/858 (44 frames left) --> Noticeably smaller than the previous setting.

--sync-lookahead 10 --rc-lookahead 200

Crash at 646/858 frames (212 frames left). --> Again mirroring the order of magnitude change in the lookahead values.

--sync-lookahead 200 --rc-lookahead 10

Crash at 624/858 frames (234 frames left). --> Very similar to the above.

Example of all the encode arguments:

tool: C:\Program Files\Hybrid\64bit\x264.exe
argumentString(1): --preset veryslow --crf 14.50 --profile high --level 5.1 --sync-lookahead 10 --rc-lookahead 200 --no-fast-pskip --vbv-maxrate 300000 --vbv-bufsize 300000 --deadzone-inter 6 --deadzone-intra 6 --threads 4 --sar 16:15 --non-deterministic --range tv --colormatrix bt470bg --demuxer y4m --input-range tv --fps 50/1 --output-depth 8 --output "C:\Users\hjatsu\Desktop\vhs\temp\2024-08-18@10_49_41_8510_06.264" -

Then I proceed to test if OpenCL is to be blamed, so tried this:

# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Very Slow", TFF=True) # new fps: 50

...instead of this:

# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Very Slow", TFF=True, opencl=True, device=1) # new fps: 50

And the result was that the test jobs FINISHED successfully 6/6. So using CPU for everything seems to be the trick?!


Decided to also test with the iGPU, so modified the OpenCL jobs a little by pointing the device param to my Intel Iris Xe instead of Nvidia RTX 3500:

# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Very Slow", TFF=True, opencl=True, device=0) # new fps: 50

And the result was exactly the same as with GPU, crashes happening at exactly the same frames.


So my take from this is that the issue must be either:
- Vapoursynth/QTGMC OpenCL implementation
- GPU drivers (unlikely because iGPU crashes too)
- Something in my system interfering with all GPU operations (no idea if even possible and what this could be)

This finding does not fully explain the crashes I observed with the initial jobs, when I was using the latest stable version that was unable to recognize my RTX 3500. But based on this test session, my only option is to just use CPU for everything I guess...?

[Selur]

When using QTGMC, using openCL only triggers Hybrid to use NNEDI3CL
https://github.com/dubhater/vapoursynth-nnedi3
instead of normal nnedi3. My guess is something with the opencl drivers is amiss.
Try using the NVIDIA studio drivers for your gpu, if the latest do not work also try older versions.

Cu Selur

[hjatsu]

Can confirm the test jobs run 6/6 without any hiccups in OpenCL mode in another machine (Win10, Intel Core i9-9900K & NVIDIA Quadro RTX 4000, same AV software but not disabled). Will continue to experiment with the drivers then. Thanks!