2nd pass won't start..."file size too small"

[Toothache]

Hi,

When doing a 2 pass encode (x265), the first pass is done without any hitches, but the second pass doesn't start. I get an error that the filesize is too small. I'm attaching the logfile.

Using Hybrid 2019.11.23.1 for windows x64.

[Selur]

What happens if you call:

C:\PROGRA~1\Hybrid\64bit\ffmpeg.exe" -y -noautorotate -nostdin -threads 8 -i "D:\convert\result.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | "C:\PROGRA~1\Hybrid\64bit\x265.exe" --preset slow --log-level 2 --input - --output-depth 10 --y4m --profile main10 --ctu 32 --no-hme --merange 26 --no-rect --max-merge 2 --tskip --no-open-gop --opt-ref-list-length-pps --bframes 5 --rc-lookahead 40 --pass 2 --bitrate 4000 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --fast-intra --psy-rd 2.20 --psy-rdoq 1.20 --splitrd-skip --deblock=-2:-2 --selective-sao 2 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --stats "D:\convert\result_new_17_23_01_7710_01.stats" --no-dynamic-refine --refine-ctu-distortion 0 --output "D:\convert\17_23_01_7710_03.265"

inside a Windows command prompt?

Cu Selur

[Toothache]

^^

I get one warning and two errors.

Warning: generating non standard YUV stream. Mjpegtools will not work.


Errors:

av_interleaved_write_frame(): Invalid argument
Error writing trailer of pipe:: Invalid argument

[Selur]

Warning: generating non standard YUV stream. Mjpegtools will not work.

Is okay, and happens because '-pix_fmt yuv420p10le -strict -1' that one was expected.

av_interleaved_write_frame(): Invalid argument
Error writing trailer of pipe:: Invalid argument

Is a problem.

I guess the same error occurs with:

C:\PROGRA~1\Hybrid\64bit\ffmpeg.exe" -y -noautorotate -nostdin -threads 8 -i "D:\convert\result.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe NUL

Are you sure the input file is okay?


Cu Selur

[Toothache]

I can't be sure that the input file is OK, but:

1) The command you gave in the above post runs fine, but I don't see an output file anywhere.
2) Hybrid runs without errors and gives me an output file if I use CRF instead of 2-pass, with all other settings being exactly the same.

The input file is a lossless x264 with huge bitrate, if that matters. (11 GB for a 5 minute clip.)

[Selur]

The command you gave in the above post runs fine, but I don't see an output file anywhere

It should not create an output file. The call just tests the decoding.

If the decoding doesn't throw any error. I seems like the problem lies with x265.

Do you get any more details when using a higher log level in x265? (using --log-level 4 instead of --log-level 2)

C:\PROGRA~1\Hybrid\64bit\ffmpeg.exe" -y -noautorotate -nostdin -threads 8 -i "D:\convert\result.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | "C:\PROGRA~1\Hybrid\64bit\x265.exe" --preset slow --log-level 4 --input - --output-depth 10 --y4m --profile main10 --ctu 32 --no-hme --merange 26 --no-rect --max-merge 2 --tskip --no-open-gop --opt-ref-list-length-pps --bframes 5 --rc-lookahead 40 --pass 2 --bitrate 4000 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --fast-intra --psy-rd 2.20 --psy-rdoq 1.20 --splitrd-skip --deblock=-2:-2 --selective-sao 2 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --stats "D:\convert\result_new_17_23_01_7710_01.stats" --no-dynamic-refine --refine-ctu-distortion 0 --output "D:\convert\17_23_01_7710_03.265"

Also check your RAM usage.

[Toothache]

I can't test anymore today, I'll run it again tomorrow and let you know.

[Selur]

I tried your calls (adjusted paths) on my system:
using

i:\Hybrid\64bit\ffmpeg.exe -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "C:\Users\Selur\Desktop\Video ERROR.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | i:\Hybrid\64bit\x265.exe --preset slow --log-level 2 --input - --output-depth 10 --y4m --profile main10 --ctu 32 --no-hme --merange 26 --no-rect --max-merge 2 --tskip --no-open-gop --opt-ref-list-length-pps --bframes 5 --rc-lookahead 40 --pass 1 --slow-firstpass --bitrate 4000 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --fast-intra --psy-rd 2.20 --psy-rdoq 1.20 --splitrd-skip --deblock=-2:-2 --selective-sao 2 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --stats "E:\Temp\result_new.stats" --output "E:\Temp\result_new.265"

for the 1st pass and

i:\Hybrid\64bit\ffmpeg.exe -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "C:\Users\Selur\Desktop\Video ERROR.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | i:\Hybrid\64bit\x265.exe --preset slow --log-level 2 --input - --output-depth 10 --y4m --profile main10 --ctu 32 --no-hme --merange 26 --no-rect --max-merge 2 --tskip --no-open-gop --opt-ref-list-length-pps --bframes 5 --rc-lookahead 40 --pass 2 --bitrate 4000 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --fast-intra --psy-rd 2.20 --psy-rdoq 1.20 --splitrd-skip --deblock=-2:-2 --selective-sao 2 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --stats "E:\Temp\result_new.stats" --no-dynamic-refine --refine-ctu-distortion 0 --output "E:\Temp\result_new.265"

for the second pass and got and error:

y4m  [info]: 854x480 fps 24/1 i420p10 sar 428:427 unknown frame count
raw  [info]: output file: E:\Output\result_new.265
x265 [info]: HEVC encoder version 3.2+15-04db2bfee5d6
x265 [info]: build info [Windows][GCC 9.2.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main 10 profile, Level-3 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 4 / wpp(15 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [error]: different scenecut setting than first pass (40 vs 40)
x265 [error]: failed to open encoder

No clue why x265 complains about the scene cut settings.

Tried this with multiple files. No clue why this error message does not appear for you.
-> since I don't see an error with the call itself I'll ask the x265 devs over at doom9s (https://forum.doom9.org/showpost.php?p=1...count=7218).
I fear this is related to the 'limit-tu: Fix bug in loading co-located CU's TU depth' fixes the dev did on '2019-11-08'.

Cu Selur

[Selur]

Seems like 2pass is broken or I'm really overlooking something.
Even using a simple settings I end up with the 'different scenecut setting than first pass (40 vs 40)'-error.

I:\Hybrid\64bit>ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | x265 --preset medium --input - --output-depth 10 --y4m --profile main10 --pass 1 --slow-firstpass --bitrate 1500  --range limited --colormatrix bt470bg --stats "E:\Temp\test_19_40_50_2910_01.stats" --output "E:\Temp\test.265"
y4m  [info]: 640x352 fps 25/1 i420p10 sar 1:1 unknown frame count
raw  [info]: output file: E:\Temp\test.265
x265 [info]: HEVC encoder version 3.2+15-04db2bfee5d6
x265 [info]: build info [Windows][GCC 9.2.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main 10 profile, Level-2.1 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 4 / wpp(6 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : ABR-1500 kbps / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing deblock sao stats-write
x265 [info]: frame I:      3, Avg QP:13.44  kb/s: 3999.20
x265 [info]: frame P:    119, Avg QP:10.20  kb/s: 3036.26
x265 [info]: frame B:    307, Avg QP:14.84  kb/s: 782.98
x265 [info]: Weighted P-Frames: Y:0.8% UV:0.8%
x265 [info]: consecutive B-frames: 2.5% 0.0% 58.2% 22.1% 17.2%

encoded 429 frames in 10.47s (40.97 fps), 1430.51 kb/s, Avg QP:13.54

I:\Hybrid\64bit>ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | x265 --preset medium --input - --output-depth 10 --y4m --profile main10 --pass 2 --bitrate 1500  --range limited --colormatrix bt470bg --stats "E:\Temp\test_19_40_50_2910_01.stats" --output "E:\Temp\19_40_50_2910_03.265"
y4m  [info]: 640x352 fps 25/1 i420p10 sar 1:1 unknown frame count
raw  [info]: output file: E:\Temp\19_40_50_2910_03.265
x265 [info]: HEVC encoder version 3.2+15-04db2bfee5d6
x265 [info]: build info [Windows][GCC 9.2.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main 10 profile, Level-2.1 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 4 / wpp(6 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [error]: different scenecut setting than first pass (40 vs 40)
x265 [error]: failed to open encoder

[Selur]

Enabling 'x265>Rate Control->Rate Distortion Optimization-> Scene cut aware quantization' seems to allow using 2pass encoding.
Seems to be a known bug in the latest x265 build, see: https://bitbucket.org/multicoreware/x265...with-32-15

Cu Selur