2nd pass won't start..."file size too small"

2nd pass won't start..."file size too small" - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: 2nd pass won't start..."file size too small" (/thread-1209.html)

Pages: 1 2

2nd pass won't start..."file size too small" - Toothache - 28.11.2019

Hi,

When doing a 2 pass encode (x265), the first pass is done without any hitches, but the second pass doesn't start. I get an error that the filesize is too small. I'm attaching the logfile.

Using Hybrid 2019.11.23.1 for windows x64.

RE: 2nd pass won't start..."file size too small" - Selur - 28.11.2019

What happens if you call:

C:\PROGRA~1\Hybrid\64bit\ffmpeg.exe" -y -noautorotate -nostdin -threads 8 -i "D:\convert\result.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | "C:\PROGRA~1\Hybrid\64bit\x265.exe" --preset slow --log-level 2 --input - --output-depth 10 --y4m --profile main10 --ctu 32 --no-hme --merange 26 --no-rect --max-merge 2 --tskip --no-open-gop --opt-ref-list-length-pps --bframes 5 --rc-lookahead 40 --pass 2 --bitrate 4000 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --fast-intra --psy-rd 2.20 --psy-rdoq 1.20 --splitrd-skip --deblock=-2:-2 --selective-sao 2 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --stats "D:\convert\result_new_17_23_01_7710_01.stats" --no-dynamic-refine --refine-ctu-distortion 0 --output "D:\convert\17_23_01_7710_03.265"

inside a Windows command prompt?

Cu Selur

RE: 2nd pass won't start..."file size too small" - Toothache - 28.11.2019

^^

I get one warning and two errors.

Warning: generating non standard YUV stream. Mjpegtools will not work.

Errors:

av_interleaved_write_frame(): Invalid argument
Error writing trailer of pipe:: Invalid argument

RE: 2nd pass won't start..."file size too small" - Selur - 28.11.2019

Quote:Warning: generating non standard YUV stream. Mjpegtools will not work.

Is okay, and happens because '-pix_fmt yuv420p10le -strict -1' that one was expected.

Quote:av_interleaved_write_frame(): Invalid argument
Error writing trailer of pipe:: Invalid argument

Is a problem.

I guess the same error occurs with:

C:\PROGRA~1\Hybrid\64bit\ffmpeg.exe" -y -noautorotate -nostdin -threads 8 -i "D:\convert\result.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe NUL

Are you sure the input file is okay?

Cu Selur

RE: 2nd pass won't start..."file size too small" - Toothache - 28.11.2019

I can't be sure that the input file is OK, but:

1) The command you gave in the above post runs fine, but I don't see an output file anywhere.
2) Hybrid runs without errors and gives me an output file if I use CRF instead of 2-pass, with all other settings being exactly the same.

The input file is a lossless x264 with huge bitrate, if that matters. (11 GB for a 5 minute clip.)

RE: 2nd pass won't start..."file size too small" - Selur - 28.11.2019

Quote:The command you gave in the above post runs fine, but I don't see an output file anywhere

It should not create an output file. The call just tests the decoding.

If the decoding doesn't throw any error. I seems like the problem lies with x265.

Do you get any more details when using a higher log level in x265? (using --log-level 4 instead of --log-level 2)

C:\PROGRA~1\Hybrid\64bit\ffmpeg.exe" -y -noautorotate -nostdin -threads 8 -i "D:\convert\result.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | "C:\PROGRA~1\Hybrid\64bit\x265.exe" --preset slow --log-level 4 --input - --output-depth 10 --y4m --profile main10 --ctu 32 --no-hme --merange 26 --no-rect --max-merge 2 --tskip --no-open-gop --opt-ref-list-length-pps --bframes 5 --rc-lookahead 40 --pass 2 --bitrate 4000 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --fast-intra --psy-rd 2.20 --psy-rdoq 1.20 --splitrd-skip --deblock=-2:-2 --selective-sao 2 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --stats "D:\convert\result_new_17_23_01_7710_01.stats" --no-dynamic-refine --refine-ctu-distortion 0 --output "D:\convert\17_23_01_7710_03.265"

Also check your RAM usage.

RE: 2nd pass won't start..."file size too small" - Toothache - 28.11.2019

I can't test anymore today, I'll run it again tomorrow and let you know.

RE: 2nd pass won't start..."file size too small" - Selur - 28.11.2019

I tried your calls (adjusted paths) on my system:
using

i:\Hybrid\64bit\ffmpeg.exe -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "C:\Users\Selur\Desktop\Video ERROR.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | i:\Hybrid\64bit\x265.exe --preset slow --log-level 2 --input - --output-depth 10 --y4m --profile main10 --ctu 32 --no-hme --merange 26 --no-rect --max-merge 2 --tskip --no-open-gop --opt-ref-list-length-pps --bframes 5 --rc-lookahead 40 --pass 1 --slow-firstpass --bitrate 4000 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --fast-intra --psy-rd 2.20 --psy-rdoq 1.20 --splitrd-skip --deblock=-2:-2 --selective-sao 2 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --stats "E:\Temp\result_new.stats" --output "E:\Temp\result_new.265"

for the 1st pass and

i:\Hybrid\64bit\ffmpeg.exe -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "C:\Users\Selur\Desktop\Video ERROR.mp4" -map 0:0 -an -sn -vf  scale=out_range=pc -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | i:\Hybrid\64bit\x265.exe --preset slow --log-level 2 --input - --output-depth 10 --y4m --profile main10 --ctu 32 --no-hme --merange 26 --no-rect --max-merge 2 --tskip --no-open-gop --opt-ref-list-length-pps --bframes 5 --rc-lookahead 40 --pass 2 --bitrate 4000 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --fast-intra --psy-rd 2.20 --psy-rdoq 1.20 --splitrd-skip --deblock=-2:-2 --selective-sao 2 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --stats "E:\Temp\result_new.stats" --no-dynamic-refine --refine-ctu-distortion 0 --output "E:\Temp\result_new.265"

for the second pass and got and error:

y4m  [info]: 854x480 fps 24/1 i420p10 sar 428:427 unknown frame count

raw  [info]: output file: E:\Output\result_new.265

x265 [info]: HEVC encoder version 3.2+15-04db2bfee5d6

x265 [info]: build info [Windows][GCC 9.2.0][64 bit] 10bit

x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

x265 [info]: Main 10 profile, Level-3 (Main tier)

x265 [info]: Thread pool created using 16 threads

x265 [info]: Slices                              : 1

x265 [info]: frame threads / pool features       : 4 / wpp(15 rows)

x265 [warning]: Source height < 720p; disabling lookahead-slices

x265 [error]: different scenecut setting than first pass (40 vs 40)

x265 [error]: failed to open encoder

No clue why x265 complains about the scene cut settings.

Tried this with multiple files. No clue why this error message does not appear for you.
-> since I don't see an error with the call itself I'll ask the x265 devs over at doom9s (https://forum.doom9.org/showpost.php?p=1891617&postcount=7218).
I fear this is related to the 'limit-tu: Fix bug in loading co-located CU's TU depth' fixes the dev did on '2019-11-08'.

Cu Selur

RE: 2nd pass won't start..."file size too small" - Selur - 28.11.2019

Seems like 2pass is broken or I'm really overlooking something.
Even using a simple settings I end up with the 'different scenecut setting than first pass (40 vs 40)'-error.

I:\Hybrid\64bit>ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | x265 --preset medium --input - --output-depth 10 --y4m --profile main10 --pass 1 --slow-firstpass --bitrate 1500  --range limited --colormatrix bt470bg --stats "E:\Temp\test_19_40_50_2910_01.stats" --output "E:\Temp\test.265"

y4m  [info]: 640x352 fps 25/1 i420p10 sar 1:1 unknown frame count

raw  [info]: output file: E:\Temp\test.265

x265 [info]: HEVC encoder version 3.2+15-04db2bfee5d6

x265 [info]: build info [Windows][GCC 9.2.0][64 bit] 10bit

x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

x265 [info]: Main 10 profile, Level-2.1 (Main tier)

x265 [info]: Thread pool created using 16 threads

x265 [info]: Slices                              : 1

x265 [info]: frame threads / pool features       : 4 / wpp(6 rows)

x265 [warning]: Source height < 720p; disabling lookahead-slices

x265 [info]: Coding QT: max CU size, min CU size : 64 / 8

x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra

x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3

x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00

x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2

x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0

x265 [info]: References / ref-limit  cu / depth  : 3 / off / on

x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1

x265 [info]: Rate Control / qCompress            : ABR-1500 kbps / 0.60

x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip signhide tmvp b-intra

x265 [info]: tools: strong-intra-smoothing deblock sao stats-write

x265 [info]: frame I:      3, Avg QP:13.44  kb/s: 3999.20

x265 [info]: frame P:    119, Avg QP:10.20  kb/s: 3036.26

x265 [info]: frame B:    307, Avg QP:14.84  kb/s: 782.98

x265 [info]: Weighted P-Frames: Y:0.8% UV:0.8%

x265 [info]: consecutive B-frames: 2.5% 0.0% 58.2% 22.1% 17.2%

encoded 429 frames in 10.47s (40.97 fps), 1430.51 kb/s, Avg QP:13.54

I:\Hybrid\64bit>ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | x265 --preset medium --input - --output-depth 10 --y4m --profile main10 --pass 2 --bitrate 1500  --range limited --colormatrix bt470bg --stats "E:\Temp\test_19_40_50_2910_01.stats" --output "E:\Temp\19_40_50_2910_03.265"

y4m  [info]: 640x352 fps 25/1 i420p10 sar 1:1 unknown frame count

raw  [info]: output file: E:\Temp\19_40_50_2910_03.265

x265 [info]: HEVC encoder version 3.2+15-04db2bfee5d6

x265 [info]: build info [Windows][GCC 9.2.0][64 bit] 10bit

x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

x265 [info]: Main 10 profile, Level-2.1 (Main tier)

x265 [info]: Thread pool created using 16 threads

x265 [info]: Slices                              : 1

x265 [info]: frame threads / pool features       : 4 / wpp(6 rows)

x265 [warning]: Source height < 720p; disabling lookahead-slices

x265 [error]: different scenecut setting than first pass (40 vs 40)

x265 [error]: failed to open encoder

RE: 2nd pass won't start..."file size too small" - Selur - 28.11.2019

Enabling 'x265>Rate Control->Rate Distortion Optimization-> Scene cut aware quantization' seems to allow using 2pass encoding.
Seems to be a known bug in the latest x265 build, see: https://bitbucket.org/multicoreware/x265/issues/524/2-pass-encoding-failure-with-32-15

Cu Selur