Posts: 53
Threads: 8
Joined: Nov 2020
Hello, I noticed another problem with AV1 encoding, this time with SVT-AV1
- What Hybrid version did you use? Windows 10 21H2 Hybrid dev 2022.04.09.1
- What did happen? If an error message came up what did it exactly say? In the "Jobs" tab it is written Progress 50% State Finished (without output file created)
- What did you do to get to the bug? Just load the video input, set to use SVT-AV1 and load a saved configuration (but same problem with default one) then start encoding
- Debug output attached
Posts: 10.551
Threads: 57
Joined: May 2017
Will look at it in a few hours after work.
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Posts: 53
Threads: 8
Joined: Nov 2020
Thanks. 
Take your time, there is no hurry.
Just the fact that you are always available to fix bugs and listen to requests on the forum means a lot, hard to find elsewhere.
Posts: 10.551
Threads: 57
Joined: May 2017
Okay, totally missed that the SVT developers changed a bunch of parameters.
-> this will take a while and SVT-HEVC is probably broken too.
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Posts: 10.551
Threads: 57
Joined: May 2017
14.04.2022, 16:54
(This post was last modified: 14.04.2022, 17:38 by Selur.)
Okay, since tons of the parameters changed I will probably have to rewrite the whole thing or I drop support for it.
Options:
--help Shows the command line options currently available
--version Shows the version of the library that's linked to the library
-i, --input Input raw video (y4m and yuv) file path, use `stdin` to read from pipe
-b, --output Output compressed (ivf) file path, use `stdout` to write to pipe
-c, --config Configuration file path
--errlog Error file path, defaults to stderr
-o, --recon Reconstructed yuv file path
--stat-file PSNR / SSIM per picture stat output file path, requires `--enable-stat-report 1`
--pred-struct-file Manual prediction structure file path
--progress Verbosity of the output, default is 1 [0: no progress is printed, 2: aomenc style output]
--no-progress Do not print out progress, default is 0 [1: `--progress 0`, 0: `--progress 1`]
--preset Encoder preset, presets < 0 are for debugging. Higher presets means faster encodes, but with a quality tradeoff, default is 10 [-2-13]
--svtav1-params colon separated list of key=value pairs of parameters with keys based on config file options
Encoder Global Options:
-w, --width Frame width in pixels, inferred if y4m, default is 0 [64-16384]
-h, --height Frame height in pixels, inferred if y4m, default is 0 [64-8704]
--forced-max-frame-width Maximum frame width value to force, default is 0 [64-16384]
--forced-max-frame-height Maximum frame height value to force, default is 0 [64-8704]
-n, --frames Number of frames to encode. If `n` is larger than the input, the encoder will loop back and continue encoding, default is 0 [0: until EOF, 1-`(2^63)-1`]
--nb Buffer `n` input frames into memory and use them to encode, default is -1 [-1: no frames buffered, 1-`(2^31)-1`]
--color-format Color format, only yuv420 is supported at this time, default is 1 [0: yuv400, 1: yuv420, 2: yuv422, 3: yuv444]
--profile Bitstream profile, default is 0 [0: main, 1: high, 2: professional]
--level Bitstream level, defined in A.3 of the av1 spec, default is 0 [0: autodetect from input, 2.0-7.3]
--enable-hdr Enable writing of HDR metadata in the bitstream, default is 0 [0-1]
--fps Input video frame rate, integer values only, inferred if y4m, default is 60 [1-240]
--fps-num Input video frame rate numerator, default is 60000 [0-2^32-1]
--fps-denom Input video frame rate denominator, default is 1000 [0-2^32-1]
--input-depth Input video file and output bitstream bit-depth, default is 8 [8, 10]
--compressed-ten-bit-format Pack 10bit video, handled between the app and library, default is 0 [0-1]
--inj Inject pictures to the library at defined frame rate, default is 0 [0-1]
--inj-frm-rt Set injector frame rate, only applicable with `--inj 1`, default is 60 [0-240]
--enable-stat-report Calculates and outputs PSNR SSIM metrics at the end of encoding, default is 0 [0-1]
--asm Limit assembly instruction set, only applicable to x86, default is max [c, mmx, sse, sse2, sse3, ssse3, sse4_1, sse4_2, avx, avx2, avx512, max]
--lp Target (best effort) number of logical cores to be used. 0 means all. Refer to Appendix A.1 of the user guide, default is 0 [0, core count of the machine]
--pin Pin the execution to the first --lp cores. Overwritten to 0 when `--ss` is set. Refer to Appendix A.1 of the user guide, default is 1 [0-1]
--ss Specifies which socket to run on, assumes a max of two sockets. Refer to Appendix A.1 of the user guide, default is -1 [-1, 0, -1]
Rate Control Options:
--rc Rate control mode, default is 0 [0: CRF or CQP (if `--aq-mode` is 0), 1: VBR, 2: CBR]
-q, --qp Initial QP level value, default is 35 [1-63]
--crf Constant Rate Factor value, setting this value is equal to `--rc 0 --aq-mode 2 --qp x`, default is 35 [1-63]
--tbr Target Bitrate (kbps), only applicable for VBR and CBR encoding, default is 7000 [1-100000]
--mbr Maximum Bitrate (kbps) only applicable for CRF encoding, default is 0 [1-100000]
--use-q-file Overwrite the encoder default picture based QP assignments and use QP values from `--qp-file`, default is 0 [0-1]
--qpfile Path to a file containing per picture QP value separated by newlines
--max-qp Maximum (highest) quantizer, only applicable for VBR and CBR, default is 63 [1-63]
--min-qp Minimum (lowest) quantizer, only applicable for VBR and CBR, default is 1 [1-63]
--aq-mode Set adaptive QP level, default is 2 [0: off, 1: variance base using AV1 segments, 2: deltaq pred efficiency]
--use-fixed-qindex-offsets Overwrite the encoder default hierarchical layer based QP assignment and use fixed Q index offsets, default is 0 [0-1]
--key-frame-qindex-offset Overwrite the encoder default keyframe Q index assignment, default is 0 [-256-255]
--key-frame-chroma-qindex-offset Overwrite the encoder default chroma keyframe Q index assignment, default is 0 [-256-255]
--qindex-offsets list of luma Q index offsets per hierarchical layer, separated by `,` with each offset in the range of [-256-255], default is `0,0,..,0`
--chroma-qindex-offsets list of chroma Q index offsets per hierarchical layer, separated by `,` with each offset in the range of [-256-255], default is `0,0,..,0`
--luma-y-dc-qindex-offset Luma Y DC Qindex Offset
--chroma-u-dc-qindex-offset Chroma U DC Qindex Offset
--chroma-u-ac-qindex-offset Chroma U AC Qindex Offset
--chroma-v-dc-qindex-offset Chroma V DC Qindex Offset
--chroma-v-ac-qindex-offset Chroma V AC Qindex Offset
--undershoot-pct Allowable datarate undershoot (min) target (percentage), default is 25, but can change based on rate control [0-100]
--overshoot-pct Allowable datarate overshoot (max) target (percentage), default is 25, but can change based on rate control [0-100]
--buf-sz Client buffer size (ms), only applicable for CBR, default is 6000 [0-10000]
--buf-initial-sz Client initial buffer size (ms), only applicable for CBR, default is 4000 [0-10000]
--buf-optimal-sz Client optimal buffer size (ms), only applicable for CBR, default is 5000 [0-10000]
--recode-loop Recode loop level, refer to "Recode loop level table" in the user guide for more info [0: off, 4: preset based]
--bias-pct CBR/VBR bias, default is 50 [0: CBR-like, 1-99, 100: VBR-like]
--minsection-pct GOP min bitrate (expressed as a percentage of the target rate), default is 0 [0-100]
--maxsection-pct GOP max bitrate (expressed as a percentage of the target rate), default is 100 [0-100]
Multi-pass Options:
--pass Multi-pass selection, pass 3 is only available for VBR, default is 0 [0: single pass encode, 1: first pass, 2: second pass, 3: third pass]
--stats Filename for multi-pass encoding, default is "svtav1_2pass.log"
--passes Number of encoding passes, default is preset dependent but generally 1 [1: one pass encode, 2: multi-pass encode]
GOP size and type Options:
--keyint GOP size (frames), default is -2 [-2: ~5 seconds, -1: "infinite" and only applicable for CRF, 0: same as -1]
--irefresh-type Intra refresh type, default is 2 [1: FWD Frame (Open GOP), 2: KEY Frame (Closed GOP)]
--scd Scene change detection control, default is 0 [0-1]
--lookahead Number of frames in the future to look ahead, not including minigop, temporal filtering, and rate control, default is -1 [-1: auto, 0-120]
--hierarchical-levels Set hierarchical levels beyond the base layer, default is 4 [3: 4 temporal layers, 4: 5 layers, 5: 6 layers]
--pred-struct Set prediction structure, default is 2 [1: low delay frames, 2: random access]
AV1 Specific Options:
--tile-rows Number of tile rows to use, `TileRow == log2(x)`, default changes per resolution but is 1 [0-6]
--tile-columns Number of tile columns to use, `TileCol == log2(x)`, default changes per resolution but is 1 [0-4]
--enable-dlf Deblocking loop filter control, default is 1 [0-1]
--enable-cdef Enable Constrained Directional Enhancement Filter, default is 1 [0-1]
--enable-restoration Enable loop restoration filter, default is 1 [0-1]
--enable-tpl-la Temporal Dependency model control, currently forced on library side, only applicable for CRF/CQP, default is 1 [0-1]
--enable-mfmv Motion Field Motion Vector control, default is -1 [-1: auto, 0-1]
--fast-decode Fast Decoder levels, default is 0 [0-1]
--enable-tf Enable ALT-REF (temporally filtered) frames, default is 1 [0-1]
--enable-overlays Enable the insertion of overlayer pictures which will be used as an additional reference frame for the base layer picture, default is 0 [0-1]
--tune Specifies whether to use PSNR or VQ as the tuning metric [0 = VQ, 1 = PSNR], default is 1 [0-1]
--scm Set screen content detection level, default is 2 [0: off, 1: on, 2: content adaptive]
--rmv Restrict motion vectors from reaching outside the picture boundary, default is 0 [0-1]
--film-grain Enable film grain, default is 0 [0: off, 1-50: level of denoising for film grain]
--enable-dnl-denoising Apply denoising when film grain is ON, default is 1 [0: no denoising, film grain data is still in frame header, 1: level of denoising is set by the film-grain parameter]
--superres-mode Enable super-resolution mode, refer to the super-resolution section in the user guide, default is 0 [0: off, 1-3, 4: auto-select mode]
--superres-denom Super-resolution denominator, only applicable for mode == 1, default is 8 [8: no scaling, 9-15, 16: half-scaling]
--superres-kf-denom Super-resolution denominator for key frames, only applicable for mode == 1, default is 8 [8: no scaling, 9-15, 16: half-scaling]
--superres-qthres Super-resolution q-threshold, only applicable for mode == 3, default is 43 [0-63]
--superres-kf-qthres Super-resolution q-threshold for key frames, only applicable for mode == 3, default is 43 [0-63]
--sframe-dist S-Frame interval (frames) (0: OFF[default], > 0: ON)
--sframe-mode S-Frame insertion mode ([1-2], 1: the considered frame will be made into an S-Frame only if it is an altref frame, 2: the next altref frame will be made into an S-Frame[default])
Color Description Options:
--color-primaries Color primaries, refer to Appendix A.2 of the user guide, default is 2 [0-12, 22]
--transfer-characteristics Transfer characteristics, refer to Appendix A.2 of the user guide, default is 2 [0-22]
--matrix-coefficients Matrix coefficients, refer to Appendix A.2 of the user guide, default is 2 [0-14]
--color-range Color range, default is 0 [0: Studio, 1: Full]
--chroma-sample-position Chroma sample position, default is 'unknown' ['unknown', 'vertical'/'left', 'colocated'/'topleft']
--mastering-display Mastering display metadata in the format of "G(x,y)B(x,y)R(x,y)WP(x,y)L(max,min)", refer to the user guide Appendix A.2
--content-light Set content light level in the format of "max_cll,max_fall", refer to the user guide Appendix A.2
Cu Selur
Ps.: yup that will take me a few days just to readup on stuff and a few weeks to implement so svt-av1 will be removed in the next release.
https://github.com/AOMediaCodec/SVT-AV1/...r_guide.md is quite a bit to read and I also will have to side check against https://aomediacodec.github.io/av1-spec/ to not add stuff that isn't fully specified. argh.
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Posts: 53
Threads: 8
Joined: Nov 2020
Okay, I understand.
Also x264 encoding crash "Crashed with exit status 0" (tried with 2 saved configuration)
Posts: 10.551
Threads: 57
Joined: May 2017
14.04.2022, 21:48
(This post was last modified: 14.04.2022, 22:16 by Selur.)
x264 aborts with:
Quote:x264 [error]: invalid width x height (32768x18432)
encoding call itself:
"C:\Program Files\Hybrid\64bit\ffmpeg.exe" -y -loglevel fatal -noautorotate -nostdin -threads 8 -ignore_editlist true -i "D:\Videos\Download di RealPlayer\AR.Drone duel in Central Park - AR.FlyingAce FREE APP!(1080p_H.264-AAC).mp4" -map 0:0 -an -sn -vf scale=32768:18432,zscale=rangein=tv:range=tv -pix_fmt yuv420p -vsync 0 -sws_flags bicublin -f rawvideo - | "C:\Program Files\Hybrid\64bit\x264.exe" --preset slower --crf 20.00 --ref 9 --keyint 10000 --sync-lookahead 12 --qcomp 0.40 --qpmax 51 --partitions i4x4,i8x8,p8x8,b8x8 --merange 32 --subme 10 --psy-rd 1.00:0.15 --aq-mode 3 --sar 1:1 --deblock -1:-1 --non-deterministic --range tv --colormatrix bt470bg --demuxer raw --input-res 32768x18432 --input-csp i420 --input-range tv --input-depth 8 --fps 25/1 --opencl --opencl-device 1 --opencl-clbin "C:\Users\Antonio\AppData\Local\Temp\x264_lookahead_2022-04-14@21_16_51_2510_02.clbin" --output-depth 8 --output "C:\Users\Antonio\AppData\Local\Temp\2022-04-14@21_16_51_2510_02.264" -
looks correct.
My guess is that x264 does not support such a high resolution.
How much RAM do you have? 16k will take over 32gb,...
"--keyint 10000" <- why not use open-gop?
Also 'opencl-clbin' also will not work with such a resolution, since the vram requirements should also be humongous.
I suspect that either x264 simply doesn't support the resolution (couldn't find anything in the x264 source code so far) or you are simply running out of RAM.
Even if I disable cabac and use the far less (effective and ressource friendly cavlc by using base profile and way faster settings) I get 99% RAM usge on a 32gb machine. (it does encode)
So seeing that you are using monsterous settings your system probably would need more than 128GB if you used base profile with simple settings, your settings probably need more double that in RAM.
-> I don't think this is bug in Hybrid.
Cu Selur
Ps.: Also note that afaik current hardware decoders are limited to max 8192x8192 so you might also in problems decoding 32k streams. 
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Posts: 53
Threads: 8
Joined: Nov 2020
x264 aborts with:
Quote:x264 [error]: invalid width x height (32768x18432)
What?! Actually is the same video file I'm using for codec tests, a normal 1080p video.
I see In the resize tab somehow it changed to upscale to that resolution. Maybe the Maximize resolution option in Misc sub-tab but that was checked in previous tests too.
In meanwhile I just did some test with SuperResolution upscaling (Vapoursynth tab).
Quote:"--keyint 10000" <- why not use open-gop?
Open-gop can be over 10000? Didn't want push too much, but anyway it is and old configuration and I think is not worth the quality change use so high value/open-gop so I'll just remove.
Quote:I suspect that either x264 simply doesn't support the resolution (couldn't find anything in the x264 source code so far) or you are simply running out of RAM.
Even if I disable cabac and use the far less (effective and ressource friendly cavlc by using base profile and way faster settings) I get 99% RAM usge on a 32gb machine. (it does encode)
So seeing that you are using monsterous settings your system probably would need more than 128GB if you used base profile with simple settings, your settings probably need more double that in RAM.
Yes, me too have 32 GB RAM (but a lot already in use) but the encoding immediately stopped so the encoder has probably given up due to the available RAM.
So sorry probably is the right behavior, just didn't' realize it changed in the resize tab.
Posts: 10.551
Threads: 57
Joined: May 2017
Quote:Open-gop can be over 10000? Didn't want push too much, but anyway it is and old configuration and I think is not worth the quality change use so high value/open-gop so I'll just remove.
OpenGOP only uses IDR frame if it makes sense for the compressibility, so you can end up with whole clips only have one IDR frame at the beginning.
Quote: Maybe the Maximize resolution option in Misc sub-tab but that was checked in previous tests too.
Yes, thats the culprit, that option only makes sense if you use a profile that is restricted otherwise Hybrid will resize to the max resolution it does support and that is 32768x32768.
Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Posts: 10.551
Threads: 57
Joined: May 2017
Side note: most of the stuff mentioned there doesn't work.
-> will probably have working SVT-AV1 encoding back tomorrow.
Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
|
