Selur's Little Message Board
[BUG] SVT-AV1 not working with vapoursynth inputs (Arch Linux) - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: [BUG] SVT-AV1 not working with vapoursynth inputs (Arch Linux) (/thread-4021.html)

Pages: 1 2

SVT-AV1 not working with vapoursynth inputs (Arch Linux) - Inky1003 - 25.02.2025

I use Hybrid 2024.12.21.1 in Arch Linux. It was compiled from AUR earlier today, 2025/02/25, UTC-3 time.

It seems that SVT-AV1 refuses to encode vspipe outputs. Tried to encode with x264, and it worked. Tried to encode with SVT-AV1 and without vapoursynth, worked. Vapoursynth and SVT-AV1 gives a blank file. Also, no errors are thrown.

Tested with both SVT-AV1-PSY (the first try) and main SVT-AV1 (to make sure It's not SVT-AV1-PSY guys fault).

Step by step guide:

1. Have a working Arch Linux with Hybrid installed.
2. Open a video file (I used an interlaced source)
3. Use SVT-AV1 as video encoder
4. Enable Vapoursynth in the Filtering guide (I also enabled QTGMC)
5. Start the job

It should tell "finished", when in fact no frames were written, and the file is empty.

Debug file:

[attachment=3006]

Long time no see!
Inky

RE: SVT-AV1 not working with vapoursynth inputs - Selur - 25.02.2025

Hybrid uses:
"/usr/bin/vspipe" "/tmp/2025-02-25@15_50_50_1010/encodingTempSynthSkript_2025-02-25@15_50_50_1010_0.vpy" - -c y4m | "/usr/bin/ffmpeg" -y -loglevel fatal -noautorotate -nostdin -threads 8 -f yuv4mpegpipe -i - -an -sn -pix_fmt yuv420p10le -strict -1 -fps_mode passthrough -f yuv4mpegpipe - | "/usr/bin/SvtAv1EncApp" --input stdin --width 720 --height 480 --fps-num 60000 --fps-denom 1001 --input-depth 10 --profile 0 --tile-rows 0 --tile-columns 0 --progress 2 --rc 0 --crf 20 --mbr 5000 --aq-mode 2 --preset 6 --matrix-coefficients 5 --color-range 0 --output "/tmp/2025-02-25@15_50_50_1010/2025-02-25@15_50_50_1010_03.av1"
Not sure why ffmpeg is used here at all. (don't really see a reason for it atm. => did a quick test, this is also done on Windows and works - might still be unnecessary)

Does the Vapoursynth Preview work?
There is no error and SvtAV1 report it is finished with the encoding. (exit code 0)
Are you sure your SvtAV1 version works properly?
Does calling:
"/usr/bin/vspipe" "/tmp/2025-02-25@15_50_50_1010/encodingTempSynthSkript_2025-02-25@15_50_50_1010_0.vpy" - -c y4m | "/usr/bin/SvtAv1EncApp" --input stdin --width 720 --height 480 --fps-num 60000 --fps-denom 1001 --input-depth 10 --profile 0 --tile-rows 0 --tile-columns 0 --progress 2 --rc 0 --crf 20 --mbr 5000 --aq-mode 2 --preset 6 --matrix-coefficients 5 --color-range 0 --output "/tmp/2025-02-25@15_50_50_1010/2025-02-25@15_50_50_1010_03.av1"
work ?
Might be some change in SvtAv1 3.x that is causing the problem.

Cu Selur

Ps.: Will look at it tomorrow. (too groggy today)

RE: SVT-AV1 not working with vapoursynth inputs - Inky1003 - 25.02.2025

The vapoursynth preview doesn't works because there is no package for vsPreview. Also, tried to use vsedit-previewer with no success. It expects only a filename and no "dash" options.

Yes, both the SVT-AV1 implementations work just fine in FFMPEG. Also, they work fine in Hybrid without Xsynths.

Quote:Does calling:
"/usr/bin/vspipe"

"/tmp/2025-02-25@15_50_50_1010/encodingTempSynthSkript_2025-02-25@15_50_50_1010_0.vpy"
- -c y4m | "/usr/bin/SvtAv1EncApp" --input stdin --width 720 --height
480 --fps-num 60000 --fps-denom 1001 --input-depth 10 --profile 0
--tile-rows 0 --tile-columns 0 --progress 2 --rc 0 --crf 20 --mbr 5000
--aq-mode 2 --preset 6 --matrix-coefficients 5 --color-range 0 --output
"/tmp/2025-02-25@15_50_50_1010/2025-02-25@15_50_50_1010_03.av1"
work ?

Just realized that the vpy file didn't even generated... But then how did x264 encoded? Also, It's not doing that anymore for some strange reason...

So the problem seems to be the vpy file not being generated.

RE: SVT-AV1 not working with vapoursynth inputs - Selur - 25.02.2025

Did a quick test with a build of SvtAv1EncApp I created on Saturday (SVT-AV1 v3.0.0 (release)).
Works fine here. My build does not report all the memory related output yours does.
Svt[info]: SVT [version]:    SVT-AV1 Encoder Lib v2.3.0-dirty
Svt[info]: SVT [build]  :    GCC 14.2.1 20240910     64 bit
Svt[info]: LIB Build date: Jan  1 2025 21:19:00
Svt[info]: -------------------------------------------
2025.02.25 - 15:50:52_Arch Linux (64bit)_2024.12.21.1 - level 9: 2025-02-25@15_50_50_1010_03_video ### /run/media/inky/vhs antigo/VHS_AV1/0-2001_new.mkv ### Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:    SVT-AV1 Encoder Lib v2.3.0-dirty
Svt[info]: SVT [build]  :    GCC 14.2.1 20240910     64 bit
Svt[info]: LIB Build date: Jan  1 2025 21:19:00
Svt[info]: -------------------------------------------
Svt[info]: Level of Parallelism: 4
Svt[info]: Number of PPCS 107
Svt[info]: [asm level selected : up to avx2]
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile    tier (auto)    level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator         : 720 / 480 / 60000 / 1001
Svt[info]: SVT [config]: bit-depth / color format                     : 10 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct                     : 6 / PSNR / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type             : 321 / 32 / key frame
Svt[info]: SVT [config]: BRC mode / rate factor / max bitrate (kbps)            : capped CRF / 20 / 5000
Svt[info]: SVT [config]: AQ mode / variance boost                     : 2 / 0
Svt[info]: -------------------------------------------
SvtMalloc[info]:     total allocated memory:       593.91 MB
SvtMalloc[info]:         malloced memory:          184.11 MB
SvtMalloc[info]:         callocated memory:        26.30 MB
SvtMalloc[info]:         allocated aligned memory: 383.50 MB
SvtMalloc[info]:     mutex count: 17737
SvtMalloc[info]:     semaphore count: 566
SvtMalloc[info]:     thread count: 70
SvtMalloc[info]:     hash table fulless: 0.034183, hash bucket is healthy
SvtMalloc[info]: (45.81 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/bitstream_unit.c:36
SvtMalloc[info]: (38.25 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/coding_unit.c:84
SvtMalloc[info]: (15.41 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/pcs.c:114
SvtMalloc[info]: (12.38 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/pcs.c:1075
SvtMalloc[info]: (10.02 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/pcs.c:115
SvtMalloc[info]: (9.76 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/pcs.c:571
SvtMalloc[info]: (9.18 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/pcs.c:1528
SvtMalloc[info]: (5.10 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/pcs.c:1531
SvtMalloc[info]: (4.20 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/md_process.c:249
SvtMalloc[info]: (2.49 MB): /usr/src/debug/svt-av1/SVT-AV1/Source/Lib/Codec/md_process.c:304
Encoding frame    0 -nan kbps -nan fpm  
SUMMARY --------------------------------- Channel 1  --------------------------------
Total Frames        Frame Rate        Byte Count        Bitrate
           0        59.94 fps                 0         -nan kbps


Channel 1
Average Speed:        -nan fps
Total Encoding Time:    0 ms
Total Execution Time:    0 ms
Average Latency:    -nan ms
Max Latency:        0 ms
SvtMalloc[info]: you have no memory leak
process finished with exitcode: 0 and exitStatus: 0
I checked my logs and I did not really change any code since 2024-12-05 in regard to SvtAV1, so me using my latest dev should not be the cause of the encoding working for me.
Atm. I suspect, that it's either an issue with your SvtAV1 build (I would recommend trying 2.3.0 or 3.0.0) or with the Vapoursynth script.

=> I'll create a new SvtAv1EncApp build tomorrow and will test with that build.

Cu Selur

RE: SVT-AV1 not working with vapoursynth inputs - Selur - 25.02.2025

Quote:So the problem seems to be the vpy file not being generated.
According to the log it was created and clean up later.
Created skript /tmp/2025-02-25@15_50_50_1010/encodingTempSynthSkript_2025-02-25@15_50_50_1010_0.vpy:
---------------------
# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = '/usr/lib/python3.12/site-packages/'
sys.path.insert(0, os.path.abspath(scriptPath))
# Import scripts
import havsfunc
import validate
# Source: '/run/media/inky/vhs antigo/VHS/0-2001.mpg'
# Current color space: YUV420P8, bit depth: 8, resolution: 720x480, frame rate: 29.97fps, scanorder: top field first, yuv luminance scale: limited, matrix: 470bg, transfer: bt.470 system m, primaries: bt.470 system m, format: mpeg-2
# Loading /run/media/inky/vhs antigo/VHS/0-2001.mpg using D2VSource
clip = core.d2v.Source(input="/tmp/2025-02-25@15_50_50_1010/2025-02-25@15_50_50_1010.d2v")
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT470_BG), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT470_BG)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 29.97fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# making sure the detected scan type is set (detected: top field first)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_TOP) # tff
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Medium", TFF=True) # new fps: 59.94
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting output color from: YUV420P8 to YUV420P10 for SvtAv1Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 59.94fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001)
# output
clip.set_output()
---------------------

2025.02.25 - 15:50:52_Arch Linux (64bit)_2024.12.21.1 - level 9: 2025-02-25@15_50_50_1010_04_cleanUp: "/tmp/2025-02-25@15_50_50_1010/encodingTempSynthSkript_2025-02-25@15_50_50_1010_0.vpy"
2025.02.25 - 15:50:52_Arch Linux (64bit)_2024.12.21.1 - level 9: JobHandler subjobFinished 2025-02-25@15_50_50_1010_04_cleanUp finished,..

Quote:The vapoursynth preview doesn't works because there is no package for vsPreview.
vsViewer code is available over at https://github.com/Selur/vsViewer. Since there are other Arch users out there, someone can probably help.

Quote:Yes, both the SVT-AV1 implementations work just fine in FFMPEG.
FFmpeg uses the library not SvtAv1EncApp.

Cu Selur

RE: SVT-AV1 not working with vapoursynth inputs - Inky1003 - 25.02.2025

But did it encoded something? Because It blames no frames, just like my tests:

SUMMARY --------------------------------- Channel 1  --------------------------------

Total Frames        Frame Rate        Byte Count        Bitrate

           0        59.94 fps                 0         -nan kbps

Also, the vpy is not being generated, even for x264. I'm not sure why It worked earlier, but It's not generating the vpy for x264 also.

RE: SVT-AV1 not working with vapoursynth inputs (Arch Linux) - Selur - 25.02.2025

Yes, I get:
SUMMARY --------------------------------- Channel 1  --------------------------------
Total Frames        Frame Rate        Byte Count        Bitrate
         429        25.00 fps            279232        130.18 kbps


Channel 1
Average Speed:        1047.619 fps
Total Encoding Time:    410 ms
Total Execution Time:    492 ms
Average Latency:    130 ms
Max Latency:        173 ms
and the output is created and is fine.

Quote:Also, the vpy is not being generated, even for x264. I'm not sure why It worked earlier, but It's not generating the vpy for x264 also.
Like I wrote, according to the logs it is. It's just deleted. Try enabling "Config->Containers->General->Keep intermediate files" to keep the intermediate files.

Cu Selur

Ps.: going to bed now. Smile

RE: SVT-AV1 not working with vapoursynth inputs (Arch Linux) - Inky1003 - 25.02.2025

Quote:FFmpeg uses the library not SvtAv1EncApp.

Arch Linux replaces svt-av1 package when installing svt-av1-psy. They are considered incompatible. That said, If I have SVT-AV1-PSY, there is automatically no main SVT-AV1 available.

About the vpy file: 

It's being generated, but still for some reason the same problems occur (again). I think maybe because Hybrid is not generating them at /tmp?

Debug files:

[attachment=3007]

RE: SVT-AV1 not working with vapoursynth inputs (Arch Linux) - Selur - 26.02.2025

Does
vspipe "/run/media/inky/vhs_antigo/VHS_AV1/encodingTempSynthSkript_2025-02-25@18_30_16_7310_0.vpy" /dev/null -c y4m --progress
work properly?

Since you don't have vsViewer, can you open the vpy file in vsedit-previewer and preview it?

Cu Selur

RE: SVT-AV1 not working with vapoursynth inputs (Arch Linux) - Inky1003 - 26.02.2025

vspipe in that command blames a module that does not exist in Python...

Quote:Script evaluation failed:
Python exception: No module named 'validate'

Traceback (most recent call last):
  File "src/cython/vapoursynth.pyx", line 3387, in vapoursynth._vpy_evaluate
  File "src/cython/vapoursynth.pyx", line 3388, in vapoursynth._vpy_evaluate
  File "encodingTempSynthSkript_2025-02-26@02_23_09_5110_0.vpy", line 12, in <module>
    import validate
ModuleNotFoundError: No module named 'validate'