Selur's Little Message Board

Full Version: Things started well but now .19 FPS and slowing
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I finally managed to get Hybrid , Vapoursynth and the addon compiled for ubuntu. It has been a challenge.

Hybrid starts but does throw one info box (see below) that when I close the UI opens and it appears to be working from a generic de-interlace test encode (24 min dvd source into mkv with x265 took about 7 minutes total) using just defaults for QTGMC (Vapoursynth) and x265 (audio is passthrough).

Upon adding some settings beyond the defaults from the test encode based on reading forum posts here and the information provided by hybrid when you mouse over settings the job (same input file as the test encode) started a little slower but over the last 4 hours has progressively gotten slower and slower.

system is pretty much idle with 1 vspipe process eating a CPU.

Job setting changes was to add bob to the deinterlace and then set framerate reduction to restore it to the source framerate

The startup info box is 
Quote:'lspci' didn't finish after 1 seconds -> aborting,... Restarting Hybrid might solve the problem,... The cause might also be some incompatibility with the tool and your system.

I get this every time I start now. Before this fresh install of ubuntu and compile of everything, I had on another ubuntu install of the same version, Hybrid starting with no issues but it wasn't seeing vapoursynth via the addon so I wiped and started over.
Installed fresh OS, added user, and then started installing/compiling just the items needed to give me Intel GPU support in encoding, then Hybrid, Vapoursynth and the addon.

System information
OS: Ubuntu 24.04.3 LTS (I need this version because Alchemest GPU support is baked in as well as some other stuff)
CPU: AMD Ryzen 7 7800X3D (IGPU enabled)
GPU: Intel Arc A380
RAM: 32 GB
Hybrid Version 2025.07.27.1

I've got all the intel drivers/media stuff installed, ffmpeg has the intel QSV support, so I am unsure why it's not detected (Handbrake can see/use it).

I am working on re-running the job that hung up with debug turned on so I can add the output.
I will add it as soon as I can.
Thank you for this software as the test deinterlace was so much better than what I was getting from other tools.
Regarding lspci:
Hybrid uses lspci to determine which GPUs are installed.
Is it installed on your system? (if not 'sudo apt install pciutils')

Regarding hardware encoding on Linux:
  • QSVEncC/VCEEncC/NVEncC those are not supported on Linux atm. and
  • I only ever wrote some experimental suppor for nvenc through ffmpeg.

Quote:system is pretty much idle with 1 vspipe process eating a CPU.
knowing nothing of the scripts used&co: no clue (debug output ?!?)
Wild guess is that the source filter is indexing the source, which can take ages if the source isn't on a fast medium (i.e. on a 1GBit or slower network)

About encoding going slower over time: no clue

Cu Selur
lspci is installed as part of the base OS of Ubuntu I can run it and $PATH includes the location of the binary.

Right before it pops up the information box, in the shell I launch Hybrid from, I see this output from the command line
Quote:QCoreApplication::postEvent: Unexpected null receiver

I did make sure I was using the same QT as the github repo documented.

Source and output are all local to the machine and on separate Samsung SSDs. Output path is an M2 device and source path is a SATA SSD.

I am attempting to run the same job again with debug on.

I do accept that this could be an ignorance/PEBKEC issue as I am not well versed in the nuances of video/encoding (I know more than enough to be dangerous  Smile ).

I am happy with the output generated by the deinterlacer defaults for the most part and was seeing if I could improve a couple items that I still see which led me to trying the blob option combined with the Reduction option to bring the framerate back to the original source frame rate.

I wasn't getting debug output with the default path (even though permissions should have allowed it) but after changing the default debug output path and checking "create report file" as well, I finally got Hybrid Debug Output.txt to generate.

Since the job slows down and never finishes I am attaching what is in the debug file.

How would I provide a sample in this situation?
Code:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = '/home/jellyfin/opt/hybrid/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Import scripts
import srestore
import qtgmc
import validate
# Source: '/working/raw/VENTURE_BROS_VOL_1_DISC_1/C1_t01.mkv'
# Current color space: YUV420P8, bit depth: 8, resolution: 720x480, frame rate: 29.97fps, scanorder: top field first, yuv luminance scale: limited, matrix: 470bg, transfer: bt.601, primaries: bt.601 ntsc, format: mpeg-2
# Loading /working/raw/VENTURE_BROS_VOL_1_DISC_1/C1_t01.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="/working/raw/VENTURE_BROS_VOL_1_DISC_1/C1_t01.mkv", format="YUV420P8", stream_index=0, cache=0, fpsnum=30000, fpsden=1001, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 29.97fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# making sure the detected scan type is set (detected: top field first)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_TOP) # tff
# Deinterlacing using QTGMC
clip = qtgmc.QTGMC(Input=clip, Preset="Fast", TFF=True) # new fps: 59.94
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting frame count and rate with sRestore
clip = srestore.sRestoreMUVs(source=clip, frate=29.9700)
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 29.97fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# output
clip.set_output()
So you are using QTGMC and sRestore with some DVD input. (aiming for 29.97fps usually is wrong in this case, but your choice)
That the initial speed reporting is 'off' is 'okay' since filling caches etc. and often having content with less motion data at the beginning usually can irritate the speed measurement.
That said, 1.3fps seems too slow for 720x480 content on your cpu.
  • Does the speed also drop if you do not use sRestore? (sRestore is a pain and sadly enforces linear processing)
  • Does it make a difference if you enable 'Filtering->Vapoursynth->Misc->Script->Limit Cache (MB)->3/4 of system ram' ?
  • Does it make a difference if you enable hardware decoding? (no clue whether that even works for lav or bestsource) Or try another source filter?


Cu Selur

Ps.: "QCoreApplication::postEvent: Unexpected null receiver" can be ignored, I fixed it locally, but it should not any any problematic side effect aside from one start up message not getting shown.
I uploaded a new dev AppImage which fixes "QCoreApplication::postEvent: Unexpected null receiver"

Cu Selur
Quote:So you are using QTGMC and sRestore with some DVD input. (aiming for 29.97fps usually is wrong in this case, but your choice)

I chose that to match the framerate supposedly of the source. I'm open to becoming educated on what I am misunderstanding.

Quote:Does the speed also drop if you do not use sRestore? (sRestore is a pain and sadly enforces linear processing)

It appears to run at "full" speed.

I do notice that all encodes at start ramp up to a high frame rate (over 200) then framerate steadily declines over the length of the encode. For this 22 minute item the ending frame rate had fallen to 156 FPS. I assume this is normal behavior and is just something to do with how frame processing, cache/buffers and calculating an estimated remaining job time looks in the UI.

Quote:Does it make a difference if you enable 'Filtering->Vapoursynth->Misc->Script->Limit Cache (MB)->3/4 of system ram' ?

It started out looking better (running at about 40 fps) but went to the same place within a couple minutes.
once it hit about 5% I was down to about 2 fps.

Quote:Does it make a difference if you enable hardware decoding? (no clue whether that even works for lav or bestsource) Or try another source filter?

Turned on hardware decode and it does not appear to have made a difference.
I can live without using the bob and rSize combo.
I attached debugs for the above attempts in case it's of any use.

Once again thank you for the help and I am impressed with and liking these tools. I just have no learned what 98% of the tools/options do so have a lot to learn as I go.

P.S. I did get a compile error for ffmpeg when compiling the vapoursynth addons and had to comment out a line that was no longer supported.

Line # 98 in  build-and-install-vapoursynth.sh
--enable-avresample
Quote: I chose that to match the framerate supposedly of the source. I'm open to becoming educated on what I am misunderstanding.
You should read up about telecine and ivtc and check whether your source is telecined or even soft telecined.
In case it's not telecined, but really interlaced (which is rather uncommon for DVDs), sticking with 2*29.97 would be what one would prefer to go for, to keep most of the motion information.

Quote:I do notice that all encodes at start ramp up to a high frame rate (over 200) then framerate steadily declines over the length of the encode.
yep, that sounds 'okay', when complex filter scripts like qtgmc are used.

Quote:P.S. I did get a compile error for ffmpeg when compiling the vapoursynth addons and had to comment out a line that was no longer supported.

Line # 98 in build-and-install-vapoursynth.sh
--enable-avresample
Haven't used that script for ages, I usually use:
1st build-vapoursynth.sh
2nd build-plugins.sh
=> I'll try to look at it in the next few days.

Cu Selur

Cu Selur
Interlace analyzer says:

Quote:Detected interlaced Material with 29,../59,.. fps, but didn't detect the material to be telecine. You might want to run 'Filtering->(De-)Interlace/Telecine->Deinterlace/Telecine Settings->Analyse'.
checking C1_t01.mkv for interlacing,...
running interlaced check,...
Starting interlaced analysis pass 1 (5%  66.5749) of C1_t01.mkv
checking C1_t01.mkv for interlacing,...
-> Interlaced analysis pass 1 finished after 00:00:11.466hrs
Starting interlaced analysis pass 2 (5%  66.5749) of C1_t01.mkv
-> Interlaced analysis pass 2 finished after 00:00:29.942hrs
-> interlaced check result: top field first
adjusting scan order due analysis to: top field first
-> finished auto routines for source number: 1
Input is completely analysed,...
analyser finished,..


I'll read up more on the subject though since I do have some badly mastered DVD sources.
Dreamworks Dragons and The Tick animated series DVDs playback on a normal DVD player hooked up to TV all the fast motion has those "interlace" looking artifacts.

Since I have the DVD's for all of this stuff and all I did was use makemkv or dvd-decrypter to back them up for encoding (if one had an issue with a media I used the other) I can pop the original discs in for comparisons.

As for building the software, Thanks for the information, I was unsure which I needed to use.

Any other "artifacting" I will make a sample and start some other thread in the appropriate area.
I meant to ask, is is possible to just do the de-interlace without encode so I can play around with different encode settings?
Quote: I meant to ask, is is possible to just do the de-interlace without encode so I can play around with different encode settings?
Aside from exporting to a lossless intermediate format: No
Deinterlacing changes the video content, so you can't simply 'patch' the input format in a lossless way.

'Filtering->(De-)Interlace/Telecine->Deinterlace/Telecine Settings->Analyse' is a heuristic which can be easily wrong.
=> best look at it with your eyes. (read: https://forum.selur.net/thread-3010.html and also read up on interlacing, telecine and field-shift and norm conversions)

Cu Selur