Preview + encode disaster ! - Printable Version

Preview + encode disaster ! - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: Preview + encode disaster ! (/thread-3355.html)

Pages: 1 2

Preview + encode disaster ! - ToiletDuck - 17.08.2023

heLL o,

Today, i tried to encode just the video from an bluray structured movie.

I can't get preview with LWlibavsource, i get this ↓ script error thrown at me Dodgy

Failed to evaluate the script:

Python exception: lsmas: failed to get video track.

Traceback (most recent call last):

File "src\cython\vapoursynth.pyx", line 3115, in vapoursynth._vpy_evaluate

File "src\cython\vapoursynth.pyx", line 3116, in vapoursynth._vpy_evaluate

File "E:\BD50\Movie\BDMV\STREAM\tempPreviewVapoursynthFile18_39_14_766.vpy", line 17, in 

clip = core.lsmas.LWLibavSource(source="E:/BD50/Movie/BDMV/STREAM/00705.m2ts", format="YUV420P8", stream_index=0, cache=0, prefer_hw=3)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "src\cython\vapoursynth.pyx", line 2847, in vapoursynth.Function.__call__

vapoursynth.Error: lsmas: failed to get video track.

And iam pretty sure that this ↑ is causing hybrid to crash at encode stage finaly !!

Also, both happends using Bluray as input or just opening the m2ts track !!

Then, when i choose FFmpegSource2 instead, and needs to create index file, its doing that at turtle slow crawling speeds of a whooping 10MB/sec using an NVme SSD pcie 4 stick for crying out loud Huh

!

Cheers,
TD

RE: Preview + encode disaster ! - Selur - 17.08.2023

No clue what causes LWlibavsource not to recognize the streams in the source, but I can't do anything about that.
You can try disabling the hardware decoding if using another source filter isn't practical for you.

Cu Selur

RE: Preview + encode disaster ! - ToiletDuck - 17.08.2023

(17.08.2023, 18:49)Selur Wrote: No clue what causes LWlibavsource not to recognize the streams in the source, but I can't do anything about that.
You can try disabling the hardware decoding if using another source filter isn't practical for you.

Cu Selur

NP, i asume you are refering to Config→Input→Decoding→Use gpu for decoding method ?
If so, that did the trick Rolleyes

!

But preview with lwlibav is terrible slow, at speeds of between 100MB/sec - 200 MB/sec (like 7200 rpm HDD speeds... coicidence ?) reading from an Nvme SSD noless that ussually operates at 1 - 2 GB/sec processsing BIG files in other apps !!!

but, i have better/faster succes using preview with FfmpegSource2 and rather quickly...
I have mentioned that ↑ several times in the past. For some reason FFmpegSource2 is XX times faster..

However, when hybrid starts an job, ffmpeg struggles to creating a index file...The speeds are soo terrible low (10MB/sec plus minus)!! ..
Pretty much the same thing happends like it wos with the TS demux/remux issue awhile back u know ?

At first , extraction of tracks after opening files wos quickly done... but at job creation NOT..
Same thing with FFMPEGsource2 now..

Atleast for very big files like BD50.. it's an returning issue .. !
NOt so much for small files from XGB ..!

THANKS for the SUGGESTION.. albeit the obvious.. my bad Sad

EDIT: Both FFmpegsource2 & Lwlibav behave normal again in hybrid , since i disable HWdecoding ^^

Cheers,
TD

RE: Preview + encode disaster ! - Selur - 18.08.2023

Strange thing is that FFmpegsource2 doesn't support hardware decoding, only LWlibavsource does, but happy it is working for you now.

Cu Selur

RE: Preview + encode disaster ! - ToiletDuck - 18.08.2023

(18.08.2023, 11:22)Selur Wrote: Strange thing is that FFmpegsource2 doesn't support hardware decoding, only LWlibavsource does, but happy it is working for you now.

Cu Selur

FFmpeg2source does work while HWdecode is enabled though.. But only to summon filter preview window you know!?

The strange part is, when i create an job and ffmpeg starts to create index file, it suddenly goes from speed 100 to zer0 Huh

and its taking forever to finish the preparations before encode !!!

I think ffmpeg2source would have done the job eventually, but it's taking ridiculus MUCH & MUCH of time imo !!
And that's happening while reading from / writing to an NVme SSD stick .. i mean c"mon !!

And again, i've noticed (maybe important) that this ↑ happends allot with BIG SIZED files (XX+/GB) , or files with long duration time (2-3 hours+) !!

When using small media files, with short duration that doesn't happend, no issues at all for an strange reason !

cheers,
TD

RE: Preview + encode disaster ! - Selur - 19.08.2023

Quote:FFmpeg2source does work while HWdecode is enabled though.. But only to summon filter preview window you know!?

FFmpeg2source has no HWDecode option, that is an LWlibavsource option,...

How long are you talking about?
For ne opening m2ts files (with vc1 which is worst combo for FFMS2) from blu-rays from a hdd (temp and output lie on m2 ssd, all have exception in the virus scanner).
Creating index (2hr film ~19GB) with:

DGDecNV: 15s
FFMS2: 25s
directly opening with LWlibavsource: 1m 34s

Creating index (2hr film ~25GB, 3 video all vc-1,ac3 m,5 subtitle, 6 audio streams) with:

DGDecNV: 19s
FFMS2: 26s
directly opening with LWlibavsource: 1m 21s

=> DGDecNV is normally the fastest of the bunch for m2ts

No clue why ffms2/ffmindex would take extremely long for you.
You can test in your system via command line the indexing calls Hybrid uses look always like this:

"PATH TO: Hybrid\64bit\vsfilters\SourceFilter\FFMS2\ffmsindex.exe" -f -t 0 "Path to source.m2ts" "Path to temp .ffindex file"

with the available option being:

FFmpegSource2 indexing app

Usage: ffmsindex [options] inputfile [outputfile]

If no output filename is specified, inputfile.ffindex will be used.

Options:

-f        Force overwriting of existing index file, if any (default: no)

-v        Set FFmpeg verbosity level. Can be repeated for more verbosity. (default: no messages printed)

-p        Disable progress reporting. (default: progress reporting on)

-c        Write timecodes for all video tracks to outputfile_track00.tc.txt (default: no)

-k        Write keyframes for all video tracks to outputfile_track00.kf.txt (default: no)

-t N      Set the audio indexing mask to N (-1 means index all tracks, 0 means index none, default: 0)

-s N      Set audio decoding error handling. See the documentation for details. (default: 0)

-u N      Set the progress update frequency in seconds. Set to 0 for every percent. (default: 1)

-o string Set demuxer options to be used in the form of 'key=val:key=val'. (default: none)

Do not be fooled by the '-f', Hybrid will not reindex if it has created the file since for this source, assuming you haven't changed the source or restarted Hyrid.
'-t 0 ' disabled the audio indexing.
=> seems fine to me.

You can also try different versions of ffms2, from:
https://github.com/FFMS/ffms2
newer 'test' builds are often linked over at: https://forum.doom9.org/showthread.php?t=127037
(those are also the only places where you can ask someone if someone is willing and capable to fix something Wink

)
All in all, I don't think there is a way to get ffms2 faster, but this is the usual range of speed these indexers work as.
If your speed strongly differ from the above, it is usually something in the system that is interfering.

Cu Selur

Ps.: An explanation mark (or mutliple), is not meant to be the standard end of a sentence in a normal conversation. I'm getting more and more tired of being yelled at by you and your broken keyboard, so either get that fixed or do not be surprised, if I stop replying to all your posts in a future post. (I do this in my free time, anything I 'earn' / get 'donated' for this is listed over at ([INFO] Thanks for the donation!
, so please don't take it wrong when I sometimes get picky with such things.)

RE: Preview + encode disaster ! - ToiletDuck - 20.08.2023

(19.08.2023, 08:54)Selur Wrote:
Quote:FFmpeg2source does work while HWdecode is enabled though.. But only to summon filter preview window you know!?
FFmpeg2source has no HWDecode option, that is an LWlibavsource option,...

How long are you talking about?
For ne opening m2ts files (with vc1 which is worst combo for FFMS2) from blu-rays from a hdd (temp and output lie on m2 ssd, all have exception in the virus scanner).
Creating index (2hr film ~19GB) with:
DGDecNV: 15s

FFMS2: 25s

directly opening with LWlibavsource: 1m 34s

Creating index (2hr film ~25GB, 3 video all vc-1,ac3 m,5 subtitle, 6 audio streams) with:
DGDecNV: 19s

FFMS2: 26s

directly opening with LWlibavsource: 1m 21s

=> DGDecNV is normally the fastest of the bunch for m2ts

No clue why ffms2/ffmindex would take extremely long for you.

Well, deselecting HWdecode does defenitely something for FFmpeg2source for me awhile back..

But get this, before i deselected it, i saw that the setting wos → DXVA2 instead of Auto for HWdecoding.
could that be it ? Could that cause slow reads or even crashes for lwlibav and/or ffmpegsource2 !?

I ask because, now it seems that it doesn't matter to hybrid what settings i've used suddenly.
Summon preview filter window now works, both with HWdecode on & off Dodgy

... EVEN when i selected DXVA2 for decoding Huh

Could the source file have something to do with it ?
Because when i took the RECODED file from that was re-encoded from that very same sourcefile, all of an sudden i get the preview window to work just fine regardless of what settings i use !?! Huh

That's weird to say the least..

That having said, i still suspects ...→ FILE SIZE perhaps !!

Because to reproduce this issue , you've mentioned to have used an "Big" media file of prox 25GB with no issues..
No issues indeed, BUT with that small difference, that your file is more close to a BD25..

The issue ussually arises when i use source files of aprox 46+GB ! ←There lies the difference .. → That could be it..

Also, you seems to get the preview windows in aproximately the same time (1,30 min) like me using an HDD ?
That doesn't sound right to me !

Especially since iam using tech that surpasses hdd by far, like X30 times faster than the fastest HDD Huh

UNLESS.. the tool that creates these index files doesn't take into account the RAW speed of an SSD.. is ← that it ?

(19.08.2023, 08:54)Selur Wrote: Ps.: An explanation mark (or mutliple), is not meant to be the standard end of a sentence in a normal conversation. I'm getting more and more tired of being yelled at by you and your broken keyboard, so either get that fixed or do not be surprised, if I stop replying to all your posts in a future post. (I do this in my free time, anything I 'earn' / get 'donated' for this is listed over at ([INFO] Thanks for the donation!
, so please don't take it wrong when I sometimes get picky with such things.)

I like the shape of → !! & ?? , i have an thing with symbols , think nothing of it Tongue

.

It's my way of telling you .. Hey Selur ... be sure NOT to OVERLOOK and READ THAT → wich is in front of this → !!?
I just want to emphasize certain subjects we discuss, or do u rather want ME TO TYPE ALL CAPITALS Big Grin

?

So that i can pass as the bedroom bully Dodgy

Don't take it personal... i!

It's not as if you suddenly go blind from reading One ! to much Selur.. Wink

Trust me.. diz duck has typed and read many ! and ? and ... in his life Wink

But if tis is your wish, i'll ..err.. my keyboard will trying to refrain itself from spawning to many symbols Wink

cheers,
td

RE: Preview + encode disaster ! - Selur - 21.08.2023

Quote:The issue ussually arises when i use source files of aprox 46+GB ! ←There lies the difference .. → That could be it..

I'll check whether I can find some test content in that regard,...

Quote: i saw that the setting wos → DXVA2 instead of Auto for HWdecoding.

Quote:Hardware decoding mode:
Whether to prefer hardware accelerated decoder to software decoder.
Have no effect if 'decoder' is specified.
- 0 : Use default software decoder.
- 1 : Use NVIDIA CUVID acceleration for supported codec, otherwise use default software decoder.
- 2 : Use Intel Quick Sync Video acceleration for supported codec, otherwise use default software decoder.
- 3 : Try hardware decoder in the order of CUVID->QSV. If none is available, then use default software decoder.

there is no mention of DXVA2 here,... -> not sure what you are referring to.

Cu Selur

RE: Preview + encode disaster ! - ToiletDuck - 21.08.2023

(21.08.2023, 05:39)Selur Wrote:
Quote:The issue ussually arises when i use source files of aprox 46+GB ! ←There lies the difference .. → That could be it..
I'll check whether I can find some test content in that regard,...

Yes, has to be something to do with the filesize , if its not the duration of the media file causing issues...
But then again, some issues couldn't simply reproduced with short duration samples.. You see what i mean ?
Like for instance the mux crashes, could only be reproduced with media cuts from minimum 5 min or so !?

(21.08.2023, 05:39)Selur Wrote:
Quote:i saw that the setting wos → DXVA2 instead of Auto for HWdecoding.

Quote:Hardware decoding mode:
Whether to prefer hardware accelerated decoder to software decoder.
Have no effect if 'decoder' is specified.
- 0 : Use default software decoder.
- 1 : Use NVIDIA CUVID acceleration for supported codec, otherwise use default software decoder.
- 2 : Use Intel Quick Sync Video acceleration for supported codec, otherwise use default software decoder.
- 3 : Try hardware decoder in the order of CUVID->QSV. If none is available, then use default software decoder.
there is no mention of DXVA2 here,... -> not sure what you are referring to.

Ah, you were refering to Lwlibav hardware decoding method Blush

While iam talking about the HWdecoding setting in Config→Input→Decoding→Use Gpu for Decoding= section from hybrid Wink

Lwlibav hardware method doesn't or better yet CAN'T do/change anything for me since that setting requires Intel and/or Nvidia hardware .
And iam ALL AMD here Cool

..

Hence, why i thought you were refering to somewhere else..
And fortunately for me, reading you wrong did helped me out ^^

P.S: BtB, lately i have noticed that the filtering tab is completely missing after launching hybrid, no vapoursynth no ffmpeg, no nothing !? What would cause that ?
When reset all settings in hybrid or loading custom profile, everything turns normal again though.. Just fyi..

Cheers,
TD

RE: Preview + encode disaster ! - Selur - 21.08.2023

Quote:While iam talking about the HWdecoding setting in Config→Input→Decoding→Use Gpu for Decoding= section from hybrid

That will be used if you don't use Avisynth or Vapoursynth and will crash is you select something that doesn't work. Wink

Largest avc m2ts source I got here has 39GB, so if 46GB is the criteria,... Confused

"00800.m2ts" 39GB

1x video avc

8 x audio

17 x subtitle

DGDecNV: 00:48

FFVideoSource: 00:18

LWLibavSource: 00:36

Strange thing is, when I indexed it with FFVideoSource, closed the preview and reindexed it, is was faster.
Could it be a Windows disk caching issue?

Cu Selur