[Feature Request] Add support for BestSource

[Dan64]

Hello Selur,

   currently Hybrid to open the sources is providing the option to use LWLibAVSource/LSMASHSource or FFMpegSource2.
   It would be nice to add support also to BestSource: https://forum.doom9.org/showthread.php?t=184255

Thanks,
Dan

[Selur]

For what?
Atm. I do not see any gain in adding BestSource.
It would also require creating (large) temporal audio files, since BestSource will output pcm audio.

Cu Selur

[Dan64]

Sorry my previous request was not clear, in effect I was referring to core.bs.VideoSource() only.

The reason is explained in the post: The time has finally come for a universal source filter that doesn't suck. FFMS2 was good back when only B-frames roamed the earth but now we have "super totally keyframes (IDR)", "referenced but hidden frames (VPx)" and "just doesn't work because FFmpeg internals changed frames"

as declared by the author: BestSource solves all of these issues by only performing linear decoding. Plus some tricks with multiple decoder instances and a large cache to hide some of the potentially slow seeking.

I'm not suggesting to set BestSource as default Video source reader, but in some case, with some particular movie, it could be useful.

[Selur]

Seems like the source of the current versions isn't public, so it would be Windows only atm.

bs.VideoSource(string source, int track = -1, bint variableformat = False, int threads = 0, int seekpreroll = 20, bint exact = True, bint enable_drefs = False, bint use_absolute_path = False, string cachepath = \<user data dir\>)

*track*: Either a positive number starting from 0 specifying the absolute track number or a negative number to select the nth audio or video track. Throws an error on wrong type or no matching track.

*adjustdelay*: Adjust audio start time relative to track number. Pass -2 to disable and -1 to be relative to the first video track if one exists.

*variableformat*: Allow format changes in the output for video. Untested.

*threads*: Number of threads to use for decoding. Pass 0 to autodetect.

*seekpreroll*: Number of frames before the requested frame to cache when seeking.

*exact*: Decode the full track to get an exact frame/sample count. Will throw an error if set to false and the frame or sample count can't be guessed at all. If too many frames/samples is guessed the end is passed with the last available frame/silence to fill up the missing data.

*enable_drefs*: Option passed to the FFmpeg mov demuxer.

*use_absolute_path*: Option passed to the FFmpeg mov demuxer.

*drc_scale*: Apply dynamic range compression to ac3 audio. 0 = None and 1.0 = Normal.

*cachepath*: The full path to the cache file. Defaults to `%LOCALAPPDATA%\bsindex.json` or `~/bsindex.json` depending on the OS.

*hwdevice*: The interface to use for hardware decoding. Depends on OS and hardware. On windows `d3d11va` and `cuda` are probably the ones most likely to work.

(from the Test 7 file)

Indexing files to get exact sample and frame counts is enabled by default but can be turned off by settings exact=False to speed up file opening. All indexing information is also stored in a single json file to not gunk up your pristine filesystems.

Indexless mode generally seems to works fairly well for avi and mp4 files. Do however note that nvops are dropped so some avi files will turn into VFR.

this also seems troubling, if you can't tell the decoder to decode to a specific frame rate and content that before was cfr might turn into vfr,...
Does that mean one would have to check the json index file each time to be sure the content stayed cfr?
=> if there's a kind of stable release I will think about it, but atm. adding this to Hybrid seems like a bad idea.

Cu Selur

[Dan64]

It is strange that since 19/07/22 was not released a stable version.

The use of BestSource is also suggested in Vapoursynth documentation: http://www.vapoursynth.com/doc/gettingstarted.html
So I guess that it is considered stable enough to suggest its use.

I any case the last decision is up to you.

[Selur]

That wiki links to: https://github.com/vapoursynth/bestsource
about vfr: https://github.com/vapoursynth/bestsource/issues/9
Seeing that there have been changes in the last few weeks it seems like myrsloik is still working on it.
They added actions which produce builds on each change, so there are updated binaries.
=> I'll look at it, but atm. I probably wouldn't recommend using it.
(one downside: you can't create the index file in advance, there is no alternative to ffindex, so first loading might be slow)
Documentation:

hwdevice: The interface to use for hardware decoding. Depends on OS and hardware. On windows d3d11va and cuda are probably the ones most likely to work. Defaults to CPU decoding. Will throw errors for formats where hardware decoding isn't possible.

also does not really list what values are available for hardware decoding,.. :/

Cu Selur

[Dan64]

Indexless mode generally seems to works fairly well for avi and mp4 files. Do however note that nvops are dropped so some avi files will turn into VFR.

this also seems troubling, if you can't tell the decoder to decode to a specific frame rate and content that before was cfr might turn into vfr,...
Does that mean one would have to check the json index file each time to be sure the content stayed cfr?

 I don't understand the implication of not being able to force the fps in the source reader. My understanding was that it is possible to change the fps in output using the encoder, also in Hybrid there are filters that are able to do that by add/removing frames.

[Selur]

cfr output means that each frame is displaced a constant time.
For example: 40ms for 25fps content.
For cfr content, a decoder simply reads the frame rate flag and then it knows how long to display each frame.
So by simply changing, the frame rate flag of a cfr stream you change the display time for each frame.

With vfr you got the video stream and for each frame in it, a time for how long it should be displayed (or until when it should be displayed, but that is basically the same).
For example: frame 0-9 10ms (~100fps), frame 10-19 40ms (~25fps),.....
So if you change the frame count or rate, you need to adjust the time codes accordingly.
(The above 20 frames have an average display time of 24.5ms which would be in average ~40,81fps.
If you now want to scale this to 50 fps, you would have to add frames in some parts and adjust the time codes for each part and maybe remove time codes and frames in other sections. And this gets more and more complicated if you think about what you can do to streams.)
Which is not way trivial!

When handling vfr content, you got:
a. the frames
b. the time codes
as long as you don't change the frame count, you can simply filter the frames and add the time codes back in.

If you want to have cfr output, you normally tell the decoder to output a specific frame rate and the decoder will then (use the time codes and simply add or remove frames to hit your desired frame rate).
If you apply stuff that changes the frame rate or frame count (like deinterlacing, frame rate interpolation) you either have to:
a. adjust the time codes to match the output (Hybrid can't do that, and I haven't ever seen someone properly doing this, so this is no option.)
b. or decode to cfr

Problem is, if your source filter changes cfr content to vfr content, you would need to have the time codes to add them back after the filtering or convert to cfr using those time codes. If you don't have them, then you will get async output.
So unless bestsource adds support for cfr output, one would have to analyse the index file to see whether it contains the time stamps (which I assume) and then add a way to handle them. :/

Also keep in mind, that basically all filters assume the input is cfr.
So unless Vapoursynth and all the (temporal) filters properly take vfr into account, you either stick with vfr (and don't do frame interpolation&co) or you decode to cfr.

also in Hybrid there are filters that are able to do that by add/removing frames.

which all assume the source if cfr

Cu Selur

Ps.: Yes, stuff like interlaced or mixed vfr content is simply 'evil' and in my mind a production mistake for which someone should get impaled by a stick.

[Dan64]

Thank for the clarification. In any case, given the time necessary to build the index, even if the source is CFR its use is warrant when is necessary an accurate frame seeking.
Performing some tests I noted that DGDecNV is the faster decoder, do you see any contraindications in configuring it as "default" decoder ?

[Selur]

If DGDecNV can decode your content, I see no problem in enabling 'Use DGDecNV when available'.

Cu Selur