Internal processing in 32 bit depth?

[shijan]

Just a quick technical question here - is it true that Hybrid, FFmpeg and VapourSynth use 32bit float internal bit depth processing for filters, resize, transcoding and all other operations?

P.S. From my personal tests i see that at least ProRres 10bit video source transcoded to same true ProRes 10bit with all hidden dynamic range data. In 8 bit depth processing hidden dynamic range data usually just clipped.

[Selur]

No. What makes you think that?

Cu Selur

[shijan]

From my personal tests i see that at least ProRres 10bit video source transcoded to same true ProRes 10bit with all hidden dynamic range data. It don't looks like upscaled from 8 bit depth. In 8 bit depth processing hidden dynamic range data usually just clipped.
So at least processing done in 12 or 16 bit? Or how does it works inside?

---

Here is another test example. Gradient with applied Levels adjustment to illustrate 8bit depth limits. There are some tiny degradation in FFmpeg Hybrid, so yeah, it is not 10000000% perfect. But same time it don't looks like crappy 8 bit processing as well.

Rendered from Resolve to 12bit ProRes444, applied Levels in Resolve


Rendered from Hybrid to 12bit ProRes444, applied Levels in Resolve


Rendered  from Resolve to 8bit Tiff, applied Levels in Resolve

[Selur]

When Avisynth or Vapoursynth are used and what filters are used Hybrid will convert to the bit depth required by the filter.
Depending on the filter 8bit, 10bit, 12bit or 16bit will be used.
Not sure how ffmpeg handles this internally.
In general color sampling precision should be kept assuming the output format supports it and the encoder is configured to keep it.
So to keep 12bit precision you need to encode to 12bit and not apply filters which require a lower precision.
-> So to know what happens in your workflow I need to know what you so.

Cu Selur

[shijan]

And here are more detailed bit depth tests. Hybrid x264/265 compression was set to near lossless quality RF1 L4.1 slow grain --ref 3 --bframes 0 --keyint 25 --min-keyint 1


Test Results:

Hybrid FFmpeg renders ProRes422HQ as 10 bit and ProRes444 also only as 10 bit (seems FFmpeg limitation https://trac.ffmpeg.org/ticket/7163)

Hybrid FFmpeg renders honest 8 bit and 10 bit x264/x265 files and even real 12 bit depth x265 files.

[Selur]

So everything behaves as expected.
Good.

Cu Selur

[shijan]

Guess i really need to know somehow what plug-ins are 8bit only So i guess i'll do real life bit depth tests, similar to codec tests. Maybe that list will be shared somehow at VapourSynth Database or arranged to filter.
For Example here are [ProRes 12 bit 444] vs [Hybrid ProRes 10 bit 422 with ColorMatrix filter] vs [Hybrid ProRes 10 bit 422 with DeRainbow Bifrost filter]
As you can see DeRainbow Bifrost filter is 8 bit only.

[shijan]

Here it is: 8 bit only filters list
(tested with Hybrid UI on macOS 10.14.6):
DeGrain -> MLDegrain
DeGrain -> TemporalDegrain
Artefacts -> DeCrawl -> DotKill
Artefacts -> DeCrawl -> DeDot
Artefacts -> DeRainbow -> BiFrost
Artefacts -> DeRainbow -> SSIQ
Artefacts -> DeCross
Line -> AntiAliasing -> SangNomMod (removed from Hybrid due conflict with SangNom)
Line -> Resizer -> Anime4KCPP

Overall situation is not so bad...

[Selur]

Simple testing doesn't really help much.
It only shows you the current state not whether that state is correct or not.
Some filters might support 10bit+, but didn't when I implemented their support.
Flash3kDB for example should be able to handle 10bit+
-> if you want to do it properly you need to
a. test the current state in Hybrid
b. in case it's 8bit and a filter check the documentation of the filter
c. in cast it's 8bit and a script check all the filters used in the script and check their documentation
to see whether this could be changed or is a limitation of the filter/script.

Cu Selur

[shijan]

Yep, maybe it is not technically perfect, but we get what we get. Filters in the list will downconvert source to 8-bit and rendered video will have only 8 bit data even if source was 10 bit and if it will be rendered to 10 bit codec.

TemporalDegrain, DotKill, DeDot, BiFrost, SSIQ, DeCross, SangNomMod are 8 bit by official tech specification. So my tests only confirm that. Not a big deal because those group of filters in most cases used for 8 bit DVD restoration.