HDR encoding

[Selur]

The filename was missing in the place of "-" that is why I thought that was the reason for crashing.

File name has nothing to do in the encoder call, it should only appear in the decoder call.
I doubt that has anything to do with why anything is crashing.

It seems Hybrid thinks the source is 8-bit (wants to dither to 8 bit instead of 10 bit)

No hybrid assumes the source to be more than 8it otherwise it wouldn't use format="YUV420P16".
It looks like it thinks you set the encoder to 8bit output and thus creates 8bit output for the encoder. (=> make sure you set the output bit depth in the encoder properly)

# dithering from 16 to 8bit for encoder

and doesn't let me modify e.g. the matrix from bt.709 to bt.2020, etc...:

Hybrid doesn't offer anything to convert bt.709 to bt.2020 it only offers to signal a specifc matrix (through the encoder).
I don't know of any way to convert SDR to HDR content.
-> if you know of a way to do a bt.709 to bt.2020 conversion using Avisynth or Vapoursynth, let me know and I can add support for it in Hybrid.

Cu Selur

Ps.: small side note normally HDR is 10bit not 12bit. 12bit is only used by Dolby Vision afaik.