Color change on encoding

[Selur]

Used latest dev (which ffmpeg version N-117721-g000e303de4-g3330b733d3+1) and encoded using UT-Video (+ 'always use Vapoursynth' + LWLibavSource as source filter).
Vapoursynth Preview didn't show any color changes.
Script used:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import validate
# Source: 'G:\099.mkv'
# Current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, frame rate: 23.976fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709, transfer: bt.709, primaries: bt.709, format: FFV1
# Loading G:\099.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="G:/099.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 709.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)
# setting color transfer (vs.TRANSFER_BT709), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709)
# setting color primaries info (to vs.PRIMARIES_BT709), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# set output frame rate to 23.976fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# output
clip.set_output()

Encoding call

vspipe "J:\tmp\2024-11-13@14_36_12_1210\encodingTempSynthSkript_2024-11-13@14_36_12_1210_0.vpy" - -c y4m  |  ffmpeg -y -noautorotate -nostdin -threads 8 -f yuv4mpegpipe -i - -an -sn -fps_mode passthrough -vcodec utvideo -pred left -slices 0 -pix_fmt yuv420p -colorspace bt709 -metadata encoding_tool="Hybrid 2024.11.12.1" "J:\tmp\2024-11-13@14_36_12_1210\UT_LWLibavSource_1_2024-11-13@14_36_12_1210_02.mkv"

Compared the outputs using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libhistogram.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import validate
# Source: 'G:\color_change_099.mkv'
# Current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, frame rate: 23.976fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709, transfer: bt.709, primaries: bt.709, format: FFV1
# Loading G:\color_change_099.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="G:/099.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
clip = core.hist.Color(clip=clip)

reencode = core.lsmas.LWLibavSource(source="G:/Output/UT_LWLibavSource.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
reencode = core.hist.Color(clip=reencode)


clip = core.std.Interleave([clip.text.Text("Org"),reencode.text.Text("Reencode")])

# output
clip.set_output()

=> I can reproduce the problem!
(https://imgsli.com/MzE3OTcy)

Looking into it.

Adding '-loglevel debug' to the ffmpeg call. I see:

Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-noautorotate' ... matched as option 'autorotate' (automatically insert correct rotate filters) with argument 0.
Reading option '-nostdin' ... matched as option 'stdin' (enable or disable interaction on standard input) with argument 0.
Reading option '-threads' ... matched as AVOption 'threads' with argument '8'.
Reading option '-f' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 'yuv4mpegpipe'.
Reading option '-i' ... matched as input url with argument '-'.
Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.
Reading option '-sn' ... matched as option 'sn' (disable subtitle) with argument '1'.
Reading option '-fps_mode' ... matched as option 'fps_mode' (set framerate mode for matching video streams; overrides vsync) with argument 'passthrough'.
Reading option '-vcodec' ... matched as option 'vcodec' (alias for -c:v (select encoder/decoder for video streams)) with argument 'utvideo'.
Reading option '-pred' ... matched as AVOption 'pred' with argument 'left'.
Reading option '-slices' ... matched as AVOption 'slices' with argument '0'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.
Reading option '-colorspace' ... matched as AVOption 'colorspace' with argument 'bt709'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-metadata' ... matched as option 'metadata' (add metadata) with argument 'encoding_tool=Hybrid 2024.11.12.1'.
Reading option 'G:\Output\UT_LWLibavSource_1_2024-11-13@14_47_27_4010_02.mkv' ... matched as output url.

and

[yuv4mpegpipe @ 000002613b1175c0] After avformat_find_stream_info() pos: 3110466 bytes read:3112960 seeks:0 frames:1
Input #0, yuv4mpegpipe, from 'fd:':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0, 1, 1001/24000: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p(progressive, center), 1920x1080, 0/1, 23.98 fps, 23.98 tbr, 23.98 tbn
Successfully opened the file.
Parsing a group of options: output url G:\Output\UT_LWLibavSource_1_2024-11-13@14_47_27_4010_02.mkv.
Applying option an (disable audio) with argument 1.
Applying option sn (disable subtitle) with argument 1.
Applying option fps_mode (set framerate mode for matching video streams; overrides vsync) with argument passthrough.
Applying option vcodec (alias for -c:v (select encoder/decoder for video streams)) with argument utvideo.
Applying option pix_fmt (set pixel format) with argument yuv420p.
Applying option metadata (add metadata) with argument encoding_tool=Hybrid 2024.11.12.1.
Successfully parsed a group of options.
Opening an output file: G:\Output\UT_LWLibavSource_1_2024-11-13@14_47_27_4010_02.mkv.
[out#0/matroska @ 000002613b128480] No explicit maps, mapping streams automatically...
[vost#0:0/utvideo @ 000002613949b300] Created video stream from input stream 0:0
[file @ 000002613949d400] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> utvideo (native))
[vost#0:0/utvideo @ 000002613949b300] Starting thread...
[vf#0:0 @ 000002613949d5c0] Starting thread...
[vist#0:0/rawvideo @ 000002613949a800] [dec:rawvideo @ 000002613949dc00] Starting thread...
[in#0/yuv4mpegpipe @ 000002613b117200] Starting thread...
[rawvideo @ 000002613b129700] PACKET SIZE: 3110400, STRIDE: 2880
detected 32 logical cores
[rawvideo @ 000002613b129700] PACKET SIZE: 3110400, STRIDE: 2880
[graph -1 input from stream 0:0 @ 000002613b176100] w:1920 h:1080 pixfmt:yuv420p tb:1001/24000 fr:24000/1001 sar:0/1 csp:unknown range:unknown
[format @ 000002613b176600] Setting 'pix_fmts' to value 'yuv420p'
[format @ 000002613b176600] Setting 'color_spaces' to value 'bt709'
[format @ 000002613b176600] Setting 'color_ranges' to value 'tv'
[auto_scale_0 @ 000002613b176ac0] w:iw h:ih flags:'' interl:0
[format @ 000002613b176600] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 000002613b12a540] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
[swscaler @ 000002613b18ab40] [swscaler @ 000002613b19ef00] YUV color matrix differs for YUV->YUV, using intermediate RGB to convert
...

[swscaler @ 000002613b18ab40] [swscaler @ 000002613b19ef00] YUV color matrix differs for YUV->YUV, using intermediate RGB to convert

Seems wrong.


Cu Selur