Selur's Little Message Board

Pages: 1 2 3

I just tried last dev, the error is gone.
Output video have all three BT709 info, no more GBR color matrix warning on closing, no visual color difference, 0% difference with diffimg.

How did you managed to find the issue ?

I only fixed the startup problem.
Didn't change anything else, but the dev version can come with different ffmpeg versions, my guess atm. is that your problem was caused by some bug in ffmpeg which they fixed.

Cu Selur

(11.11.2024, 19:32)Selur Wrote: [ -> ]I only fixed the startup problem.
Didn't change anything else, but the dev version can come with different ffmpeg versions, my guess atm. is that your problem was caused by some bug in ffmpeg which they fixed.

Cu Selur

That would make sense, I didn't thought about that. Well, problem fixed it's nice Smile

Bad news, the bug is still there, but only when applying any vapoursynth filters. I have investigated and I think you're right, it seems to be ffmpeg, because all codecs encoded by ffmpeg have the issue.

Apparently, the issue is from the last ffmpeg 7.1 which is probably used in Hybrid 2024-11-11

With ffmpeg 7.1 (manually located on Config > Tools) or internal Hybrid ffmpeg, this is the result :

- any codec without vapoursynth filter = OK
- any codec + avisynth filters = OK
- UT Video + vapoursynth filters = colors changed
- FFV1 + vapoursynth filters = colors changed
- FFvhuff + vapoursynth filters = color changed
- x264/x265 lossless + vapoursynth filters = OK

With ffmpeg 7.0.2 (manually located on Config > Tools) with Hybrid 2024-11-11

- any codec without vapoursynth filter = OK
- any codec + avisynth filters = OK
- UT Video + vapoursynth filters = OK
- FFV1 + vapoursynth filters = OK
- FFvhuff + vapoursynth filters = OK
- x264/x265 lossless + vapoursynth filters = OK

With Hybrid 2024-09-29 (probably using a nightly builf of ffmpeg between 7.0.2 and 7.1)

- any codec without vapoursynth filter = OK
- any codec + avisynth filters = OK
- UT Video + vapoursynth filters = colors changed
- FFV1 + vapoursynth filters = OK
- FFvhuff + vapoursynth filters = color changed
- x264/x265 lossless + vapoursynth filters = OK

So if you try the latest Hybrid, with it's internal ffmpeg + enable any vapoursynth filter, you should have the color change issue.

Note : The Vapoursynth preview with all filters is fine, the issue seems to be on encoding.

1. How do you test the color difference?
2. Did you try another source filter? (BestSource?FFMS2?LWLibavSource?)
3. Did you see any difference in the encoding calls? (seems to be 'just' ffmpeg based encoders)

Quote:The Vapoursynth preview with all filters is fine, the issue seems to be on encoding

The problem should then be with ffmpeg or the ffmpeg call.

Cu Selur

Used latest dev (which ffmpeg version N-117721-g000e303de4-g3330b733d3+1) and encoded using UT-Video (+ 'always use Vapoursynth' + LWLibavSource as source filter).
Vapoursynth Preview didn't show any color changes.
Script used:

Code:
# Imports

import vapoursynth as vs

# getting Vapoursynth core

import sys

import os

core = vs.core

# Import scripts folder

scriptPath = 'F:/Hybrid/64bit/vsscripts'

sys.path.insert(0, os.path.abspath(scriptPath))

# loading plugins

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")

# Import scripts

import validate

# Source: 'G:\099.mkv'

# Current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, frame rate: 23.976fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709, transfer: bt.709, primaries: bt.709, format: FFV1

# Loading G:\099.mkv using LWLibavSource

clip = core.lsmas.LWLibavSource(source="G:/099.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)

frame = clip.get_frame(0)

# setting color matrix to 709.

clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)

# setting color transfer (vs.TRANSFER_BT709), if it is not set.

if validate.transferIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709)

# setting color primaries info (to vs.PRIMARIES_BT709), if it is not set.

if validate.primariesIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709)

# setting color range to TV (limited) range.

clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)

# making sure frame rate is set to 23.976fps

clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)

# making sure the detected scan type is set (detected: progressive)

clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive

# set output frame rate to 23.976fps (progressive)

clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)

# output

clip.set_output()

Encoding call

Code:
vspipe "J:\tmp\2024-11-13@14_36_12_1210\encodingTempSynthSkript_2024-11-13@14_36_12_1210_0.vpy" - -c y4m  |  ffmpeg -y -noautorotate -nostdin -threads 8 -f yuv4mpegpipe -i - -an -sn -fps_mode passthrough -vcodec utvideo -pred left -slices 0 -pix_fmt yuv420p -colorspace bt709 -metadata encoding_tool="Hybrid 2024.11.12.1" "J:\tmp\2024-11-13@14_36_12_1210\UT_LWLibavSource_1_2024-11-13@14_36_12_1210_02.mkv"

Compared the outputs using:

Code:
# Imports

import vapoursynth as vs

# getting Vapoursynth core

import sys

import os

core = vs.core

# Import scripts folder

scriptPath = 'F:/Hybrid/64bit/vsscripts'

sys.path.insert(0, os.path.abspath(scriptPath))

# loading plugins

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libhistogram.dll")

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")

# Import scripts

import validate

# Source: 'G:\color_change_099.mkv'

# Current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, frame rate: 23.976fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709, transfer: bt.709, primaries: bt.709, format: FFV1

# Loading G:\color_change_099.mkv using LWLibavSource

clip = core.lsmas.LWLibavSource(source="G:/099.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)

clip = core.hist.Color(clip=clip)

reencode = core.lsmas.LWLibavSource(source="G:/Output/UT_LWLibavSource.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)

reencode = core.hist.Color(clip=reencode)

clip = core.std.Interleave([clip.text.Text("Org"),reencode.text.Text("Reencode")])

# output

clip.set_output()

=> I can reproduce the problem! Big Grin

(https://imgsli.com/MzE3OTcy)

Looking into it.

Adding '-loglevel debug' to the ffmpeg call. I see:

Code:
Splitting the commandline.

Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.

Reading option '-noautorotate' ... matched as option 'autorotate' (automatically insert correct rotate filters) with argument 0.

Reading option '-nostdin' ... matched as option 'stdin' (enable or disable interaction on standard input) with argument 0.

Reading option '-threads' ... matched as AVOption 'threads' with argument '8'.

Reading option '-f' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 'yuv4mpegpipe'.

Reading option '-i' ... matched as input url with argument '-'.

Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.

Reading option '-sn' ... matched as option 'sn' (disable subtitle) with argument '1'.

Reading option '-fps_mode' ... matched as option 'fps_mode' (set framerate mode for matching video streams; overrides vsync) with argument 'passthrough'.

Reading option '-vcodec' ... matched as option 'vcodec' (alias for -c:v (select encoder/decoder for video streams)) with argument 'utvideo'.

Reading option '-pred' ... matched as AVOption 'pred' with argument 'left'.

Reading option '-slices' ... matched as AVOption 'slices' with argument '0'.

Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.

Reading option '-colorspace' ... matched as AVOption 'colorspace' with argument 'bt709'.

Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.

Reading option '-metadata' ... matched as option 'metadata' (add metadata) with argument 'encoding_tool=Hybrid 2024.11.12.1'.

Reading option 'G:\Output\UT_LWLibavSource_1_2024-11-13@14_47_27_4010_02.mkv' ... matched as output url.

and

Code:
[yuv4mpegpipe @ 000002613b1175c0] After avformat_find_stream_info() pos: 3110466 bytes read:3112960 seeks:0 frames:1

Input #0, yuv4mpegpipe, from 'fd:':

  Duration: N/A, start: 0.000000, bitrate: N/A

  Stream #0:0, 1, 1001/24000: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p(progressive, center), 1920x1080, 0/1, 23.98 fps, 23.98 tbr, 23.98 tbn

Successfully opened the file.

Parsing a group of options: output url G:\Output\UT_LWLibavSource_1_2024-11-13@14_47_27_4010_02.mkv.

Applying option an (disable audio) with argument 1.

Applying option sn (disable subtitle) with argument 1.

Applying option fps_mode (set framerate mode for matching video streams; overrides vsync) with argument passthrough.

Applying option vcodec (alias for -c:v (select encoder/decoder for video streams)) with argument utvideo.

Applying option pix_fmt (set pixel format) with argument yuv420p.

Applying option metadata (add metadata) with argument encoding_tool=Hybrid 2024.11.12.1.

Successfully parsed a group of options.

Opening an output file: G:\Output\UT_LWLibavSource_1_2024-11-13@14_47_27_4010_02.mkv.

[out#0/matroska @ 000002613b128480] No explicit maps, mapping streams automatically...

[vost#0:0/utvideo @ 000002613949b300] Created video stream from input stream 0:0

[file @ 000002613949d400] Setting default whitelist 'file,crypto,data'

Successfully opened the file.

Stream mapping:

  Stream #0:0 -> #0:0 (rawvideo (native) -> utvideo (native))

[vost#0:0/utvideo @ 000002613949b300] Starting thread...

[vf#0:0 @ 000002613949d5c0] Starting thread...

[vist#0:0/rawvideo @ 000002613949a800] [dec:rawvideo @ 000002613949dc00] Starting thread...

[in#0/yuv4mpegpipe @ 000002613b117200] Starting thread...

[rawvideo @ 000002613b129700] PACKET SIZE: 3110400, STRIDE: 2880

detected 32 logical cores

[rawvideo @ 000002613b129700] PACKET SIZE: 3110400, STRIDE: 2880

[graph -1 input from stream 0:0 @ 000002613b176100] w:1920 h:1080 pixfmt:yuv420p tb:1001/24000 fr:24000/1001 sar:0/1 csp:unknown range:unknown

[format @ 000002613b176600] Setting 'pix_fmts' to value 'yuv420p'

[format @ 000002613b176600] Setting 'color_spaces' to value 'bt709'

[format @ 000002613b176600] Setting 'color_ranges' to value 'tv'

[auto_scale_0 @ 000002613b176ac0] w:iw h:ih flags:'' interl:0

[format @ 000002613b176600] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format'

[AVFilterGraph @ 000002613b12a540] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed

[swscaler @ 000002613b18ab40] [swscaler @ 000002613b19ef00] YUV color matrix differs for YUV->YUV, using intermediate RGB to convert

...

Quote:[swscaler @ 000002613b18ab40] [swscaler @ 000002613b19ef00] YUV color matrix differs for YUV->YUV, using intermediate RGB to convert

Seems wrong.

Cu Selur

Nice !

using latest ffmpeg from: https://www.gyan.dev/ffmpeg/builds/ also shows the problem (both with 'master' and 'release')
FFmpegs changelog does not hint to something that should do this.

Think I got, it seems like "-colorspace bt709" now need to be added as input parameters too if pipe input is used.
=> I'll look into it

Uploaded a dev version which now sets vui stuff for input&output when using ffmpeg.
Let me know whether that fixes the problem for you too.

Cu Selur

I just tried last build with all FFMPEG codecs + Vapoursynth filters and everything is now fixed Smile

Pages: 1 2 3

Lipomo

Selur

Lipomo

Lipomo

Selur

Selur

Lipomo

Selur

Selur

Lipomo