[HELP] AviSynth 32 bit - ColorMatrix filter not working for video?

[Selur]

Okay, to many things to look at.
I can't keep the overview. + don't have much time this week.

So let's first stick to preview only, Vapoursynth and the current version.
(Vapoursynth since, the code is a bit easier there.)

First think I noticed when looking at the image:
- It's not limited luma range! It's 28-255, so full range.
- so using ColorMatrix while clamping the colors on input and output to 16-235 will cause color changes.

So whatever created the video stream didn't properly flag the input and might have messed up the colors during the rgb->yuv444 conversion.
Opening the image with Luma range PC and Color Matric bt709 and then opening the normal preview with FFvHuFF and Color space rgb select uses the following script for preview:

# Imports
import vapoursynth as vs
core = vs.get_core()
# source: 'C:/Users/Selur/Desktop/Image Test/Test-AdobeRGB.png'
# current color space: RGB24, bit depth: 8, resolution: 1440x1080, fps: 25, color matrix: 709, yuv luminance scale: full, scanorder: progressive
# Loading C:\Users\Selur\Desktop\Image Test\Test-AdobeRGB.png using vsImageReader
clip = core.imwri.Read(["C:/Users/Selur/Desktop/Image Test/Test-AdobeRGB.png"])
clip = core.std.Loop(clip=clip, times=100)
# Input color space is assumed to be RGB24
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# set output frame rate to 25.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

(color range is still 28/28/30-255/255/255)
enabling ColorMatrix with 'Rec.601/..' to 'Rec.709' and Clipping set to 'no clipping' uses:

# Imports
import vapoursynth as vs
core = vs.get_core()
# source: 'C:/Users/Selur/Desktop/Image Test/Test-AdobeRGB.png'
# current color space: RGB24, bit depth: 8, resolution: 1440x1080, fps: 25, color matrix: 709, yuv luminance scale: full, scanorder: progressive
# Loading C:\Users\Selur\Desktop\Image Test\Test-AdobeRGB.png using vsImageReader
clip = core.imwri.Read(["C:/Users/Selur/Desktop/Image Test/Test-AdobeRGB.png"])
clip = core.std.Loop(clip=clip, times=100)
# Input color space is assumed to be RGB24
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# adjusting color space from RGB24 to YUV444P8 for VsColorMatrix
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, matrix_s="709", range_s="full")
# ColorMatrix: adjusting color matrix from 470bg to 709
clip = core.resize.Bicubic(clip=clip, matrix_in_s="470bg", matrix_s="709", range_in=0, range=0)
# adjusting output color from: YUV444P8 to RGB24 for FFvHuffModel (rgb@8-bit)
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="limited")
# set output frame rate to 25.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

(color range is still 28/28/30-255/255/255)
now using the FilterPreview:

# Imports
import vapoursynth as vs
core = vs.get_core()
# source: 'C:/Users/Selur/Desktop/Image Test/Test-AdobeRGB.png'
# current color space: RGB24, bit depth: 8, resolution: 1440x1080, fps: 25, color matrix: 709, yuv luminance scale: full, scanorder: progressive
# Loading C:\Users\Selur\Desktop\Image Test\Test-AdobeRGB.png using vsImageReader
clip = core.imwri.Read(["C:/Users/Selur/Desktop/Image Test/Test-AdobeRGB.png"])
clip = core.std.Loop(clip=clip, times=100)
# Input color space is assumed to be RGB24
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
original = clip
# adjusting color space from RGB24 to YUV444P8 for VsColorMatrix
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, matrix_s="709", range_s="full")
# ColorMatrix: adjusting color matrix from 470bg to 709
clip = core.resize.Bicubic(clip=clip, matrix_in_s="470bg", matrix_s="709", range_in=1, range=1)
# adjusting output color from: YUV444P8 to RGB24 for FFvHuffModel (rgb@8)
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adjusting for FilterView
if original.format.id != clip.format.id:
if (original.format.color_family == vs.RGB and clip.format.color_family != vs.RGB):
   original = core.resize.Bicubic(original, format=clip.format.id, matrix_s="709", range_s="full")
elif (original.format.color_family == clip.format.color_family):
   original = core.resize.Bicubic(original, format=clip.format.id, range_s="full")
else:
   original = core.resize.Bicubic(original, format=clip.format.id, matrix_in_s="709", range_s="full")
stacked = core.std.StackHorizontal([original,clip])
# set output frame rate to 25.000fps
stacked = core.std.AssumeFPS(clip=stacked, fpsnum=25, fpsden=1)
# Output
stacked.set_output()

Send you a link to my latest version, which should fix the 'unnkown color format' problem.
(btw. you can check the preview script in your temp folder while the Preview is open)

Cu Selur

[The_Tinkerer]

Yeah, this is really a headache.

So again, the reason I'm using ColorMatrix 709->601 is because I process a DVD source with another program that shifts the color matrix to 709, so the colors look off. I take the result (which is an image stream) and when processing through Hybrid for final scaling/sharpening/etc., I use the ColorMatrix 709->601 to shift the color matrix back, so that the colors match the source DVD.

In Hybrid 2020.11.03.1 (and 2020.10.28.1), the ColorMatrix filter doesn't do anything, so the colors don't get shifted back.

HOWEVER, what I think is probably what I SHOULD be doing instead of using (probably mis-using ) the ColorMatrix filter, is simply opening the image stream like this:



Now, the Avisynth script has convert lines like ConvertToYV12(matrix="Rec601").

This appears to get the colors to match the source DVD MUCH MORE precisely than ColorMatrix. And it WORKS in Hybrid 2020.11.03.1.

The test image I was using above is not a good test image. It's just for an example, and is not even from a video source... it's an Adobe test image that shipped with Photoshop 3.0 in the mid 1990's!

Here's a GIF! The first frame is taken direct from the DVD, and is upscaled in IrfanView (Lanczos). The second frame is after it's been processed with Topaz Video Enhance AI, which upscales it using machine learning. Looks good! But, the colors are shifted, as you can see:



It took me a while to figure out what was going on. At first I assumed it's just the way the "machine learning" works and is trying to do some kind of color correction. But after trial-and-error investigation, I found that what actually appears to be going on is that the color matrix is getting converted from Rec.601 to Rec.709. Sometimes it can actually look a bit better. But sometimes it doesn't look good. In either case, it's not correct to the source DVD. Topaz VEAI has been like this for quite some time, and it happens when it outputs RGB24 PNG image streams.

So before, I was trying to fix this by using ColorMatrix.

But, if just simply open the image stream with bt470bg selected for color matrix (instead of bt709 which is what I was doing before), I get this script:

ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\ImageSeq.dll")
# loading source: E:/USER/fin/Kaiji/wip/Episode 01/032370.png
#  color sampling RGB24@8, matrix: bt470, scantyp: progressive, luminance scale: limited
ImageReader(file="E:\USER\fin\Kaiji\wip\EPB7DA~1\032370.png", end=100, fps=23.976, use_DeviL=true, pixel_type="RGB24")
# current resolution: 1440x960
# filtering
# color modifications
Levels(0,1.02,255,0,255)
# adjust color to YV12 (color matrix: Rec601)
ConvertToYV12(matrix="Rec601")
PreFetch(4)
# setting output fps to 23.976fps
AssumeFPS(24000,1001)
#  output: color sampling YV12@8, matrix: bt470, scantyp: progressive, luminance scale: limited
return last

(Side note: Topaz VEAI slightly lowers the gamma; that's why I've bumped it up to 1.02 in Levels. In the above GIF, I bumped the gamma up on the Topaz VEAI result to only show the color matrix shift.)

So now, with ConvertToYV12(matrix="Rec601") in the script (instead of ConvertToYUY2(interlaced=false, matrix="Rec709") and ColorMatrix(mode="Rec.709->Rec.601")), the output much better matches the source DVD:



I'm not sure if ColorMatrix is supposed to give results like the above. I would think, intuitively, that it should. But maybe I'm misusing it... I'm not sure if it's really designed to go from Rec.709->Rec.601, even though it can be set like that. It does nothing in Hybrid 2020.10.28.1 and 2020.11.03.1, and I don't think that's correct behavior; it should probably be fixed at some point. I probably won't be using ColorMatrix anymore, so that means no more giant posts from me about it, which I'm sure is a relief.

---

By the way, in case you (or anyone) finds it useful, here's the DVD source frame, and the Topaz VEAI frame (gamma bumped up to 1.02) shown above.

[attachment=1124]

[Selur]

There's a difference between a mislabed source and that ColorMatrix does when 709->601 is selected,...
If your source is just mislabel simply using the right color matrix on import is the correct way to go.

Also the Topaz output doesn't look impressive.
That looks doesn't look better than "Waifu2x nvk" (Vapoursynth resizer) + aWarpSharp2 and some color change.

Cu Selur