Cleaning old 640x480 AVI clips

[Selur]

probably since qtgmc adds some sharpening, so more smoothing and sharping,...

[Al74]

probably since qtgmc adds some sharpening, so more smoothing and sharping,...

Probably you are right. In my clips I also see some aliasing/jumping lines, I must see if any other filters could help. Or maybe NNEDI3 while upscaling could do a job...
Do you use any other AI upscale than the NNEDI3 one?

[Selur]

Always depends on the source and the other filters I used. But most of the time when upscaling normal content I use nnedi3 when upscaling since I don't want the denoising&co the machine learning resizers create. (Sometimes I use FeMaSR or RealESRGAN with general model.)

Cu Selur

[Al74]

Always depends on the source and the other filters I used. But most of the time when upscaling normal content I use nnedi3 when upscaling since I don't want the denoising&co the machine learning resizers create. (Sometimes I use FeMaSR or RealESRGAN with general model.)

Cu Selur

So I shall try RealESRGAN but remembered it somehow used to create a little bit of artificial footage. Maybe it has changed recently. And now my AVI clips are much cleaner to maybe it would also influence the way how this machine learning app. works.

[Al74]

send you a new link

As written to PM, thank you. Tomorrow I shall try different settings. Great you have fixed the issue with FineDeHalo filter.

Hi Selur,
Today I have played with dev version of Hybrid and must say that you have fixed all the problems I have encoutered applying filters I wanted to use. That is why I asked you about certain things in the past.
So to sum up, now I can use Filter queue using the followng filters: YAHR, FineDeHalo and QTGMC in Denoise option. Moreover, I can also use Preset Very slow and in the case of Match Preset and Match Preset 2 I can also use Very slow settings plus Source matching 2x refined! That is really nice. And probably even better path than described by me a few pages above.
PS In Filter queue I have tried to add QTGMC filter twice and to my suprise the output is sharper than in the case of single use of it. Why? I have thought it would be the opposite since usually denoiseing leads to softening of the image.
Please find enclosed the samples:
https://we.tl/t-cS8R62RYgH

Today I have been playing, choosing different denoising filters to remove noise and other strange artifacts from my immortal AVI files... I have recalled that filtering these files caused some loss of dynamic range. Hence, I have performed the following test: Namely, I have taken an AVI filter, and then in Hybrid, using the Filter Queue option, I have set the following filter order: YAHR 1/16, FineDeHalo, and Denoise by QTGMC filter. I have chosen FFV1 and mkv and rendered them. I have compared it with the original AVI, and the rendered file has a worse dynamic range; the highlights have been blown, and in the shadows, the details have gone. Why?
Then, using the Shutter Encoder, I changed the AVI to FFV1 and repeated the procedure. It turns out that the output is very similar to the original AVI file from a dynamic range standpoint. Hence, the question is: why has using the original AVI file and then different filters led to the loss of dynamic range?
Moreover, I have taken the original avi file, and in Hybrid, I have converted it to an FFV1 file without applying any filters, and again, I have repeated the procedure. The result was like in the case of AVI and then Hybrid filters.
Hence, it seems that to keep the original dynamic range, using for instance the Shutter encoder is crucial. Maybe other methods are also possible, but this step is just necessary.
Has anybody observed the same issue?
Below, I attach the file to play with or just look at.
https://we.tl/t-p7IBF2nNRq

[Selur]

No clue what you understand under "dynamic range".
Comparing 'MVI_5394.AVI' with 'MVI_5394 Hybrid FFV1.mkv' in vsViewer using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libhistogram.dll")
# source: 'C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 640x480, fps: 30, scanorder: progressive, yuv luminance scale: full, matrix: 470bg
# Loading C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv using LWLibavSource
clip1 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394.AVI", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
clip1 = core.hist.Levels(clip=clip1)
clip2 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394 Hybrid FFV1.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
clip2 = core.hist.Levels(clip=clip2)
clip = core.std.StackHorizontal([clip1, clip2])
# Output
clip.set_output()

I don't see the problem:

the levels seem to be identical.
Same when using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libhistogram.dll")
# source: 'C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 640x480, fps: 30, scanorder: progressive, yuv luminance scale: full, matrix: 470bg
# Loading C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv using LWLibavSource
clip1 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394.AVI", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
#clip1 = core.hist.Levels(clip=clip1)
clip1 = core.hist.Color2(clip=clip1)
clip2 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394 Hybrid FFV1.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
#clip2 = core.hist.Levels(clip=clip2)
clip2 = core.hist.Color2(clip=clip2)
clip = core.std.StackHorizontal([clip1, clip2])
# Output
clip.set_output()

=> atm. it seems like a problem with your comparison method.
Wild guess: Could be that whatever you use during the comparison might not respect "luminance scale: full", but assue luminance scale: limited.

Cu Selur

[Al74]

No clue what you understand under "dynamic range".
Comparing 'MVI_5394.AVI' with 'MVI_5394 Hybrid FFV1.mkv' in vsViewer using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libhistogram.dll")
# source: 'C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 640x480, fps: 30, scanorder: progressive, yuv luminance scale: full, matrix: 470bg
# Loading C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv using LWLibavSource
clip1 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394.AVI", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
clip1 = core.hist.Levels(clip=clip1)
clip2 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394 Hybrid FFV1.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
clip2 = core.hist.Levels(clip=clip2)
clip = core.std.StackHorizontal([clip1, clip2])
# Output
clip.set_output()

I don't see the problem:

the levels seem to be identical.
Same when using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libhistogram.dll")
# source: 'C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 640x480, fps: 30, scanorder: progressive, yuv luminance scale: full, matrix: 470bg
# Loading C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv using LWLibavSource
clip1 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394.AVI", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
#clip1 = core.hist.Levels(clip=clip1)
clip1 = core.hist.Color2(clip=clip1)
clip2 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394 Hybrid FFV1.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
#clip2 = core.hist.Levels(clip=clip2)
clip2 = core.hist.Color2(clip=clip2)
clip = core.std.StackHorizontal([clip1, clip2])
# Output
clip.set_output()

=> atm. it seems like a problem with your comparison method.
Wild guess: Could be that whatever you use during the comparison might not respect "luminance scale: full", but assue luminance scale: limited.

Cu Selur

I have compared the files in DaVinci Resolve side by side and even when playing them one after another in VLC, the difference was clear. I include the files once again, and also a short clip rendered from DaVinci and add a more infomation about the clips.
BTW, dynamic range - how much information is in the shadows and highlights.
PS Compare MVI_5394 YAHR 116 FineDehalo+QTGMC  with  MVI_5394_FFV1 YAHR 116 FineDehalo+QTGMC
The first file, original AVI one,  was directly put into Hybrid and processed as described above and the second, original AVI one, was processed using Shuter Encoder to create FFV1 file and then it was put into Hybrid and processed as described above. Look at the railroad sleepers ahead and below the train (shadows) and the boat on the left (quite bright). In the clip which was directly put to Hybrid the boat just looks as a totally blown away and the railroad sleepers are also much darker.
To make it clearer, I have changed the names of the clips and added them to DaVInci to show as as well the file AVI changed into FFV1 both in Shutter Encoder and Hybrid. And FFV1 files from both method look the same BUT after Hybrid filter application the difference starts to appear. I have aobserved it one year ago using ProRes output clips - head of a few people was much brighter than on the original file.
So looking at the mp4 from DaVinci, the order of short films is as follows:
1. MVI_5394_ AVI to FFV1 in Shutter Encoder
2. MVI_5394 AVI to FFV1 in Hybrid
Summary: The look the same.
3. MVI_5394_ AVI to FFV1 in Shutter Encoder then to Hybrid +  YAHR+DeHalo+QTGMC
4. MVI_5394 AVI directly  to Hybrid + YAHR+DeHalo+QTGMC
Summary: Clip no 4 has blown highlights and much darker shadows=much worse dynamic range.
I also include original AVI file to play with.
https://we.tl/t-O2eHmnTosL

---

No clue what you understand under "dynamic range".
Comparing 'MVI_5394.AVI' with 'MVI_5394 Hybrid FFV1.mkv' in vsViewer using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libhistogram.dll")
# source: 'C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 640x480, fps: 30, scanorder: progressive, yuv luminance scale: full, matrix: 470bg
# Loading C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv using LWLibavSource
clip1 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394.AVI", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
clip1 = core.hist.Levels(clip=clip1)
clip2 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394 Hybrid FFV1.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
clip2 = core.hist.Levels(clip=clip2)
clip = core.std.StackHorizontal([clip1, clip2])
# Output
clip.set_output()

I don't see the problem:

the levels seem to be identical.
You compared the wrong files, these are identical.
Same when using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libhistogram.dll")
# source: 'C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 640x480, fps: 30, scanorder: progressive, yuv luminance scale: full, matrix: 470bg
# Loading C:\Users\Selur\Desktop\MVI_5394 Hybrid FFV1.mkv using LWLibavSource
clip1 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394.AVI", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
#clip1 = core.hist.Levels(clip=clip1)
clip1 = core.hist.Color2(clip=clip1)
clip2 = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/MVI_5394 Hybrid FFV1.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
#clip2 = core.hist.Levels(clip=clip2)
clip2 = core.hist.Color2(clip=clip2)
clip = core.std.StackHorizontal([clip1, clip2])
# Output
clip.set_output()

=> atm. it seems like a problem with your comparison method.
Wild guess: Could be that whatever you use during the comparison might not respect "luminance scale: full", but assue luminance scale: limited.

Cu Selur

I have compared the files in DaVinci Resolve side by side and even when playing them one after another in VLC, the difference was clear. I include the files once again, and also a short clip rendered from DaVinci and add a more infomation about the clips.
BTW, dynamic range - how much information is in the shadows and highlights.
PS Compare MVI_5394 YAHR 116 FineDehalo+QTGMC  with  MVI_5394_FFV1 YAHR 116 FineDehalo+QTGMC
The first file, original AVI one,  was directly put into Hybrid and processed as described above and the second, original AVI one, was processed using Shuter Encoder to create FFV1 file and then it was put into Hybrid and processed as described above. Look at the railroad sleepers ahead and below the train (shadows) and the boat on the left (quite bright). In the clip which was directly put to Hybrid the boat just looks as a totally blown away and the railroad sleepers are also much darker.
To make it clearer, I have changed the names of the clips and added them to DaVInci to show as well the file AVI changed into FFV1 both in Shutter Encoder and Hybrid. And FFV1 files from both methods look the same BUT after the Hybrid filter application, the difference starts to appear. I observed it one year ago using ProRes output clips - the heads of a few people were much brighter than on the original file.
So looking at the mp4 from DaVinci, the order of short films is as follows:
1. MVI_5394_ AVI to FFV1 in Shutter Encoder
2. MVI_5394 AVI to FFV1 in Hybrid
Summary: They look the same.
3. MVI_5394_ AVI to FFV1 in Shutter Encoder then to Hybrid +  YAHR+DeHalo+QTGMC
4. MVI_5394 AVI directly  to Hybrid + YAHR+DeHalo+QTGMC
Summary: Clip no 4 has blown highlights and much darker shadows=much worse dynamic range.
I also include original AVI file to play with.
https://we.tl/t-O2eHmnTosL

[Selur]

just looking at the meta data for the files (+histogram of source file):
"MVI_5394.avi" is 4:2:2 with no information about luma scale. (Since the source if mjpg, Hybrid will assume pc scale, which looking at the histogram is correct)
"MVI_5394 AVI to FFV1 in Hybrid.mkv" is 4:2:0 with pc (=full=0-255) luma scale.
"MVI_5394_ AVI to FFV1 in Shutter Encoder.mkv" is 4:2:2 with tv (=limited=16-235) luma scale. (also has bt470 color matrix signaling)
"MVI_5394 AVI directly to Hybrid + YAHR+DeHalo+QTGMC.mkv" is 4:2:0 with no luma scale signaling.
=> I agree that those files differ.
Color difference are to be expected when using different luma scaling and differ color spaces.
As a side note: no clue whether VLC respects the luma scale info (I hopw so) and what it assumes if it isn't present.

So my conclusion is that:
a. Hybrid is correct in using pc scale (you can convert pc to tv scale by limiting to scaling).
b. Hybrid is changed the color sampling based on the configuration you set in the encoder.
=> nothing to fix or look into for me.
Looking at the Histogram of MVI_5394_ AVI to FFV1 in Shutter Encoder.mkv

and MVI_5394.AVI:

you can clearly see that:
a. MVI_5394 isn't limited scale (otherwise the purple borders would be empty)
b. Shutter encoder scaled the color to fit into tv scale, which Hybrid does not unless it is told too.


Cu Selur

[Al74]

just looking at the meta data for the files (+histogram of source file):
"MVI_5394.avi" is 4:2:2 with no information about luma scale. (Since the source if mjpg, Hybrid will assume pc scale, which looking at the histogram is correct)
"MVI_5394 AVI to FFV1 in Hybrid.mkv" is 4:2:0 with pc (=full=0-255) luma scale.
"MVI_5394_ AVI to FFV1 in Shutter Encoder.mkv" is 4:2:2 with tv (=limited=16-235) luma scale. (also has bt470 color matrix signaling)
"MVI_5394 AVI directly  to Hybrid + YAHR+DeHalo+QTGMC.mkv" is 4:2:0 with no luma scale signaling.
=> I agree that those files differ.
Color difference are to be expected when using different luma scaling and differ color spaces.
As a side note: no clue whether VLC respects the luma scale info (I hopw so) and what it assumes if it isn't present.

So my conclusion is that:
a. Hybrid is correct in using pc scale (you can convert pc to tv scale by limiting to scaling).
b. Hybrid is changed the color sampling based on the configuration you set in the encoder.
=> nothing to fix or look into for me.
Looking at the Histogram of MVI_5394_ AVI to FFV1 in Shutter Encoder.mkv

and MVI_5394.AVI:

you can clearly see that:
a. MVI_5394 isn't limited scale (otherwise the purple borders would be empty)
b. Shutter encoder scaled the color to fit into tv scale, which Hybrid does not unless it is told too.


Cu Selur

Hmm, but when looking at them, the difference is visible. So what do you suggest to do in Hybrid not to limit dynamic range vs original. The file is not maybe the most representative but one can easily see the difference. During the weekend I shall try to use maybe other settings in Hybrid if possible and maybe find another file to show such a behaviour.
MVI_5394.AVI  and MVI_5394_ AVI to FFV1 in Shutter Encoder look the same.
The only difference is in the case of AVI to Hybrid and filter use and mkv render, the same is in the case of ProRes. I must find other clips and maybe another way of treatment but which one..

[Selur]

Please, stop using full-quotes, it really makes reading your posts tiring.
If you want to keep the color sampling to 4:2:2 while using FFV1, configure 'Color space' to 'i422'.
If you want to convert pc to tv scale, by scaling you can use 'Filtering->Vapoursynth->Color->Misc->Range conversion' and set it to 'full to limited (pc ->tv)':

or use 'Filtering->Vapoursynth->Basic->Levels' set 'In(min/max)' to '0/255', set 'Out (min/max)' to '16/235' and set 'Range signaling' to 'limited'.

(there are other ways to archive this in Hybrid, but these are the easiest)

Cu Selur