Chapter selection → *.MKV ..

[ToiletDuck]

Howdy,

First of, this issue happened with dvd source (*.vob/ifo) before, but has been resolved in the meanwhile.

Now , the same happends with mkv files created / extracted from dvd source !  
When selecting a range, hybrid ignores it and does the complete media file !

Acording the debug, it selects chap 1-2 (3 min) ,even though i choose 1-1 (1:30 min), but finaly does the complete mumbo jumbo when i hit the start button ◔̯◔ !

Proof is in the pudding/debug   ↓




Ta Ta
Toilet

[Selur]

Looking at the log and the Vapoursynth script:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import site
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# Adding torch dependencies to PATH
path = site.getsitepackages()[0]+'/torch_dependencies/bin/'
ctypes.windll.kernel32.SetDllDirectoryW(path)
path = path.replace('\\', '/')
os.environ["PATH"] = path + os.pathsep + os.environ["PATH"]
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/KNLMeansCL/KNLMeansCL.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# defining beforeDeinterlace-function - START
def beforeDeinterlace(clip):
  clip = core.std.Transpose(clip)
  return clip
# defining beforeDeinterlace-function - END

# defining beforeQTGMCFilter-function - START
def beforeQTGMCFilter(clip):
  clip = core.std.Transpose(clip)
  return clip
# defining beforeQTGMCFilter-function - END

# Import scripts
import G41Fun
import havsfunc
# source: 'F:\Temp-\AIRWOLF\SEIZOEN 4\0001.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 720x576, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: top field first
# Loading F:\Temp-\AIRWOLF\SEIZOEN 4\0001.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/Temp-/AIRWOLF/SEIZOEN 4/0001.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer (to 470bg), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
  clip = core.std.SetFrameProps(clip, _Transfer=5)
# Setting color primaries info (to 5), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
  clip = core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2) # tff
# cutting from frame 0 to 2264  - WARNING: This might cause synch issues
clip = core.std.Trim(clip=clip, first=0, last=2264)
clip = beforeDeinterlace(clip)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Slower", TFF=True) # new fps: 25
# Making sure content is preceived as frame based
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
clip = clip[::2] # selecting previously even frames
# adjusting color space from YUV420P8 to RGB24 for vsLevels
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
# Color Adjustment using Levels on RGB24 (8 bit)
clip = core.std.Levels(clip=clip, min_in=0, max_in=255, min_out=0, max_out=255)
# adjusting color space from RGB24 to YUV444P8 for vsRemoveGrain
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, matrix_s="470bg", range_s="limited")
# removing grain using RemoveGrain
clip = core.rgvs.RemoveGrain(clip=clip, mode=24)
# denoising using KNLMeansCL
clip = havsfunc.KNLMeansCL(clip=clip, d=0, a=8, s=2, h=1.00)
clip = beforeQTGMCFilter(clip)
# Denoising using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Placebo", InputType=3, TR2=2, Sharpness=0.2, TFF=False, SourceMatch=2, Lossless=0, EZDenoise=0.35, NoisePreset="Fast", MatchPreset="Fast", MatchPreset2="Fast", MatchEnhance=1.00)
# sharpening using DetailSharpen
clip = G41Fun.DetailSharpen(clip=clip, sstr=2.000, ldmp=0.250, mode=0, med=True)
from vsrealesrgan import realesrgan as RealESRGAN
# adjusting color space from YUV444P8 to RGBH for vsRealESRGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# resizing using RealESRGAN
clip = RealESRGAN(clip=clip, model=5, device_index=0, trt=True, trt_cache_path=r"F:\Temp-\AIRWOLF\SEIZOEN 4", denoise_strength=0.25) # 2880x2304
# resizing 2880x2304 to 1920x1080
# adjusting resizing
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, range_s="limited")
clip = core.fmtc.resample(clip=clip, w=1920, h=1080, kernel="sinc", interlaced=False, interlacedd=False)
# adjusting output color from: RGBS to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

Directly before the deinterlacing:

clip = core.std.Trim(clip=clip, first=0, last=2264)

is properly added.

btw. youe script is badly wrong.

# defining beforeDeinterlace-function - START
def beforeDeinterlace(clip):
  clip = core.std.Transpose(clip)
  return clip
# defining beforeDeinterlace-function - END

# defining beforeQTGMCFilter-function - START
def beforeQTGMCFilter(clip):
  clip = core.std.Transpose(clip)
  return clip
# defining beforeQTGMCFilter-function - END

You should add the Transpose once before beforeQTGMCFilter and then directly after it. Not before the deinterlacing!


You stopped the processing after 376 frames,...

2024.02.13 - 14:55:45_Windows 11 Version 23H2 (64bit)_2024.01.14.1 - level 9: x264 output: 376 frames: 13.72 fps, 3257.13 kb/s
2024.02.13 - 14:55:45_Windows 11 Version 23H2 (64bit)_2024.01.14.1 - level 9: on_stopAllPushButton_click

this does not look like anything is wrong, with Hybrid.

Cu Selur

[ToiletDuck]

Looking at the log and the Vapoursynth script:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import site
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# Adding torch dependencies to PATH
path = site.getsitepackages()[0]+'/torch_dependencies/bin/'
ctypes.windll.kernel32.SetDllDirectoryW(path)
path = path.replace('\\', '/')
os.environ["PATH"] = path + os.pathsep + os.environ["PATH"]
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/KNLMeansCL/KNLMeansCL.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# defining beforeDeinterlace-function - START
def beforeDeinterlace(clip):
  clip = core.std.Transpose(clip)
  return clip
# defining beforeDeinterlace-function - END
# defining beforeQTGMCFilter-function - START
def beforeQTGMCFilter(clip):
  clip = core.std.Transpose(clip)
  return clip
# defining beforeQTGMCFilter-function - END
# Import scripts
import G41Fun
import havsfunc
# source: 'F:\Temp-\AIRWOLF\SEIZOEN 4\0001.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 720x576, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: top field first
# Loading F:\Temp-\AIRWOLF\SEIZOEN 4\0001.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/Temp-/AIRWOLF/SEIZOEN 4/0001.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer (to 470bg), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
  clip = core.std.SetFrameProps(clip, _Transfer=5)
# Setting color primaries info (to 5), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
  clip = core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2) # tff
# cutting from frame 0 to 2264  - WARNING: This might cause synch issues
clip = core.std.Trim(clip=clip, first=0, last=2264)
clip = beforeDeinterlace(clip)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Slower", TFF=True) # new fps: 25
# Making sure content is preceived as frame based
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
clip = clip[::2] # selecting previously even frames
# adjusting color space from YUV420P8 to RGB24 for vsLevels
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
# Color Adjustment using Levels on RGB24 (8 bit)
clip = core.std.Levels(clip=clip, min_in=0, max_in=255, min_out=0, max_out=255)
# adjusting color space from RGB24 to YUV444P8 for vsRemoveGrain
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, matrix_s="470bg", range_s="limited")
# removing grain using RemoveGrain
clip = core.rgvs.RemoveGrain(clip=clip, mode=24)
# denoising using KNLMeansCL
clip = havsfunc.KNLMeansCL(clip=clip, d=0, a=8, s=2, h=1.00)
clip = beforeQTGMCFilter(clip)
# Denoising using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Placebo", InputType=3, TR2=2, Sharpness=0.2, TFF=False, SourceMatch=2, Lossless=0, EZDenoise=0.35, NoisePreset="Fast", MatchPreset="Fast", MatchPreset2="Fast", MatchEnhance=1.00)
# sharpening using DetailSharpen
clip = G41Fun.DetailSharpen(clip=clip, sstr=2.000, ldmp=0.250, mode=0, med=True)
from vsrealesrgan import realesrgan as RealESRGAN
# adjusting color space from YUV444P8 to RGBH for vsRealESRGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# resizing using RealESRGAN
clip = RealESRGAN(clip=clip, model=5, device_index=0, trt=True, trt_cache_path=r"F:\Temp-\AIRWOLF\SEIZOEN 4", denoise_strength=0.25) # 2880x2304
# resizing 2880x2304 to 1920x1080
# adjusting resizing
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, range_s="limited")
clip = core.fmtc.resample(clip=clip, w=1920, h=1080, kernel="sinc", interlaced=False, interlacedd=False)
# adjusting output color from: RGBS to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

Directly before the deinterlacing:

clip = core.std.Trim(clip=clip, first=0, last=2264)

is properly added.
btw. youe script is badly wrong.

# defining beforeDeinterlace-function - START
def beforeDeinterlace(clip):
  clip = core.std.Transpose(clip)
  return clip
# defining beforeDeinterlace-function - END
# defining beforeQTGMCFilter-function - START
def beforeQTGMCFilter(clip):
  clip = core.std.Transpose(clip)
  return clip
# defining beforeQTGMCFilter-function - END

You should add the Transpose once before beforeQTGMCFilter and then directly after it. Not before the deinterlacing!

I would like to think my script is to the point .. yes..  
But at the same time you are confusing me , when you write that my script is wrong  ..

You still can remember why exactly i put the "line" where it should belong don't you, my Majesty ?
I mean, you've suggested it exactly as you read it in the script .. !?

i.e: → "Vertico Line Noise" ←  ..

If i change it back like you suggested now , and put it B4 QTGMCfilter & after (B4 Cnr2), the Vertical Line Noise is Back with (ง ͠° ͟ل͜ ͡°)ง A VENGEANCE  !!!
Yep, the noise is all over the place again when i do that..

So again, why would i put it before & after QTGMCfilter all of a sudden ?? 

That doesn't solve the issue with the chapter selection .. fyi ..

You stopped the processing after 376 frames,...

2024.02.13 - 14:55:45_Windows 11 Version 23H2 (64bit)_2024.01.14.1 - level 9: x264 output: 376 frames: 13.72 fps, 3257.13 kb/s
2024.02.13 - 14:55:45_Windows 11 Version 23H2 (64bit)_2024.01.14.1 - level 9: on_stopAllPushButton_click

this does not look like anything is wrong, with Hybrid.
Cu Selur

Ofcourse i have aborted the job, i don't intend to let hybrid process an full media file that would take several hours (ESRgan upscaling) !!

I know, the totall amount of frames matches with the duration of the chapter wich is 90 seconds...
Yet.. either debug is lying to both of us !!  Or this is yet another "global profile" thingie  .. again !!  

Except, loading a re-saved profile didn't solved it .. this time !  
that having said, restarting hybrid (using defaults setting) solved it ... 

Except for loading a profile, i started hybrid, set paths to source , checked auto rename file , choosed a container format, loaded the media file, selected chapter 1-1 .. and hybrid complied .. this time..

Hence why i suspect a profile issue  .. again ಠ~ಠ  !?

 

Ta Ta
Toilet(°^°)

---

EDIT:  Important to know..  To exclude any particular hybrid config setting that may cause this.. i have loaded two instances of Hybrid..  One with default settings and the other wiht my loaded custom saved global preset.

I have compared all vital settings side by side , and have found NOT one box checked on or off differently ..  Also in Vapoursynth, all config settings are pretty much the same.  Apart from the settings from the filter section ofcourse , wich has nothing to do with chapter selection imo ...

So it seems, loaded profiles affect hybrid in such way it sabotages hybrid functioning  !


Ta Ta
TD

[Selur]

full quote again: I asked you to not do that
about the debug output: if you abort the debug output before the issue is present, how do you expect the debug output to be helpful?
about the filtering: before I suggested adding the transpose-calls to rotate the image, so that QTGMC(Filter) will detect these as remnance of deinterlacing and try to remove them (the later transpose is used to rotate the image back). You are applying both QTGMC and QTGMC on the image,...

=> without a debug output that actually shows the problem I can't reproduce the problem here and thus have no way to fix the problem.

Cu Selur

[ToiletDuck]

full quote again: I asked you to not do that   

ehh? Say what    ?

about the debug output: if you abort the debug output before the issue is present, how do you expect the debug output to be helpful?

i ONLY the LONELY abort when the encode process has started... The Chapter selection process happends before the encode process .. is it not ?  
So therefor, that portion should be already been recorded / covered in the debug file .. right    ?  Since debugging recording works in realtime from the looks of it.. 

about the filtering: before I suggested adding the transpose-calls to rotate the image, so that QTGMC(Filter) will detect these as remnance of deinterlacing and try to remove them (the later transpose is used to rotate the image back). You are applying both QTGMC and QTGMC on the image,...
=> without a debug output that actually shows the problem I can't reproduce the problem here and thus have no way to fix the problem.
Cu Selur

Right, i get that the line is used to rotate the image up side down.. 

What do you actually mean by "applying to both QTGMC & QTGMC !?  
Do you mean iam using that filter for cleaning up & rotating at the same time?  Meaning i should put that transpose line in front of Cnr2 instead of QTGMC now ?
That it ?  Is that what you want me to do ?

If so, i particulary placed  and for an very GOOD REASON too the transpose before QTGMCfilter Selur..  

When i do it like you now suggests, and put it behind QTGMC i get noise between scene changes !!!
CHECK THIS ↓ out 


 
Only when i put transpose →before← QTGMCfilter , i get rid of that phenomenon !
So again, the line "transpose" works only correct when add before "Deinterlace" and Before "QTGMCfilter" in this method .. ¯\_(ツ)_/¯


SEeee   ↓ 



Regarding the chapter selection issue, i'll post a full debug later on from a small test sample as by your request My benevolent Royal Majesty Sir Selur ☝(°ロ°)

Ta Ta
Toilet

[Selur]

i ONLY the LONELY abort when the encode process has started... The Chapter selection process happends before the encode process .. is it not ?
So therefor, that portion should be already been recorded / covered in the debug file .. right Huh ? Since debugging recording works in realtime from the lo

Like I wrote the debug output shows everything is correctly done by Hybrid.

About the positioning of the transpose, I can only say: that is not how it works here on the sample you provided. There it works like I suggested, without issues.
Also on just a theoretical approach transposing before deinterlacing does not make sense for deinterlacing.

=> If it works for you that is fine.
Since I can't reproduce your issues, from the looks of it you can't reproduce it either or share details for me to reproduce it, I count the issue as non-existant.

Cu Selur

[ToiletDuck]

i ONLY the LONELY abort when the encode process has started... The Chapter selection process happends before the encode process .. is it not ? 
So therefor, that portion should be already been recorded / covered in the debug file .. right  Huh  ?  Since debugging recording works in realtime from the lo

Like I wrote the debug output shows everything is correctly done by Hybrid.

If thatis so, the debug is bugged too and is lying to both of us imo ..  
Since and again, for some reason chapter selecction doesn't trigger properly , AFTER i've loaded a global profile .. !

About the positioning of the transpose, I can only say: that is not how it works here on the sample you provided. There it works like I suggested, without issues.
Also on just a theoretical approach transposing before deinterlacing does not make sense for deinterlacing.
=> If it works for you that is fine.
Since I can't reproduce your issues, from the looks of it you can't reproduce it either or share details for me to reproduce it, I count the issue as non-existant.
Cu Selur

Now you've lost me there, because transpose b4 deinterlace is actually what you have suggested back there the first time !?? 

Quote of the day ↓

How do i proceed with your approach ?
Probebly have to add custom script to be able to do a back flip with the clip, right ?

Correct, you have to add two custom additions containing:

clip = core.std.Transpose(clip)

, one before and one after the deinterlacing. (That is my I linked to the scipt I used.)
Cu Selur

  So, didn't you say right there "BEFORE & AFTER → Deinterlace !??  .. And you were right back there...  
Also, the pictures i just have showned / posted in this thread proves it , doesn't it ?


Ta ta
TD