Jobs go over 100% indefinitely

[protectron]

As stated, jobs go well over 100%. I watched one go over 200% before I cancelled it.
Debug attached.

[Selur]

Usually happens If there is a proplem with the length detection of the source. Will look at it this evening.

[Selur]

MediaInfo reports the source as:

Video
Count                                    : 392
Count of stream of this kind             : 1
Kind of stream                           : Video
Kind of stream                           : Video
Stream identifier                        : 0
StreamOrder                              : 0
ID                                       : 1
ID                                       : 1
ID in the original source medium         : 224
ID in the original source medium         : 224 (0xE0)
Unique ID                                : 1
Format                                   : MPEG Video
Format                                   : MPEG Video
Commercial name                          : MPEG-2 Video
Format version                           : Version 2
Format profile                           : Main@Main
Format settings                          : CustomMatrix / BVOP
Format settings, BVOP                    : Yes
Format settings, BVOP                    : Yes
Format settings, Matrix                  : Custom
Format settings, Matrix                  : Custom
Format_Settings_Matrix_Data              : 080808080808080808080808080808080808080808080808080808080909090808080808080808080909090A09090909090A0A0A0A0A0C0C0C0C0E0E0F111115 / 08080808080808080808080808080808080808080808080808080808080808080808080808080808080808080808080808080808080808080808080808080808
Format settings, GOP                     : Variable
Internet media type                      : video/MPV
Codec ID                                 : V_MPEG2
Codec ID/Info                            : MPEG 1 or 2 Video
Codec ID/Url                             : http://ffdshow-tryout.sourceforge.net/
Duration                                 : 1355087.066666
Duration                                 : 22 min 35 s
Duration                                 : 22 min 35 s 87 ms
Duration                                 : 22 min 35 s
Duration                                 : 00:22:35.087
Duration                                 : 00:22:41:21
Duration                                 : 00:22:35.087 (00:22:41:21)
Bit rate mode                            : VBR
Bit rate mode                            : Variable
Bit rate                                 : 5508177
Bit rate                                 : 5 508 kb/s
Maximum bit rate                         : 9800000
Maximum bit rate                         : 9 800 kb/s
Width                                    : 720
Width                                    : 720 pixels
Height                                   : 480
Height                                   : 480 pixels
Sampled_Width                            : 720
Sampled_Height                           : 480
Pixel aspect ratio                       : 0.889
Display aspect ratio                     : 1.333
Display aspect ratio                     : 4:3
Frame rate mode                          : VFR
Frame rate mode                          : Variable
Frame rate                               : 24.120
Frame rate                               : 24.120 FPS
Original frame rate                      : 29.970
Original frame rate                      : 29.970 (30000/1001) FPS
FrameRate_Original_Num                   : 30000
FrameRate_Original_Den                   : 1001
Frame count                              : 32685
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Chroma subsampling                       : 4:2:0
Bit depth                                : 8
Bit depth                                : 8 bits
Compression mode                         : Lossy
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.661
Delay                                    : 0
Delay                                    : 00:00:00.000
Delay                                    : 00:00:00:00
Delay                                    : 00:00:00.000 (00:00:00:00)
Delay, origin                            : Container
Delay, origin                            : Container
Delay_Original                           : 35999000
Delay_Original                           : 9 h 59 min
Delay_Original                           : 9 h 59 min 59 s 0 ms
Delay_Original                           : 9 h 59 min
Delay_Original                           : 09:59:59.000
Delay_Original                           : 10:02:59:00
Delay_Original                           : 09:59:59.000 (10:02:59:00)
Delay_Original_Settings                  : drop_frame_flag=0 / closed_gop=1 / broken_link=0
Delay_Original_DropFrame                 : No
Delay_Original_Source                    : Stream
Time code of first frame                 : 09:59:59:00
TimeCode_DropFrame                       : No
Time code source                         : Group of pictures header
GOP, Open/Closed                         : Open
GOP, Open/Closed                         : Open
GOP, Open/Closed of first frame          : Closed
GOP, Open/Closed of first frame          : Closed
Stream size                              : 933007480
Stream size                              : 890 MiB (96%)
Stream size                              : 890 MiB
Stream size                              : 890 MiB
Stream size                              : 890 MiB
Stream size                              : 889.8 MiB
Stream size                              : 890 MiB (96%)
Proportion of this stream                : 0.96478
Language                                 : en
Language                                 : English
Language                                 : English
Language                                 : en
Language                                 : eng
Language                                 : en
Default                                  : No
Default                                  : No
Forced                                   : No
Forced                                   : No
Buffer size                              : 229376
Original source medium                   : DVD-Video
intra_dc_precision                       : 10

since it's from a DVD it's either interlaced or telecine.

Hybrid assues it's telecine (could also be soft telecine), thus IVTC should be applied. You chose to use QTGMC (thus deinterlacing the source)
(you should overwrite the scan type to tff or bff if your source really isn't telecined, but interlaced)

for filtering:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/DCTFilter.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeblockFilter/Deblock/Deblock.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeCrawlFilter/DotKill/DotKill.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import havsfunc
# source: 'C:\Users\Michael\Desktop\Courage the Cowardly Dog [remux]\Season 1\Courage the Cowardly Dog S01E01E02 A Night at the Katz Motel. Cajun Granny Stew.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading C:\Users\Michael\Desktop\Courage the Cowardly Dog [remux]\Season 1\Courage the Cowardly Dog S01E01E02 A Night at the Katz Motel. Cajun Granny Stew.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Michael/Desktop/Courage the Cowardly Dog [remux]/Season 1/Courage the Cowardly Dog S01E01E02 A Night at the Katz Motel. Cajun Granny Stew.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer (to 470bg), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
  clip = core.std.SetFrameProps(clip, _Transfer=5)
# Setting color primaries info (to 5), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
  clip = core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
# converting interlaced to full-height progressive for filtering (vsDotKill) (using: QTGMC(preset="Fast"))
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", Lossless=2, TFF=True) # new fps: 59.94
# Making sure content is preceived as frame based
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
clip = core.dotkill.DotKillS(clip=clip)
# denoising using MCTemporalDenoise
clip = havsfunc.MCTemporalDenoise(i=clip, settings="low", ncpu=1)
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 59.94fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001)

is used.
Strange thing the output fps in the encoder is set to 29.97 <- this looks wrong.
During the encoding Hybrid assumes that the output is 1090.59 seconds long
(Frame count 32685 @ 29.97 fps)
22 min 35 s which would be 18min 10.50 sec.
Which seems to be correct assuming that Hybrid things the chapter selection is 1-4 (and thus limits the length to 00:18:10.591)

Seems like something is wrong with the frame count, frame count should be doubled due to the bobbing with QTGMC.
This explains the problem with the progress indication.
Sadly I can't reproduce the problem here.
For me both the frame count and frame rate of the output get adjusted properly.

=> I'll send you a link to my latest dev, via pm.
Can you reproduce the issue with that version.
If you can, can you share your source with me and a tell me the steps you took to get this result?
I have no clue what could causte the output frame count and rate not to get adjusted.

Cu Selur

[protectron]

Dev version did not work I'm afraid. I had deinterlace/telecine set to "off" in all cases by the way. I did not enable QTGMC or ITVC.
New debug attached.

[Selur]

I had deinterlace/telecine set to "off" in all cases by the way.

the sample you send me is soft telecined and has tons of rainbows&co.
Hybrid will still deinterlace if you use filtering that requires progressive content.

If your have an NVIDIA graphic card:
a. enable Filtering->Vapoursynth->Misc->Source->Use DGDecNV when available.
b. set "Overwrite input scan type to" "(soft) telecine"
c. if there are still some combing artifacts use either Filtering->Vapoursynth->Frame->Misc->Vinverse or Vinverse2 or Filtering->Vapoursynth->DeNoise->QTMC in mode 1.

If you have no NVIDIA graphic card:
a. set "Overwrite input scan type to" "telecine" (the source for me is already detected as telecine)
b. if there are still some combing artifacts use either Filtering->Vapoursynth->Frame->Misc->Vinverse or Vinverse2 or Filtering->Vapoursynth->DeNoise->QTMC in mode 1.

Cu Selur

Ps.: If you have a modern NVIDIA card using BasicVSR++ from the torch-Addon might also help with the denoising&co.
PPs.: going to bed now, but will try to reproduce the frame rate&coun issue tomorrow after work.
had a quick look, I see the problem: Hybrid needs to add Deinterlacing for progressive filtering (source is detected as not progressive), then doesn't reinterlace since the output encoder is set to progressive, but the frame rate and count isn't adjusted properly. => will fix tomorrow (by throwing away frames)

[Selur]

I created a dev version, which will make sure that when Deinterlacer is set to 'none', overwrite scan type will be forcefully set to 'progressive', which should fix the problem.
Send you a link via pm.

Cu Selur

[protectron]

Here’s the result of some testing:
 
When I set deinterlace handling to “TIVTC” with DotKill enabled, it processed the file at 125% and playback was fine.
When I set deinterlace handling to “none” with DotKill enabled, it processed the file at 125%, but the video introduced combing that is not present in the source file.
When I set deinterlace handling to “none” with DotKill disabled, it processed the file at 125%, but the video playback was too fast and ended before the audio.
 
While TITVC did work, it reduced the frame rate from 29.97 to 23.97, which I do not want, as parts of the video are true 30fps such as the intro and moving background animations at 19:35.

[Selur]

While TITVC did work, it reduced the frame rate from 29.97 to 23.97, which I do not want, as parts of the video are true 30fps such as the intro and moving background animations at 19:35.

Then overwrite the scan type to TFF, this way Hybrid will treat the source as interlaced and deinterlace to same rate.

125% seems to be 'correct' since the source has more frames than mediainfo reports:

Frame count                              : 32685

but opening the source in Vapoursynth/Avisynth I get 40612 frames, which naturally screws all progress indication and file size calculations.

No clue what MakeMKV did to this file, extractin the time codes, there should be one per frame (even if the source was vfr) I get 32687 time codes, thus there should be no more frames than this.

Seems like either MediaInfo changed the was it reports the frame count or makemkv changed the way it writes the headers.
I'm away from my pc for today, but may be I can figure something out tomorrow.

About deinterlacing, another thing you can try is keeping keeping the scan type as telecine or tff, apply TFM and move it above the deinterlacer in the FilterOrder.


Cu Selur

=> Frame count problem, seem to be an issue with the source filters. Using FFMS2 or BestSource as source filter seems to work with the frame count, using DGDecNV or LWLibAVSource returns 40k instead or 32k frames.