Selur's Little Message Board

Hello,

So I have a M2V file that I'm upresing and sending out to ProRes. All my filters work just fine, but I'm pretty positive that the M2V file has 3:2 pulldown applied and so I use VITVC to rid the file of it. When I get it back, the 22 or 24 minute episode suddenly becomes 18 minutes. All the frames are there, as far as I can tell, it's just way too fast.

I'm pretty sure it's a VFR issue and I tried clicking CFR Output in the Config tab, but that doesn't seem to do anything.

Mac OS Sierra (10.12.6) - Hybrid 2018.08.05.1

I can send a sample video privately if needed but the Debug and Report text files are attached to this post. Let me know if I can offer up anymore information.

Thank you for your help.

a. try using a temp folder with a normal path. "/var/folders/tg/7dxp1cw503x4wrv507ypxqm40000gn/T" looks like some temporary path.
b. Does the vapoursynth preview work properly?
The Vapoursynth script looks fine.

Code:
# Imports

import os

import sys

import vapoursynth as vs

core = vs.get_core()

# Import scripts folder

scriptPath = '/Applications/Hybrid.app/Contents/MacOS/vsscripts'

sys.path.append(os.path.abspath(scriptPath))

# Import scripts

import mvsfunc

import muvsfunc

import edi_rpow2

import hysteria

import havsfunc

# Loading /Volumes/Ghost In The Shell Fix/IGPX Discotek/IGPX for Jose (1)/Episode 1/Episode 1 16.9.M2V using D2VSource

clip = core.d2v.Source(input="/var/folders/tg/7dxp1cw503x4wrv507ypxqm40000gn/T/09_45_20_5310.d2v", rff=False)

# making sure input color matrix is set as unspec

clip = core.resize.Point(clip, matrix_in_s="unspec",range_s="limited")

# making sure frame rate is set to 30000/1001

clip = core.std.AssumeFPS(clip, fpsnum=30000, fpsden=1001)

# Setting color range to TV (limited) range.

clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

clip = core.vivtc.VFM(clip=clip, order=0, mode=1)

clip = core.vivtc.VDecimate(clip=clip)

# denoising using MCTemporalDenoise

clip = havsfunc.MCTemporalDenoise(i=clip, settings="very low")

# Debanding using Flash3kDB

clip = core.f3kdb.Deband(clip, keep_tv_range=True)

clip = havsfunc.LSFmod(input=clip, Smode=4, Smethod=2, Lmode=22, soft=-2, edgemaskHQ=True)

# DeHaloing using DeHalo_Alpha

clip = havsfunc.DeHalo_alpha(clip)

 # line darkening using Hysteria

clip = hysteria.Hysteria(clip=clip,strength=2.00)

# resizing clip to 2048x1366

# resizing using NNEDI3

clip = edi_rpow2.nnedi3_rpow2(clip=clip, rfactor=4, nsize=2, nns=4, pscrn=4)

# adjusting resizing

clip = core.fmtc.resample(clip=clip, w=2048, h=1366, kernel="spline64", interlaced=False, interlacedd=False)

# Anti Aliasing using NNedia3aa

clip = muvsfunc.nnedi3aa(clip)

# adjusting output color from: YUV420P16 to YUV422P10 for ProResModel (i422)

clip = core.resize.Bicubic(clip=clip, format=vs.YUV422P10, range_s="limited")

# Output

clip.set_output()

c. The encoding call:

Code:
"/usr/local/bin/vspipe" "/var/folders/tg/7dxp1cw503x4wrv507ypxqm40000gn/T/encodingTempSynthSkript_23_57_37_7010.vpy" - --y4m | "/Applications/Hybrid.app/Contents/MacOS/ffmpeg" -y -threads 8 -f yuv4mpegpipe -i - -an -sn   -vf zscale=rangein=tv:range=tv -pix_fmt yuv422p10le -strict -1 -vsync 0   -vcodec prores_ks -profile:v 3 -vtag apch -f mov "/Volumes/Ghost In The Shell Fix/IGPX Discotek/IGPX for Jose (1)/Episode 1/Episode 1 - PAV - VITIC_23_57_37_7010_02.mov"

looks fine to me and doesn't throw an error.

Quote:I'm pretty sure it's a VFR issue and I tried clicking CFR Output in the Config tab, but that doesn't seem to do anything.

If the source doesn't have vfr time codes there is nothing to do.

Since the encding works fine and the script looks fine too, I see no reason for the output to be shorter than the source unless the deocder has some problem with the source. (Vapoursynth Preview should show whether there some frames are dropped,..)

If you send me a link to a sample which causes the problem for you via PM I can look at the source under Windows. (Don't have a mac any more, which is why I stopped offering Mac builds.)
My current guess is that the source is interlaced and not telecined and thus applying IVTC causes Hybrid to drop frames which are needed for normal playback and causes the illusion of the content played back faster.

Cu Selur

Okay, had a look at the m2v file and the mov.
1st thing I noticed: the m2v is progressive and some idiot added interlaced subtitles over the original picture. So applying IVTC or Deinterlacing both would be the wrong approach. What I would recommend would be to deinterlace only the parts where the CG text was layed over the clip and keep the rest as it is. (Alternatively deinterlace everything and live with the additional smoothing and artifacts.)
2nd the source has 33912 frames, the mov only has 4373 so that explains the play-length difference

So then I tried the following:
A.
1. enabling 'Filtering->(De-)Interlace/Telecine->Deinterlace/Telecine Settings->Overwrite input scan type' and setting it to 'bottom field first' and changing the Deinterlacer to 'Yadif -fast'.
2. encoding to H.264 using x264 (preset ultrafast), just to see whether the frame count of the output would differ from the input (more than a few frames, a few might be due to rounding errors in the calculations)
Result A: encoded 33912 frames (playback length 18:52min)

B.
1. enabling 'Filtering->(De-)Interlace/Telecine->Deinterlace/Telecine Settings->Overwrite input scan type' and setting it to 'bottom field first' and changing the Deinterlacer to 'Yadif -fast'.
Result A: encoding speed ~71fps, most cpu usage goes to the deinterlacer
2. encoding to ProRes (prores_ks -profile:v 3; hope using profile 3 while sticking with an SD resolution doesn't cause problems), just to see whether the frame count of the output would differ from the input (more than a few frames, a few might be due to rounding errors in the calculations)
Result B: encoded 33912 frames
Strangely something was off with the playback length which was 23:07min.
Frame rate : 24.455 FPS
Minimum frame rate : 14.981 FPS
Maximum frame rate : 30.000 FPS
Problem is that 29.97 isn't a valid frame rate for ProRes at that resolution&profile (https://www.apple.com/final-cut-pro/docs..._Paper.pdf), but since the frame rate is just a flag I ignore that.

Conclusion 1: the deinterlacing isn't causing the frame count difference between output and input.

C.
My next try was to adjust the Vapoursynth settings to the settings you used (aside from the deinterlaced where I stuck with QTGMC, since IVTC really would be a bad idea).
One thing I noticed is that there seems to be a bug in your old version which causes:

Code:
havsfunc.LSFmod(input=clip, Smode=4, Smethod=2, Lmode=22, soft=-2, edgemaskHQ=True)

Problem is soft=-2 is wrong.
If should be:

Code:
havsfunc.LSFmod(input=clip, Smode=4, Smethod=2, Lmode=22, undershoot=2, edgemaskHQ=True)

(I guess)
So all in all I ended up with:

Code:
# Imports

import os

import sys

import ctypes

# Loading Support Files

Dllref = ctypes.windll.LoadLibrary("I:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")

Dllref = ctypes.windll.LoadLibrary("I:/Hybrid/64bit/vsfilters/Support/OpenCL.dll")

import vapoursynth as vs

core = vs.get_core()

# Import scripts folder

scriptPath = 'I:/Hybrid/64bit/vsscripts'

sys.path.append(os.path.abspath(scriptPath))

# Loading Plugins

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DebandFilter/Flash3kDeband/flash3kyuu_deband.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/DCTFilter.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DeblockFilter/Deblock/Deblock.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/fft3dfilter.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/EEDI3.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/temporalsoften.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/scenechange.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/d2vSource/libd2vsource.dll")

# Import scripts

import mvsfunc

import muvsfunc

import edi_rpow2

import havsfunc

# Loading C:\Users\Selur\Desktop\Episode 1 16.9.M2V using D2VSource

clip = core.d2v.Source(input="E:/Temp/m2v_d3896d070973f6089c36d704cc1fe14d_41.d2v", rff=False)

# making sure input color matrix is set as unspec

clip = core.resize.Point(clip, matrix_in_s="unspec",range_s="limited")

# making sure frame rate is set to 30000/1001

clip = core.std.AssumeFPS(clip, fpsnum=30000, fpsden=1001)

# Setting color range to TV (limited) range.

clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

# setting field order to what QTGMC should assume

clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=1)

# Deinterlacing using QTGMC

clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False)

clip = clip[::2]

# denoising using MCTemporalDenoise

clip = havsfunc.MCTemporalDenoise(i=clip, settings="very low")

# Debanding using Flash3kDB

clip = core.f3kdb.Deband(clip, keep_tv_range=True)

clip = havsfunc.LSFmod(input=clip, Smode=4, Smethod=2, Lmode=22, undershoot=2, soft=False, edgemaskHQ=True)

# DeHaloing using DeHalo_Alpha

clip = havsfunc.DeHalo_alpha(clip)

# resizing clip to 2048x1366

# resizing using NNEDI3

clip = edi_rpow2.nnedi3_rpow2(clip=clip, rfactor=4, qual=1)

# adjusting resizing

clip = core.fmtc.resample(clip=clip, w=2048, h=1366, kernel="lanczos", interlaced=False, interlacedd=False)

# Anti Aliasing using NNedia3aa

clip = muvsfunc.nnedi3aa(clip)

# adjusting output color from: YUV420P16 to YUV422P10 for ProResModel (i422)

clip = core.resize.Bicubic(clip=clip, format=vs.YUV422P10, range_s="limited")

# Output

clip.set_output()

which should be near enough to your setting to get a bit further.
Side notes:
- not sure whether ProRes supports a different PAR then 1:1. (resizing to compensate the PAR would result in a resolution of 2048x1152)
- encoding speed of ~9 fps :/
Result C: encoded 33912 frames, file size ~41GB, frame before re-muxing with ffmbc is a cfr of 29.97fps, same for the output of ffmbc, so the strange frame rate before really way due to the violation of the profile.

Conclusion 2: will look into LSFmod since there seems to be something screwed/off with the settings in the current version of Hybrid.
Conclusion 3: works properly here

=> all in all I sadly have no clue what is causing the extrem drop in frames with your encode, my current guess is that something causing problems for ffmpeg (changing the temp folder might help)

Cu Selur

(23.01.2019, 21:25)Selur Wrote: [ -> ]=> all in all I sadly have no clue what is causing the extrem drop in frames with your encode, my current guess is that something causing problems for ffmpeg (changing the temp folder might help)

Cu Selur

Ok, I'm gonna try out your recommendations and try to figure out what's going on. Thank you for all your help, I think you've put me on the right track to correcting this.

g0ren

Selur

Selur

g0ren