[HELP] M2V to ProRes - File plays back way too fast

[Selur]

Okay, had a look at the m2v file and the mov.
1st thing I noticed: the m2v is progressive and some idiot added interlaced subtitles over the original picture. So applying IVTC or Deinterlacing both would be the wrong approach. What I would recommend would be to deinterlace only the parts where the CG text was layed over the clip and keep the rest as it is. (Alternatively deinterlace everything and live with the additional smoothing and artifacts.)
2nd the source has 33912 frames, the mov only has 4373 so that explains the play-length difference

So then I tried the following:
A.
1. enabling 'Filtering->(De-)Interlace/Telecine->Deinterlace/Telecine Settings->Overwrite input scan type' and setting it to 'bottom field first' and changing the Deinterlacer to 'Yadif -fast'.
2. encoding to H.264 using x264 (preset ultrafast), just to see whether the frame count of the output would differ from the input (more than a few frames, a few might be due to rounding errors in the calculations)
Result A: encoded 33912 frames (playback length 18:52min)

B.
1. enabling 'Filtering->(De-)Interlace/Telecine->Deinterlace/Telecine Settings->Overwrite input scan type' and setting it to 'bottom field first' and changing the Deinterlacer to 'Yadif -fast'.
Result A: encoding speed ~71fps, most cpu usage goes to the deinterlacer
2. encoding to ProRes (prores_ks -profile:v 3; hope using profile 3 while sticking with an SD resolution doesn't cause problems), just to see whether the frame count of the output would differ from the input (more than a few frames, a few might be due to rounding errors in the calculations)
Result B: encoded 33912 frames
Strangely something was off with the playback length which was 23:07min.
Frame rate : 24.455 FPS
Minimum frame rate : 14.981 FPS
Maximum frame rate : 30.000 FPS
Problem is that 29.97 isn't a valid frame rate for ProRes at that resolution&profile (https://www.apple.com/final-cut-pro/docs..._Paper.pdf), but since the frame rate is just a flag I ignore that.

Conclusion 1: the deinterlacing isn't causing the frame count difference between output and input.

C.
My next try was to adjust the Vapoursynth settings to the settings you used (aside from the deinterlaced where I stuck with QTGMC, since IVTC really would be a bad idea).
One thing I noticed is that there seems to be a bug in your old version which causes:

havsfunc.LSFmod(input=clip, Smode=4, Smethod=2, Lmode=22, soft=-2, edgemaskHQ=True)

Problem is soft=-2 is wrong.
If should be:

havsfunc.LSFmod(input=clip, Smode=4, Smethod=2, Lmode=22, undershoot=2, edgemaskHQ=True)

(I guess)
So all in all I ended up with:

# Imports
import os
import sys
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("I:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
Dllref = ctypes.windll.LoadLibrary("I:/Hybrid/64bit/vsfilters/Support/OpenCL.dll")
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DebandFilter/Flash3kDeband/flash3kyuu_deband.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/TTempSmooth/TTempSmooth.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/DCTFilter.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DeblockFilter/Deblock/Deblock.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/fft3dfilter.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/EEDI3.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/temporalsoften.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/d2vSource/libd2vsource.dll")
# Import scripts
import mvsfunc
import muvsfunc
import edi_rpow2
import havsfunc
# Loading C:\Users\Selur\Desktop\Episode 1 16.9.M2V using D2VSource
clip = core.d2v.Source(input="E:/Temp/m2v_d3896d070973f6089c36d704cc1fe14d_41.d2v", rff=False)
# making sure input color matrix is set as unspec
clip = core.resize.Point(clip, matrix_in_s="unspec",range_s="limited")
# making sure frame rate is set to 30000/1001
clip = core.std.AssumeFPS(clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# setting field order to what QTGMC should assume
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=1)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False)
clip = clip[::2]
# denoising using MCTemporalDenoise
clip = havsfunc.MCTemporalDenoise(i=clip, settings="very low")
# Debanding using Flash3kDB
clip = core.f3kdb.Deband(clip, keep_tv_range=True)
clip = havsfunc.LSFmod(input=clip, Smode=4, Smethod=2, Lmode=22, undershoot=2, soft=False, edgemaskHQ=True)
# DeHaloing using DeHalo_Alpha
clip = havsfunc.DeHalo_alpha(clip)
# resizing clip to 2048x1366
# resizing using NNEDI3
clip = edi_rpow2.nnedi3_rpow2(clip=clip, rfactor=4, qual=1)
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=2048, h=1366, kernel="lanczos", interlaced=False, interlacedd=False)
# Anti Aliasing using NNedia3aa
clip = muvsfunc.nnedi3aa(clip)
# adjusting output color from: YUV420P16 to YUV422P10 for ProResModel (i422)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV422P10, range_s="limited")
# Output
clip.set_output()

which should be near enough to your setting to get a bit further.
Side notes:
- not sure whether ProRes supports a different PAR then 1:1. (resizing to compensate the PAR would result in a resolution of 2048x1152)
- encoding speed of ~9 fps :/
Result C: encoded 33912 frames, file size ~41GB, frame before re-muxing with ffmbc is a cfr of 29.97fps, same for the output of ffmbc, so the strange frame rate before really way due to the violation of the profile.

Conclusion 2: will look into LSFmod since there seems to be something screwed/off with the settings in the current version of Hybrid.
Conclusion 3: works properly here

=> all in all I sadly have no clue what is causing the extrem drop in frames with your encode, my current guess is that something causing problems for ffmpeg (changing the temp folder might help)

Cu Selur