Optical flow using Hybrid -- Possible?

[markfilipak]

Not sure what you mean by 'optical flow', but Hybrid can use Inteframe for frame interpolation.
Filtering->Vapoursynth/Avisynth->Frame->Interpolation->Frame Interpolation

Abount your script: no clue why you interpolate to 120000/1001fps instead directly to 120fps and then speed up to 120fps.

Cu Selur

Thank you for replying and for all the help you've given to so many people here and elsewhere. If you ever need help, I hope you will ask me.

Good questions.

I've not seen "optical flow" defined but from the pictures I've seen it appears to be,
1, object discrimination based on raw frame-to-frame pixel-block (4x4, 6x6, 8x8, etc.) pattern matching followed by,
2, motion vector (MV) generation followed by,
3, interframe interpolation using those generated MVs.
The optical flow process interpolates between decoded pictures as opposed to interpolating between the MVs found in the undecoded source frames (which, it appears, is never done because creating an entirely new transport stream by tiddling undecoded frames is too complicated or has never been attempted or has never been thought of).

I do 24000/1001fps-to-12000/1001fps interpolation because it's exactly 1-to-5.
I don't convert the 24000/1001fps source frames directly to 120fps for 2 reasons:
1, Instead of 1-to-5, the interpolation would be 1001-to-5000: not an integer and much too complex, and
2, After interpolation, I force the resulting 12000/1001fps to 120fps by changing the time base (TB) and the presentation time stamps (PTSs), only, not the number of frames. That reduces running time by T/1.001 thereby matching the speed and running time seen in movie theaters -- all without a 2nd transcoding.

If I may anticipate an additional question, I convert 24000/1001fps to 120000/1001fps instead of to 60000/1001fps because,
1, 1-to-5 conversion (to 120000/1001fps, a 5x interpolation) works/looks better than 2-to-5 conversion (to 60000/1001fps, a 2.5x interpolation), and
2, The final 120fps files are 1/8-th the size of the 60fps files -- amazing but true! -- and
3, 60Hz TVs simply drop the odd numbered frames -- 120fps to 60fps -- thereby completely avoiding judder -- no telecine!

Here is a simplfied example -- single-audio stream, single-subtitle stream -- of my workflow (in which the audio codec is dts and the subtitles are stream 3):

VSPIPE --y4m "FOO.VPY" - | FFMPEG -i pipe: -i "SOURCE" -filter_complex "[0:v]settb=expr=1/120,setpts=expr=N[v],[1:a:0]atempo=1.001[a0]" -map "[v]" -map "[a0]" -codec:v hevc_nvenc -preset 1 -rc 0 -spatial_aq 1 -bluray-compat 1 -multipass 2 -codec:a:0 dts -r 120 "TEMPTARGET.MKV"

MKVMerge --output "TARGET.MKV" --sync 3:0,1000/1001 "TEMPTARGET.MKV"

FOO.VPY is:

from vapoursynth import core
import havsfunc as havsfunc
oClip=core.ffms2.Source(source=r'SOURCE')
oClip=havsfunc.InterFrame(oClip, ...interframe.parameters.here...)
oClip=core.std.AssumeFPS(oClip, fpsnum=120000, fpsden=1001)
oClip.set_output()

Currently, the Windows cmd script is 443 lines and is totally automatic -- one click. It uses FFprobe to gather stream numbers and whatnot but I will probably probe via MediaInfo in the future.

Regarding interpolation, it appears Hybrid currently asks InterFrame to do frame repeating, only, not MV interpolation.

If you'd like to see the script, I'd prefer to show it via private message because it's still somewhat rough and I don't want people to use it until it's finished.