[INFO] Feature, add/update rife frame interpolation

[Brutoloco]

I don't know if this has anything to do with it,
but it turns out that FPS: 23.976 is actually 24000/1001 = 23.97602397602398 fps.

Those would be the actual numbers,

I don't know if this has anything to do with how the Hybrid handles those numbers. Where Hybrid says fps is 23,976 it's actually all of those numbers.

60fps/23.97602397602398fps = 2.5025 (this should be the correct multiplier)

[Selur]

Nope that is unrelated to the crash, but yes "2.5025" should be used to avoid potential sync issues later on. (if the output frame count differs through this)
(doesn't crash where with gpu_threads=1 to 5)

Cu Selur

Ps.: If I find some time on Sunday, I'll look into adding an additional choice where the user can set a target frame rate and Hybrid will calculate the multiplier for RIFEv4. -> forget that after a short thought I realized, that's not possible here, since I don't know what the rest of the script will look like.

[Selur]

Okay, I could implement this through a python wrapper (to use multiplier = targetfps * clip.fps.denumerator / clip.fps.numerator), but since this might also lead to problems later on, I made a feature request over at https://github.com/HolyWu/vs-rife/issues/17, maybe HolyWu agrees and implements direct support for this in the filter.

Cu Selur

[Selur]

send you a new link to a dev version,...

[Brutoloco]

I have tried that of only multiplying by 2.5025. the video of 2,976.... It has to be a long video to be noticed. There is no crash. The audio gets progressively out of sync at the end of the video. However the video lasts the same. The audio has not been altered. 

Partial FPS multiplication for a 23,976... video to 60fps... works unexpectedly, changing irregularly the original place of the FPS, unknown cause.

BUT STILL NOT TESTED YOUR NEW DEV VERSION

Post edit, checking new version:

I don't know what's the reason of outsync frames, but I know a way to reproduce the problem without having to encoding. The problem is visible using vapoursynth preview

- Put any video at 23,976 fps and open preview, without changing any setting (keep synth auto refresh deactivated)
- Choose any part of the video using TIME, not frame, for example I choose minute 22:00.
- Now, enable rifev4 fps to 60fps or multiply 2,5025
- Enable Synth auto refresh, and put the same time --> 22:00 for example

Check that the frame at that time now, is many frames away from the first frame. There is a delay.

This does not happen if I combine "rifev4 x2.5", and "Filtering, speed change, scaling frame rate output: 60fps". --> sync ok

But If I combine rife v4 x2.5025(or60fps) and scaling frame rate output to 60. The sync out persist. --> sync not ok

If I combine rife v4 x2.51(more than 60fps) and scaling frame rate output to 60. -> sync ok

[Selur]

Argh,... will to some testing over the weekend.
Does the frame count change in the vapoursynth preview between using:
a. RIFEv4 + 2.51 + scaling frame rate output to 60
b. RIFEv4 + 2.5025
I suspect this whole thing comes down to rounding issue in RIFE. :/

Cu Selur

[Selur]

Starting with 24000/1001 fps content and looking at:

Frame: 71928 (= 24000/1001fps * 50*60sec)
with a multiplier of 2, I get frame 143856 (= 2*24000/1001fps * 50*60sec), which is the same.
-> everything seems alright.

Frame: 71928 (= 24000/1001fps * 50*60sec)
with a multiplier of 2.5025, I get frame 180000 (= 2.5025 * 24000/1001fps fps * 50*60sec), which is the same.
-> everything seems alright.

Frame 143856 (= 24000/1001fps * 100*60sec)
with a multiplier of 2.5025 I get frame 360000 (= 2.5025 * 24000/1001fps fps * 100*60sec), which is not the same.
Looking through the frames 359640 seems to be the correct one which corresponds to a multiplier of 2.5.

=> My guess is that that either multiplier can only have 1 or 2 decimals or there is a bug in vs-rife. (reported this to the vs-rife author: https://github.com/HolyWu/vs-rife/issues/17)

So the only workaround I can think of atm. would be to get 2.5025 by using 2.51 (two decimals) in RIFE and then drop frames to archive the desired mult,... which essentially is what you did by using 2.51 and then scaling the frame rate output to 60fps.


Cu Selur

[Selur]

Send you a lnk to a dev version which uses the described workaround. (seems to work in my tests)

Cu Selur

[Brutoloco]

Thanks, I suspected all along that the problem was in the plugin itself.

I think it is better to multiply by x2.5

23.97602397602398 x 2.5 = 59.94005994005994

And round up by adding missing frames by copying some frames to make it exact 60fps. That is done by the Filtering->Speed change -> Scale output frame rate adjust to 60fps

Otherwise, as you say, with x2.51, then cutting with the same fuction, I think some of the original frames are cropped, this other way the original frames are intact in the result, which is what I wanted.

In any case, the difference in the result does not seem to be appreciable.

[Selur]

Hmm,.. I will think about it.
Also:

Filtering->Speed change -> Scale output frame rate adjust to 60fps

Does the same as Hybrid does atm., so may be I'll add an option for the user to select whether RIFE should use a higher or lower multiplier.

Cu Selur