[INFO] Feature, add/update rife frame interpolation

[Brutoloco]

Hi,

Version 4.0 of the RIFE frame interpolation engine in Vulkan allows to multiply by a fractional number, this allows to obtain an output in FPS of the exact value you want with very decent quality without having to crop frames by eliminating excess images.

I think that would be a very useful feature, the fractional option seems to only work on Vulkan API.

https://github.com/HomeOfVapourSynthEvol...cnn-Vulkan
https://github.com/nihui/rife-ncnn-vulkan

Thanks

[Selur]

Added this to current dev version at the end of May.
Can send you a link to my current dev version if you want.

Cu Selur

[Brutoloco]

Yes I would like to try it. Thanks.

[Selur]

Send you a link via pm.

Cu Selur

[Brutoloco]

Well, I've tested it, it works great.

Although it only supports 2 decimals.

This seems to be a problem converting the standard film=23,976fps to the standard 60fps.
23.976 x 2.5 = 59.94
23.976 x 2.51= 60.17976

I'm not 100% sure, but I think this ncnn vulkan engine supports up to 6 decimals.

Greetings and thank you.

[Selur]

Adding more decimals shouldn't be a problem.
-> I'll compile a new version and send you a link when I'm back home.

Cu Selur

[Brutoloco]

It works well and also very efficiently.
By multiplying film format (23.976) by 2.502502, the output is set to 60fps without touching anything else.

Awesome.

[Brutoloco]

I have found problems with the interpolated rife v4, I don't know if they can be fixed, but I mention them, I have done some tests before writing here, to try to find out what could fail.

Occasionally it crashes, but that's not the main problem.

When I interpolate x2.502503, to convert 23.976fps to 60fps, the audio is out of sync. The desynchronization is irregular, it cannot be corrected with a delay. The duration of the video is the same than the original, but the frames do not match in the same places.

The audio is a file that does not go through the hybrid. Ruled out that the hybrid could do something to the audio.

And at the moment, the only way I've found to avoid desynchronization is to convert to more frames, for example, multiply x2.51 or x2.6, and simultaneously crop the output to 60fps, in:

Speed Change -> Output Frame rate ->60fps

Sometimes, this make a video from 22mins, last more seconds, but the audio is still synchronized.

After all, this seems to increase the likelihood of videos crashing. But if they don't crash, the result has the audio in sync.

I have tested all types file "always extract timecodes from input", didin't change the result.

[Selur]

No clue why RIFE would be crashing.
If your source if vfr frame and you use frame interpolation, Hybrid should tell the source filter to output cfr content and drop the time codes, since they don't help.
My guess would be that the issue is the source decoders conversion to cfr.
-> what does your Vapoursynth script look like?

Cu Selur

[Brutoloco]

I'll try to upload a log later, now I just found out, or so I think, that it crashes right at the end of the video. I haven't checked, but I'm thinking it might be crashing always at the end of the video, when it hits 99.99% conversion but I'm not sure.

And that the video remains in the temporary folder, in this case it is a .h265. It can be packaged as mkv or something else, and it works.

This crash error has happened to me with two different types of video. At different conversion speed. It doesn't seem to have anything to do with the audio out-of-sync problem. This issue seems to have to do with these fractional FPS multipliers, but I don't understand why it does this.