[HELP] Issue when using RIFE interpolation

[kht]

Hi, Selur.

I'm currently using the latest dev 2024.11.01.1 at Windows 11 23H2. When I try to interpolate video from 23.98 to 59.94 fps (2.5x), it keep crashed with exit code of -22. Using RIFEmlrt with 4.26, tried 4.25, 4.25_lite, 4.25_heavy models but no luck.

I've tried to read through the debug output but unable to catch the error part of the encoding. Please enlighten me.

Latest stable version (2024.09.29) don't have this issue though. Attach with debug output.

Regards.

[Selur]

Check the Vapoursynth Preview.
I think I might have fixed it a few minutes ago, but I'm still working on it.

Cu Selur

[Selur]

Not sure your problem is the one I fixed, so best check the Vapoursynth Preview and share what it's complaining about.
(Uploading new versions, but that will take ~1hr,..)

Cu Selur

[kht]

When I click VS Preview, this is the error code

2024-11-01 23:15:26.447
Failed to evaluate the script:
Python exception: fractional multi requires plugin akarin (https://github.com/AkarinVS/vapoursynth-plugin/releases), version v0.96g or later.

Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate
File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate
File "E:\Temp\Hybrid\tempPreviewVapoursynthFile23_15_25_109.vpy", line 46, in
clip = vsmlrt.RIFE(clip, multi=Fraction(2.5), model=426, backend=Backend.TRT(fp16=True,device_id=0,verbose=True,use_cuda_graph=True, num_streams=1,builder_optimization_level=3,engine_folder="E:/Temp/Hybrid")) # new fps: 59.94
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:/Tools/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 1263, in RIFE
raise RuntimeError(
RuntimeError: fractional multi requires plugin akarin (https://github.com/AkarinVS/vapoursynth-plugin/releases), version v0.96g or later.

[Selur]

That is fixed in the new release. (will be up in ~45min, uploading takes its time)

Cu Selur

[Selur]

New dev (and torch-addon) are online, the dev should fix the akarin dependency.

Cu Selur

[kht]

It's not working too.

The dependency error is solved. But it will throw an error on resolution divisible by 32, increasing to 1920x1088 will not work but 1920x1056 is working. However, when I start the job, trtexec have this error and I'm not sure what to do about this.

[11/02/2024-01:21:17] [E] Error[4]: IBuilder::buildSerializedNetwork: Error Code 4: API Usage Error (IElementWiseLayer /Mul: broadcast dimensions must be conformable)

When using stable version, the standard resolution (1920x1080, 3840x2160) work flawlessly, so is this the new limitation of the features? When I look for the source code of the vs-mlrt, it shows that latest stable (15.5) will force to use resolution divisible by 32/64.

if (model_major, model_minor) >= (4, 26):
        tilesize_requirement = 64
    else:
        tilesize_requirement = 32

However, if I commented the code above, it works even with standard resolution on rife-4.25, rife-4.26.

Do you have any idea on these? Attached with trtexec and debug log.

https://pixeldrain.com/u/wvSBGUtV

Regards.

[Selur]

Argh, I thought the mod restrictions were only for non TRT.
=> Looking into it

[Selur]

Okay, restrictions are:

model	mod
4.00-4.25	32
4.25_lite	128
4.25_heavy	64
4.26	64

=> uploaded a new dev version which should respect this

Cu Selur

[kht]

Latest dev is working. As you mentioned, standard resolution is now working with TRT backend without restriction.

Thank you.