[BUG] FillDrops with RIFE

[zspeciman]

Please, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.

Hello, I'm using the latest version:  Hybrid Windows Installer: rev 2024.01.12.1
When enabling FillDrops it works fine on methods  MV & SVP.  However when I enable RIFE, no matter which version of, it crashes.
I've never used FillDrops before, so I don't know if it worked in earlier versions.  
Please help

[Selur]

It does work fine here.
The Vapoursynth script used:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import filldrops
# source: 'K:\DeOldify-master\video\result\MoreAboutEnglishWoody.mp4'
# current color space: YUV420P8, bit depth: 8, resolution: 640x480, fps: 25, color matrix: 470bg, yuv luminance scale: full, scanorder: progressive
# Loading K:\DeOldify-master\video\result\MoreAboutEnglishWoody.mp4 using LibavSMASHSource
clip = core.lsmas.LibavSMASHSource(source="K:/DeOldify-master/video/result/MoreAboutEnglishWoody.mp4")
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer (to 470bg), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
  clip = core.std.SetFrameProps(clip, _Transfer=5)
# Setting color primaries info (to 5), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
  clip = core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
clip = filldrops.FillSingleDrops(clip=clip, method="rife", rifeModel=22)
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

does look fine too.
(side note: when using FillDrops and similar, I would strongly suggest check the thresholds to be sure they are not too low or high.)

Strangely, the encoding does stop directly and isn't complaining about a decoding issue.

"C:\Program Files\Hybrid\64bit\Vapoursynth\vspipe.exe" "C:\Users\Yoyo-Desktop\AppData\Local\Temp\encodingTempSynthSkript_2024-01-16@11_14_18_9510_0.vpy" - -c y4m | "C:\Program Files\Hybrid\64bit\x264.exe" --preset veryfast --pass 1 --bitrate 1500 --profile high --level 4.1 --direct auto --b-adapt 0 --sync-lookahead 18 --qcomp 0.50 --rc-lookahead 40 --qpmax 51 --aq-mode 0 --sar 1:1 --non-deterministic --range pc --stats "C:\Users\Yoyo-Desktop\AppData\Local\Temp\MoreAboutEnglishWoodyRife_1_2024-01-16@11_14_18_9510_03.stats" --demuxer y4m --input-csp yuvj420p --input-range pc --fps 25/1 --output-depth 8 --output NUL -

If you call the encoding call inside a Windows Command Line interface (not powershell), does it show any errors that Hybrid might have missed?
Does the Vapoursynth preview work properly, or does it show an error?

Does RIFE work on its own (Filtering->Vapoursynth->Frame->Interpolation->Frame Interpolation->RIFE)

Cu Selur

Ps.: what gpu and driver version are you using?
PPs.: going to bed now, but send you a link to my current dev so we can be sure we both use the same version.

[zspeciman]

I'm using Nvidia GeForce GTX 1080 Ti
Driver 31.0.15.4633
Rife 4.6 for frame interpolation does work on its own.  I took a video at 25fps and doubled it to 50fps, no issues

For Filldrops, I left all at defaults which was Threshold 0.3000 and SCD at 0.15

The Vapoursynth Preview does not work. It opens the preview window, but when I press play, it closes the window
This is the error I get on the older Hybrid rev 2024.01.12.1

Vapoursynth preview error: p16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
Vapoursynth preview error: Force GTX 1080 Ti]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1 [0 NVIDIA GeForce GTX 1080 Ti]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1 [0 NVIDIA GeForce GTX 1080 Ti]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0

This is the error I get on the newest Hybrid you've sent me:
Vapoursynth preview error: =1/1/1 [0 NVIDIA GeForce GTX 1080 Ti] subgroup=32 basic/vote/ballot/shuffle=1/1/1/1 [0 NVIDIA GeForce GTX 1080 Ti] fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0

[Selur]

Okay, since RIFE V4.6 does work on its own, check whether using 'rife-v4.6 (ensemble=False)' work in FillDrops?
You can also try whether using an older RIFE version from https://github.com/styler00dollar/Vapour...n/releases helps. (older version can't use the newer models, but should work fine otherwise).

Cu Selur

[zspeciman]

For the current version, none of the RIFE versions work for FillDrops, ensemble=True or False.  Preview doesn't work.

I don't know how to implement older versions of RIFE with the link you've sent, but I took your suggestion with different approach. 
I tried different versions of Hybrid that I've downloaded in the past.

I get the Vapoursynth Preview for FillDrops in both of these and I was able to process a short video without a crash. Success
Hybrid_2022.03.20.1 upto RIFE 3.1
Hybrid_2023.03.17.1 upto RIFE 4.6


No Preview in these, and it crashes
Hybrid_2023.12.22.1 
Hybrid_2024.01.12.1

[Selur]

The problem is probably related to your old graphic card (nearly 8 years old) and the drivers used.
RIFE and some of the other GPU based tools more and more require newer CUDA SDK support, which in turn requires newer graphic card drivers. :/
Sadly, I don't think I can do much about it.
As a workaround, you could replace the Hybrid\64bit\vsfilters\FrameFilter\RIFE-folder in the current version with one from Hybrid_2023.03.17.1, this way you would be able to use up to RIFE 4.6 with a current Hybrid version. (or just test the different RIFE releases from https://github.com/styler00dollar/Vapour...n/releases)

Cu Selur

[zspeciman]

Thank you, your suggestion worked!!!
To my surprise Filldrops MV method did a much better job than SVP and RIFE.  Neither one SVP nor RIFE has removed those stubborn "almost exact duplicates" frames and replaced them with interpolated frame, no matter what Threshold I've used.  But MV did exactly that.

[Selur]

Depending on the content, one or the other interpolation method can be better.

[zspeciman]

Is the difference between FillDrops and ReplaceSingle that in ReplaceSingle you manually specify the frames that are duplicate whereas FillDrops will auto detect it for you?
Is there a media player that has a Frame counter?

[Selur]

FillDrops replaces single duplicates if one frame and it's neighbour are identical. (it won't work properly if you have multiple identical frames in a row)
ReplaceSingle, will place single frames, listed by the user, through interpolating their neighbours.
Yes, the replacement process is the same. But ReplaceSingle doesn't care whether the replaced is a duplicate, FillDrops does.

Using a media player is a bad choice (as they don't need to be frame accurate), you should better use the Vapoursynth Preview.

Cu Selur