[HELP] rife - encoding freezes

[humanoid86]

what's wrong with the new rife-app? when you click Start, the old version without any brakes, immediately processed the file, and this new one, at the encoding stage, reloads the video stream, slows down and freezes, then encodes

https://drive.google.com/file/d/1DVHA3zc...drive_link

https://drive.google.com/file/d/1T1-eqia...linkPlease, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.

[Selur]

Using 4k input, and this script:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")
# Import scripts
import validate
# Source: 'G:\TestClips&Co\files\MPEG-4 H.264\4k\4k_sample_4096x2160.mp4'
# Current color space: YUV420P8, bit depth: 8, resolution: 4096x2160, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 2020ncl, format: AVC
# Loading G:\TestClips&Co\files\MPEG-4 H.264\4k\4k_sample_4096x2160.mp4 using DGSource
clip = core.dgdecodenv.DGSource("J:/tmp/mp4_b30946f06bbf6d3e0309b51cccf4cf0c_853323747.dgi")# 25 fps, scanorder: progressive
frame = clip.get_frame(0)
# setting color matrix to 2020ncl.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT2020_NCL)
# setting color transfer (vs.TRANSFER_BT2020_10), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT2020_10)
# setting color primaries info (to vs.PRIMARIES_BT2020), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT2020)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
clip = core.misc.SCDetect(clip=clip,threshold=0.150)
# adjusting color space from YUV420P8 to RGBS for vsRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="2020ncl", range_s="limited")
# adjusting frame count&rate with RIFE, target fps: 60fps
clip = core.rife.RIFE(clip, model=35, fps_num=60, fps_den=1, uhd=True, sc=True) # new fps: 60
# adjusting output color from: RGBS to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="2020ncl", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 60fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=60, fpsden=1)
# output
clip.set_output()

encoding stalls a bit, but then starts, and constantly works, no pauses.
Using a Geforce RX 4800 (with 560.81 Studio drivers): VRAM usage ~10,8GB (video decode is running at ~70.90% and 3D usage is at ~90-95%)

Since I can't see the settings and you didn't post them:
What does the Vapoursynth script look like?
(side note: not enabling 'Scene change' is usually a mistake, unless your whole clip is just once scene)
What GPU are you using?
How is the CPU and GPU (VRAM/3D/Decode) usage?
What drivers are you using?
Is a virus scanner running in the background? (aside from Windows defender?)
Any hardware tuning tools running in the background?
How are the CPU and GPU temperatures?
(for me GPU is ~45-55°C and CPU is clocked at 5.27GHz and at ~70-72°C)
What is your RAM usage? (mine is 18-18.7GB running the above script and NVEncC as encoder)

=> not having any issues here
Wild guess is that something is interfering or bottlenecking the decoding.

Looking over at https://github.com/styler00dollar/Vapour...cnn-Vulkan I see no changes in the past 6 months that should influence the speed.
This does not look like an issue with the rife filter itself, but either your system or your other settings.

Cu Selur

Ps.: rife-app (https://grisk.itch.io/rife-app) is something different from Hybrid.

[humanoid86]

the problem is that when it comes to the encoding stage, it thinks for a long time, with the sound of the loading process, then it freezes, then it starts processing.

CPU Intel i7 12700
GPU NVIDIA RTX3060-12
RAM 64GB DDR5
SSD 256+1024
WinOS 10x64 22H2

https://drive.google.com/file/d/1P-Vn5dK...drive_link

https://drive.google.com/file/d/1WLv0XdD...drive_link

[Selur]

Encoding call:

"E:\Pro321\Hybrid2\Hybrid\64bit\Vapoursynth\vspipe.exe" "C:\Users\79142\AppData\Local\Temp\encodingTempSynthSkript_2024-09-07@21_03_15_4110_0.vpy" - -c y4m |  "E:\Pro321\Hybrid2\Hybrid\64bit\x265.exe" --input - --fps 60/1 --output-depth 8 --y4m --profile main --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --lookahead-slices 0 --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --no-mcstf --range limited --colormatrix bt709 --sar 1:1 --output "C:\Users\79142\AppData\Local\Temp\2024-09-07@21_03_15_4110_03.265"

and used script:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'E:/Pro321/Hybrid2/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="E:/Pro321/Hybrid2/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife.dll")
core.std.LoadPlugin(path="E:/Pro321/Hybrid2/Hybrid/64bit/vsfilters/SourceFilter/BestSource/BestSource.dll")
# Import scripts
import validate
# Source: 'E:\COOKIE 111\CHAERYEONG111_new.mp4'
# Current color space: YUV420P8, bit depth: 8, resolution: 2160x3840, frame rate: 30fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709
# Loading E:\COOKIE 111\CHAERYEONG111_new.mp4 using BestSource (NOT recommended and untested!)
clip = core.bs.VideoSource(source="E:/COOKIE 111/CHAERYEONG111_new.mp4", cachepath="C:/Users/79142/AppData/Local/Temp/CHAERYEONG111_new.json", track=0)
frame = clip.get_frame(0)
# setting color matrix to 709.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)
# setting color transfer (vs.TRANSFER_BT2020_10), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT2020_10)
# setting color primaries info (to vs.PRIMARIES_BT2020), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT2020)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 30fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting color space from YUV420P8 to RGBS for vsRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="709", range_s="limited")
# adjusting frame count&rate with RIFE, target fps: 60fps
clip = core.rife.RIFE(clip, model=66, fps_num=60, fps_den=1, uhd=True) # new fps: 60
# adjusting output color from: RGBS to YUV420P8 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="709", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 60fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=60, fpsden=1)
# output
clip.set_output()

(side note: Totally forgot to remove the 'NOT recommended and untested!' warning.)
both seem fine.
Side note: I would set BestSource to use 'cuda' decoding, decoding 4k hevc on cpu can slow things down quite a bit and if there's a dedicated hardware decoder why not use it.

Encoding starts at: "21:03:54"

2024.09.07 - 21:03:54_Windows 10 Version 22H2 (64bit)_2024.09.03.1 - level 9: x265 output: y4m  [info]: 2160x3840 fps 60/1 i420p8 sar 1:1 unknown frame count
raw  [info]: output file: C:\Users\79142\AppData\Local\Temp\2024-09-07@21_03_15_4110_03.265
x265 [info]: HEVC encoder version 3.6+117-858bf15
x265 [info]: build info [Windows][GCC 14.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main profile, Level-5.1 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 4 / wpp(60 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 25 / 250 / 40 / 5.00
x265 [info]: Cb/Cr QP Offset                     : -2 / -2
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 0.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: tools: limit-modes rd=3 ssim-rd psy-rd=2.50 rdoq=2 psy-rdoq=10.00
x265 [info]: tools: rskip mode=1 signhide tmvp b-intra strong-intra-smoothing
x265 [info]: tools: deblock(tC=-1:B=-1) sao

first frame is processed 4 seconds later:

2024.09.07 - 21:03:58_Windows 10 Version 22H2 (64bit)_2024.09.03.1 - level 9: x265 output: 1 frames: 0.03 fps, 239274.72 kb/s

and looking at the further outputs, I see no stalling.
There does not seem to be any 'pausing' or similar, which should show in a drop in the average frame rate.

The overall encoding speed also seems like an expected speed:

2024.09.07 - 21:34:03_Windows 10 Version 22H2 (64bit)_2024.09.03.1 - level 9: x265 output:                                                                                
x265 [info]: frame I:     48, Avg QP:24.07  kb/s: 70279.85
x265 [info]: frame P:   3116, Avg QP:25.22  kb/s: 53047.96
x265 [info]: frame B:   8380, Avg QP:28.34  kb/s: 19125.44
x265 [info]: Weighted P-Frames: Y:2.2% UV:0.8%

encoded 11544 frames in 1808.81s (6.38 fps), 28494.64 kb/s, Avg QP:27.48

the problem is that when it comes to the encoding stage, it thinks for a long time, with the sound of the loading process, then it freezes, then it starts processing.

Can you elaborate on that, since the debug output does not show anything like this.
Might be some 'glitch' in the progress indication.
If you look at the gpu and cpu usage, can you see those freezes there somehow?

Cu Selur

[humanoid86]

probably because of the driver, I have an SD of 560.81

I'll change to 5060/5070 when released

[Selur]

Maybe, these are the same drivers I use, but I have another card.

[humanoid86]

if there was bad overclocking and bad cooling, then the problems would be everywhere, and not with this function.

I don't like overclocking, minus the warranty. I did it once a long time ago, and I didn't like it, constant monitoring is needed

[humanoid86]

Why doesn’t QTGMC < TFF/BFF or Progressive work when Rife-app is activated??? these modes should help with FPS with transitions

[Selur]

No clue about rife-app (https://grisk.itch.io/rife-app), I have never used it.
QTGMC works fine in combination with Filtering->Vapoursynth->Frame->Frame Interpolation->RIFE here.

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/workspace/Hybrid/debug/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("F:/workspace/Hybrid/debug/64bit/vsfilters/Support/libfftw3f-3.dll")
# loading plugins
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/FrameFilter/RIFE/librife.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/Support/EEDI3m_opencl.dll")# vsQTGMC
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/ResizeFilter/nnedi3/NNEDI3CL.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")
# Import scripts
import havsfunc
import validate
# Source: 'G:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\bff.m2v'
# Current color space: YUV420P8, bit depth: 8, resolution: 720x480, frame rate: 29.97fps, scanorder: bottom field first, yuv luminance scale: limited, matrix: 470bg, format: mpeg-2
# Loading G:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\bff.m2v using DGSource
clip = core.dgdecodenv.DGSource("J:/tmp/m2v_478bc6d007ec94bfc67367d30d9093a4_853323747.dgi",fieldop=0)# 29.97 fps, scanorder: bottom field first
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 29.97fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# making sure the detected scan type is set (detected: bottom field first)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_BOTTOM) # bff
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False, opencl=True) # new fps: 59.94
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting color space from YUV420P8 to RGBS for vsRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# clip, new color sampling: RGBS, new bit depth: 32
# adjusting frame count&rate with RIFE, target fps: 120fps
clip = core.rife.RIFE(clip, model=46, fps_num=120, fps_den=1) # new fps: 120
# adjusting output color from: RGBS to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
# clip, new color sampling: YUV420P10, new bit depth: 10
# set output frame rate to 120fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=120, fpsden=1)
# output
clip.set_output()

Interlaced source, it deinterlaced to 59.94 and then interpolated to 120ps with RIFE.

=> No clue, what you are on about.

Cu Selur

Ps.: on a progressive source QTGMC-deinterlacer isn't applied since applying a deinterlacer on progressive content will damage the content.

[humanoid86]

No clue about rife-app (https://grisk.itch.io/rife-app), I have never used it.
QTGMC works fine in combination with Filtering->Vapoursynth->Frame->Frame Interpolation->RIFE here.

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll

Interlaced source, it deinterlaced to 59.94 and then interpolated to 120ps with RIFE.

hmm...