[HELP] rife - encoding freezes

[HELP] rife - encoding freezes - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: [HELP] rife - encoding freezes (/thread-3859.html)

Pages: 1 2

rife - encoding freezes - humanoid86 - 07.09.2024

Huh

what's wrong with the new rife-app? when you click Start, the old version without any brakes, immediately processed the file, and this new one, at the encoding stage, reloads the video stream, slows down and freezes, then encodes

https://drive.google.com/file/d/1DVHA3zcLH_BYzAwOGwzkl6YvPrk2aG5o/view?usp=drive_link

https://drive.google.com/file/d/1T1-eqia-S8Ap5Rvd8U_mgBWNMaFe_wZS/view?usp=drive_linkPlease, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.

RE: rife - Selur - 07.09.2024

Using 4k input, and this script:

# Imports

import vapoursynth as vs

# getting Vapoursynth core

import sys

import os

core = vs.core

# Import scripts folder

scriptPath = 'F:/Hybrid/64bit/vsscripts'

sys.path.insert(0, os.path.abspath(scriptPath))

# loading plugins

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife.dll")

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")

# Import scripts

import validate

# Source: 'G:\TestClips&Co\files\MPEG-4 H.264\4k\4k_sample_4096x2160.mp4'

# Current color space: YUV420P8, bit depth: 8, resolution: 4096x2160, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 2020ncl, format: AVC

# Loading G:\TestClips&Co\files\MPEG-4 H.264\4k\4k_sample_4096x2160.mp4 using DGSource

clip = core.dgdecodenv.DGSource("J:/tmp/mp4_b30946f06bbf6d3e0309b51cccf4cf0c_853323747.dgi")# 25 fps, scanorder: progressive

frame = clip.get_frame(0)

# setting color matrix to 2020ncl.

clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT2020_NCL)

# setting color transfer (vs.TRANSFER_BT2020_10), if it is not set.

if validate.transferIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT2020_10)

# setting color primaries info (to vs.PRIMARIES_BT2020), if it is not set.

if validate.primariesIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT2020)

# setting color range to TV (limited) range.

clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)

# making sure frame rate is set to 25fps

clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)

# making sure the detected scan type is set (detected: progressive)

clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive

clip = core.misc.SCDetect(clip=clip,threshold=0.150)

# adjusting color space from YUV420P8 to RGBS for vsRIFE

clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="2020ncl", range_s="limited")

# adjusting frame count&rate with RIFE, target fps: 60fps

clip = core.rife.RIFE(clip, model=35, fps_num=60, fps_den=1, uhd=True, sc=True) # new fps: 60

# adjusting output color from: RGBS to YUV420P10 for NVEncModel

clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="2020ncl", range_s="limited", dither_type="error_diffusion")

# set output frame rate to 60fps (progressive)

clip = core.std.AssumeFPS(clip=clip, fpsnum=60, fpsden=1)

# output

clip.set_output()

encoding stalls a bit, but then starts, and constantly works, no pauses.
Using a Geforce RX 4800 (with 560.81 Studio drivers): VRAM usage ~10,8GB (video decode is running at ~70.90% and 3D usage is at ~90-95%)

Since I can't see the settings and you didn't post them:
What does the Vapoursynth script look like?
(side note: not enabling 'Scene change' is usually a mistake, unless your whole clip is just once scene)
What GPU are you using?
How is the CPU and GPU (VRAM/3D/Decode) usage?
What drivers are you using?
Is a virus scanner running in the background? (aside from Windows defender?)
Any hardware tuning tools running in the background?
How are the CPU and GPU temperatures?
(for me GPU is ~45-55°C and CPU is clocked at 5.27GHz and at ~70-72°C)
What is your RAM usage? (mine is 18-18.7GB running the above script and NVEncC as encoder)

=> not having any issues here
Wild guess is that something is interfering or bottlenecking the decoding.

Looking over at https://github.com/styler00dollar/VapourSynth-RIFE-ncnn-Vulkan I see no changes in the past 6 months that should influence the speed.
This does not look like an issue with the rife filter itself, but either your system or your other settings.

Cu Selur

Ps.: rife-app (https://grisk.itch.io/rife-app) is something different from Hybrid.

RE: rife - humanoid86 - 07.09.2024

the problem is that when it comes to the encoding stage, it thinks for a long time, with the sound of the loading process, then it freezes, then it starts processing.

CPU Intel i7 12700
GPU NVIDIA RTX3060-12
RAM 64GB DDR5
SSD 256+1024
WinOS 10x64 22H2

https://drive.google.com/file/d/1P-Vn5dK8EfHSZ0hEABVXmxoHIRTNHBA2/view?usp=drive_link

https://drive.google.com/file/d/1WLv0XdD__MF8lvsZw_qmP4xNPvG2nuD2/view?usp=drive_link

RE: rife - Selur - 07.09.2024

Encoding call:

"E:\Pro321\Hybrid2\Hybrid\64bit\Vapoursynth\vspipe.exe" "C:\Users\79142\AppData\Local\Temp\encodingTempSynthSkript_2024-09-07@21_03_15_4110_0.vpy" - -c y4m |  "E:\Pro321\Hybrid2\Hybrid\64bit\x265.exe" --input - --fps 60/1 --output-depth 8 --y4m --profile main --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --lookahead-slices 0 --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --no-mcstf --range limited --colormatrix bt709 --sar 1:1 --output "C:\Users\79142\AppData\Local\Temp\2024-09-07@21_03_15_4110_03.265"

and used script:

# Imports

import vapoursynth as vs

# getting Vapoursynth core

import sys

import os

core = vs.core

# Import scripts folder

scriptPath = 'E:/Pro321/Hybrid2/Hybrid/64bit/vsscripts'

sys.path.insert(0, os.path.abspath(scriptPath))

# loading plugins

core.std.LoadPlugin(path="E:/Pro321/Hybrid2/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife.dll")

core.std.LoadPlugin(path="E:/Pro321/Hybrid2/Hybrid/64bit/vsfilters/SourceFilter/BestSource/BestSource.dll")

# Import scripts

import validate

# Source: 'E:\COOKIE 111\CHAERYEONG111_new.mp4'

# Current color space: YUV420P8, bit depth: 8, resolution: 2160x3840, frame rate: 30fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709

# Loading E:\COOKIE 111\CHAERYEONG111_new.mp4 using BestSource (NOT recommended and untested!)

clip = core.bs.VideoSource(source="E:/COOKIE 111/CHAERYEONG111_new.mp4", cachepath="C:/Users/79142/AppData/Local/Temp/CHAERYEONG111_new.json", track=0)

frame = clip.get_frame(0)

# setting color matrix to 709.

clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)

# setting color transfer (vs.TRANSFER_BT2020_10), if it is not set.

if validate.transferIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT2020_10)

# setting color primaries info (to vs.PRIMARIES_BT2020), if it is not set.

if validate.primariesIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT2020)

# setting color range to TV (limited) range.

clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)

# making sure frame rate is set to 30fps

clip = core.std.AssumeFPS(clip=clip, fpsnum=30, fpsden=1)

# making sure the detected scan type is set (detected: progressive)

clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive

# adjusting color space from YUV420P8 to RGBS for vsRIFE

clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="709", range_s="limited")

# adjusting frame count&rate with RIFE, target fps: 60fps

clip = core.rife.RIFE(clip, model=66, fps_num=60, fps_den=1, uhd=True) # new fps: 60

# adjusting output color from: RGBS to YUV420P8 for x265Model

clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="709", range_s="limited", dither_type="error_diffusion")

# set output frame rate to 60fps (progressive)

clip = core.std.AssumeFPS(clip=clip, fpsnum=60, fpsden=1)

# output

clip.set_output()

(side note: Totally forgot to remove the 'NOT recommended and untested!' warning.)
both seem fine.
Side note: I would set BestSource to use 'cuda' decoding, decoding 4k hevc on cpu can slow things down quite a bit and if there's a dedicated hardware decoder why not use it. Smile

Encoding starts at: "21:03:54"

2024.09.07 - 21:03:54_Windows 10 Version 22H2 (64bit)_2024.09.03.1 - level 9: x265 output: y4m  [info]: 2160x3840 fps 60/1 i420p8 sar 1:1 unknown frame count

raw  [info]: output file: C:\Users\79142\AppData\Local\Temp\2024-09-07@21_03_15_4110_03.265

x265 [info]: HEVC encoder version 3.6+117-858bf15

x265 [info]: build info [Windows][GCC 14.2.0][64 bit] 8bit+10bit+12bit

x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

x265 [info]: Main profile, Level-5.1 (Main tier)

x265 [info]: Thread pool created using 16 threads

x265 [info]: Slices                              : 1

x265 [info]: frame threads / pool features       : 4 / wpp(60 rows)

x265 [info]: Coding QT: max CU size, min CU size : 64 / 8

x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra

x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3

x265 [info]: Keyframe min / max / scenecut / bias  : 25 / 250 / 40 / 5.00 

x265 [info]: Cb/Cr QP Offset                     : -2 / -2

x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2

x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0

x265 [info]: References / ref-limit  cu / depth  : 3 / off / off

x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 0.0 / 32 / 1

x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60

x265 [info]: tools: limit-modes rd=3 ssim-rd psy-rd=2.50 rdoq=2 psy-rdoq=10.00

x265 [info]: tools: rskip mode=1 signhide tmvp b-intra strong-intra-smoothing

x265 [info]: tools: deblock(tC=-1:B=-1) sao

first frame is processed 4 seconds later:

2024.09.07 - 21:03:58_Windows 10 Version 22H2 (64bit)_2024.09.03.1 - level 9: x265 output: 1 frames: 0.03 fps, 239274.72 kb/s

and looking at the further outputs, I see no stalling.
There does not seem to be any 'pausing' or similar, which should show in a drop in the average frame rate.

The overall encoding speed also seems like an expected speed:

2024.09.07 - 21:34:03_Windows 10 Version 22H2 (64bit)_2024.09.03.1 - level 9: x265 output:                                                                                 

x265 [info]: frame I:     48, Avg QP:24.07  kb/s: 70279.85

x265 [info]: frame P:   3116, Avg QP:25.22  kb/s: 53047.96

x265 [info]: frame B:   8380, Avg QP:28.34  kb/s: 19125.44

x265 [info]: Weighted P-Frames: Y:2.2% UV:0.8%

encoded 11544 frames in 1808.81s (6.38 fps), 28494.64 kb/s, Avg QP:27.48

Quote:the problem is that when it comes to the encoding stage, it thinks for a long time, with the sound of the loading process, then it freezes, then it starts processing.

Can you elaborate on that, since the debug output does not show anything like this.
Might be some 'glitch' in the progress indication.
If you look at the gpu and cpu usage, can you see those freezes there somehow?

Cu Selur

RE: rife - encoding freezes - humanoid86 - 08.09.2024

probably because of the driver, I have an SD of 560.81

I'll change to 5060/5070 when released

RE: rife - encoding freezes - Selur - 08.09.2024

Maybe, these are the same drivers I use, but I have another card.

RE: rife - encoding freezes - humanoid86 - 08.09.2024

if there was bad overclocking and bad cooling, then the problems would be everywhere, and not with this function.

I don't like overclocking, minus the warranty. I did it once a long time ago, and I didn't like it, constant monitoring is needed

RE: rife - encoding freezes - humanoid86 - 28.09.2024

Why doesn’t QTGMC < TFF/BFF or Progressive work when Rife-app is activated??? these modes should help with FPS with transitions

Huh

RE: rife - encoding freezes - Selur - 28.09.2024

No clue about rife-app (https://grisk.itch.io/rife-app), I have never used it.
QTGMC works fine in combination with Filtering->Vapoursynth->Frame->Frame Interpolation->RIFE here.

# Imports

import vapoursynth as vs

# getting Vapoursynth core

import ctypes

import sys

import os

core = vs.core

# Import scripts folder

scriptPath = 'F:/workspace/Hybrid/debug/64bit/vsscripts'

sys.path.insert(0, os.path.abspath(scriptPath))

# Loading Support Files

Dllref = ctypes.windll.LoadLibrary("F:/workspace/Hybrid/debug/64bit/vsfilters/Support/libfftw3f-3.dll")

# loading plugins

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/FrameFilter/RIFE/librife.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/Support/EEDI3m_opencl.dll")# vsQTGMC

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/ResizeFilter/nnedi3/NNEDI3CL.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/Support/libmvtools.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/Support/scenechange.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/Support/fmtconv.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")

# Import scripts

import havsfunc

import validate

# Source: 'G:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\bff.m2v'

# Current color space: YUV420P8, bit depth: 8, resolution: 720x480, frame rate: 29.97fps, scanorder: bottom field first, yuv luminance scale: limited, matrix: 470bg, format: mpeg-2

# Loading G:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\bff.m2v using DGSource

clip = core.dgdecodenv.DGSource("J:/tmp/m2v_478bc6d007ec94bfc67367d30d9093a4_853323747.dgi",fieldop=0)# 29.97 fps, scanorder: bottom field first

frame = clip.get_frame(0)

# setting color matrix to 470bg.

clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)

# setting color transfer (vs.TRANSFER_BT601), if it is not set.

if validate.transferIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)

# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.

if validate.primariesIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)

# setting color range to TV (limited) range.

clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)

# making sure frame rate is set to 29.97fps

clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)

# making sure the detected scan type is set (detected: bottom field first)

clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_BOTTOM) # bff

# Deinterlacing using QTGMC

clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False, opencl=True) # new fps: 59.94

# Making sure content is preceived as frame based

clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive

# adjusting color space from YUV420P8 to RGBS for vsRIFE

clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")

# clip, new color sampling: RGBS, new bit depth: 32

# adjusting frame count&rate with RIFE, target fps: 120fps

clip = core.rife.RIFE(clip, model=46, fps_num=120, fps_den=1) # new fps: 120

# adjusting output color from: RGBS to YUV420P10 for NVEncModel

clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")

# clip, new color sampling: YUV420P10, new bit depth: 10

# set output frame rate to 120fps (progressive)

clip = core.std.AssumeFPS(clip=clip, fpsnum=120, fpsden=1)

# output

clip.set_output()

Interlaced source, it deinterlaced to 59.94 and then interpolated to 120ps with RIFE.

=> No clue, what you are on about.

Cu Selur

Ps.: on a progressive source QTGMC-deinterlacer isn't applied since applying a deinterlacer on progressive content will damage the content.

RE: rife - encoding freezes - humanoid86 - 28.09.2024

(28.09.2024, 07:08)Selur Wrote: No clue about rife-app (https://grisk.itch.io/rife-app), I have never used it.
QTGMC works fine in combination with Filtering->Vapoursynth->Frame->Frame Interpolation->RIFE here.

core.std.LoadPlugin(path="F:/workspace/Hybrid/debug/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll

Interlaced source, it deinterlaced to 59.94 and then interpolated to 120ps with RIFE.

hmm...