[BUG] Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash

[BUG] Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: [BUG] Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash (/thread-3670.html)

Pages: 1 2

Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Euangelos - 10.04.2024

Hi
I am on 2950x Threadripper Windows 11 22H2 (All Redisributables are installed 2005 to 2022)

Avisynth TemporalDergrain 2 FFT3DFilter works up to "Previous and current frame 3d wiener filter" above that i.e "two previous etc" crash with log see attachment

Avisynth TemporalDergrain 2 FFT3DGPU works up to "Only current frame 2d wiener filter" above that i.e "Previous and current frame 3d wiener filter" crash with log see attachment

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Selur - 10.04.2024

Problem seems to only occur if PostFFT is set to ff3dfilter or ff3dgpu.
I checked, Hyrid does use v2.6.6 from https://avisynth.nl/index.php/TemporalDegrain2 so it does not seem to be a bug in Hybrid itself.

with fft3dfilter, the error is:

neo_fft3d: bt must be -1 (Sharpen), 0 (Kalman), 1..5 (Wiener)

with FFT3DGPU, the error is:

Valid modes for bt are -1,0,1,2,3,4

Looking at the script, I see:

out    = (postFFT == 0) ? (postDither > 0 ? in.ConvertBits(16) : in) : \
             (postFFT == 1) ? neo_fft3d ( postDither > 0 ? in.ConvertBits(16) : in, y=LumaP?3:2, u=ChromaP?3:2, v=ChromaP?3:2, sigma=postSigma, bt=postTD, ncpu=fftThreads, bw=postBlkSz, bh=postBlkSz ) : \
             (postFFT == 11)? FFT3DFilter( postDither > 0 ? in.ConvertBits(16) : in, plane=postPlane, sigma=postSigma, bt=postTD, ncpu=fftThreads, bw=postBlkSz, bh=postBlkSz ) : \
             (postFFT == 2) ? FFT3DGPU( postDither > 0 ? in.ConvertBits(16) : in, plane=postPlane, sigma=postSigma*2/3, bt=postTD, precision=2, mode=1, bw=postBlkSz, bh=postBlkSz ) : \
             (postFFT == 3) ? neo_dfttest( postDither > 0 ? in.ConvertBits(16) : in, y=LumaP?3:2, u=ChromaP?3:2, v=ChromaP?3:2, sigma=postSigma*4, tbsize=postTD, dither=dftDither, threads=fftThreads, sbsize=postBlkSz, sosize=postBlkSz*9/12, slocation=dftsfile) : \
             (postFFT == 13)? dfttest( postDither > 0 ? in.ConvertBits(16) : in, Y=LumaP, U=ChromaP, V=ChromaP, sigma=postSigma*4, tbsize=postTD, threads=fftThreads, dither=dftDither, sbsize=postBlkSz, sosize=postBlkSz*9/12, sfile=dftsfile ) : \
             (postFFT == 4) ? DT_KNLMeansCL( postDither > 0 ? in.ConvertBits(16) : in, a=2, d=postTR, h=postSigma, Luma = LumaP, Chroma = ChromaP, device_type="GPU", device_id=devId) : \
             (postFFT == 5) ? DT_BM3D( postDither > 0 ? in.ConvertBits(16) : in, radius=postTR, sigma=postSigma, chroma=ChromaP, CUDA=cuda, device_id=devId ) : \
             (postFFT == -1)? HQDn3D( postDither > 0 ? in.ConvertBits(16) : in, 0,0,4,1, u=ChromaP?3:2, v=ChromaP?3:2) : NOP()

the problem does happen when neo_fft3d (postFFT ==1), fft3dgpu (postFFT == 2), or fft3dfilter (postFFT == 11) is used.
In all these cases, 'bt' is set to postTD which is set in line 236&267 of the script:

postTR  = (postFFT > 0) ? postTR : 0
  postTD  = postTR * 2 + 1

so for

neo_fft3d (postFFT ==1), postTD would be 3
fft3dgpu (postFFT == 2), postTD would be 5
fft3dfilter (postFFT == 11), postTD would be 23

Lookint at what the filters support:

fft3dfilter (postFFT == 11) supports

bt - block temporal size, number of frames (-1, 0, 1, 2 or 3, default = 3): 0 - all previous frames (switch Kalman filter mode); 1 - only current frame (spatial 2D Wiener filter); 2 - previous and current frame (3D Wiener filter); 3 - previous, current and next frame (3D Wiener filter) 4 - two previous, current and next frame (3D Wiener filter) 5 - two previous, current and two next frames (3D Wiener filter) -1 - sharpen only (2D);
source: https://avisynth.org.ru/fft3dfilter/fft3dfilter.html
fft3dgpu (postFFT == 2), supports:

bt = 3 Block temporal size, number of frames. -1 : sharpen only (2D) 0 : all previous frames (switch Kalman filter mode) 1 : only current frame (spatial 2D Wiener filter) 2 : previous and current frame (3D Wiener filter) 3 : previous, current and next frame (3D Wiener filter) 4 : two previous, current and next frame (3D Wiener filter)
source: http://avisynth.nl/index.php/FFT3DGPU
neo_fft3d (postFFT ==1) supports:

int bt = 3 Block temporal size, number of frames. 0 : all previous frames (switch Kalman filter mode) 1 : only current frame (spatial 2D Wiener filter) 2 : previous and current frame (3D Wiener filter) 3 : previous, current and next frame (3D Wiener filter) 4 : two previous, current and next frame (3D Wiener filter) 5 : two previous, current and two next frames (3D Wiener filter) -1 : sharpen only (2D)
source: http://avisynth.nl/index.php/Neo_FFT3D

So 'postTD = postTR * 2 + 1' makes no sense. for these.
Replacing it with:

postTD  = (postFFT == 11 || postFFT == 1 || postFFT == 2) ? postTR : postTR * 2 + 1

and adjusting the mapping for postFFT in Hybrid might work.

=> looking into it.

Cu Selur

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Selur - 10.04.2024

Send you a link to a dev version where I change the TemporalDegrain2 script and adjusted Hybrid.
Seems to work, but test it and let me know whether it works for you too.

Cu Selur

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Euangelos - 11.04.2024

I included debug from 2 clips... to see it go in the middle of the clip the first frame is fine...

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Selur - 11.04.2024

Okay, iirc. the problem was that if you look at the preview an navigate to the middle, the position of the filtered and the possition of the unfiltered does not align, correct?

Script for first clip:

# loading source: D:\6.0 VR\Test\Test 3min.mov
# color sampling YUY2@10, matrix: bt601, scantyp: progressive, luminance scale: limited
Source = LWLibavVideoSource("D:\6.0 VR\Test\Test 3min.mov",cache=false,format="YUV422P10", prefer_hw=0)
# current resolution: 576x432
SourceFiltered = Source
SourceFiltered = SourceFiltered.ConvertToYUV420(interlaced=false)
SourceFiltered = SourceFiltered.TemporalDegrain2(limitFFT=2,postFFT=1,postTR=5)
# stacking horizontal for filter preview
# adjust color to RGB32 (for preview)
SourceFiltered = SourceFiltered.ConvertToRGB32(matrix="Rec601")
# adjust color to RGB32 (for preview)
Source = Source.ConvertToRGB32(matrix="Rec601")
StackHorizontal(Source.Subtitle("Original"), SourceFiltered.Subtitle("Filtered"))
PreFetch(16)

Scrript for second clip:

# loading source: D:\6.0 VR\Test\Test 12min.mp4
# color sampling YV12@8, matrix: bt709, scantyp: progressive, luminance scale: limited
Source = LWLibavVideoSource("D:\6.0 VR\Test\Test 12min.mp4",cache=false,format="YUV420P8", prefer_hw=0,repeat=true)
# current resolution: 1440x1080
SourceFiltered = Source
SourceFiltered = SourceFiltered.TemporalDegrain2(limitFFT=2,postFFT=1,postTR=5)
# stacking horizontal for filter preview
# adjust color to RGB32 (for preview)
SourceFiltered = SourceFiltered.ConvertToRGB32(matrix="Rec709")
# adjust color to RGB32 (for preview)
Source = Source.ConvertToRGB32(matrix="Rec709")
StackHorizontal(Source.Subtitle("Original"), SourceFiltered.Subtitle("Filtered"))
# setting output fps to 24.000fps
AssumeFPS(24,1)
PreFetch(16)
#  output: color sampling RGB32@8, matrix: bt709, scantyp: progressive, luminance scale: limited
return last

The scripts seem fine anly thing that differs is that the source clips.

So questions are know:
a. Is it a bug in the source filter LWLibavVideoSource (dev version uses different version), or
b. is it something I changed in TemporalDegrain2
c. What in the source triggers the problem.

=> I'll look at it after work, but what you can test, is whether this async between 'Original' and 'Filtered' also happens with other filters on your system.

Cu Selur

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Euangelos - 11.04.2024

I have tested lets say 50-60% of the filters this is the first time I noticed

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Selur - 11.04.2024

To be sure: Did you test them with the last dev version?
Like I wrote the source filter was updated, so if it's a bug in the source filter, you would not have had the issue before.

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Selur - 11.04.2024

side note: don't use TemporalDegrain2(limitFFT=2,postFFT=1,postTR=5) on 4k content, with these settings its terribly slow (and eats ~11GB of RAM).

I can reproduce the problem and it's caused by the changed I made, not sure why so far.
This will take me a while to figure out how to properly change TemporalDegrain2.avsi
Maybe I should simply restrict PostTR to avoid the bugs in the script.

Cu Selur

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Selur - 11.04.2024

Okay, I think I found a solution.
Replace Hybrid/32bit/avisynthPlugins/TemporalDegrain2.avsi and Hybrid/32bit/Avisynth/avisynthPlugins/TemporalDegrain2.avsi with the attached file should fix the problem.

Cu Selur

RE: Avisynth TemporalDergrain 2 FFT3DFilter&FFT3DGPU Crash - Euangelos - 12.04.2024

The frames are synchronised now Smile

but the preview window sometimes with 2-3 filters does not open, while in the Jobs queue the encoding takes place normally...