BasicVSR++ temporal jumps at group boundaries

[ywcz1997]

I'm using VapourSynth/Other/BasicVSR++ to restore a DVD source. The output has noticeable temporal inconsistency – the background jumps every few frames, especially in long shots.

I suspect this is due to chunk processing. Increasing length to 150 makes those 150 frames coherent, but the jump reappears exactly between frame 150 and 151. It seems the model doesn't handle boundaries well.

Is there a way to process with overlapping windows (like a sliding window with crossfade) to smooth out boundary jumps? Or any other recommended approach to maintain temporal continuity across groups?

[Selur]

BasicVSR++ had nothing for this,... see: https://github.com/HolyWu/vs-basicvsrpp
Maybe something for this could be done through the newly scene change base 'only apply to' controls,... or something similar,...

Can you share a clip which allows to reproduce the problem?

Cu Selur

[ywcz1997]

I've attached a short clip showing the problem. I processed the DVD source with BasicVSR++ using length=15 and length=150. In both cases, pay close attention to the mist/fog in the background – you'll see noticeable jumps at regular intervals.

[Selur]

off to work now, but will look at it later after work,..

[Selur]

Looking at your encode:
Frame with index 149 => 150
       
I see the problem, BasicVSR++ is too aggressive, due to the low resolution. 704x480 with PAR 10:11 is 640x480 or 704x528 depending on how you compensate the PAR.
Only way around that is to feed it a higher resolution video. BasicVSR++ is less aggressive on higher resolution input.
Using BasicVSR++ ( "BasicVSRPP(clip, model=4, length=150)" with FP16), but telling Hybrid beforehand to resize width&height by a factor of 2 (lower values might also work; you probably will have to test). Not knowing what your goal is, I let Hybrid downscale afterward. (Controls for this are only visible if "Filtering->Vapoursynth->Misc->UI->Show 'Gimmick'-controls" is enabled)
Frame with index 149 => 150:
       
The downside with this is, that the filtering is a lot slower, since 4x the amount of pixel are processed.
So I would recommend: Stick with length 15 and try what resize factor causes BasicVSR++ not to over filter and is still acceptable in speed.
(use Vapoursynth Preview to check before encoding)

Cu Selur

Ps.: larger length values in BasicVSR++ usually mainly help if you have long lasting defects,..