[BUG] VSGAN models produce green frames, and more

[Selur]

I'm uploading a new R65 download (just used the old download and used the steps I mentioned).
If that version also throws the same error, you might have installed something that is causing issues,...
(upload should be finished in ~1hr)

Cu Selur

[Woozy6528]

I'm uploading a new R65 download (just used the old download and used the steps I mentioned).
If that version also throws the same error, you might have installed something that is causing issues,...

Hey, you are amazing. Danke. : )

This also doesn't work, BUT: I do see the issue now (well, THIS issue)... My own install (thanks to your post 🙂) and also your install (in today's R65 package) show file sizes of 9 bytes for 4 of the 7 models - including the one I always use (General). So I tried the install again - same issue. They don't download properly:



So I copied the two General models from your R68 package, and changed the underscores to hyphens in their file names - and now it works. : )

Cheers

[Selur]

Thanks for the Info.
Deleted the R65 from my GoogleDrive, since the next step is R69 and keeping old stuff simply takes to much space.

[Woozy6528]

Hi there,

I'm digging this up again regarding VS-Basic-VSR++, which still doesn't work on my GTX 1650 even with the new VapourSynth release from your Google Drive experimental folder, unlike the older R65 release(s), which worked no problem.

I just found out it does work with R68 - if I reduce the sequence length to 1 frame. Choosing more than 1 frame results in the usual errors - in Preview I get this:



And if I were to start the job, Hybrid would finish it without recognizing something went wrong:



Maybe this is helpful. : )

[Selur]

Maybe this is helpful. : )

No, some screenshots do not help me.
Maybe a debug output of you calling the Vapoursynth Preview includes some hints.
I suspect that some of the dependencies simply do not support cards from the 1XXX branch.
So you probably have to stick to the old R65 release. (since upgrading the gpu probably isn't possible)

Cu Selur

[humanoid86]

Green frames are direct proof of incompatibility.

in your case: your video card is too ancient/old for the new Vapoursynth code

you need a fresh/new GPU, minimum nvidia rtx30

+update the GPU driver (not sure that will help)

++or use the old version(Vapoursynth) and forget about the update

[Woozy6528]

I might have figured out what the issue is with those green frames I get using VSGAN on my system. Hybrid is setting the color range to RGBH. When I add the VSGAN filter myself using the Custom tab, and set the color range to RGBS, it totally works - no green frames. I reckon I could've seen that a long time ago. ; )

Why BasicVSR++ doesn't work, I haven't figured out though. Here is a script comparison between Hybrid 2024.06.09.1 with Vapoursynth R65_2024.07.17 (left pane, BasicVSR++ works) and Hybrid 2024.12.21.1 with Vapoursynth R70_2024.12.27 (right pane, BasicVSR++ doesn't work). Maybe it's helpful.



It's still somehow related to the time component. If I set Length for BasicVSR++ to 1 frame, it works. Having said all this, I don't use BasicVSR++ anymore, so I don't really care much.

Cheers

[Selur]

The latest dev also comes with a newer BasicVSR++ version which drops mmc dependencies, maybe that changes something.

[Woozy6528]

The latest dev also comes with a newer BasicVSR++ version which drops mmc dependencies, maybe that changes something.

I'm downloading the dev version and latest Vapoursynth right now, and I'll test this later. Thanks!

[Selur]

Hmm,.. I got the same issue now here on a source.
Encoding with BasicVSR++ and strength > 1 causes a crash, seems to be the combination of some filters.

clip = core.tivtc.TFM(clip=clip)
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting color space from YUV420P8 to RGBH for vsBasicVSRPPFilter
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# Quality enhancement using BasicVSR++
from vsbasicvsrpp import basicvsrpp as BasicVSRPP
clip = BasicVSRPP(clip=clip, model=4)
# adjusting color space from RGBH to YUV444P16 for vsGLSLAdaptiveSharpen
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, matrix_s="470bg", range_s="limited")
with open("F:/Hybrid/64bit/vsfilters/GLSL/parameterized/adaptive-sharpen.glsl") as glslf:
  glsl = glslf.read()
glsl = glsl.replace('#define curve_height    1.0', '#define curve_height    1.0000')
glsl = glsl.replace('#define anime_mode      false', '#define anime_mode    true')
glsl = glsl.replace('#define overshoot_ctrl  false', '#define overshoot_ctrl  true')
glsl = glsl.replace('#define video_level_out false', '#define video_level_out true')
clip = core.placebo.Shader(clip=clip, shader_s=glsl, width=clip.width, height=clip.height)
# Using Anime4k Darken GLSL filter for line darkening
with open("F:/Hybrid/64bit/vsfilters/GLSL/parameterized/Anime4K_Darken_HQ.glsl") as glslf:
  glsl = glslf.read()
glsl = glsl.replace('#define STRENGTH 1.5', '#define STRENGTH 1.5')
clip = core.placebo.Shader(clip=clip, shader_s=glsl, width=clip.width, height=clip.height)
# Resizing using 10 - bicubic spline
clip = core.fmtc.resample(clip=clip, kernel="spline16", w=960, h=720, interlaced=False, interlacedd=False) # resolution 960x720 before YUV444P16 after YUV444P16
# adjusting output color from: YUV444P16 to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited", dither_type="error_diffusion")
# set output frame rate to 23.976fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# output

works

clip = core.tivtc.TFM(clip=clip)
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# denoising using MCTemporalDenoise
clip = havsfunc.MCTemporalDenoise(i=clip, settings="very high", ncpu=1, cuda=True)
# adjusting color space from YUV420P8 to RGBH for vsBasicVSRPPFilter
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# Quality enhancement using BasicVSR++
from vsbasicvsrpp import basicvsrpp as BasicVSRPP
clip = BasicVSRPP(clip=clip, model=4)
# adjusting color space from RGBH to YUV444P16 for vsGLSLAdaptiveSharpen
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, matrix_s="470bg", range_s="limited")
with open("F:/Hybrid/64bit/vsfilters/GLSL/parameterized/adaptive-sharpen.glsl") as glslf:
  glsl = glslf.read()
glsl = glsl.replace('#define curve_height    1.0', '#define curve_height    1.0000')
glsl = glsl.replace('#define anime_mode      false', '#define anime_mode    true')
glsl = glsl.replace('#define overshoot_ctrl  false', '#define overshoot_ctrl  true')
glsl = glsl.replace('#define video_level_out false', '#define video_level_out true')
clip = core.placebo.Shader(clip=clip, shader_s=glsl, width=clip.width, height=clip.height)
# Using Anime4k Darken GLSL filter for line darkening
with open("F:/Hybrid/64bit/vsfilters/GLSL/parameterized/Anime4K_Darken_HQ.glsl") as glslf:
  glsl = glslf.read()
glsl = glsl.replace('#define STRENGTH 1.5', '#define STRENGTH 1.5')
clip = core.placebo.Shader(clip=clip, shader_s=glsl, width=clip.width, height=clip.height)
# Resizing using 10 - bicubic spline
clip = core.fmtc.resample(clip=clip, kernel="spline16", w=960, h=720, interlaced=False, interlacedd=False) # resolution 960x720 before YUV444P16 after YUV444P16
# adjusting output color from: YUV444P16 to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited", dither_type="error_diffusion")
# set output frame rate to 23.976fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# output
clip.set_output()

crashes. (only during encoding preview works fine)

Okay, seems like the problem for me lied with using dfttest2, without it the encoding works fine. So my guess is that your problem is either through something using some cuda (NVRTC) stuff during the encoding.

Cu Selur