ArtCNN issue/feature request - Printable Version +- Selur's Little Message Board (https://forum.selur.net) +-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html) +--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html) +--- Thread: ArtCNN issue/feature request (/thread-4261.html)

ArtCNN issue/feature request - GunplaXross2 - 10.11.2025 When trying to run ArtCNN models in VSMLRT, be it TRT, DML, or ONNX Cuda modes, it produces an error.  According to the github,  "ArtCNN is supported by vs-mlrt".  It seems to be complaining about the type of data being fed to it, so possibly may need an override for planes to feed, or a workaround?  "vapoursynth.Error: operator (): expects 1 input planes" The common models that people use are R16F96  Relevant github: https://github.com/Artoriuz/ArtCNN The R16F96 and R8F64 are commonly used in encoding tasks as of recent, so it'd be useful to have them available.  Specifically, the regular and chroma variants; a la ArtCNN_R8F64 and ArtCNN_R8F64_Chroma.   artcnn_HybridDebugOutput.zip (Size: 3,5 KB / Downloads: 2) Edit: Just realize the files didn't have the 1x/2x at the beginning, even when renaming the files, same plane error occurs. RE: ArtCNN issue/feature request - Selur - 11.11.2025 Hybrid atm. uses vsmlrt.inference to call general models, for ArtCNN (like with for example RIFE, SCUNET, DPIR a separate class needs to be used) one needs to call ArtCNN: def ArtCNN(     clip: vs.VideoNode,     tiles: typing.Optional[typing.Union[int, typing.Tuple[int, int]]] = None,     tilesize: typing.Optional[typing.Union[int, typing.Tuple[int, int]]] = None,     overlap: typing.Optional[typing.Union[int, typing.Tuple[int, int]]] = None,     model: ArtCNNModel = ArtCNNModel.ArtCNN_C16F64,     backend: backendT = Backend.OV_CPU() ) -> vs.VideoNode: and only specific models are supported: class ArtCNNModel(enum.IntEnum):     ArtCNN_C4F32 = 0     ArtCNN_C4F32_DS = 1     ArtCNN_C16F64 = 2     ArtCNN_C16F64_DS = 3     ArtCNN_C4F32_Chroma = 4     ArtCNN_C16F64_Chroma = 5     ArtCNN_R16F96 = 6     ArtCNN_R8F64 = 7     ArtCNN_R8F64_DS = 8     ArtCNN_R8F64_Chroma = 9     ArtCNN_C4F16 = 10     ArtCNN_C4F16_DS = 11     ArtCNN_R16F96_Chroma = 12     ArtCNN_C4F16_DN = 13     ArtCNN_C4F32_DN = 14     ArtCNN_R8F64_JPEG420 = 15     ArtCNN_R8F64_JPEG444 = 16 Atm. you would need to write some custom code to use this. ArtCNN seems to expect either GRAYH or GRAYS input,... Something like: # adjusting color space from YUV420P8 to GRAYH for ArtCNN clip = core.resize.Bicubic(clip=clip, format=vs.GRAYH, matrix_in_s="470bg") # filtering using VSMLRT clip = vsmlrt.ArtCNN(clip,model=2, backend=Backend.TRT_RTX(fp16=True,device_id=0,num_streams=1,verbose=True,use_cuda_graph=True,workspace=1073741824,builder_optimization_level=3,engine_folder="J:/TRT")) does seem to do something, but no clue atm. how to properly hand this. Are ArtCNN Models meant to be only applied on a specific plane of the source? Looking at https://github.com/Artoriuz/ArtCNN it seems like different models expect different input types. => I'll try to look at it in the next few days (probably won't get around to it today), if you can provide some links to documentation (and/or examples) how to use these, please do. Cu Selur Ps.: I'll also have to update the vs-mlrt add-on since atm. it only comes with a few of the ArtCNN models. RE: ArtCNN issue/feature request - Selur - 11.11.2025 Had a quick go at it, and I can easily implement support for it. ArtCNN_C4F32 = 0 supports GRAYS, GRAYH     ArtCNN_C4F32_DS =  supports GRAYS, GRAYH     ArtCNN_C16F64 = 2 supports GRAYS, GRAYH     ArtCNN_C16F64_DS = 3 supports GRAYS, GRAYH     ArtCNN_C4F32_Chroma = 4 supports YUV444PS, YUV44PH     ArtCNN_C16F64_Chroma = 5 supports YUV444PS, YUV44PH     ArtCNN_R16F96 = 6 supports GRAYS, GRAYH     ArtCNN_R8F64 = 7 supports GRAYS, GRAYH     ArtCNN_R8F64_DS = 8 supports GRAYS, GRAYH     ArtCNN_R8F64_Chroma = 9 supports YUV444PS, YUV44PH     ArtCNN_C4F16 = 10 supports GRAYS, GRAYH     ArtCNN_C4F16_DS = 11 supports GRAYS, GRAYH     ArtCNN_R16F96_Chroma = 12 supports YUV444PS, YUV44PH     ArtCNN_C4F16_DN = 13 supports GRAYS, GRAYH     ArtCNN_C4F32_DN = 14 supports GRAYS, GRAYH     ArtCNN_R8F64_JPEG420 = 15 supports RGBS, RGBH     ArtCNN_R8F64_JPEG444 = 16 supports RGBS, RGBH But I still need to know what to do with the Gray color spaces. Should just the luma be filtered and the chroma should get copied? Should each plane be filtered differently? I suspect: # Imports import vapoursynth as vs # getting Vapoursynth core import logging import site import sys import os core = vs.core # Limit frame cache to 48449MB core.max_cache_size = 48449 # Import scripts folder scriptPath = 'F:/Hybrid/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) os.environ["CUDA_MODULE_LOADING"] = "LAZY" # Force logging to std:err logging.StreamHandler(sys.stderr) # loading plugins core.std.LoadPlugin(path="F:/Hybrid/64bit/vs-mlrt/vstrt_rtx.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vs-mlrt/vstrt.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vs-mlrt/vsort.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/BestSource/BestSource.dll") # Import scripts from importlib.machinery import SourceFileLoader vsmlrt = SourceFileLoader('vsmlrt', 'F:/Hybrid/64bit/vs-mlrt/vsmlrt.py').load_module() from vsmlrt import Backend import validate # Source: 'G:\TestClips&Co\files\test.avi' # Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual # Loading G:\TestClips&Co\files\test.avi using BestSource) clip = core.bs.VideoSource(source="G:/TestClips&Co/files/test.avi", cachepath="J:/tmp/test_bestSource", track=0, hwdevice="opencl") frame = clip.get_frame(0) # setting color matrix to 470bg. clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG) # setting color transfer (vs.TRANSFER_BT601), if it is not set. if validate.transferIsInvalid(clip):   clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601) # setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set. if validate.primariesIsInvalid(clip):   clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG) # setting color range to TV (limited) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED) # making sure frame rate is set to 25fps clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # making sure the detected scan type is set (detected: progressive) clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE)# progressive # changing range from limited to full range for vsVSMLRTFilter clip = core.resize.Bicubic(clip, format=vs.YUV420P8, range_in_s="limited", range_s="full") # setting color range to PC (full) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL) # adjusting color space from YUV420P8 to YUV444PH for vsVSMLRTFilter clip = core.resize.Bicubic(clip=clip, format=vs.YUV444PH) # filtering using VSMLRT clipY,clipU,clipV = core.std.SplitPlanes(clip)# split YUV planes to filter Y only clip = vsmlrt.ArtCNN(clipY,model=3, backend=Backend.TRT_RTX(fp16=True,device_id=0,num_streams=1,verbose=True,use_cuda_graph=True,workspace=1073741824,builder_optimization_level=3,engine_folder="J:/TRT")) clip = core.std.ShufflePlanes(clips=[clipY, clipU, clipV], planes=[0, 0, 0], colorfamily=vs.YUV)# merge YUV planes # changing range from full to limited range for vsVSMLRTFilter clip = core.resize.Bicubic(clip, format=vs.YUV444PH,range_in_s="full", range_s="limited") # setting color range to TV (limited) range. clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED) # no resizing since resolution is already archived # adjusting output color from: YUV444PH to YUV420P10 for NVEncModel clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, dither_type="error_diffusion") # set output frame rate to 25fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) # output clip.set_output() would be a correct way to apply one of the GRAY only models, but I'm not sure. => Can you answer this? Cu Selur Ps.: Based on my assumption, I uploaded a new dev and vs-mlrt add-on with ArtCNN support in VSMLRT(Filter).