ArtCNN issue/feature request

[Selur]

Hybrid atm. uses vsmlrt.inference to call general models, for ArtCNN (like with for example RIFE, SCUNET, DPIR a separate class needs to be used) one needs to call ArtCNN:

def ArtCNN(
    clip: vs.VideoNode,
    tiles: typing.Optional[typing.Union[int, typing.Tuple[int, int]]] = None,
    tilesize: typing.Optional[typing.Union[int, typing.Tuple[int, int]]] = None,
    overlap: typing.Optional[typing.Union[int, typing.Tuple[int, int]]] = None,
    model: ArtCNNModel = ArtCNNModel.ArtCNN_C16F64,
    backend: backendT = Backend.OV_CPU()
) -> vs.VideoNode:

and only specific models are supported:

class ArtCNNModel(enum.IntEnum):
    ArtCNN_C4F32 = 0
    ArtCNN_C4F32_DS = 1
    ArtCNN_C16F64 = 2
    ArtCNN_C16F64_DS = 3
    ArtCNN_C4F32_Chroma = 4
    ArtCNN_C16F64_Chroma = 5
    ArtCNN_R16F96 = 6
    ArtCNN_R8F64 = 7
    ArtCNN_R8F64_DS = 8
    ArtCNN_R8F64_Chroma = 9
    ArtCNN_C4F16 = 10
    ArtCNN_C4F16_DS = 11
    ArtCNN_R16F96_Chroma = 12
    ArtCNN_C4F16_DN = 13
    ArtCNN_C4F32_DN = 14
    ArtCNN_R8F64_JPEG420 = 15
    ArtCNN_R8F64_JPEG444 = 16

Atm. you would need to write some custom code to use this.
ArtCNN seems to expect either GRAYH or GRAYS input,...
Something like:

# adjusting color space from YUV420P8 to GRAYH for ArtCNN
clip = core.resize.Bicubic(clip=clip, format=vs.GRAYH, matrix_in_s="470bg")
# filtering using VSMLRT
clip = vsmlrt.ArtCNN(clip,model=2, backend=Backend.TRT_RTX(fp16=True,device_id=0,num_streams=1,verbose=True,use_cuda_graph=True,workspace=1073741824,builder_optimization_level=3,engine_folder="J:/TRT"))

does seem to do something, but no clue atm. how to properly hand this. Are ArtCNN Models meant to be only applied on a specific plane of the source?
Looking at https://github.com/Artoriuz/ArtCNN it seems like different models expect different input types.


=> I'll try to look at it in the next few days (probably won't get around to it today), if you can provide some links to documentation (and/or examples) how to use these, please do.

Cu Selur

Ps.: I'll also have to update the vs-mlrt add-on since atm. it only comes with a few of the ArtCNN models.