Selur's Little Message Board

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89

Hello Selur,

I do agree with you that moving inside vsdeoldify the grain/noise removal it will be possible to use a more "bold" approach.
I opted for writing the following degrain function:

Code:
def vs_degrain(clip: vs.VideoNode = None, strength: int = 0) -> vs.VideoNode:

    if strength == 0:

        return clip

    else: 

        import havsfunc

    match strength:

        case 1:       

          dstr = 1.0

        case 2:

          dstr = 1.5

        case 3:

          dstr = 2.5

        case 4:

          dstr = 3.5

        case _:

            raise vs.Error("ddeoldify: not supported strength value: " + mode)      

    clip = clip.resize.Bicubic(format=vs.YUV444PS, matrix_s="709", range_s="full")   

    clip = havsfunc.KNLMeansCL(clip=clip, d=1, a=2, s=4, h=dstr, device_type="gpu", device_id=0)

    clip = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="full", dither_type="error_diffusion") 

    return clip

I decided to use KNLMeansCL because it is fast and can remove both grain and noise. I simplified the input and to the functions ddeoldify_main() and ddeoldify() I added only 1 parameter, that control the strenght of grain/noise removal, called "degrain_strength" (if = 0, is disabled).

I attached the new RC7.

Thanks,
Dan

Will look at it tomorrow.

Okay, took a look.
Unless I overlook something, this is different from what I suggested. (filtering for deoldify, but keeping the luma details)
=> not implementing this
This is the same as applying the denoising before deoldify which Hybrid already can do, but in Hybrid you can use any denoiser not just KNLMeansCL.
This does make sense when deoldify is used outside of Hybrid, but in Hybrid, adding this seems like a step back.

Cu Selur

The implemented denoise/degrain will preserve the luma details.

The main code is the following:

Code:
def ddeoldify(

    clip: vs.VideoNode, method: int = 2, mweight: float = 0.5, deoldify_p: list = [0, 24, 1.0, 0.0], ddcolor_p: list = [1, 24, 1.0, 0.0, True], dotweak: bool = False, dotweak_p: list = [0.0, 1.0, 1.0, False, 0.2, 0.5, 1.5, 0.5], ddtweak: bool = False, ddtweak_p: list = [0.0, 1.0, 2.5, True, 0.3, 0.6, 1.5, 0.5, "300:360|0.8,0.1"],  degrain_strength: int = 0, cmc_tresh: float = 0.2, lmm_p: list = [0.2, 0.8, 1.0], alm_p: list = [0.8, 1.0, 0.15], cmb_sw: bool = False, device_index: int = 0, torch_dir: str = model_dir) -> vs.VideoNode:

...

...

chroma_resize = True  # always enabled

...

...

if chroma_resize:

        frame_size = min(max(ddcolor_rf, deoldify_rf) * 16, clip.width)     # frame size calculation for inference()  

        clip_orig = clip;

        clip = clip.resize.Spline64(width=frame_size, height=frame_size) 

try:

     d_clip = vs_degrain(clip, degrain_strength)

except:

    vs.core.log_message(2, "Degrain filter: havsfunc.KNLMeansCL is not installed/loaded properly") 

    d_clip = clip  

clip = d_clip

...

...

if chroma_resize:

        return _clip_chroma_resize(clip_orig, clip_colored)

    else:

        return clip_colored

Since it was decided to remove the flag "chroma_resize", now this flag is always enabled.
This implies that no matter all the changes that are made to the luma by the filters, at the end the original luma is always restored.

Dan

P.S.
I discovered a bug in managing the exception on vs_degrain, I attached the correct version.

Ahh,..
chroma_resize_enabled is always true, which calls _clip_chroma_resize, which calls vs_recover_clip_luma
which is:

Code:
def vs_recover_clip_luma(orig: vs.VideoNode = None, clip: vs.VideoNode = None) -> vs.VideoNode:

    def copy_luma_frame(n, f):    

        img_orig = frame_to_image(f[0])

        img_clip = frame_to_image(f[1])        

        img_m = chroma_post_process(img_clip, img_orig)

        return image_to_frame(img_m, f[0].copy())                

    clip = clip.std.ModifyFrame(clips=[orig, clip], selector=copy_luma_frame)

    return clip

def chroma_post_process(img_m: Image, orig: Image) -> Image:

    img_np = np.asarray(img_m)

    orig_np = np.asarray(orig)

    img_yuv = cv2.cvtColor(img_np, cv2.COLOR_RGB2YUV)

    # copy the chroma parametrs "U", "V", of "img_m" in "orig" 

    orig_yuv = cv2.cvtColor(orig_np, cv2.COLOR_RGB2YUV)

    orig_copy = np.copy(orig_yuv)

    orig_copy[:, :, 1:3] = img_yuv[:, :, 1:3]

    img_np_new = cv2.cvtColor(orig_copy, cv2.COLOR_YUV2RGB)

    return Image.fromarray(img_np_new)

which hopefully does the correct thing and is similar fast do Vapoursynth ShufflePlanes. Smile

Cu Selur

Why are you using havsfunc?

Code:
clip = havsfunc.KNLMeansCL(clip=clip, d=1, a=2, s=4, h=dstr, device_type="gpu", device_id=0)

better use KNLMeansCL directly.

Code:
clip = core.knlm.KNLMeansCL(clip=clip, d=1, a=2, s=4, h=dstr, channels='Y', device_type="gpu", device_id=0)

No need to add another dependency. Smile

https://github.com/Selur/VapoursynthScri...c.py#L5948
is only used to filter 'Y' and 'UV' separately to speed it up, but in your case there is no need to filter 'UV' at all. Wink

Cu Selur

Ps.: uploaded a test version which does include KNLMeans, but not havsfunc.

Thanks for the advice, I will update the code.

Dan

No, problem will release a new Hybrid tomorrow then, assuming you release a new DeOldify.

Cu Selur

Ok, I will also release the new version tomorrow.

Thanks,
Dan

(04.05.2024, 12:55)Selur Wrote: [ -> ]Ps.: uploaded a test version which does include KNLMeans, but not havsfunc.

I rewrote the "degrain" function as following

Code:
def vs_degrain(clip: vs.VideoNode = None, strength: int = 0, device_id: int = 0) -> vs.VideoNode:

    if strength == 0:

        return clip

    match strength:

        case 1:       

          dstr = 1.5

          dtmp = 1

        case 2:

          dstr = 2.5

          dtmp = 1

        case 3:

          dstr = 3.5

          dtmp = 1

        case 4:

          dstr = 5.5

          dtmp = 2

        case 5:

          dstr = 8.5

          dtemp = 2

        case _:

            raise vs.Error("ddeoldify: not supported strength value: " + strength)      

    clip = clip.resize.Bicubic(format=vs.YUV444PS, matrix_s="709", range_s="full")   

    clip = vs.core.knlm.KNLMeansCL(clip=clip, d=dtemp, a=2, s=4, h=dstr, channels='Y', device_type="gpu", device_id=device_id)

    clip = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="full", dither_type="error_diffusion") 

    return clip

I prefer that in Hybrid the Degrain will display only integer values from 1 to 5.
So that in the future I can decide to implement a different mapping and/or denoise/degrain function, without the need to change the input.

If you prefer you can call this parameter also "DegrainFactor".

Thanks

Uploaded a new test version.

Cu Selur

Dan64

Selur

Selur

Dan64

Selur

Selur

Dan64

Selur

Dan64

Selur