This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm whether you accept or reject these cookies being set.

A cookie will be stored in your browser regardless of choice to prevent you being asked this question again. You will be able to change your cookie settings at any time using the link in the footer.

Deoldify Vapoursynth filter
Hello Selur,

   In the current dev are missing the following values in BlackWhiteTune:
'Light+Custom',
'Medium+Custom',
'Strong+Custom'
'Light+Restore',
'Medium+Restore',
'Strong+Restore'
 
If you don't like this solution, you can add above "B&W mode" another combo-box called "B&W Restore" with the following values

'none'
'custom'
'default'
 
The parameter "B&W Restore" in the code will be called BlackWhiteRestore and should have the same behaviour of "B&W mode"
 
Please let me know what do you prefer so that I can provide a new RC with the new parameter.

Thanks,
Dan
Reply
Quote: In the current dev are missing the following values in BlackWhiteTune:
That is correct, like I wrote:
Quote:That seems unnecessary, since using 'light/medium/strong' + enabling 'Exemplar Models' is the same as 'light/medium/strong+Custom'.
And 'light/medium/strong+Restore' does not make any sense to me. (since it should be a preset for 'exemplar models')
From what you described, those are presets for other options, which still could be adjusted manually.

Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply
Just to clarify better these are the situations that need to be managed in Hybrid:

1) standard HAVC without BlackWhiteTune adjustments  -> managed with BlackWhiteTune='none'
2) standard HAVC with BlackWhiteTune adjustments  -> managed with BlackWhiteTune != 'none'
3) standard HAVC with BlackWhiteTune Restore (it is a more complex strategy that will be explained in the guide)  -> settings = ?
4) apply BlackWhiteTune adjustments on a clip already colored -> managed with BlackWhiteTune != 'none'  and DeepExMethod = 5
5) apply BlackWhiteTune Restore  (with user parameters) on a clip already colored -> settings ? 
6) apply BlackWhiteTune Restore (with optimal parameters) on a clip already colored -> settings ?

with the current GUI I'm able to manage only 3 cases out of 6.

Dan
Reply
I thought the GUI would only cover 1&2. Smile

But from the looks of it, I don't really understand what BlackWhiteTune should do.
I assumed it would always be applied after on the colored image.
Quote:This parameter allows to improve contrast and luminosity of frames colored with HAVC.

Quote:in the case are used the settings with "+custom" will be used the parameters specified in the "Exemplar Model", in this case should be used the ColorMNet model (but the user can try also the other models). If are used the settings with "+restore", the user settings will be ignored and will be used the internal settings.
Is my conclusion that:
a. BlackWhiteTune with '+custom' requires 'Exemplar Model' turned on, correct?
(So in the gui, if you select a '+custom' BlackWhiteTune value, the 'Exemplar Model' would need to be forced to be enabled. User can't disable it.)
If this is correct, the 'custom' BlackWhiteTune values should not be there, since adding an option to basically enforcing 'Exemplar Model' doesn't make sense.
If this is not correct, try to explain it, please.

b. BlackWhiteTune with '+restore' requires 'Exemplar Model' turned on and disabled and also set to specific settings, correct?
(So in the gui, if you select a '+restore' BlackWhiteTune value, the 'Exemplar Model' would need to be forced to be enabled. User can't disable it. And the content of the GroupBox would need to stay disabled while forced to specific values.)
If this is correct, then this the 'restore' values should not be there, instead there should an additional sub option i.e. AutoBlackWhite which is basically enforcing preset settings for 'Exemplar Model'.
If this is wrong, try to explain it, please.

Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply
Ok, I will try to explain better.

Case: 1 & 2 are simply to understand and should be the cases more frequent.
Case: 4, I added this case to be able to quickly adjust the colored movies with the previous version of HAVC, in this case is not necessary to recolor again the clip.
Case: 3, This is a complex strategy (which requires at last a RTX 4070ti) which split the color processes in 2 steps: 
       1) in the first step will be used HAVC with Deep-Exemplar and freq=5, just to speed-up the first coloring step (BlackWhiteTune will not be used in this step);    
       2) in the second step the clip previously colored will be stabilized using ColorMNet, which provides dull colors, and will be applied BlackWhiteTune to improve the final colors 
Case: 6, is similar to case 4, but in this case  BlackWhiteTune will be applied using the process described in Case 3.2
Case: 5, is similar to 6, but in this case the stabilization parameters will be taken from the GUI.

The need to add  "B&W Restore"  is to manage the cases 3, 5, 6. In all these cases the "Exemplar Models" will be used twice with different values. 
To simplify the GUI management I proposed to add  "B&W Restore".

Dan
Reply
Case 1&2: should be covered by Hybrid as it is atm., correct?

Quote: Case: 4, I added this case to be able to quickly adjust the colored movies with the previous version of HAVC, in this case is not necessary to recolor again the clip.
What? Please rephrase that.
Did you want to write:
Quote:I added this case to be able to quickly adjust the already colored movies, which were colored using the previous version of HAVC.
In this case, it is not necessary to recolor again the clip.
If that is what you wanted to say, then this should be a separate filter in Hybrid (not a option of HAVC_main) which should be called using for example with something like
clip = havc.ColorPostProcessing(clip, tune=.., mode=...)
.

Quote: Case: 6, is similar to case 4, but in this case BlackWhiteTune will be applied using the process described in Case 3.2
If this too is meant to be applied to improve and stabilize an already colored clip, then this too should not be part of HAVC_main.
The ColorPostProcessing, should be extended to
clip = havc.ColorPostProcessing(clip, tune=.., mode=..., stabilize=True)
.
(side note: such a filter would be located under Color->Misc not Color->Coloring in Hybrid and maybe named 'ColorImprovement(HAVC)')

Quote:Case: 5, is similar to 6, but in this case the stabilization parameters will be taken from the GUI.
Okay,.. then you should extend ColorPostProcessing with the necessary options.


Quote:Case: 3, This is a complex strategy (which requires at last a RTX 4070ti) which split the color processes in 2 steps:
1) in the first step will be used HAVC with Deep-Exemplar and freq=5, just to speed-up the first coloring step (BlackWhiteTune will not be used in this step);
2) in the second step the clip previously colored will be stabilized using ColorMNet, which provides dull colors, and will be applied BlackWhiteTune to improve the final colors
Sounds to me, like this should be: done with two filter calls (user enabling two separate filters)
1. 'HAVC' (havc.HAVC_main(...))
2. 'ColorImprovement(HAVC)' (havc.ColorPostProcessing(...))

Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply
Hello Selur,

  I attached the new RC3. I added the function ColorPostProcessing
 
def ColorPostProcessing(clip: vs.VideoNode, BlackWhiteTune: str = 'Medium', BlackWhiteMode: int = 2,
                        ReColor: bool = True, ScThreshold: float = 0.10, ScNormalize: bool = True,
                        DeepExVivid: bool = True,  ScMinFreq: int = 0) -> vs.VideoNode:
    """HAVC Color Post Processing function

        :param clip:               clip to process, only RGB24 format is supported.
        :param BlackWhiteTune:     This parameter allows to improve contrast and luminosity of frames colored with HAVC.
                                   Allowed values are:
                                        'None' (default)
                                        'Light',
                                        'Medium',
                                        'Strong'
        :param BlackWhiteMode:     Method used by BlackWhiteTune to perform colors adjustments.
                                   Allowed values are:
                                          0 : Apply Contrast Limited Adaptive Histogram Equalization on Luma
                                          1 : Apply Simple Histogram Equalization on all RGB channels
                                          2 : Apply CLAHE on all RGB channels (default)
                                          3 : method=0 and method=1 are merged
                                          4 : Automatic brightness and contrast optimization with ScaleAbs
                                          5 : Automatic brightness and contrast optimization with saturation arithmetic
        :param ReColor:            If True the clip will re-colored with ColorMNet
        :param ScThreshold:        Scene change threshold used to generate the reference frames to be used by
                                   ColorMNet. It is a percentage of the luma change between the previous and the
                                   current frame. range [0-1], default 0.10. If =0 are not generate reference frames.
                                   default = 0.10
        :param ScNormalize:        If true the frames are normalized before using misc.SCDetect(), the normalization
                                   will increase the sensitivity to smooth scene changes, range [True, False],
                                   default: True
        :param DeepExVivid:        if enabled (True) the ColorMNet memory is reset at every reference frame update
                                   range [True, False], default: True
        :param ScMinFreq:          if > 0 will be generated at least a reference frame every "ScMinFreq" frames.
                                   range [0-1500], default: 0.
    """


  This function not only is able to perform color adjustments, but if the boolean parameter ReColor is enabled (to be used only if the input was colored with HAVC), the input clip will be stabilized using ColorMNet, all the remaing parameters are related to the ReColor process.

   I hope that will be able to converge to a solution.
 
Dan


Attached Files
.zip   vsdeoldify-5.5.0_RC3.zip (Size: 452,3 KB / Downloads: 11)
Reply
Won't get around to implement anything today. (will work on it tomorrow)
With this I would:
a. keep HAVC as it is in the latest test version (with BlackWhiteTune&BlackWhiteMode)
b. add an additional filter 'ColorPostProcessing (HAVC)' I'll put in the FilterOrder directly behind HAVC.

About the current documentation:
1.
def ColorPostProcessing(clip: vs.VideoNode, BlackWhiteTune: str = 'Medium', BlackWhiteMode: int = 2,
                        ReColor: bool = True, ScThreshold: float = 0.10, ScNormalize: bool = True,
                        DeepExVivid: bool = True,  ScMinFreq: int = 0) -> vs.VideoNode:
    """HAVC Color Post Processing function

        :param clip:               clip to process, only RGB24 format is supported.
        :param BlackWhiteTune:     This parameter allows to improve contrast and luminosity of frames colored with HAVC.
                                   Allowed values are:
                                        'None' (default)
                                        'Light',
                                        'Medium',
                                        'Strong'
the description and the code "BlackWhiteTune: str = 'Medium" vs "'None' (default)" do not match.

2. ReColor
ReColor:            If True the clip will re-colored with ColorMNet
vs.
Quote:if the boolean parameter ReColor is enabled (..), the input clip will be stabilized using ColorMNet, ...
Will the clip get recolored or will colors get stabilized?

Quote:to be used only if the input was colored with HAVC
Why? Shouldn't it also work with other content?

Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply
I attached the new RC4
I changed the description and added a new parameter Strength
def ColorPostProcessing(clip: vs.VideoNode, BlackWhiteTune: str = 'Medium', BlackWhiteMode: int = 2,
                        ReColor: bool = True, Strength: int = 1, ScThreshold: float = 0.10, ScNormalize: bool = True,
                        DeepExVivid: bool = True,  ScMinFreq: int = 0) -> vs.VideoNode:
    """HAVC Color Post Processing function

        :param clip:               clip to process, only RGB24 format is supported.
        :param BlackWhiteTune:     This parameter allows to improve contrast and luminosity of frames colored with HAVC.
                                   Allowed values are:
                                        'Light',
                                        'Medium', (default)
                                        'Strong'
        :param BlackWhiteMode:     Method used by BlackWhiteTune to perform colors adjustments.
                                   Allowed values are:
                                          0 : Apply Contrast Limited Adaptive Histogram Equalization on Luma
                                          1 : Apply Simple Histogram Equalization on all RGB channels
                                          2 : Apply CLAHE on all RGB channels (default)
                                          3 : method=0 and method=1 are merged
                                          4 : Automatic brightness and contrast optimization with ScaleAbs
                                          5 : Automatic brightness and contrast optimization with saturation arithmetic
        :param ReColor:            If True the clip will re-colored with ColorMNet to enforce color temporal
                                   stabilization. To be used if the clip was colored using an AI automatic video
                                   colorizer like HAVC.
        :param Strength:           Color temporal stabilization strength, using high level the colors will be more stable but
                                   will be also more washed.
                                   Allowed values are:
                                        1 = VeryLow  (default)
                                        2 = Low
                                        3 = Med
                                        4 = High
                                        5 = VeryHigh
        :param ScThreshold:        Scene change threshold used to generate the reference frames to be used by
                                   ColorMNet. It is a percentage of the luma change between the previous and the
                                   current frame. range [0-1], default 0.10. If =0 are not generate reference frames.
                                   default = 0.10
        :param ScNormalize:        If true the frames are normalized before using misc.SCDetect(), the normalization
                                   will increase the sensitivity to smooth scene changes, range [True, False],
                                   default: True
        :param DeepExVivid:        if enabled (True) the ColorMNet memory is reset at every reference frame update
                                   range [True, False], default: True
        :param ScMinFreq:          if > 0 will be generated at least a reference frame every "ScMinFreq" frames.
                                   range [0-1500], default: 0.
    """

   The stabilization is not related to the flickering effect, but to temporal instability of colors, for this reason I explained that the ReColor post-process should be used if the clip was colored using an AI automatic video colorizer like HAVC, which are affected by this problem. In this case the BlackWhiteTune will be applied after ColorMNet and this should be able to mitigate the washed color effect produced by ColorMNet .

Dan


Attached Files
.zip   vsdeoldify-5.5.0_RC4.zip (Size: 452,45 KB / Downloads: 10)
Reply
Updated Hybrid_havc_test.
When not enabling 'ReColor' I just get a white image as output, independent of the BlackWhiteMode.

Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply


Forum Jump:


Users browsing this thread: 5 Guest(s)