Selur's Little Message Board

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77

using:

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

coloring isn't good, but the chroma lag seems to be fixed.
You can get the file here.

using:

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

I get:

Code:
2024-09-22 09:44:34.029

Failed to evaluate the script:

Python exception: HAVC_deepex: no reference frames found in 'J:/tmp', allowed format is: ref_nnnnnn.[png|jpg]

Traceback (most recent call last):

File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate

File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate

File "J:\tmp\tempPreviewVapoursynthFile09_44_12_474.vpy", line 44, in 

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 280, in HAVC_main

clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 510, in HAVC_deepex

clip = SceneDetectFromDir(clip_ref, sc_framedir=sc_framedir, merge_ref_frame=merge_ref_frame,

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\vsslib\vsutils.py", line 156, in SceneDetectFromDir

raise vs.Error(

vapoursynth.Error: HAVC_deepex: no reference frames found in 'J:/tmp', allowed format is: ref_nnnnnn.[png|jpg]

not setting 'ScFrameDir' I get:

Code:
2024-09-22 09:46:45.803

Failed to evaluate the script:

Python exception: HAVC_deepex: method != 0 but sc_framedir is unset

Traceback (most recent call last):

File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate

File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate

File "C:\Users\Selur\Desktop\Vapoursynth Scripts\test.vpy", line 44, in 

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 280, in HAVC_main

clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 425, in HAVC_deepex

raise vs.Error("HAVC_deepex: method != 0 but sc_framedir is unset")

vapoursynth.Error: HAVC_deepex: method != 0 but sc_framedir is unset

Cu Selur

(22.09.2024, 09:47)Selur Wrote: [ -> ]using:

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
coloring isn't good, but the chroma lag seems to be fixed.
You can get the file here.

using:

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)
I get:

Code:
2024-09-22 09:44:34.029 Failed to evaluate the script: Python exception: HAVC_deepex: no reference frames found in 'J:/tmp', allowed format is: ref_nnnnnn.[png|jpg]
not setting 'ScFrameDir' I get:

Code:
2024-09-22 09:46:45.803 Failed to evaluate the script: Python exception: HAVC_deepex: method != 0 but sc_framedir is unset

The error messages should be clear.

In the first case has been provided sc_framedir but in this folder are not available reference images in the format: ref_nnnnnn.[png|jpg].
For example valid ref frames are ref_000000.png or ref_000310.jpg. It is mandatory to prefix all the reference images by the text "ref_", the frame number must be 6 digit long padded on the left with zeros, and the extension can be ".png" or ".jpg".

In the second case, has been specified DeepExMethod=1, which implies "HAVC + RF same as video", but not reference frame directory has been provided.

Dan

Quote: In the second case, has been specified DeepExMethod=1, which implies "HAVC + RF same as video", but not reference frame directory has been provided.

No.
First I used:

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

which worked fine. 'ScFrameDir' is only added since you wanted it to be always there.

Then I used:

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

"HAVC + RF same as video": DeepExMethod=1 + ScFrameDir="J:/tmp"
and DeOlddify complained about

Code:
HAVC_deepex: no reference frames found in 'J:/tmp', allowed format is: ref_nnnnnn.[png|jpg]

When I removed ScFrameDir="J:/tmp" from the call, DeOldify complains about:

Code:
HAVC_deepex: method != 0 but sc_framedir is unset

Maybe I misunderstood 'DeepExMethod=1', but shouldn't it create the reference images itself?
So complaining about the format seems wrong.

Cu Selur

The feature that you are referring to is enabled only if the parameter "only_ref_frames" is set to True.

With the current GUI this parameter can be set only as shown in the picture below (see check box "Reference frames only")

[Image: attachment.php?aid=2755]

Currently your GUI don't allow to set "Ref FrameDir" when Method is in: 0, 1, 2

So I had, first set method = 3, set the reference dir and then change the method to 0 (please allow always the setting of this field, with the option to clear it)

I attached a new RC12 where now it is possible to set the flag "only_ref_frames" using a command like

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="ref_ext_stable", DeepExModel=1, VideoTune="Stable", DeepExOnlyRefFrames=True, ScThreshold=0.035)

As explained this option has been enabled to check the reference frames that are generated when is selected the method = 0 (HAVC).

The best way to use this feature is to open the preview window, open with vsedit or veViewer the script generated automatically and press "F7" or select "Script->Benchmark".
There is not reason to encode the generated movie, the Benchmark will run the script and the reference frames will be generated without encoding.

For example if you run the command described previously you will see that the following frames were not colored properly

[Image: attachment.php?aid=2757]

You can create a new directory called "ref_fix" and put inside this directory these pictures

[Image: attachment.php?aid=2759]

Then you can use a command like this

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExRefMerge=0, ScFrameDir="ref_fix", DeepExModel=1, VideoTune="Stable", DeepExMethod=1, ScThreshold=0.035)

to use DeepExMethod=1 -> HAVC + RF same as video

In this case the reference frames will be generated using HAVC but the 2 frames added in the directory "ref_fix" will override the frames generated automatically by HAVC, thus providing an improved colored image.

Dan

Quote:Currently your GUI don't allow to set "Ref FrameDir" when Method is in: 0, 1, 2
So I had, first set method = 3, set the reference dir and then change the method to 0 (please allow always the setting of this field, with the option to clear it)

I adjusted the code to always allow changing the "Ref FrameDir". (If nothing is set, Hybrid will use the temp folder)
Updated the deoldify test version.

I have no clue what the screenshot was to do with my settings and the error I got, since those are totally different settings.

Quote:now it is possible to set the flag "only_ref_frames" using a command like

This is driving me nuts. You are changing the logic while the GUI does not reflect the old logic properly. Angry

So in which cases should be set now? atm. it only gets set for HAVC_deepex if method is 0.

At this point I have no clue, whether the GUI sets things properly or not. Sad

Cu Selur

Since you are the only one out there who knows how to use it, I need feedback on what to change so that:
a. one can enable the option without the preview or the encoding crashing
b. one can not set stuff that causes crashes. If one specifies to demanding settings for one's hardware that is 'okay' to crash. But enabling feature X should always create correct calls.

Cu Selur

Uploaded new test version, where 'Reference frames only' is enabled by default.
using:

Code:
# Imports

import vapoursynth as vs

# getting Vapoursynth core

import sys

import os

core = vs.core

# Import scripts folder

scriptPath = 'F:/Hybrid/64bit/vsscripts'

sys.path.insert(0, os.path.abspath(scriptPath))

# loading plugins

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")

core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/BestSource/BestSource.dll")

# Import scripts

import adjust

import validate

# Source: 'G:\TestClips&Co\files\test.avi'

# Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual

# Loading G:\TestClips&Co\files\test.avi using BestSource)

clip = core.bs.VideoSource(source="G:/TestClips&Co/files/test.avi", cachepath="J:/tmp/test.json", track=0, hwdevice="cuda")

frame = clip.get_frame(0)

# setting color matrix to 470bg.

clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)

# setting color transfer (vs.TRANSFER_BT601), if it is not set.

if validate.transferIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)

# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.

if validate.primariesIsInvalid(clip):

  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)

# setting color range to TV (limited) range.

clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)

# making sure frame rate is set to 25fps

clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)

# making sure the detected scan type is set (detected: progressive)

clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive

# adjusting color using Tweak

clip = adjust.Tweak(clip=clip, hue=0.00, sat=0.00, cont=1.00, coring=True)

# changing color matrix from '470bg' to '709' for vsDeOldify

clip = core.resize.Bicubic(clip, matrix_in_s="470bg", matrix_s="709")

# changing range from limited to full range for vsDeOldify

clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")

# setting color range to PC (full) range.

clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)

# adjusting color space from YUV420P8 to RGB24 for vsDeOldify

clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")

# adding colors using DeOldify

from vsdeoldify import HAVC_main

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0, DeepExOnlyRefFrames=True, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

# internally changing color matrix for YUV<>RGB to '470bg' undoing color matrix change for vsDeOldify

# changing range from full to limited range for vsDeOldify

clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

# adjusting output color from: RGB24 to YUV420P10 for NVEncModel

clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")

# set output frame rate to 25fps (progressive)

clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)

# output

clip.set_output()

Cu Selur

Quote: I adjusted the code to always allow changing the "Ref FrameDir". (If nothing is set, Hybrid will use the temp folder)

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExOnlyRefFrames=True, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

with RC12, I get:

Code:
2024-09-22 12:54:46.928

Failed to evaluate the script:

Python exception: HAVC_deepex: only_ref_frames is enabled but method != 0 (HAVC)

Traceback (most recent call last):

File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate

File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate

File "J:\tmp\tempPreviewVapoursynthFile12_54_29_135.vpy", line 48, in 

clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExOnlyRefFrames=True, ScFrameDir="J:/tmp", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 282, in HAVC_main

clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset,

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 427, in HAVC_deepex

raise vs.Error("HAVC_deepex: only_ref_frames is enabled but method != 0 (HAVC)")

vapoursynth.Error: HAVC_deepex: only_ref_frames is enabled but method != 0 (HAVC)

If ScFrameDir is not set need to be passed ScFrameDir=None.
If (depending on the method selected) will be issued an error, the user have to fix the input.

Just to clarify further, DeepExOnlyRefFrames=True can be used only with DeepExMethod=0.
Then is necessary to run the encode (the encode video is not usable) or the Benchmark (non encode video is generated).

After having created the ref frames and eventually fixed the colors of wrong frames, you can put "only" the fixed frames in another directory, for example "temp_fix"

The you can colorize the video using both the ref frames automatically generated by HAVC than the fixed ref frames with the following command

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=1, DeepExRefMerge=0, DeepExOnlyRefFrames=False, ScFrameDir="J:/tmp_fix", DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

your command raised the error because you left DeepExOnlyRefFrames=True, it need to be set to False for the "True" encoding.

Dan

Quote:Just to clarify further, DeepExOnlyRefFrames=True can be used only with DeepExMethod=0.
Then is necessary to run the encode (the encode video is not usable) or the Benchmark (non encode video is generated).

So DeepExOnlyRefFrames is a two pass process which is not supported by Hybrid, correct?
=> So atm. the correct way to handle it, is to remove that option from Hybrid.

Uploaded a new test version which:
1. removes DeepExOnlyRefFrames (might come back in case Hybrid supports 2pass handling)
2. uses 'None' if Ref. FrameDir isn't set by the user.

Now, enabling DeOldify with default values in Hybrid creates:

Code:
clip = HAVC_main(clip=clip)

additionally enabling 'Exemplar Models', creates:

Code:
clip = HAVC_main(clip=clip, EnableDeepEx=True, DeepExMethod=0, DeepExRefMerge=0, ScFrameDir=None, DeepExModel=0, DeepExEncMode=0, DeepExMaxMemFrames=0)

Is this the correct behavior? (yes? no?)

So if any method other than 'HAVC' is used, the user must set 'Ref. FrameDir' to a folder with reference frames.

Cu Selur

Selur

Dan64

Selur

Dan64

Selur

Selur

Selur

Selur

Dan64

Selur