Selur's Little Message Board

Full Version: Using Stable Diffision models for Colorization
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4 5 6 7 8 9 10 11
Hello Selur,

  see my post on github: https://github.com/vapoursynth/vapoursyn...4620507489
  let me know, if you can replicate my results (the clip preview will frozen after 100-500 frames).
  The havc_test is a simplified version (all the functions regarding colorization and network connections are no-op).
  Honestly on my side, in some case, the preview is freezing already on BestSource call.

Dan
using:
Code:
# Imports import sys import os from pathlib import Path import vapoursynth as vs # Ensure local module is found script_dir = Path(__file__).parent.resolve() if str(script_dir) not in sys.path: sys.path.insert(0, str(script_dir)) # getting Vapoursynth core core = vs.core # loading plugins core.std.LoadPlugin(path="F:/Hybrid/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/akarin/libakarin.dll") core.std.LoadPlugin(path="F:/Hybrid/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/libbestsource.dll") # Import scripts import vshavc_test as havc clip = core.bs.VideoSource(source="sample.mp4") # colorize using CMNet with external DiT (HAVC) clip = havc.HAVC_cmnet2dit(clip, max_memory_frames=10) # output clip.set_output()
(copied tcanny into the Support folder)
with:
Code:
F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vapoursynth>vspipe.exe c:\Users\Selur\Desktop\CMNET2DIT_test.vpy -c y4m --info Warning: Start detecting referenence frames Warning: Plugin F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vshavc_test\vsfilters\ColorFilter\Retinex\Retinex.dll is using API3 which is deprecated and will be removed shortly. Warning: Detection completed
it seems to be stuck no cpu or gpu usage,..
This is the bug...

Dan
Is this even related to get_frame ? For me it freezes while using "--info", so this might be a more fundamental issue. (I assume that '--info' doesn't really calls get_frame)
Try the version released in this post: https://github.com/vapoursynth/vapoursyn...4624305102

Dan
freezes too.
To me the "--info" works

Code:
PS D:\PProjects\vs-havc_dev\vs_test_debug> D:\Programs\Hybrid\64bit\Vapoursynth\Lib\site-packages\vapoursynth\vspipe.exe CMNET2DIT_test.vpy -c y4m --info Information: VideoSource track #0 index progress 4% Warning: call: disable_warnings Warning: call: convert_format_RGB24 Warning: call: torch.hub.set_dir Warning: Start detecting referenence frames Warning: Plugin D:\PProjects\vs-havc_dev\vs_test_debug\vshavc_test\vsfilters\Support\akarin.dll is using API3 which is deprecated and will be removed shortly. Warning: Detection completed Warning: call: vs_colormnet2dit Warning: call: vs_colormnet2dit_local Warning: call: RefImageReader2.get_clip_ref_list(start_frame=0) Warning: call: ModifyFrame(colormnet_dit_color) Output Index: 0 Type: Video Width: 720 Height: 540 Frames: 2544 FPS: 24000/1001 (23.976 fps) Format Name: YUV420P8 Color Family: YUV Alpha: No Sample Type: Integer Bits: 8 SubSampling W: 1 SubSampling H: 1

Is the preview that freeze after 100-800 frame. Also using vspipe for encoding it does not work, it freeze at frame 562 (while with R74 works as expected).

Code:
D:\Programs\Hybrid\64bit\Vapoursynth\Lib\site-packages\vapoursynth\vspipe.exe D:\PProjects\vs-havc_dev\vs_test_debug\CMNET2DIT_test.vpy - --outputindex 0 -c y4m | d:\programs\hybrid\64bit\x265.exe --preset fast --input - --fps 24000/1001 --y4m --crf 20.00 --output "D:\PProjects\vs-havc_dev\vs_test_debug\CMNET2DIT_test.265" Information: VideoSource track #0 index progress 4% Warning: call: disable_warnings Warning: call: convert_format_RGB24 Warning: call: torch.hub.set_dir Warning: Start detecting referenence frames Warning: Plugin D:\PProjects\vs-havc_dev\vs_test_debug\vshavc_test\vsfilters\Support\akarin.dll is using API3 which is deprecated and will be removed shortly. Warning: Detection completed Warning: call: vs_colormnet2dit Warning: call: vs_colormnet2dit_local Warning: call: RefImageReader2.get_clip_ref_list(start_frame=0) Warning: call: ModifyFrame(colormnet_dit_color) Warning: colormnet_dit_color(frame=0) Warning: colorize_frame(ti=0) Warning: colormnet_dit_color(frame=1) Warning: colorize_frame(ti=1) Warning: colormnet_dit_color(frame=2) Warning: colorize_frame(ti=2) Warning: colormnet_dit_color(frame=3) Warning: colorize_frame(ti=3) Warning: colormnet_dit_color(frame=4) Warning: colorize_frame(ti=4) Warning: colormnet_dit_color(frame=5) Warning: colorize_frame(ti=5) Warning: colormnet_dit_color(frame=6) Warning: colorize_frame(ti=6) Warning: colormnet_dit_color(frame=7) Warning: colorize_frame(ti=7) Warning: colormnet_dit_color(frame=8) Warning: colorize_frame(ti=8) Warning: colormnet_dit_color(frame=9) Warning: colorize_frame(ti=9) Warning: colormnet_dit_color(frame=10) ..... .... Warning: colorize_frame(ti=559) Warning: colormnet_dit_color(frame=560) Warning: colorize_frame(ti=560) Warning: colormnet_dit_color(frame=561) Warning: colorize_frame(ti=561) Warning: colormnet_dit_color(frame=562) Warning: perm_mem_win.adjust(562)


Dan

P.S.
no answers from Fredrik Mellbin and I provided all the necessary info for a serious debugging
Ah okay, tested just in vsViewer this time.
Give him some time.
a. today is a holiday depending on where you live
b. it's not that this is the only problem in Vapoursynths issue tracker
c. Vapoursynth is a huge project.
=> If he doesn't give an update in a week or so, I would ask further, but not reacting for a few days is something that is normal in my eyes. Smile

Cu Selur
Hi Selur,

  in the case it could be useful I added a GUI to the DiTServer, see post #1.425

Dan
Nice. Smile
Thanks for the info.

Cu Selur
Pages: 1 2 3 4 5 6 7 8 9 10 11