Selur's Little Message Board

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142

In effect there is a bug on local encoding mode.
The new RC5 (attached) fix it.

Thanks,
Dan

P.S.
On my side using local mode with max_memory_frames=4, I reach a pick on VRAM usage of 15.9 GB.

(24.05.2026, 19:53)Selur Wrote: [ -> ]Restarted the job and it worked again.
But at the end of the encoding I always get:

Code:
Exception occurred during processing of request from ('127.0.0.1', 5402) Exception occurred during processing of request from ('127.0.0.1', 5403) Traceback (most recent call last): Traceback (most recent call last): File "socketserver.py", line 697, in process_request_thread File "socketserver.py", line 362, in finish_request File "socketserver.py", line 697, in process_request_thread File "socketserver.py", line 362, in finish_request File "socketserver.py", line 766, in __init__ File "socketserver.py", line 766, in __init__ File "http\server.py", line 438, in handle File "http\server.py", line 438, in handle File "http\server.py", line 404, in handle_one_request File "socket.py", line 720, in readinto File "http\server.py", line 404, in handle_one_request ConnectionResetError: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen File "socket.py", line 720, in readinto ---------------------------------------- ConnectionResetError: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen
This should be captured, since otherwise folks might think this is a crash.

Cu Selur

I can try to remove the trace, but the connection error is normal in this case.

Dan

Using RC5 with:

Code:
# colorize using CMNet with external DiT (HAVC)
clip = havc.HAVC_cmnet2dit(clip, dit_engine_params={"host": "127.0.0.1", "port": 8765, "model_precision": "int4"})
# adjusting color space from YUV420P8 to RGB24 for vsAutoWhite
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# adjusting colors using AutoWhite
clip = color.AutoWhite(clip)

worked.
Switching to 'local'

Code:
# colorize using CMNet with external DiT (HAVC)
clip = havc.HAVC_cmnet2dit(clip, encode_mode=1, dit_engine_params={"host": "127.0.0.1", "port": 8765, "model_precision": "int4"})
# adjusting color space from YUV420P8 to RGB24 for vsAutoWhite
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# adjusting colors using AutoWhite
clip = color.AutoWhite(clip)

this time it worked. Initial colorizing of the ref frames (6 in my case) was done as fast as before, then it seemed 'stuck' (no progress), but since the GPU was running at 50% I waited a bit and it worked, now the encoding was really slow. So seems like 'local' is working now too, just rather slow.

Cu Selzur

I updated DiTServerRPC

I implemented a clean client-disconnect logging that suppress the alarming traceback that socketserver prints by default when a client closes the connection unexpectedly.

Please update your version.

Dan

Updated and tried it ('remote').
1st try: client didn't connect (cpu: 0%, gpu: 0%)
restarting job, didn't help.
2nd try (restarting server and then the job): now client connected, encoding started and finished.
3rd try, now with 'local': client connected, encoding started with ~0.5-1.2fps (instead of ~11fps during encoding with 'remote') but then stopped after encoding a few frames.
Having to restart the server sometimes is a bit annoying, so using this in succession with multiple files is something I wouldn't recommend.
Also, the strange stopping in the middle of the processing, is annoying.
Will be offline till tomorrow, but can so more testing tomorrow.

Cu Selur

In the case it could be useful, I attached the script that I'm using to run the server.
Copy it in the same folder where is located your server.
Then create a link on your desktop to this file and assign to it a name and an icon that you like.

Inside the script you need to updated the following variables, to match your folder structure

Code:
set PYTHON_EXE=D:\PProjects\DiTServerRPC\.venv\Scripts\python.exe
set SERVER_DIR=D:\PProjects\DiTServerRPC
set LOGFILE=D:\PProjects\DiTServerRPC\dit_server.log

change the default precision to "int4" (in the script is "fp4")

Code:
if /i "%PRECISION%"=="" set PRECISION=int4

Dan

P.S.
I don't understand why you are using "virtualenv" while in the instructions provided is requested to use "venv", should be the same, but I don't have tested the installation using "virtualenv".

In RC3 I reduced the timeout for client connection from 600sec to 60sec.
In the attached RC6 I restored the 600sec timeout.

Dan

Quote: I don't understand why you are using "virtualenv" while in the instructions provided is requested to use "venv", should be the same, but I don't have tested the installation using "virtualenv".

you are right virtualenv isn't used and isn't needed

---

started server, started script in vsViewer
First frame took 41s
The first output from the server takes a long time ~41s without any indication.
=> the server should indicate something when a client is connected,...
now it seems to be stuck again, vsViewer is still hidden,...

Something is still not okay with the communication. Since both the Python and the VapourSynth Viewer process ar at 0& cpu&gpu, I killed the VapourSynthViewer process.
Then I saw:

Code:
:02,360 [INFO] Connection 127.0.0.1:7423 closed by the client (ConnectionResetError: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen)
2026-05-25 09:22:02,360 [INFO] Connection 127.0.0.1:7424 closed by the client (ConnectionResetError: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen)

(isn't one time enough?)

Restarted vsViewer, frames get created in the server, but twice as much as before. No clue why,... did you change something else?
closed vsViewer, reopened it, first 6 frames get colorized, then the server is stuck again,..
Stopped the server (and vViewer) after a few minutes.
also called:

Code:
set LOGFILE=F:\dit_server.log"

before starting the server again.
started server and vsViewer gain, client doesn't connect (<>nothing is happening),.. killed vsViewer and restarted it.
Now it seems to be working,...

=> this too instable to be usable atm. (at least on my system)

Cu Selur

Ps.: no "F:\dit_server.log" was created,...

The preview is not the right way to test this filter, because the underlying task is slow, while the preview expects almost real-time processing.

For example before running this code, using max_memory_frames=10

Code:
# colorize using CMNet with external DiT (HAVC)
clip = havc.HAVC_cmnet2dit(clip, render_speed="auto", render_vivid=False, sc_thresh=0.035, sc_tht_ssim=0.80, sc_min_int=25, sc_tht_offset=2, sc_min_freq=0, encode_mode=0, max_memory_frames=10, dit_engine_params={"host": "127.0.0.1", "port": 8765, "model_precision": "fp4"}, retry_threshold=0.00)

You need to wait that the server is fully loaded as shown below

Code:
============================================================
 DiT Colorize RPC Server
 Precision   : fp4
 Config      : D:\PProjects\DiTServerRPC\qwen_config_fp4.json
 Listening on: 127.0.0.1:8765
 Log file    : D:\PProjects\DiTServerRPC\dit_server.log
============================================================

2026-05-25 18:28:01,828 [INFO] module_dir             : D:\PProjects\DiTServerRPC
2026-05-25 18:28:01,828 [INFO] dit_colorize_main.py   : found
2026-05-25 18:28:01,828 [INFO] Loading pipeline from config: D:\PProjects\DiTServerRPC\qwen_config_fp4.json
2026-05-25 18:28:01,828 [INFO] Loading pipeline: nunchaku-qwen fp4 r32 steps=4
Loading SVDQuant FP4 transformer from: nunchaku-ai/nunchaku-qwen-image-edit-2509/lightning-251115/svdq-fp4_r32-qwen-image-edit-2509-lightning-4steps-251115.safetensors
The config attributes {'pooled_projection_dim': 768} were passed to NunchakuQwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 21.28it/s]
Loading pipeline components...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:01<00:00,  4.68it/s]
Optimizing VRAM ...
D:\PProjects\DiTServerRPC\.venv\Lib\site-packages\nunchaku\models\transformers\transformer_qwenimage.py:620: UserWarning: Skipping moving the model to GPU as offload is enabled
  warn("Skipping moving the model to GPU as offload is enabled", UserWarning)
2026-05-25 18:29:32,546 [INFO] Pipeline loaded successfully.
2026-05-25 18:29:32,546 [INFO] HAVC Colorize RPC Server listening on 127.0.0.1:8765
2026-05-25 18:29:32,546 [INFO] Press Ctrl+C to stop.

After that you can ask for the preview.

Before the preview window will be visible, on client side will be performed the following tasks:

1) the clip is parsed to create the reference image list (on my side, on a clip with 2544 frames this task requires about 80sec)
2) colorize 5 pairs (on my side this activity requires about 74sec)

Code:
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:04<00:00,  2.06s/it]
2026-05-25 18:31:32,458 [INFO] colorize_frame_pair_shm: 38.22s (19.11s/frame)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:04<00:00,  2.13s/it]
2026-05-25 18:31:50,511 [INFO] colorize_frame_pair_shm: 10.44s (5.22s/frame)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:03<00:00,  1.94s/it]
2026-05-25 18:31:58,968 [INFO] colorize_frame_pair_shm: 8.15s (4.08s/frame)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:03<00:00,  1.94s/it]
2026-05-25 18:32:07,565 [INFO] colorize_frame_pair_shm: 8.22s (4.11s/frame)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:03<00:00,  1.94s/it]
2026-05-25 18:32:16,147 [INFO] colorize_frame_pair_shm: 8.28s (4.14s/frame)

After that you finally will see the preview window, and on the client window you will se the following message

Code:
2026-05-25 18:32:18.198
Start CMNET2 server, listening on : ('127.0.0.1', 55752)

If you run play you will see the colorized frames, with some long interruption due to the need to ask to the server to colorize other reference frames.

As you can see the preview for this filter is not very useful, you should disable it, or if you want keep it you should force the preview with a limited number of reference frame (for example 2 or 4) and show an overly text that advise the user the the preview is very slow and limited to 2 or 4 reference frames.

The best way to use this filter is just encode the clip and wait for the encode to be completed.

Dan

P.S.
On your hardware I expect that this pipeline should work just fine, but don't expect miracles in the preview (better advise the users of this limitation)

You should remove the double quote

Code:
set LOGFILE=F:\dit_server.log"

(25.05.2026, 09:37)Selur Wrote: [ -> ]Something is still not okay with the communication. Since both the Python and the VapourSynth Viewer process ar at 0& cpu&gpu, I killed the VapourSynthViewer process.
Then I saw:

Code:
:02,360 [INFO] Connection 127.0.0.1:7423 closed by the client (ConnectionResetError: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen) 2026-05-25 09:22:02,360 [INFO] Connection 127.0.0.1:7424 closed by the client (ConnectionResetError: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen)
(isn't one time enough?)

The two messages correspond to two TCP connections opened by the client. xmlrpc.client.ServerProxy creates a new TCP connection for each RPC call — in
the client, there are 2 (is_pipeline_loaded(), colorize_frame()). With HTTP/1.1 enabled on the server side, a connection already served can remain open idle
(keep-alive) waiting for a new request, even if the client transport closes it immediately after reading the response.
When the client terminates, all open TCP connections are closed by the operating system.
Each of the two remaining connections (those of is_pipeline_loaded() and colorize_frame()) produces its own ConnectionResetError. Hence the two messages.

This is not a problem — it's the expected behavior when using the HTTP/1.1 protocol.

Dan

Quote: The preview is not the right way to test this filter, because the underlying task is slow, while the preview expects almost real-time processing.

Not really, preview works fine for me with nowhere near real-time processing, but I get what you want to say.

=> I leave it as it is and stop bugging you, since the chance that I'll use it are next to null.
Also, I'll redirect anybody how wants to use it to you anyway. Big Grin

Cu Selur

I understand your point of view, but I'd like to know if you tried to color at least a small clip of about 10min.
Just to be sure that there are not other issues.
You should try to just encode the clip adding it to Hybrid's job queue.

Thanks,
Dan

Dan64

Selur

Dan64

Selur

Dan64

Dan64

Selur

Dan64

Selur

Dan64