Deoldify Vapoursynth filter

[Dan64]

It is strange the code in vsdeoldify\fastai\ge_doc should never be called because is not used by vs-deoldify (dead code).

I attached a new version RC5.

Before unzip the new version you have to manually delete the following files in vsdeoldify\fastai\ge_doc:

• nbtest.py
  
• gen_notebooks.py
  
• doctest.py
  
• convert2html.py
  

Dan

P.S.
The second error is due to torch cache settings. It is unable to download or to find resnet50.

[Selur]

I have no clue why it's not able to download (I made sure to disable my firewall).
I still get:

2024-09-19 18:20:25.242
Failed to evaluate the script:
Python exception: 'NoneType' object has no attribute 'write'

Traceback (most recent call last):
File "src\\cython\\vapoursynth.pyx", line 3365, in vapoursynth._vpy_evaluate
File "src\\cython\\vapoursynth.pyx", line 3366, in vapoursynth._vpy_evaluate
File "C:\Users\Selur\Desktop\sample\thing2_async.vpy", line 43, in
clip = HAVC_deepex(clip=clip, method=4, ref_merge=0, dark=True, smooth=False, max_memory_frames=8,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\__init__.py", line 540, in HAVC_deepex
clip_colored = vs_colormnet(clip, clip_ref, image_size=-1, enable_resize=enable_resize,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\vsslib\vsmodels.py", line 44, in vs_colormnet
return vs_colormnet_async(clip, clip_ref, image_size, enable_resize, frame_propagate, max_memory_frames)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\__init__.py", line 203, in vs_colormnet_async
colorizer = colormnet_colorizer(image_size=image_size, vid_length=vid_length, enable_resize=enable_resize,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\__init__.py", line 45, in colormnet_colorizer
return ColorMNetRender(image_size=image_size, vid_length=vid_length, enable_resize=enable_resize,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\colormnet_render.py", line 80, in __init__
self._colorize_init(image_size, vid_length)
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\colormnet_render.py", line 134, in _colorize_init
self.network = ColorMNet(self.config, self.config['model']).cuda().eval()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\model\network.py", line 30, in __init__
self.key_encoder = KeyEncoder_DINOv2_v6()
^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\model\modules.py", line 161, in __init__
network = resnet.resnet50(pretrained=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\colormnet\model\resnet.py", line 170, in resnet50
load_weights_add_extra_dim(model, model_zoo.load_url(model_urls['resnet50']), extra_dim)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\hub.py", line 855, in load_state_dict_from_url
sys.stderr.write(f'Downloading: "{url}" to {cached_file}\n')
^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'write'

Ahhh,... download only works if called by VSPipe, not by vsViewer.
But now I get:

F:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\sample\thing2_async.vpy -c y4m NUL

Warning: F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\kornia\feature\lightglue.py:44: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd(cast_inputs=torch.float32)

Warning: F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\deepex\models\vgg19_gray.py:130: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  model.load_state_dict(torch.load(vgg19_gray_path))

Downloading: "https://download.pytorch.org/models/resnet50-19c8e357.pth" to C:\Users\Selur/.cache\torch\hub\checkpoints\resnet50-19c8e357.pth
100%|█████████████████████████████████████████████████████████████████████████████| 97.8M/97.8M [00:07<00:00, 12.9MB/s]
Downloading: "https://github.com/facebookresearch/dinov2/zipball/main" to C:\Users\Selur/.cache\torch\hub\main.zip
Downloading: "https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_pretrain.pth" to C:\Users\Selur/.cache\torch\hub\checkpoints\dinov2_vits14_pretrain.pth
100%|█████████████████████████████████████████████████████████████████████████████| 84.2M/84.2M [00:06<00:00, 12.8MB/s]
Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to C:\Users\Selur/.cache\torch\hub\checkpoints\resnet18-5c106cde.pth
100%|█████████████████████████████████████████████████████████████████████████████| 44.7M/44.7M [00:03<00:00, 12.8MB/s]
Error: Failed to retrieve frame 0 with error:
Traceback (most recent call last):
  File "src\\cython\\vapoursynth.pyx", line 3194, in vapoursynth.publicFunction
  File "src\\cython\\vapoursynth.pyx", line 3196, in vapoursynth.publicFunction
  File "src\\cython\\vapoursynth.pyx", line 829, in vapoursynth.FuncData.__call__
  File "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\vsslib\mcomb.py", line 63, in set_clip_frame
    return image_to_frame(ref_img, f.copy())
                          ^^^^^^^
UnboundLocalError: cannot access local variable 'ref_img' where it is not associated with a value

But after that calling the script in vsViewer works.

These models are already in "F:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\models\checkpoints", how can I stop vsdeoldify from downloading the files again and playing them somewhere else? (deleted "C:\Users\Selur/.cache" now)
I'd like to have these files in the torch-addon and not have deoldify download them.

Cu Selur

[Dan64]

It is strange, one the nets are downloaded they are not downloaded again.

This is my folder models:



and this is my folder checkpoints



do you have the file: trusted_list ?

Dan

[Selur]

Models:

checkpoints:

=> look like yours, so vsdeoldify should not download anything, right?

Cu Selur

[Dan64]

yes, as you can see my nets are date 12/9 and has never been updated again.

Dan

[Selur]

Okay, then how can I get vsdeoldify not to download stuff that is already there?

[Dan64]

the cache is managed by torch, I don't know if there is an option to don't update it, see caching-logic

Dan

[Selur]

Hmm,... so is this something new with colormnet, since before vsdeolodfy worked fine without the .cache folder.
(not planning to add support for vsdeoldify, if it needs downloading additional stuff, that I can't put in the torch addon)

Cu Selur

[Dan64]

It is strange, on my side I don't have this behaviour. 
A noted that your nets are dated 15/9 and not 19/9 as should be if they has been updated today. 
Are your sure that in this case is using another temporary folder ?

[Selur]

Now it downloaded the models to:

C:\Users\Selur/.cache\torch\hub\checkpoints\

see previous post.