Multiple issues

[Selur]

However, even after that, attempt to mp4 output caused endless stay in the "Waiting" mode

would need a debug output of that

[Selur]

DVD input now shows nothing in "Filter View"/"Crop View" with Vapoursunth

should be fixed, problem was that the wrong indexer and source filter was used

QT Prores now shows in Vapoursynth "Filter View" promptly, however after assign "Despot" - "Filter View" becomes unresponsive, filtered result not showing.

Found the problem: DeSpot only contains 8bit or 16bit YUV no 9/10bit.
-> should be fixed now

QT Prores "Dehalo"/"Dehalo_alpha"/"BlindDehalo3" shows some kind of mask even with "Show Mask:None"

Your debug outptu shows:

# Imports
import os
import sys
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/CTMF/CTMF.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import havsfunc
# source: 'K:\Gulliver\GULLIVER HD TRANSFER\Gulliver0001.mov'
# current color space: YUV422P10, bit depth: 10, resolution: 1920x1080, fps: 23.976, color matrix: 709, yuv luminance scale: limited, scanorder: progressive
# Loading K:\Gulliver\GULLIVER HD TRANSFER\Gulliver0001.mov using LibavSMASHSource
clip = core.lsmas.LibavSMASHSource(source="K:/Gulliver/GULLIVER HD TRANSFER/Gulliver0001.mov")
# making sure input color matrix is set as 709
clip = core.resize.Point(clip, matrix_in_s="709",range_s="limited")
# making sure frame rate is set to 23.976
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
original = clip
clip = havsfunc.FineDehalo(clip)
# adjusting output color from: YUV422P10 to YUV420P8 for x264Model (i420@10)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")
# adjusting for FilterView
if (original.format.id != clip.format.id):
  if (original.format.color_family == vs.RGB and clip.format.color_family != vs.RGB):
    original = core.resize.Bicubic(clip=original, format=clip.format.id, matrix_s="709", range_s="limited")
  elif (original.format.color_family == clip.format.color_family):
    original = core.resize.Bicubic(clip=original, format=clip.format.id, range_s="limited")
  else:
    original = core.resize.Bicubic(clip=original, format=clip.format.id, matrix_in_s="709", range_s="limited")
stacked = core.std.StackHorizontal([original,clip])
# set output frame rate to 23.976fps
stacked = core.std.AssumeFPS(clip=stacked, fpsnum=24000, fpsden=1001)
# Output
stacked.set_output()

I used:

# Imports
import os
import sys
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/CTMF/CTMF.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import havsfunc
# source: 'F:\TestClips&Co\files\ProRes\SDR - ProRes 422.mov'
# current color space: YUV422P10, bit depth: 10, resolution: 3840x2160, fps: 59.9401, color matrix: 709, yuv luminance scale: limited, scanorder: progressive
# Loading F:\TestClips&Co\files\ProRes\SDR - ProRes 422.mov using LibavSMASHSource
clip = core.lsmas.LibavSMASHSource(source="F:/TestClips&Co/files/ProRes/SDR - ProRes 422.mov")
# making sure input color matrix is set as 709
clip = core.resize.Point(clip, matrix_in_s="709",range_s="limited")
# set output frame rate to 59.940fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
original = clip
clip = havsfunc.FineDehalo(clip)
# adjusting output color from: YUV422P10 to YUV420P8 for x264Model (i420@10)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")
# adjusting for FilterView
if (original.format.id != clip.format.id):
  if (original.format.color_family == vs.RGB and clip.format.color_family != vs.RGB):
    original = core.resize.Bicubic(clip=original, format=clip.format.id, matrix_s="709", range_s="limited")
  elif (original.format.color_family == clip.format.color_family):
    original = core.resize.Bicubic(clip=original, format=clip.format.id, range_s="limited")
  else:
    original = core.resize.Bicubic(clip=original, format=clip.format.id, matrix_in_s="709", range_s="limited")
stacked = core.std.StackHorizontal([original,clip])
# set output frame rate to 59.940fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001)
# Output
stacked.set_output()

-> can't reproduce the issue here.
Would need a sample which allows ne to reproduce the issue.

QT Prores is not showing in Avisynth "Filter View"/"Crop View" at all.

"Crop View": Avisynth isn't used during Crop View and Mencoder/MPlayer to not support ProRes4444
Nothing I can do about it.
"Filter View: Works fine here.
If you use 32bit Avisynth (the default), 4k and intensive filtering might not work due to memory limitation.
-> would need details, since I can't reproduce the issue here.

Avisynth "Despot" shows highlighted noise instead of remove it .

I can reproduce it, happens when Hybrid converts to YUY2 and feeds that to DeSpot, works fine with YV16.
-> should be fixed now


=> will send you a link to a new dev version in ~20min

Cu selur

[serg]

Well, here  the links to videos that you can use to reproduce the issues mentioned above.
 
1. "BP_test.avi" -  https://drive.google.com/file/d/1PXkarSI...sp=sharing
2. "ProRes_test.mov" - https://drive.google.com/file/d/1ZvI8D44...sp=sharing

Now:
1. Open "BP_test.avi" and assign Avisynth "DeSpot", navigate "Next"/"Previous" - see result.
2. Quit "Hybrid" (I noticed that changing sources multiple times without "Quit" makes "Hybrid" unresponsive)
3. Open "ProRes_test.mov" and assign Vapursynth any "Dehalo" filter - see result.
4. Open "ProRes_test.mov" and assign Avisynth  "DeSpot", navigate "Next"/"Previous" - see result.
It looks like Avisynth still use ffmpeg for QT Prores (it takes  approx 5mins to open 20 mins file)

DVD input works (you need just wait ~5 mins)

[Selur]

DVD input works (you need just wait ~5 mins)

No clue why the indexing takes so long for you.
You might want to check that:
a. no anti virus or firewall solution is interfering
b. you source lies on a fast drive
Indexing requires to read the dvd once, but that shouldn't take 5min on modern hardware.

Got the files, will try to reproduce the issues this evening. (long work day today)

It looks like Avisynth still use ffmpeg for QT Prores (it takes approx 5mins to open 20 mins file)

Other than Vapoursynth, Avisynth does use FFVideoSource by default.
-> Try whether enabling "Filtering->Avisynth->misc->Sourcefilter Settings->LibavVideoSoure instead of FFmpegSource" helps, when enabled Hybrid does use LibavVideoSoure here.

Cu Selur

[serg]

DVD input works (you need just wait ~5 mins)

No clue why the indexing takes so long for you.
You might want to check that:
a. no anti virus or firewall solution is interfering
b. you source lies on a fast drive
Indexing requires to read the dvd once, but that shouldn't take 5min on modern hardware.

Got the files, will try to reproduce the issues this evening. (long work day today)

It looks like Avisynth still use ffmpeg for QT Prores (it takes  approx 5mins to open 20 mins file)

Other than Vapoursynth, Avisynth does use FFVideoSource by default.
-> Try whether enabling "Filtering->Avisynth->misc->Sourcefilter Settings->LibavVideoSoure instead of FFmpegSource" helps, when enabled Hybrid does use LibavVideoSoure  here.

Cu Selur

Enabling "Filtering->Avisynth->misc->Sourcefilter Settings->LibavVideoSoure instead of FFmpegSource" doesn't help.
It seems to me that the Avisynth  has an issue with QT Prores because Avisynth doesn't have LSMASHSource over LWLibAVSource option like Vapoursynth.
If in Vapursynth I disable "Prefer LSMASHSource over LWLibAVSource (that enabled by default), leaving "LibavVideoSoure instead of FFmpegSource" enabled, it behaves exactly as Avisynth (takes long time to load)

[serg]

Is there is GPU support option for VP9 encoding?

[Selur]

No, since I don't know of any gpu based VP9 (or avs) encoder which Hybrid could use.

Cu Selur

[Selur]

It seems to me that the Avisynth  has an issue with QT Prores because Avisynth doesn't have LSMASHSource over LWLibAVSource option like Vapoursynth.

Enable "Filtering->Avisynth->Misc->Sourcefilter Settings->Use L-Smash for mp4 input" and Hybrid should use LSMASHSource.


---
About raw rgb and DeSpot: I can reproduce the issue with your sample, problem seems happen when DeSpot is fed with YUY2 content when YV12 content is used the problem does not occur.
Seems like newer Despot Avisynth versions do not support YUY2 anymore.
-> I'll limit it to YV12 which should fix the issue.

About the Vapoursynth + FineDeHalo screenshot -> Can't reproduce the problem here with and of the DeHalo-Filters.

=> will send you a link to a new version for the 'Raw rgb avi + DeSpot'-problem in ~20min.

Cu Selur

[serg]

Enable "Filtering->Avisynth->Misc->Sourcefilter Settings->Use L-Smash for mp4 input" and Hybrid should use LSMASHSource - works.
Avisynth "Despot" has no effect. Open "BP_test.avi" and assign Avisynth "DeSpot", go to frame 1 (just more noticeable - small white spots still there)
Vapoursynth "Despot" works as suppose (You can compare using the same "BP_test.avi".)
QT Prores "Dehalo"/"Dehalo_alpha"/"BlindDehalo3" surprisingly works fine now.

[Selur]

Avisynth "Despot" has no effect. Open "BP_test.avi" and assign Avisynth "DeSpot", go to frame 1 (just more noticeable - small white spots still there)

They are !different! filters which just share the same name and try to remove spots.

https://github.com/theChaosCoder/lostfun...ostfunc.py which is a port of http://forum.doom9.net/showthread.php?p=...ost1402690

vs

http://www.avisynth.nl/index.php/DeSpot

-> nothing I can do about that, about porting the (Not my fault the Vapoursynth author used the same name for his script as the filter that exists for Avisynth.

Also this has nothing to do with:

QT Prores "Dehalo"/"Dehalo_alpha"/"BlindDehalo3" surprisingly works fine now.

I didn't touch those filters at all,..