"Custom" filters dont work properly

[serg]

Trying to include filters in to the "Custom" section Avisynth (tried 32 bit and 64 bit - result the same).
Case 1:
1. Video source - TIVTC
2. Before "Restore" (just after deinterlace):
LoadPlugin("C:\AVSplugins\64bit\mvtools2.dll")
LoadPlugin("C:\AVSplugins\64bit\masktools2.dll")
Import("C:\AVSplugins\DeStripe.avs")
TurnLeft()
DeStripe(2,3,16)
DeStripe(2,2,16)
DeStripe(2,1,12)
TurnRight()

mvtools2.dll and masktools2.dll has been copied from "Hybrid" plugins folder.
Took around 1 min doing something and finally shows (see attachment)
Running the same script with previously TIVTCed in the "Hybrid" video and AvsPmod gives perfect result with the "speed of light"

Case 2: (above script has been removed ):
1. Video source - TIVTC
2.Before "Restore":

LoadPlugin("C:\AVSplugins\64bit\mvtools2.dll")
LoadPlugin("C:\AVSplugins\64bit\masktools2.dll")
Import("C:\AVSplugins\cdeblend.avs")
cdeblend()

Result - no effect.
Running the same script with previously TIVTCed in the "Hybrid" video and AvsPmod gives perfect result with the "speed of light"

[Selur]

Looking at the cdeblend debug output the used script for the preview is:

ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("C:\PROGRA~1\Hybrid\64bit\Avisynth\AVISYN~1\LoadDll.dll")
LoadDLL("C:\Program Files\Hybrid\64bit\Avisynth\avisynthPlugins\libfftw3f-3.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\64bit\Avisynth\AVISYN~1\LSMASHSource.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\64bit\Avisynth\AVISYN~1\TIVTC.dll")
# loading source: D:\Hybrid_test\Gill\face_10bit.mov
#  color sampling YUY2@10, matrix: bt601, scantyp: telecine, luminance scale: limited
Source = LWLibavVideoSource("D:\HYBRID~1\Gill\FACE_1~1.MOV",cache=false,format="YUV422P16", prefer_hw=0)
# current resolution: 720x486
# deinterlacing
SourceFiltered = Source
Source = Source.TFM(mChroma=true)
Source = Source.TDecimate(cycleR=1,cycle=5,noblend=true,nt=0,blockx=32,blocky=32,chroma=true,denoise=false,ssd=false,display=false)
Source = Source.AssumeFrameBased()
SourceFiltered = SourceFiltered.TFM(mChroma=true)
SourceFiltered = SourceFiltered.TDecimate(cycleR=1,cycle=5,noblend=true,nt=0,blockx=32,blocky=32,chroma=true,denoise=false,ssd=false,display=false)
SourceFiltered = SourceFiltered.AssumeFrameBased()

# CUSTOM SCRIPT PART - position: Restore - START
function CustomRestore(clip clp) {
  last=clp
  LoadPlugin("C:\AVSplugins\64bit\mvtools2.dll")
  LoadPlugin("C:\AVSplugins\64bit\masktools2.dll")
  Import("C:\AVSplugins\cdeblend.avs")
  cdeblend()
  return last
}
# CUSTOM SCRIPT PART - position: Restore - END

SourceFiltered = SourceFiltered.CustomRestore() # loading custom script content

#  color sampling YV16@16, matrix: bt601, scantyp: progressive, luminance scale: limited
# filtering
# stacking horizontal for filter preview
# adjust color to RGB32 (for preview)
SourceFiltered = SourceFiltered.ConvertToRGB32(matrix="Rec601")
# adjust color to RGB32 (for preview)
Source = Source.ConvertToRGB32(matrix="Rec601")
StackHorizontal(Source, SourceFiltered)
PreFetch(2)
# setting output fps to 23.976fps
AssumeFPS(24000,1001)
#  output: color sampling RGB32@16, matrix: bt601, scantyp: progressive, luminance scale: limited
return last

and looking at the documentation of C-deblend ('Now superseded by Srestore.') I see that it requires YV12, so you need to:
a. convert to YV12, haven't tested whether you might even have to convert to 8bit for c-blend to work
b. let Hybrid know you did so
Instead of:

LoadPlugin("C:\AVSplugins\64bit\mvtools2.dll")
  LoadPlugin("C:\AVSplugins\64bit\masktools2.dll")
  Import("C:\AVSplugins\cdeblend.avs")
  cdeblend()

LoadPlugin("C:\AVSplugins\64bit\mvtools2.dll")
LoadPlugin("C:\AVSplugins\64bit\masktools2.dll")
Import("C:\AVSplugins\cdeblend.avs")
ConvertToYV12()
cdeblend()
# colorformat YV12

(added 'ConvertToYV12()' and '# colorformat YV12' to the custom section)
Same with DeStripe, it also requires YV12 so save adjustments to the custom code have to be made.

The scripts you used in AvsPmod worked probably since the source probably then was already YV12.

-> Custom section does seem to work properly, problem seemed to be is that the user made a mistake. Doesn't seem like I need to change any of the code in Hybrid.

Cu Selur

[serg]

Looking at the cdeblend debug output the used script for the preview is:

ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("C:\PROGRA~1\Hybrid\64bit\Avisynth\AVISYN~1\LoadDll.dll")
LoadDLL("C:\Program Files\Hybrid\64bit\Avisynth\avisynthPlugins\libfftw3f-3.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\64bit\Avisynth\AVISYN~1\LSMASHSource.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\64bit\Avisynth\AVISYN~1\TIVTC.dll")
# loading source: D:\Hybrid_test\Gill\face_10bit.mov
#  color sampling YUY2@10, matrix: bt601, scantyp: telecine, luminance scale: limited
Source = LWLibavVideoSource("D:\HYBRID~1\Gill\FACE_1~1.MOV",cache=false,format="YUV422P16", prefer_hw=0)
# current resolution: 720x486
# deinterlacing
SourceFiltered = Source
Source = Source.TFM(mChroma=true)
Source = Source.TDecimate(cycleR=1,cycle=5,noblend=true,nt=0,blockx=32,blocky=32,chroma=true,denoise=false,ssd=false,display=false)
Source = Source.AssumeFrameBased()
SourceFiltered = SourceFiltered.TFM(mChroma=true)
SourceFiltered = SourceFiltered.TDecimate(cycleR=1,cycle=5,noblend=true,nt=0,blockx=32,blocky=32,chroma=true,denoise=false,ssd=false,display=false)
SourceFiltered = SourceFiltered.AssumeFrameBased()

# CUSTOM SCRIPT PART - position: Restore - START
function CustomRestore(clip clp) {
  last=clp
  LoadPlugin("C:\AVSplugins\64bit\mvtools2.dll")
  LoadPlugin("C:\AVSplugins\64bit\masktools2.dll")
  Import("C:\AVSplugins\cdeblend.avs")
  cdeblend()
  return last
}
# CUSTOM SCRIPT PART - position: Restore - END

SourceFiltered = SourceFiltered.CustomRestore() # loading custom script content

#  color sampling YV16@16, matrix: bt601, scantyp: progressive, luminance scale: limited
# filtering
# stacking horizontal for filter preview
# adjust color to RGB32 (for preview)
SourceFiltered = SourceFiltered.ConvertToRGB32(matrix="Rec601")
# adjust color to RGB32 (for preview)
Source = Source.ConvertToRGB32(matrix="Rec601")
StackHorizontal(Source, SourceFiltered)
PreFetch(2)
# setting output fps to 23.976fps
AssumeFPS(24000,1001)
#  output: color sampling RGB32@16, matrix: bt601, scantyp: progressive, luminance scale: limited
return last

and looking at the documentation of C-deblend ('Now superseded by Srestore.') I see that it requires YV12, so you need to:
a. convert to YV12, haven't tested whether you might even have to convert to 8bit for c-blend to work
b. let Hybrid know you did so
Instead of:

  LoadPlugin("C:\AVSplugins\64bit\mvtools2.dll")
  LoadPlugin("C:\AVSplugins\64bit\masktools2.dll")
  Import("C:\AVSplugins\cdeblend.avs")
  cdeblend()

LoadPlugin("C:\AVSplugins\64bit\mvtools2.dll")
LoadPlugin("C:\AVSplugins\64bit\masktools2.dll")
Import("C:\AVSplugins\cdeblend.avs")
ConvertToYV12()
cdeblend()
# colorformat YV12

(added 'ConvertToYV12()' and '# colorformat YV12' to the custom section)
Same with DeStripe, it also requires YV12 so save adjustments to the custom code have to be made.

The scripts you used in AvsPmod worked probably since the source probably then was already YV12.

-> Custom section does seem to work properly, problem seemed to be is that the user made a mistake. Doesn't seem like I need to change any of the code in Hybrid.

Cu Selur

ConvertToYV12() as per your siggestion

Doesn't make any difference for cdeblend() - still no effect. And what about DeStripe()?

[Selur]

Same with DeStripe, it also requires YV12 so save adjustments to the custom code have to be made.

so before the DeStripe() call you should add a convertToYV12 and after it a '# colorformat YV12'

Cu Selur

[serg]

Same with DeStripe, it also requires YV12 so save adjustments to the custom code have to be made.

so before the DeStripe() call you should add a convertToYV12 and after it a '# colorformat YV12'

Cu Selur

That works, thank you.
Had to add ConvertToRGB24() before ConvertToYV12() (not sure if this correct approach) .
Otherwise - error "ConvertToYV12: only 8 bit sources allowed..." 
Any plan to make path to "Hybrid" plugins folder transparent for "Custom"?

[Selur]

Any plan to make path to "Hybrid" plugins folder transparent for "Custom"?

No sure what you m
You can use for example:

Import("%FILTERPATH%\daa.avs")
daa()

and Hybrid will replace the

%FILTERPATH%

with the path to the 32bit or 64bit avisynthPlugins-Path, so for me:

Import("I:\Hybrid\64bit\Avisynth\avisynthPlugins\daa.avs")

or

Import("I:\Hybrid\32bit\avisynthPlugins\daa.avs")

There will be no auto-loading, if you need auto-loading you need another tool than Hybrid.

Had to add ConvertToRGB24() before ConvertToYV12()

Make sure to take the ColorMatrix and ChromaSampling into Account when converting between YUV.. and RGB.. otherwise you will change the colors.

Cu Selur