[GUIDE] VapourSynth + QTGMC Deinterlace + Hybrid UI FAQ for macOS

[shijan]

This VapourSynth + Plugins + Scripts + Hybrid [2020.07.27] DMG package includes:
VapourSynth-R50
downloaded from https://forum.doom9.org/showthread.php?t=175522
All VapourSynth Plugins, Dependencies and Libraries
downloaded from https://forum.doom9.org/showthread.php?t=175522
D2V Witch v4 (it required by Hybrid to process VOB files and DVDs)
downloaded from https://forum.doom9.org/showthread.php?t=175522
JESExtensifier 3.0.5 (it helps to change aspect ratio and other metadata tags inside MOV container)
downloaded from http://jes.weiss.no
Hybrid 2018.08.05.1 (with updates)
downloaded from https://www.videohelp.com/software/Hybrid
Updates installed inside Hybrid:
Scripts for Hybrid
downloaded from https://github.com/Selur/VapoursynthScriptsInHybrid
FFmpeg 4.3.1
downloaded from https://evermeet.cx/ffmpeg/
MKVToolNix-48.0.0
downloaded from https://www.videohelp.com/software/MKVToolNix

DOWNLOAD from Dropbox: https://www.dropbox.com/sh/hzy3f0zyuyu4y...O5zla?dl=0
DOWNLOAD from Google Drive: https://drive.google.com/drive/folders/1...sp=sharing




Quick FAQ for Beginners:

VapourSynth http://www.vapoursynth.com/ is a cross-platform Open Source application for video manipulation with command line interface. It also can be used as a plugin inside other video editing apps, video players and UI wrappers. It require Python https://www.python.org/ environment to work. VapourSynth support external Plugins (.dylib extension on macOS). Some plugins may require additional Dependencies Libraries (also .dylib extension on macOS). It also may require external Script files (.py extension) to automate some processes.
Current version of VapourSynth (including Python, Plugins, Dependencies Libraries and Scripts) is wrapped inside single Framework container. On macOS it is installed to:
/Library/Frameworks/VapourSynth.framework
Dependencies Libraries installed to:
/Library/Frameworks/VapourSynth.framework/lib/
External Plugins installed to:
/Library/Frameworks/VapourSynth.framework/lib/vapoursynth/
Scripts installed to:
/Library/Frameworks/VapourSynth.framework/lib/python3.8/site-packages/
Some legacy apps may read Dependencies Libraries and Pugins from legacy location:
/usr/local/lib/
Currently to fix this incompatibility problem VapourSynth installs to that directory special alias links from "libvapoursynth-script.dylib" and "libvapoursynth.dylib" files.

QTGMC http://avisynth.nl/index.php/QTGMC is a series of scripts for VapourSynth that combine chain of different plugins and produce Motion Compensation deinterlace with double frame rate and amazing artifacts-free quality.

Hybrid http://www.selur.de/ is a cross-platform UI wrapper for VapourSynth and a bunch of other video tools http://www.selur.de/licence It use well known FFmpeg library https://evermeet.cx/ffmpeg/ to read and write ProRes MOV files as well as many other file types. Starting from 2018 Hybrid is no more updated for macOS, but seems it still works well on macOS 10.14.6. Some important components like FFmpeg and MKVToolNix located inside Hybrid.app can be easily updated manually.
Hybrid combines a lot of video tools inside but it don't contains pre-installed VapourSynth. To make QTGMC work you need to install and adjust VapourSynth with all required plugins and scripts separately.




Prepare to Installation FAQ:

Do NOT install VapourSynth with Homebrew https://brew.sh/. It seems Homebrew installation scripts are outdated and some sources don't compiled as they should.
Do NOT mix it with other VapourSynth versions or with different installers made by other people.
If you use any other VapourSynth installations before - delete old versions before install this one.

Disable SIP (System Integrity Protection):
Restart computer
Hold down Command-R to reboot into Recovery Mode
Click Utilities, select Terminal and type:

csrutil disable

Press Return or Enter on keyboard and restart computer

Allow install apps from unidentified developers by typing in Terminal:

sudo spctl --master-disable

Press Return or Enter on keyboard.
Type your password.
Press Return or Enter on keyboard.

Disable Gatekeeper turning back on automatically after 30 days by typing in Terminal:

sudo defaults write /Library/Preferences/com.apple.security GKAutoRearm -bool NO

Press Return or Enter on keyboard.
Type your password.
Press Return or Enter on keyboard.

Disable quarantine for downloaded apps by typing in Terminal:

sudo defaults write com.apple.LaunchServices LSQuarantine -bool NO

Press Return or Enter on keyboard.
Type your password.
Press Return or Enter on keyboard.

For more details see additional FAQs:
https://www.imore.com/how-open-apps-anyw...and-mojave
https://www.imore.com/how-turn-system-in...tion-macos
https://apple.stackexchange.com/question...-on-mojave
https://apple.stackexchange.com/question...t-to-the-t

I personally recommend to use 10.14.6 Mojave and don't see any practical reason move to macOS 10.15 Catalina with insane security restrictions on every step and without 32-bit apps support.


Installation FAQ:

1. Open VapourSynth + Plugins + Scripts + Hybrid DMG package
2. Install "VapourSynth-R50.pkg"
3. Open "Apps" folder and move all .app files to "Applications" alias
4. Open "Scripts" folder and move all .py files to "site-packages" alias
5. Open "Plugins" folder and move all .dylib files and nnedi3_weights.bin file to "vapoursynth" alias

VapourSynth installer creates "VapourSynth" folder on your Desktop with quick alias links to Plugins, Dependencies, Scripts and Uninstall script. You can move this folder to any location, if you want to keep it.

When you start Hybrid first time, it may give you a message: “Hybrid.app” can’t be opened because the identity of the developer cannot be confirmed.

To fix this problem go to "Applications" folder, mouse right-click on "Hybrid.app", select in menu "Open", and next press "Open" button in pop-up window.

Sometimes after installing new plugins or doing other internal changes "Hybrid" may force quit after start. Just open it again and it should work well.


Usage FAQ:

Filtering -> QTGMC (Vapoursynth)
Filtering -> Overwrite input scan type to: (For Hi8 tape sources set it to "Top Field List", For DV tape sources set it to "Bottom Field List". If rendered video plays back with with "jumping/strobing" effect, change Field List order setting)
Filtering -> Preset: Slow (in "Custom mode" you can also experiment with other settings)
Filtering -> Bob (use for double frame rate deinterlace mode)
Open File
Set Output Folder
Render file (Command-F1 keyboard shortcut)
Base -> Video: ProRes
Base -> Audio: passthrough all
Base -> Default container: mov
Base -> Generate
ProRes -> Profile: Even if you render to SD, set preset to ProRes422HQ (HD)
Filtering -> Filtering -> ColorMatrix: From Rec.601 to Rec.709 (apply only If you upscale SD to HD with Crop/Resize tools in Hybrid)
Filtering -> VapourSynth -> Line -> Resizer -> NNEDI3
Neural network resizer NNEDI3. It removes jaggies upscale artifacts on contrast diagonal lines, but don't adds artificial oversharpening. Recommended settings: Neurons 32(64 max), Level 1(2 max), Network 2. On higher settings it is very slow. NNEDI3 use its own internal resize algorithm and don't depends of resize algorithm selected in "Crop/Resize" tab.

Some tips to speedup workflow:
Instead of "Open File" dialog it is possible to drag and drop file directly to path field.
For batch processing you can select multiple files in "Open File" dialog or drag and drop multiple files directly to path field.
Config -> Internals -> FFmped/MEncoder threads: (set it according your CPU)
Config -> Output -> Container Settings -> Standard output container: mov
Config -> Output -> Filename Generation -> Suffix:
Config -> Path -> Default input path:
Config -> Path -> Default temp path:
Config -> Path -> Adjust: Output path to source path
Config -> Profiles -> Auto save defaults (this allow to remember all program settings on next launch)
Jobs -> Misc -> Delete jobs when finished

It is also possible to change font size and use native system buttons instead of custom UI icons (see more info here https://forum.selur.net/showthread.php?tid=10):
1. Create hidden folder ".hybrid" in home directory (~/.hybrid/)
2. Create TXT file and rename it to "misc.ini"
3. Put misc.ini inside ".hybrid" folder (~/.hybrid/misc.ini)
4. Add this text to "misc.ini" file:

[General]
disableIcons=true
fontSize=14
toolTipFontSize=14

Bugs and Fixes:

Hybrid on macOS can't read interlaced AVI DV files.
Problem fix: Remux AVI to MOV container without recompression. You can do it with legacy QuickTime 7.6.6 Player (File -> Save as self contained movie) or something similar.

Mistake in ProRes presets. ProRes422HQ (SD) use standard basic ProRes422 compression.
Problem fix: If you want your result in ProRes422HQ, always set preset to ProRes422HQ (HD), even if your footage is SD.

Hybrid/FFmpeg ProRes files don't match exactly to original Apple ProRes specs. No correct codec names, no color tags, etc...
Problem fix: Edit tags with JESExtensifier or with AMCDXVideoPatcher http://mogurenko.com/

Hybrid FFmpeg renders ProRes422HQ as 10 bit and ProRes444 also only as 10 bit. Seems like known FFmpeg limitation https://trac.ffmpeg.org/ticket/7163

Hybrid adds additional empty frame to MP4 x264/265 transcoded video. Probably due outdated MP4Box/GPAC muxer components.
No problem when transcode to MKV or MOV container with same x264/265 settings.
Problem fix: Config -> Output -> Container Settings -> FFmpeg instead of MP4Box

Hybrid can't passthrough 24bit audio if use default FFmbc muxer.
Problem Fix: Use FFmpeg muxer instead of FFmbc muxer.
Config -> Output -> Container Settings -> uncheck FFmbc for .mov muxing
Or you can just transcode from 24bit to 24bit instead of passthrough.

Preview window in this version of Hybrid seems totally broken. When i click VapourSynth Preview button in Filtering tab, VS icon jumps in Dock and disappears. No any error log and no any Preview window. Other (non Vapoursynth) Preview button works well.
Preview button in Crop/Resize tab also cause permanent program hang and crash.
Problem fix: Don't use Preview window in Hybrid.

Too small font size in some Tooltips and Help Info panel.
Partial problem fix: Copy/Paste text to external TXT document and read it there.

When switching QTGMC presets to custom mode, Sharpness parameter is always inactive. Even if Sharpness is unchecked in UI, it is always force used in VapourSynth script, so QTGMC always use Sharpness=1.0. This bug was confirmed by developer of the Hybrid.
Problem fix (if you want to experiment with Sharpness strength):
Export QTGMC preset from Hybrid.
Open .json file in Text Editor.
Change "vsQTGMCSharpness": "1.0" to "vsQTGMCSharpness": "0.0"
Save changes.
Apply same preset in Hybrid.

[Selur]

as a side note:
There are also pre-compiled versions of MP4Box available (https://gpac.wp.imt.fr/downloads/gpac-nightly-builds/), which are part of gpac.
Note that it is probably not enough to replace just the mp4box binary, some of the libraries in MacOS/lib-folder might have to be updated too (they too come with gpac).

[shijan]

Ok, i will try to update it too. One quesion here:
/Applications/Hybrid.app/Contents/MacOS/lib folder contains 28 .dylib files
/Applications/GPAC/Contents/MacOS/lib folder contains 63 .dylib files

Should i update only 28 old files or should place all 63 new files to Hybrid?

Here is full list of new files:

liba52.0.dylib
libass.9.dylib
libavcodec.58.dylib
libavdevice.58.dylib
libavfilter.7.dylib
libavformat.58.dylib
libavresample.4.dylib
libavutil.56.dylib
libbluray.2.dylib
libcrypto.1.1.dylib
libfaad.2.dylib
libffi.6.dylib
libfontconfig.1.dylib
libfreetype.6.dylib
libfribidi.0.dylib
libgif.dylib
libglib-2.0.0.dylib
libgmp.10.dylib
libgnutls.30.dylib
libgpac.dylib
libgraphite2.3.dylib
libharfbuzz.0.dylib
libhogweed.4.dylib
libidn2.0.dylib
libintl.8.dylib
libjpeg.9.dylib
liblept.5.dylib
liblzma.5.dylib
libmad.0.dylib
libmp3lame.0.dylib
libnettle.6.dylib
libogg.0.dylib
libopencore-amrnb.0.dylib
libopencore-amrwb.0.dylib
libopenjp2.7.dylib
libopus.0.dylib
libp11-kit.0.dylib
libpcre.1.dylib
libpng16.16.dylib
libpostproc.55.dylib
librtmp.1.dylib
librubberband.2.dylib
libSDL2-2.0.0.dylib
libsnappy.1.dylib
libsoxr.0.dylib
libspeex.1.dylib
libssl.1.1.dylib
libswresample.3.dylib
libswscale.5.dylib
libtasn1.6.dylib
libtesseract.4.dylib
libtheora.0.dylib
libtheoradec.1.dylib
libtheoraenc.1.dylib
libtiff.5.dylib
libunistring.2.dylib
libvidstab.1.1.dylib
libvorbis.0.dylib
libvorbisenc.2.dylib
libwebp.7.dylib
libwebpmux.3.dylib
libx264.155.dylib
libx265.179.dylib

[Selur]

Most of the libs that are in the lib folder of gpac should not be needed since mp4box does not usw them.
Also some of the libs in Hybrids lib folder are from Mediainfo iirc. -> when updating mp4box only replacing a few of the 28 libs should be enough.

Cu Selur

[shijan]

Ok, so i select same 28 files now. What exact dylib files from this list should i replace to keep things safe and stable?

Also in addition to libx264.155.dylib i see new libx265.179.dylib file inside GPAC.app It feels like may be something required by new MP4Box version? Or just ignore it?
Hybrid contains only libx264.148.dylib

liba52.0.dylib
libavcodec.58.dylib
libavdevice.58.dylib
libavfilter.7.dylib
libavformat.58.dylib
libavresample.4.dylib
libavutil.56.dylib
libcrypto.1.1.dylib
libfaad.2.dylib
libfreetype.6.dylib
libgpac.dylib
libjpeg.9.dylib
libmad.0.dylib
libmp3lame.0.dylib
libogg.0.dylib
libpng16.16.dylib
libpostproc.55.dylib
libSDL2-2.0.0.dylib
libssl.1.1.dylib
libswresample.3.dylib
libswscale.5.dylib
libtheora.0.dylib
libvorbis.0.dylib
libx264.155.dylib

[Selur]

Seither x264 nor x265 libs are required by mp4box, No clue which libs are needed since it is simply to long ago that I used a Mac.

[shijan]

VapourSynth + Plugins + Scripts + Hybrid package update 2020.03.23
Changes:
Package now includes D2V Witch.app required by Hybrid to process VOB files.
Updated FAQ.
Links to discussion forums.

VapourSynth + Plugins + Scripts + Hybrid package update 2020.03.28
Changes:
Package now includes JESExtensifier.app (it help to change aspect ratio and other metadata tags inside .MOV container)
Updated FAQ.

VapourSynth + Plugins + Scripts + Hybrid package update 2020.03.29
Changes:
VapourSynth R49
Updated FAQ.
Download from same Dropbox folder: https://www.dropbox.com/sh/hzy3f0zyuyu4y...O5zla?dl=0

[shijan]

Here are few more macOS related questions:
1. I have AMD RX 580 GPU, and in Hybrid UI in many filters i see option to use GPU and OpenCL. But checkmarks are always inactive. Is it possible somehow to turn those things on on macOS? Or maybe they inactive due some limitation in VapourSynth plugins? Or they are for Nvidia cards only?



2. When i switch QTGMC presets to custom mode, Sharpneess value is always inactive. Is this a bug?

[Selur]

1. about OpenCL support in some filters/scripts: as far as I know there is no needi3cl available for MacOS X which is why all options that relate to it are disabled.
Same for some other filters.
2. yes, that's a bug. (happens when Vapoursynth is available but not Avisynth)

Cu Selur

[shijan]

Strange. There is a libnnedi3cl.dylib as well as libnnedi3.dylib in plugins folder.

2. yes, that's a bug. (happens when Vapoursynth is available but not Avisynth)

That is odd problem