Selur's Little Message Board

Since Vapoursynth supports audio for quite some time so I collected what filters&o are out there and I though others might be interested in it too

External plugins

Plugin	Namespace	Functionality	Notes
VS-AudioTools	atools	Convert, Crossfade, Delay, FadeIn, FadeOut, FindPeak, Mix, Normalize, SineTone	Most comprehensive dedicated audio utility plugin found
AudioBoost	ab	Soft clipping, saturation, nonlinear amplification	WIP, VapourSynth compatibility need to be tested
AudioResample	Historical	Sample-rate and sample-format conversion	Functionality now available natively through Vapoursynth
Faveworm	fw	Visualization	Analog oscilloscope emulation, waveform rendering, trigger mode, scrolling mode, X-Y mode
BestSource	bs	Audio/video source using FFmpeg	Source plugin, sample-accurate audio access

Native VapourSynth audio functions

Function	Category	Functionality
AssumeSampleRate	Format	Assume a different sample rate
AudioGain	Gain	Linear amplification, per-channel gain
AudioLoop	Editing	Repeat audio samples
AudioMix	Mixing	Arbitrary channel mixing matrix, multiple input clips
AudioResample	Conversion	Sample-rate, sample-type, bit-depth and channel-layout conversion
AudioReverse	Editing	Reverse audio samples
AudioSplice	Editing	Concatenate audio clips
AudioTrim	Editing	Sample-accurate trimming
BlankAudio	Generation	Generate silence, sine, square, triangle or sawtooth signals
SetAudioCache	Infrastructure	Configure audio-node caching
ShuffleChannels	Channels	Extract, reorder and combine channels
SplitChannels	Channels	Split multichannel audio into mono clips

Audio-processing capabilities

Feature	Native	External
Gain adjustment	Yes	atools
Peak normalization	No	atools
Loudness normalization	No	None identified
Soft clipping	No	AudioBoost
Dynamic compression	No	No conventional compressor identified
Equalization	No	None identified
Noise reduction	No	None identified
Time stretching	No	None identified
Pitch shifting	No	None identified
Reverb / convolution	No	None identified
Fades / crossfades	No dedicated filter	atools
Delay	No dedicated DSP filter	atools
Mixing / downmixing	Yes	atools
Resampling	Yes	Native
Dithering	Yes	Native

If someone knows of a filter or feature I haven't listed, please let me know, so I can update this overview.
Thanks!

Cu Selur

Thank you for the highly useful post!

Maybe you could post a few sample scripts combining audio and video with audio filtering for those of us who are clueless about it?

I wrote an example of audio and video cutting at the same time a while ago:

Code:
# Imports
import math
import sys
import os
import vapoursynth as vs
# using Vapoursynth R78
# getting Vapoursynth core
core = vs.core
# Limit frame cache to 48449MB
core.max_cache_size = 48449
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/Vapoursynth/Lib/site-packages/vapoursynth/plugins2/libbestsource.dll")
# Import scripts
import validate
# Source: 'g:\Test.mkv'
# clip current meta; color space: YUV420P8, bit depth: 8, resolution: 1920x1080, fps: 23.976, color matrix: 709, color primaries: Unspecific, color transfer: BT.709, yuv luminance scale: limited, scanorder: progressive, full height: true (Source)
# Loading 'g:\Test.mkv' using BestSource
clip = core.bs.VideoSource(source="G:/Test.mkv", cachepath="J:/tmp/Test_bestSource", track=0)
frame = clip.get_frame(0)
# setting color matrix to 709.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)
# setting color transfer (vs.TRANSFER_BT709), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709)
# setting color primaries info (to vs.PRIMARIES_BT709), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709)
# setting color range to TV (limited) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_LIMITED})
# making sure frame rate is set to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# making sure the detected scan type is set (detected: progressive)
video = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # scan type: progressive

# Format: ac-3
# Length: 2567.840
# Bit rate: 384
# Channels: 6
# Sample rate: 48000
# Raw Audio: true
# Codec: pcm
# Channel Mapping: 3/2/0.1
# Dialog Normalization: -27
# Video delay: 0
# New Video delay: 0
# Audio only delay: 0
audio = core.bs.AudioSource("G:/Test.mkv")


def trim_av(video: vs.VideoNode, audio: vs.AudioNode, first: int = 0, last: int = None) -> tuple:
    """
    Trims both video and audio simultaneously based on video frame numbers.
    Follows VapourSynth's core.std.Trim behavior (inclusive of 'last').
    """
    # 1. Trim the video node
    v_trimmed = core.std.Trim(video, first=first, last=last)
    
    # 2. Calculate frame rate and sample rate
    fps = video.fps.numerator / video.fps.denominator
    sample_rate = audio.sample_rate
    
    # 3. Calculate corresponding audio samples
    # First sample starts at the beginning of the 'first' frame
    audio_first = math.floor(first * sample_rate / fps)
    
    if last is not None:
        # Last sample ends at the end of the 'last' frame
        audio_last = math.floor((last + 1) * sample_rate / fps) - 1
    else:
        audio_last = None
        
    # 4. Trim the audio node
    a_trimmed = core.std.AudioTrim(audio, first=audio_first, last=audio_last)
    
    return v_trimmed, a_trimmed

# Cut List frame numbers based on progressive input
cut_list = [
    (12995, 13035),
    (13071, 13123),
    (13180, 13270),
    (13369, 13413),
    (13541, 14944),
    (17796, 18033),
    (19986, 20060),
    (31344, 31553),
    (60053, 60764)
]

# Do the cutting
video_cuts = []
audio_cuts = []

for start, end in cut_list:
    v_cut, a_cut = trim_av(video, audio, first=start, last=end)
    video_cuts.append(v_cut)
    audio_cuts.append(a_cut)

# combine cuts
video = core.std.Splice(video_cuts)
audio = core.std.AudioSplice(audio_cuts)

# apply additional audio filtering

# adjusting output color from YUV420P8 to YUV420P10 for NVEncModel
video = core.resize.Bicubic(clip=video, format=vs.YUV420P10)
# set output frame rate to 23.976fps (progressive)
video = core.std.AssumeFPS(clip=video, fpsnum=24000, fpsden=1001)

# output
video.set_output(0)
audio.set_output(1)

Cu Selur

Ps.: I'm also uploading a new dev version (will be up in ~ 1hr) which can play audio in the Vapoursynth Preview and add some visualization using Faveworm (this still needs some tuning).
(control this under Filtering->Vapoursynth->Misc->Preview)
It also has support to decode and cut audio using Vapoursynth, but those scripts are usually not inside the same script as the video.
Pss.: once can use mkv output with vspipe and feed the output to i.e. ffmpeg or play it with ffplay.

Selur

videoh

Selur