Analyzers and Visualizers

Overview

doc_visualizers_ctx

All visualizer goes with a pair of nodes Initxxx and Getxxx. Init nodes must be called before of the Get nodes to reserve the buffer analysis data, and after the audio source initialization. During the audio playback, you can use the player controls (Start/Stop/Pause) to interact with the analysis process.

doc_visualizers_overview

Visualizer nodes flow

doc_flow_graph_visualizers

Channel Result Object

Channel mode and “by Channel” nodes
To be able to do multiple operations with each channel audio data the Channel Mode has been added to the configuration parameters of our analysis. So now we have 3 options:
  • All in one: Accumulates all channels data into only one array, to get the analysis result can be used the old Getxxx nodes

  • Select one: Returns only the channel specified in Channel option. Now to obtain only a Right Channel analysis you must select Channel Mode: Select one and Channel: 1

doc_visualizers_channelselectone
  • Split all: Returns each channel analysis into one array. You must use the new Getxxx By Channel nodes to retrieve the results. The output of this nodes are a Channel Result Object Reference. You can access to the number of channels and the data for each channel using the channel number as index.

doc_visualizers_bychannels
To interact with the result object we have 2 nodes
Get Num Channels
Returns number of channels stored in the object
Get Channel Result
Returns the analysis result of a specific channel.

Channel

Index of the channel

Run Time Analysis nodes

Amplitude

One of the most basic audio visualizer, just to retrieve the amplitude values of the audio wave.

doc_anim_amplitude
doc_visualizers_bp_amplitude
Init Amplitude Config
Configures the wave amplitude analyzer.

Channel Mode

All in one, Select one, Split all

Channel

Number of channel to be analyzed (Only valid for Select one option)

Num Time Bands

Number of bands to display, it determines the resolution of the wave display

Time Window

Interval Time showed by the display in seconds

Get Amplitude
Returns the amplitude values of the wave
Get Amplitude By Channel
Returns the amplitude values of the wave. The output is a Channel Result Object Reference.

Frequency spectrum

Visualize the frequency spectrum analysis. This analyzer include two frequency data distribution Linear and Logarithmic, but you can define too your own distribution feeding the init node with your own bounds for each band.

doc_anim_amplitude
doc_visualizers_bp_spectrum
Init Spectrum Config
Configures the spectrum frequency analyzer.

Type

Linear, Logarithmic

Channel Mode

All in one, Select one, Split all

Channel

Number of channel to be analyzed (Only valid for Select one option)

Num Freq Bands

Number of bands to distribute the frequencies

Time Window

Time of the frequency sample

History Size

Number of samples used to calculate the average frequencies (Default: 1)

UsePeakValues

Returns peak value of the band instead of average value

Num Peaks

Num of frequency peaks to be calculated by Get Spectrum Peaks

Init Spectrum Config WLimits
Configures the spectrum frequency analyzer. The frequency bounds of each band can be defined.

Channel Mode

All in one, Select one, Split all

Channel

Number of channel to be analyzed (Only valid for Select one option)

Band Limits

TArray of Vector2D to define the bounds of each band, must be sorted and no overlap allowed (Closed Intervals)

Time Window

Time of the frequency sample

History Size

Number of samples used to calculate the average frequencies (Default: 1)

UsePeakValues

Returns peak value of the band instead of average value

Num Peaks

Num of frequency peaks to be calculated by Get Spectrum Peaks

doc_visualizers_bp_spectrum_frequency
Get Spectrum
Returns the frequency spectrum values, and the average values and Calculate Averages Frequencies option is Enabled.

Calculate Averages Frequencies

Calculates average frequency values too

Get Spectrum By Channel
Returns the frequency spectrum values, and the average values and Calculate Averages Frequencies option is Enabled. The output is a Channel Result Object Reference.

Calculate Averages Frequencies

Calculates average frequency values too

doc_visualizers_bp_spectrum_peaks
Get Spectrum Peaks
Returns the frequency spectrum peaks, retrieve as many peaks as Num Peaks has been set on Init Spectrum Config node, ordered by energy.

Frequency Peaks

Value in Hz for the frequency peaks

Frequency Peaks Energy

Energy value for the frequency peaks

Get Spectrum Peaks By Channel
Returns the frequency spectrum peaks, retrieve as many peaks as Num Peaks has been set on Init Spectrum Config node, ordered by energy. The output is a Channel Result Object Reference.

Frequency Peaks

Value in Hz for the frequency peaks

Frequency Peaks Energy

Energy value for the frequency peaks

Beat Tracking

This beat tracking analyzer has a basic tracker configuration but you can configure your own tracking bounds too as in the spectrum analyzer. The pre-configured beat tracking use the next frequency limits to track the most valuable ranges of the spectrum.

  • Bass range: 0 - 87 Hz

  • Low Mid Range: 500 - 700 Hz

  • High Mid Range: 5000 - 12000 Hz

BPM Tracking

Now you can obtain the beat-per-minute result of each beat tracking band too.
This analysis returns the partial result of the bpm scan interval configurated and the overall bpm result for the band.
doc_anim_beat
doc_visualizers_bp_bt
Init Beat Tracking Config
Configures the beat tracking visualyzer.

Channel Mode

All in one, Select one, Split all

Channel

Number of channel to be analyzed (Only valid for Select one option)

Time Window

Time of the frequency sample

History Size

Number of samples used to calculate the average frequencies

Calculate BPM

Enable Beat-per-minute analysis

BPM History Size

Number of beat samples used to calculate the bpm. (Number of calls to GetBeatTracking node)

Threshold

Factor used to trigger the beat, values greater than Threshold * Average trigger this beat

Get Beat Tracking
Returns the basic Beat Tracking results in Bass, Low Mid range and High Mid range, the current band spectrum values, the current BPM counter result for each band and the Total BPM
Get Beat Tracking By Channel
Returns the basic Beat Tracking results in Bass, Low Mid range and High Mid range, the current band spectrum values, the current BPM counter result for each band and the Total BPM. The output is a Channel Result Object Reference.
Get Beat Tracking WThreshold
Returns the basic Beat Tracking results in Bass, Low Mid range and High Mid range, and the current band spectrum values, the current BPM counter result for each band and the Total BPM, using the input threshold values.

Override Threshold

TArray to override the threshold factor for each band

Get Beat Tracking WThreshold By Channel
Returns the basic Beat Tracking results in Bass, Low Mid range and High Mid range, and the current band spectrum values, the current BPM counter result for each band and the Total BPM, using the input threshold values. The output is a Channel Result Object Reference.

Override Threshold

TArray to override the threshold factor for each band

doc_visualizers_bp_bt_limits

A basic beat tracking over a specific frequency bands, the band bounds can be defined, must be sorted and no overlap allowed (Closed Intervals)

Init Beat Tracking Config WLimits
Configures the beat tracking visualyzer

Channel Mode

All in one, Select one, Split all

Channel

Number of channel to be analyzed (Only valid for Select one option)

Band Limits

TArray of Vector2D to define the bounds of each band, must be sorted and no overlap allowed

Time Window

Time of the frequency sample

History Size

Number of samples used to calculate the average frequencies

Calculate BPM

Enable Beat-per-minute analysis

BPM History Size

Number of beat samples used to calculate the bpm. (Number of calls to GetBeatTracking node)

Threshold

Factor used to trigger the beat, values greater than Threshold * Average trigger this beat

Get Beat Tracking WLimits
Returns the Beat Tracking results for the bands defined in the configuration, the current frequency spectrum values for each band, the current BPM counter result for each band and the Total BPM.
Get Beat Tracking WLimits By Channel
Returns the Beat Tracking results for the bands defined in the configuration, the current frequency spectrum values for each band, the current BPM counter result for each band and the Total BPM. The output is a Channel Result Object Reference.
Get Beat Tracking WLimits WThreshold
Returns the basic Beat Tracking results in Bass, Low Mid range and High Mid range, and the current band spectrum values, the current BPM counter result for each band and the Total BPM, using the input threshold values.

Override Threshold

TArray to override the threshold factor for each band

Get Beat Tracking WLimits WThreshold By Channel
Returns the basic Beat Tracking results in Bass, Low Mid range and High Mid range, and the current band spectrum values, the current BPM counter result for each band and the Total BPM, using the input threshold values. The output is a Channel Result Object Reference.

Override Threshold

TArray to override the threshold factor for each band

doc_visualizers_bp_bt_utils

Some nodes to return intermediate values

Get Beat Tracking Average
Returns the frequency average values for each band
Get Beat Tracking Average By Channel
Returns the frequency average values for each beat band. The output is a Channel Result Object Reference.
Get Beat Tracking Average and Variance
Returns the frequency average and variance for each beat band
Get Beat Tracking Average and Variance By Channel
Returns the frequency average and variance for each beat band. The output is a Channel Result Object Reference.

Pitch Tracking

Now you can obtain the fundamental frequency of an audio in real-time. It’s an implementation using YIN algorithm.

doc_anim_pitch
doc_visualizers_bp_pt
Init Pitch Tracking Config
Configures the pitch tracking analyzer.

Channel Mode

All in one, Select one, Split all

Channel

Number of channel to be analyzed (Only valid for Select one option)

Time Window

Time of the analysis sample

Threshold

Yin algorithm threshold (Default 0.19)

Get Pitch Tracking
Returns the pitch tracking result, the fundamental frequency value
Get Pitch Tracking By Channel
Returns the pitch tracking result (fundamental frequency value). The output is a Channel Result Object Reference.

Analysis Utils

doc_visualizers_utils
Frequency to MIDI
Converts a frequency value in Hz into MIDI note value.

Frequency Hz

Frequency value in hertz

Amplitude to DB
Converts normalized amplitude values to decibel scale (without device calibration), just calculated as dB = 20 * log(Amplitude)

Amplitude

Amplitude normalized value

Offline Analysis nodes

Amplitude

doc_visualizers_bp_off_ctx.png

We need to initialize an Extractor instance to be able to use the offline analysis nodes

doc_visualizers_bp_off_amplitude.png
Get Amplitude Section
Returns the analysis result for a specific section of the audio file. We can set the resolution of the wave.

Channel Mode

All in one, Select one, Split all

Channel

Number of channel to be analyzed (Only valid for Select one option)

Start Time

Start position of the section (in seconds)

End Time

End position of the section (in seconds)

Resolution

This is the number of samples grouped in each bar. With small values we can obtain more detail of the wave but more results too

Get Amplitude Section by Channel
Returns the analysis result for a specific section of the audio file. We can set the resolution of the wave. The output is a Channel Result Object Reference.

Channel

Number of channel to be analyzed (Only valid for Select one option)

Start Time

Start position of the section (in seconds)

End Time

End position of the section (in seconds)

Resolution

This is the number of samples grouped in each bar. With small values we can obtain more detail of the wave but more results too