Inputs

The input or source determines and controls the origin of the audio data. We have multiple available inputs, and each one can manage different audio feeding method. For example, to use audio files we are going to need the Player or the Extractor Object. For a microphone we have the Capturer or the OVR, depends on the target platform. We have too the ability to listen the audio from other devices using the Loopback input (only works on Windows platforms for now), and with the Stream Object we can inject directly the audio data from other sources.

Player Input

This plugin includes a basic audio player to control the loaded audio playback.
Current player features:
  • Play/Stop/Pause controls

  • Load audio files from the content folder

  • Allow seek playback

  • Allow adjust playback volume

  • Access to audio metadata

Initialization

doc_player_init_ctx
We need to initialize some parameters before be able to play any sound, we need to provide the sound path at this moment.
We can configure the buffer playback size too. By default the buffer has a value of 1024 frames, must be enought small to feed the audio playback and get smooth playback progress updates.

Warning

If you experience audio gliches on Android platform the buffer size must be increased. The optimal value may vary between devices.

doc_player_init_bp
Init Player Audio
Initializes the player audio source instance.

File Name

Path to the audio file

Init Player Audio Ex
Initializes the player audio source instance (Extended version).

File Name

Path to the audio file

Player Buffer Size

Player buffer size (power of 2)

Unload Player Audio
Closes the player audio device
doc_player_init_bp_device
Set Default Device Player Audio
Sets a specific player audio device to play the sound file. Must be called before Init Player Audio nodes.

Audio Device Name

Audio device name (From Get Output Audio Device node output)

Get Default Device Player Audio
Returns the player audio device

Players controls

doc_player_controls_ctx

After the player initialization we can use the player nodes to control the sound playback, We can start or stop the playback, change the volume, set the playback position…

doc_player_controls_bp_play
Play
Starts audio playback from the beginning but if the audio has been paused previously, continue the playback from this position
Play Ex
Starts audio playback from the beginning but if the audio has been paused previously, continue the playback from this position. Each loop starts from StartTime position.

Loops

Number of loops (0-infinite playback)

StartTime

Time in seconds to start the playback

Stop
Stops audio playback
Is Playing
Returns player playing state
doc_player_controls_bp_pause
Pause
Pauses player playback if the device is in playing state UnPauses player playback if the device has been paused previously
Set Paused
Sets the player state on pause/unpause state.

Pause

New pause state

Is Paused
Returns player pause state
doc_player_controls_bp_loops
Get Remaining Loops
Returns the number of remaining loops to be played
doc_player_controls_bp_total
Get Total Duration
Returns total player playback duration
doc_player_controls_bp_time
Get Playback Time
Returns current player playback position
Set Playback Time
Sets current player playback position.

Progress

New playback position in seconds

doc_player_controls_bp_volume
Get Playback Volume
Returns current player playback volume
Set Playback Volume
Sets a new volume in range 0.0 - 1.0.

Volume

New volume in range 0.0 - 1.0

Player Events

doc_player_events_ctx

We can track some event related with the audio playback, like the end of a loop or the total file playback end.

Bindable events

doc_player_events_bp_finish
On Playback Finished
Event to track the player playback end. Triggered when the playback reaches the end of the file. All loops has been played.
On Playback Loop Finished
Event to track the loop playback end. Triggered when the playback reaches the end of one loop

Player Utils

doc_player_utils_ctx

Some player extra functions

doc_player_utils_bp_meta
Get Metadata
Returns metadata if the file has the info.

Filename

Filename of the audio file

Extension

Extension of the audio file

MetaType

ID3_V1 , ID3_V2.3 , ID3_V2.4

Title

Title of the song

Artist

Artist

Album

Album

Year

Year

Genre

Genre

Get Metadata Art
Extract the AlbumArt pictures into the selected folder. The format of the file name will be {Prefix}{number_of_file}.{image_extension}. Prefix1.png, Prefix2.png,…

Prefix

Prefix used to name the pictures.

Folder

Destination folder, must exists

Number of files

Number of stored pictures

doc_player_utils_bp_info
Get Player Device Audio Info
Returns the parameters used to configure the player device.

Device Name

Name of the device in the system devices list

Sample Rate

Number of samples per second

Num Channels

Number of channels

Buffer Samples

Number of samples of the playback buffer

Format

Format of the values that represent each sample (Fixed/Float)

Bit Depth

Number of bits used for each sample

Player Supported formats

This player support multiple audio formats, with wav files we are using the memory file directly so no wait time is needed, but with compressed formats a decode process is done when the audio file is loaded, so have the audio ready for the playback can take some time.

Wav files , multichannel supported 1 (mono), 2 (stereo), 4 (quad), and 6 (5.1)
- PCM signed 16bit
- PCM signed 32 bit
- PCM float 32 bit
Mp3 files , multichannel (ID3 Tag v1/v2)
- CBR Constant Bitrate
- VBR Variable Bitrate
Ogg files , multichannel
- Vorbis
- Flac

Microphone Input

This plugin supports a real-time audio capture from a microphone source
Current capturer features:
  • Real Time capture

  • Volume Override

Initialization

doc_capturer_init_ctx

We need to initialize some parameters before be able to capture sound

doc_capturer_init_bp
Init Capturer Audio
Initializes the capturer audio source instance
Init Capturer Audio Ex
Initializes the capturer audio source instance (Extended version).

Sample Rate

Samples per second of the capture

Bit Depth

Bits per sample

Audio Format

Sample number format (fixed/float)

Capturer Buffer Seconds

Number of seconds of capture history

Player Buffer Size

Player buffer size (power of 2)

Unload Capturer Audio
Closes the capturer audio devices
doc_capturer_init_bp_device
Set Default Devices Capturer Audio
Sets the audio device to capture and play the sound. Must be called before Init Capturer Audio nodes.

Capturer Device Name

Audio device name to capture the sound (From Get Input Audio Device node output)

Player Device Name

Audio device name to play the sound (From Get Output Audio Device node output)

Get Default Devices Capturer Audio
Returns the capturer and local player audio device.

Capturer Device Name

Audio device name to capture the sound

Player Device Name

Audio device name to play the sound

Capturer controls

doc_capturer_controls_ctx

After initialization we can use the capturer nodes to control the start or end of the capture

doc_capturer_controls_bp
Start Capture
Starts audio capture (and playback).

Playback Capturer

Enable to playback the capture sound at the same time

Stop Capture
Stops audio capture
Is Capturing
Returns player playing state
doc_capturer_controls_bp_time
Get Capture Time
Returns total capture elapsed time
doc_player_controls_bp_volume
Get Capture Volume
Returns current capture volume.

Capture Volume

Microphone normalized volume

Playback Volume

Playback normalized volume

Set Capture Volume
Sets a new volume for microphone capture and playback in range 0.0 - 1.0

Capture Volume

New capture volume in range 0.0 - 1.0

Playback Volume

New playback volume in range 0.0 - 1.0

Capturer Utils

doc_capturer_utils_ctx

Some capturer extra functions

doc_capturer_utils_bp_info
Get Capturer Devices Audio Info
Retrieves the capture/playback audio device info.

Capturer Device Name

Name of the capturer device in the system devices list

Capturer Sample Rate

Number of samples per second

Capturer Num Channels

Number of channels

Capturer Buffer Samples

Number of samples of the playback buffer

Capturer Format

Format of the values that represent each sample (Fixed/Float)

Capturer Bit Depth

Number of bits used for each sample

Player Device Name

Name of the player device in the system devices list

Player Sample Rate

Number of samples per second

Player Num Channels

Number of channels

Player Buffer Samples

Number of samples of the playback buffer

Player Format

Format of the values that represent each sample (Fixed/Float)

Player Bit Depth

Number of bits used for each sample

Loopback Input

This plugin can listen an audio system device (Only Windows platform) and take the output audio to do the analysis. If no Audio device is set, then listen on default system audio device.
Current Loopback features:
  • Real Time capture

Initialization

doc_loopback_init_ctx

We need to initialize some parameters before be able to capture sound

doc_loopback_init_bp
Init Loopback Audio
Initializes the loopback audio source instance
Init Loopback Audio Ex
Initializes the loopback audio source instance (Extended version).

Capturer Buffer Seconds

Number of seconds of capture history

Unload Loopback Audio
Closes the loopback listener
doc_loopback_init_bp_device
Set Default Device Loopback Audio
Sets the audio device to listen. Must be called before Init Loopback Audio nodes.

Source Device Name

Audio device name to listen (From Get Output Audio Device node output)

Get Default Device Loopback Audio
Returns the listened audio device.

Source Device Name

Audio device name to capture the sound

Loopback controls

doc_loopback_controls_ctx

After initialization we can use the next loopback nodes to control the start/end of the capture

doc_loopback_controls_bp
Start Loopback
Starts audio capture
Stop Loopback
Stops audio capture
Is Loopback Capturing
Returns player playing state
doc_loopback_controls_bp_time
Get Loopback Capture Time
Returns total capture elapsed time

Loopback Utils

doc_loopback_utils_ctx

Some loopback extra functions

doc_loopback_utils_bp_info
Get Loopback Device Audio Info
Retrieves audio device info.

Device Name

Name of the capturer device in the system devices list

Sample Rate

Number of samples per second

Num Channels

Number of channels

Buffer Samples

Number of samples of the playback buffer

Format

Format of the values that represent each sample (Fixed/Float)

Bit Depth

Number of bits used for each sample

Extractor Input

With this source we can load an audio file to do offline analysis and extract the header information.
We can’t use the realtime analysis nodes to retrieve the results of this source (see Offline analysis nodes)

Initialization

doc_extractor_init_ctx

We need to uncompress the audio file before be able to do the analysis.

doc_extractor_init_bp
Init Extractor Audio
Initializes the extractor audio source instance. With this node the audio data will be uncompressed.

File Name

Path to the audio file

Only Header

The audio data will not be uncompressed, but we can access to the metadata info

Unload Extractor Audio
Free the memory for the uncompressed audio data, an analysis can’t be done after call this node.

Extractor Utils

doc_extractor_utils_ctx

Some extractor extra functions

doc_extractor_utils_bp
Get Audio Metadata
Returns metadata if the file has the info.

Filename

Filename of the audio file

Extension

Extension of the audio file

MetaType

ID3_V1 , ID3_V2.3 , ID3_V2.4

Title

Title of the song

Artist

Artist

Album

Album

Year

Year

Genre

Genre

Get Audio Metadata Art
Extract the AlbumArt pictures into the selected folder. The format of the file name will be {Prefix}{number_of_file}.{image_extension}. Prefix1.png, Prefix2.png,…

Prefix

Prefix used to name the pictures

Folder

Destination folder, must exists

Number of files

Number of stored pictures

Stream Input

With this input we can load raw audio data directly from other thirdparty sources.

Initialization

doc_stream_init_ctx
We need to initialize this object with the parameters needed to read the raw data
doc_stream_init_bp
Init Stream Audio
Initializes the stream audio source instance.

Num Channels

Number of channels

Sample Rate

Samples per second

Bit Depth

Bits per sample

Audio Format

Sample number format

PlayBack Buffer Size

Player buffer size (power of 2)

Enable Playback

Playback the recived audio

Unload Stream Audio
Closes the stream listener audio device
doc_stream_init_bp_device
Set Default Device Stream Audio
Sets default stream playback device.

Player Device Name

Friendly Name of audio device (GetOutputAudioDeviceNames output)

Get Default Device Stream Audio
Returns the player audio device

Stream controls

doc_stream_controls_ctx

After the initialization we can feed the object using raw audio data

doc_stream_controls_bp_feed
Feed Stream Capture
Sends the stream audio data to the analyzer.

Stream Data

TArray with the raw audio data

Close Stream Capture
Closes audio loopback capture and stops the playback
Is Stream Capturing
Returns player playing state
doc_stream_controls_bp_time
Get Stream Capture Time
Returns current loopback total capture time

Stream Utils

doc_stream_utils_ctx

Some extractor extra functions

doc_stream_utils_bp_info
Get Stream Devices Audio Info
Retrieves the stream playback audio device info.

Player Device Name

Name of the player device in the system devices list

Player Sample Rate

Number of samples per second

Player Num Channels

Number of channels

Player Buffer Samples

Number of samples of the playback buffer

Player Format

Format of the values that represent each sample (Fixed/Float)

Player Bit Depth

Number of bits used for each sample

OVR Input

This Object can be used instead of Capturer to capture the audio from a microphone of an Oculus device.

Initialization

doc_ovr_init_ctx
We can initialize this object with some parameters to customize the audio output
doc_ovr_init_bp
Init OVR Audio
Initializes the OVR audio source instance.

Sample Rate

Samples per second of the capture

Audio Format

Sample number format (fixed/float)

Capturer Buffer Seconds

Number of seconds of capture history

Player Buffer Size

Player buffer size (power of 2)

Unload OVER Audio
Closes the OVR audio source
doc_ovr_init_bp_device
Set Default Devices OVR Audio
Sets default OVR devices.

Capturer Device Name

Audio device name to capture the sound (From Get Input Audio Device node output)

Player Device Name

Audio device name to play the sound (From Get Output Audio Device node output)

Get Default Devices OVR Audio
Returns the default OVR device names (use after InitOVRAudio).

Capturer Device Name

Audio device name to capture the sound

Player Device Name

Audio device name to play the sound

OVR Controls

doc_ovr_controls_ctx

After initialization we can use this nodes to control the start or end of the capture

doc_ovr_controls_bp_start
Start OVR Capture
Starts OVR audio capture (and playback).

Playback Capturer

Enable to playback the capture sound at the same time

Stop OVR Capture
Stops OVR audio capture
Is OVR Capturing
Returns OVR Capture state
doc_ovr_controls_bp_time
Get OVR Capture Time
Returns total OVR capture time

OVR Utils

doc_ovr_utils_ctx

Some OVR extra functions

doc_ovr_utils_bp_info
Get OVR Devices Audio Info
Retrieves the capture/playback audio device info.

Capturer Device Name

Name of the capturer device in the system devices list

Capturer Sample Rate

Number of samples per second

Capturer Num Channels

Number of channels

Capturer Buffer Samples

Number of samples of the playback buffer

Capturer Format

Format of the values that represent each sample (Fixed/Float)

Capturer Bit Depth

Number of bits used for each sample

Player Device Name

Name of the player device in the system devices list

Player Sample Rate

Number of samples per second

Player Num Channels

Number of channels

Player Buffer Samples

Number of samples of the playback buffer

Player Format

Format of the values that represent each sample (Fixed/Float)

Player Bit Depth

Number of bits used for each sample

Manager Utils

doc_manager_utils_ctx

Some extra functions that can be used without an audio source

doc_manager_utils_bp_project
Get Folder Files
Returns a list with the name of the files of a folder and if it is a valid folder.

Folder path

Path to the folder

Extension

Extension filter

Get Project Folder
Returns current project root folder path
doc_manager_utils_bp_devices
Get Output Audio Devices
Returns a list with the name of available audio devices to play audio
Get Input Audio Devices
Returns a list with the name of available audio devices to capture audio

Input nodes flow

Once the audio input source has been initializated any control node can be used to modify the audio playback

doc_flow_graph_input