Realtime Audio Mixer for Python

Warning

This is work in progress!

Goal: Reliable low-latency audio playback and recording with Python, using PortAudio via the sounddevice module.

The audio callback is implemented in C (and compiled with the help of CFFI) and doesn’t invoke the Python interpreter, therefore avoiding waiting for things like garbage collection and the GIL.

All PortAudio platforms and host APIs are supported. Runs on any Python version where CFFI is available.

Features:

• playback of multiple signals at the same time (that’s why it’s called “mixer”)
• play from buffer, play from ringbuffer
• record into buffer, record into ringbuffer
• multichannel support
• NumPy arrays with data type 'float32' can be easily used (via the buffer protocol) as long as they are C-contiguous
• fixed latency playback, (close to) no jitter (optional)
  • to be verified …
• sample-accurate playback/recording (with known offset)
  • to be verified …
• non-blocking callback function, using PortAudio ringbuffers
• all memory allocations/deallocations happen outside the audio callback

Planned features:

• meticulous reporting of overruns/underruns
• loopback tests to verify correct operation and accurate latency values
• fade in/out?
• loop?
• playlist/queue?

Out of scope:

• reading from/writing to files (use e.g. the soundfile module)
• realtime signal processing (inside the audio callback)
• signal generators
• multiple mixer instances (some PortAudio host APIs only support one stream at a time)
• resampling (apart from what PortAudio does)
• fast forward/rewind
• panning/balance
• audio/video synchronization

Somewhat similar projects:

• https://github.com/nwhitehead/swmixer
• https://github.com/nvahalik/PyAudioMixer
• http://www.pygame.org/docs/ref/mixer.html

Installation

On Windows, macOS, and Linux you can install a precompiled wheel with:

python3 -m pip install rtmixer

This will install rtmixer and its dependencies, including sounddevice.

Note

On Linux, to use sounddevice and rtmixer you will need to have PortAudio installed, e.g. via sudo apt install libportaudio2. On other platforms, PortAudio comes bundled with sounddevice.

Developers can install in editable mode with some variant of:

git clone https://github.com/spatialaudio/python-rtmixer
cd python-rtmixer
git submodule update --init
python3 -m pip install -e .

Usage

See the list of examples on GitHub.

API Documentation

Reliable low-latency audio playback and recording.

http://python-rtmixer.readthedocs.io/

class rtmixer.Mixer(**kwargs)

PortAudio output stream for realtime mixing.

Takes the same keyword arguments as sounddevice.OutputStream, except callback (a callback function implemented in C is used internally) and dtype (which is always 'float32').

Uses default values from sounddevice.default (except dtype, which is always 'float32').

Has the same methods and attributes as sounddevice.OutputStream (except write() and write_available), plus the following:

actions

The set of active “actions”.

cancel(action, time=0, allow_belated=True)

Initiate stopping a running action.

This creates another action that is sent to the callback in order to stop the given action.

This function typically returns before the action is actually stopped. Use wait() (on either one of the two actions) to wait until it’s done.

fetch_and_reset_stats(time=0, allow_belated=True)

Fetch and reset over-/underflow statistics of the stream.

play_buffer(buffer, channels, start=0, allow_belated=True)

Send a buffer to the callback to be played back.

After that, the buffer must not be written to anymore.

play_ringbuffer(ringbuffer, channels=None, start=0, allow_belated=True)

Send a RingBuffer to the callback to be played back.

By default, the number of channels is obtained from the ring buffer’s elementsize.

stats

Get over-/underflow statistics from an inactive stream.

To get statistics from an active stream, use fetch_and_reset_stats().

wait(action, sleeptime=10)

Wait for action to be finished.

Between repeatedly checking if the action is finished, this waits for sleeptime milliseconds.

class rtmixer.Recorder(**kwargs)

PortAudio input stream for realtime recording.

Takes the same keyword arguments as sounddevice.InputStream, except callback (a callback function implemented in C is used internally) and dtype (which is always 'float32').

Uses default values from sounddevice.default (except dtype, which is always 'float32').

Has the same methods and attributes as Mixer, except that play_buffer() and play_ringbuffer() are replaced by:

record_buffer(buffer, channels, start=0, allow_belated=True)

Send a buffer to the callback to be recorded into.

record_ringbuffer(ringbuffer, channels=None, start=0, allow_belated=True)

Send a RingBuffer to the callback to be recorded into.

By default, the number of channels is obtained from the ring buffer’s elementsize.

class rtmixer.MixerAndRecorder(**kwargs)

PortAudio stream for realtime mixing and recording.

Takes the same keyword arguments as sounddevice.Stream, except callback (a callback function implemented in C is used internally) and dtype (which is always 'float32').

Uses default values from sounddevice.default (except dtype, which is always 'float32').

Inherits all methods and attributes from Mixer and Recorder.

class rtmixer.RingBuffer(elementsize, size=None, buffer=None)

PortAudio’s single-reader single-writer lock-free ring buffer.

C API documentation:

http://portaudio.com/docs/v19-doxydocs-dev/pa__ringbuffer_8h.html

Python wrapper:

https://github.com/spatialaudio/python-pa-ringbuffer

Instances of this class can be used to transport data between Python code and some compiled code running on a different thread.

This only works when there is a single reader and a single writer (i.e. one thread or callback writes to the ring buffer, another thread or callback reads from it).

This ring buffer is not appropriate for passing data from one Python thread to another Python thread. For this, the queue.Queue class from the standard library can be used.

Parameters:

• elementsize (int) – The size of a single data element in bytes.
• size (int) – The number of elements in the buffer (must be a power of 2). Can be omitted if a pre-allocated buffer is passed.
• buffer (buffer) – optional pre-allocated buffer to use with RingBuffer. Note that if you pass a read-only buffer object, you still get a writable RingBuffer; it is your responsibility not to write there if the original buffer doesn’t expect you to.

advance_read_index(size)

Advance the read index to the next location to be read.

Parameters:size (int) – The number of elements to advance. Returns:The new position. Return type:int

Note

This is only needed when using get_read_buffers(), the methods read() and readinto() take care of this by themselves!

advance_write_index(size)

Advance the write index to the next location to be written.

Parameters:size (int) – The number of elements to advance. Returns:The new position. Return type:int

Note

This is only needed when using get_write_buffers(), the method write() takes care of this by itself!

elementsize

Element size in bytes.

flush()

Reset buffer to empty.

Should only be called when buffer is not being read or written.

get_read_buffers(size)

Get buffer(s) from which we can read data.

When done reading, use advance_read_index() to make the memory available for writing again.

Parameters:size (int) – The number of elements desired. Returns:

• The number of elements available for reading (which might be less than the requested size).
• The first buffer.
• The second buffer.

Return type:(int, buffer, buffer)

get_write_buffers(size)

Get buffer(s) to which we can write data.

When done writing, use advance_write_index() to make the written data available for reading.

Parameters:size (int) – The number of elements desired. Returns:

• The room available to be written or the given size, whichever is smaller.
• The first buffer.
• The second buffer.

Return type:(int, buffer, buffer)

read(size=-1)

Read data from the ring buffer into a new buffer.

This advances the read index after reading; calling advance_read_index() is not necessary.

Parameters:size (int, optional) – The number of elements to be read. If not specified, all available elements are read. Returns:A new buffer containing the read data. Its size may be less than the requested size. Return type:buffer

read_available

Number of elements available in the ring buffer for reading.

readinto(data)

Read data from the ring buffer into a user-provided buffer.

This advances the read index after reading; calling advance_read_index() is not necessary.

Parameters:data (CData pointer or buffer) – The memory where the data should be stored. Returns:The number of elements read, which may be less than the size of data. Return type:int

write(data, size=-1)

Write data to the ring buffer.

This advances the write index after writing; calling advance_write_index() is not necessary.

Parameters:

• data (CData pointer or buffer or bytes) – Data to write to the buffer.
• size (int, optional) – The number of elements to be written.

Returns:

The number of elements written.

Return type:

int

write_available

Number of elements available in the ring buffer for writing.

Index

Index