Kamaelia docs : Kamaelia.Audio.RawAudioMixer

Multi-source Raw Audio Mixer

A component that mixes raw audio data from an unknown number of sources, that can change at any time. Audio data from each source is buffered until a minimum threshold amount, before it is included in the mix. The mixing operation is a simple addition. Values are not scaled down.

Example Usage

Mixing up to 3 sources of audio (sometimes a source is active, sometimes it isn't):

Graphline(
    MIXER = RawAudioMixer( sample_rate=8000,
                           channels=1,
                           format="S16_LE",
                           readThreshold=1.0,
                           bufferingLimit=2.0,
                           readInterval=0.1),
                         ),
    A = pipeline( SometimesOn_RawAudioSource(), Entuple(prefix="A") ),
    B = pipeline( SometimesOn_RawAudioSource(), Entuple(prefix="B") ),
    C = pipeline( SometimesOn_RawAudioSource(), Entuple(prefix="C") ),

    OUTPUT = RawSoundOutput( sample_rate=8000,
                             channels=1
                             format="S16_LE",
                           ),
           linkages = {
               (A, "outbox") : (MIXER, "inbox"),
               (B, "outbox") : (MIXER, "inbox"),
               (C, "outbox") : (MIXER, "inbox"),

               (MIXER, "outbox") : (OUTPUT, "inbox"),
           },
         ).run()

Each source is buffered for 1 second before it is output. If more than 2 seconds of audio are buffered, then samples are dropped.

How does it work?

Send (id, raw-audio) tuples to RawAudioMixer's inbox. Where 'id' is any value that uniquely distinguishes each source of audio.

RawAudioMixer buffers each source of audio, and mixes them together additively, outputting the resulting stream of audio data.

Constructor arguments:

  • sample_rate, channels, format The format of audio to be mixed. The only format understood at the moment is "S16_LE"
  • readThreshold number of seconds of audio that will be buffered before RawAudioMixer starts mixing it into its output.
  • bufferingLimit maximum number of seconds of audio that will be buffered. If more piles up then some audio will be lost.
  • readInterval number of seconds between each time RawAudioMixer outputs a chunk of audio data.

RawAudioMixer buffers each source of audio separately. If the amount of audio in any buffer exceeds the 'buffering limit' then the oldest samples buffered will be lost.

When one or more buffered sources reaches the 'read threshold' then they are mixed together and output. How often audio is output is determined by setting the 'read Interval'.

Mixing is done additively and is not scaled down (ie. it is a sum() function, not an average() ). Therefore, ensure that the sum of the sources being mixed does not exceed the range of values that samples can take.

Why the buffering, thresholds, and read intervals? It is done this way so that RawAudioMixer can mix without needing to know what sources of audio there are, and whether they are running or stopped. It also enables RawAudioMixer to cope with audio data arriving from different sources at different times.

You may introduce new audio sources at any time - simply send audio data tagged with a new, unique identifier.

You may stop an audio source at any time too - simply stop sending audio data. The existing buffered data will be output, until there is not left.

If there is not enough audio in any of the buffers (or perhaps there are no sources of audio) then RawAudioMixer will not output anything, not even 'silence'.

If a shutdownMicroprocess or producerFinished message is received on this component's "control" inbox this component will cease reading in data from any audio sources. If it is currently outputting audio from any of its buffers, it will continue to do so until these are empty. The component will then forward on the shutdown message it was sent, out of its "signal" outbox and immediately terminate.

TODO:

  • Needs a timeout mechanism to discard very old data (otherwise this is effectively a memory leak!)
    • If an audio source sends less than the readThreshold amount of audio data, then stops; then this data never gets flushed out.

Test documentation

Tests passed:

  • Multiple sources of audio will be mixed
  • If there is no input (no incoming audio data), there will be no output (no outgoing audio data)
  • A single source of audio will pass through

Kamaelia.Audio.RawAudioMixer.RawAudioMixer

class RawAudioMixer(Axon.ThreadedComponent.threadedcomponent)

RawAudioMixer([sample_rate][,channels][,format][,readThreshold][,bufferingLimit][,readInterval]) -> new RawAudioMixer component.

Mixes raw audio data from an unknown number of sources, that can change at any time. Audio data from each source is buffered until a minimum threshold amount, before it is included in the mix. The mixing operation is a simple addition. Values are not scaled down.

Send (uniqueSourceIdentifier, audioData) tuples to the "inbox" inbox and mixed audio data will be sent out of the "outbox" outbox.

Keyword arguments:

  • sample_rate -- The sample rate of the audio in Hz (default=8000)
  • channels -- Number of channels in the audio (default=1)
  • format -- Sample format of the audio (default="S16_LE")
  • readThreshold -- Duration to buffer audio before it starts being used in seconds (default=1.0)
  • bufferingLimit -- Maximum buffer size for each audio source in seconds (default=2.0)
  • readInterval -- Time between each output chunk in seconds (default=0.1)

Methods defined here

Warning!

You should be using the inbox/outbox interface, not these methods (except construction). This documentation is designed as a roadmap as to their functionalilty for maintainers and new component developers.

Feedback

Got a problem with the documentation? Something unclear that could be clearer? Want to help improve it? Constructive criticism is very welcome - especially if you can suggest a better rewording!

Please leave you feedback here in reply to the documentation thread in the Kamaelia blog.

-- Automatic documentation generator, 05 Jun 2009 at 03:01:38 UTC/GMT


Kamaelia is an open source project originated from and guided by BBC Research. For more information browse the site or get in contact.

This is an ongoing community based development site. As a result the contents of this page is the opinions of the contributors of the pages involved not the organisations involved. Specificially, this page may contain personal views which are not the views of the BBC. (the site is powered by a wiki engine)

(C) Copyright 2008 Kamaelia Contributors, including the British Broadcasting Corporation, All Rights Reserved.

This web site is powered by the same code created for the bicker manor project. For more details, contact Michael Sparks at BBC Research directly (cf contact)