Modules Overview

This file provides an overview of the current Kamaelia files, what they are and where appropriate, how they work. This is designed to be a living document, but not designed as raw API documentation. (There are better ways of building API documentation) Essentially the purpose of this documentis to act as "the next layer up" the documentation layers - stating how and why the different parts are joined together.

• Base Modules
• Internet Adaption Modules
• Network Protocol Modules
• UDP Components
• Utility Modules

Base Modules

Kamaelia.
|-- __init__.py
|-- KamaeliaExceptions.py
|-- KamaeliaIPC.py

The purpose of __init__.py is as usual to allow the kamaelia modules to be imported in the traditional manner. It is empty however, so explicit naming of submodules is needed after import.

KamaeliaExceptions.py contains a number of exceptions covering different potential failure points in the modules. These exceptions generally inherit from AxonException.

KamaeliaIPC.py however defines some specific payloads for intercomponent communication. These are generally control messages used by various Internet Adaption components for signalling various events - normally relating to new or closed socket events. The IPC messages inherit from Axon.IPC, and are mainly notify events. Currently the only producerFinished class is socketShutdown - issued by Internet Abstraction Modules.

Internet Adaption Modules

Kamaelia.
|-- Internet.
| |-- __init__.py
| |-- socketConstants.py
| |-- TCPServer.py
| |-- TCPClient.py
| |-- ConnectedSocketAdapter.py
| |-- Selector.py
| |-- InternetConnection.py
| `-- InternetHandlingTests.py

These modules provide the basic primitives relating to network handling. Essentially the

purpose of each file defines a basic primitive in a normal network system.

As with the base Kamaelia package, __init__.py is empty and simply exists to allow package import as follows:

import Kamaelia.Internet

Clearly socketConstants.py contains a bunch of common constants - these aren't provided by the socket module and largely cover error conditions, and are defined to increase code readability.

The internet abstraction primitives can divided thus:

• Connection Factories
• TCPServer.py acts as a factory spawning new connections which are handled by their own components and will often have a protocol handler associated with this.
• TCPClient.py also acts as a factory, but only spawns one connected socket. It acts as a data passthrough proxy for the subcomponent.
Neither connection factory directly handles any data to/from a connected socket.
• Specific Socket Handling:
• ConnectedSocketAdapter.py manages any connected socket.
• Any data recieved on its DataSend inbox is sent to the socket in a non-blocking fashion, and any data recieved from the socket in a non-blocking fashion, is sent to the component outbox outbox.
• It only attempts to get data from the socket when it recieves an appropriate message on its dataReady inbox.
• When it recieves a producerFinished message on its control port after having sent all outstanding dataSend messages, it goes into a shutdown mode. This results in shutdown messages being sent to the FactoryFeedback and signal outboxes. This is due to the ownership of the connected sockets being elsewhere.
This simplicity allows the to be used ConnectedSocketAdapter by both the connection factories.
• Multiple socket handling.
• Selector.py is designed for dealing with checking to see if any network connections are active, and which ConnectedSocketAdapters need to be told they can/should attempt to recieve data. When it detects a server socket has a connection ready to be accepted it simply sends a message to a dynamically allocated outbox to send the appropriate TCPServer factory a message to tell it to accept the connection.
In practice server sockets and data sockets are treated pretty much the same way - the difference of interpretation is handled by the socket adaptor or the factory. Whilst it hasn't been written this allows the "select" based implementation of Selector to be replaced by a polling approach without many code changes. (Indeed this could easily be done on the fly)
• Other code:
• InternetConnection.py is legacy code and due to be ditched
• InternetHandlingTests.py is the current test code being used to drive integration between client/server/selector service. This is a relatively complex example:
• It creates a SimpleServer running and EchoProtocol, and a client running an EchoCheckerProtocolComponent, and a TCPClient. The EchoCheckerProtocolComponent is wired in to handle the application communications. The client connects to the server making a complete loop.

Network Protocol Modules

Kamaelia.
|-- Protocol.
| |-- __init__.py
| |-- AudioCookieProtocol.py
| |-- EchoProtocolComponent.py
| |-- FortuneCookieProtocol.py
| |-- HTTPServer.py
| |-- SimpleVideoCookieServer.py

This directory contains a collection of modules, each implementing a simple of not so simple network protocol. Some of this are designed as toy protocols and were used during the design of the system to flesh out key areas. As before, __init__.py is simply an empty file.

The only protocol here with a potential to be non-trivial is the HTTPServer.py protocol. The implementation inside that component at this point in time is deliberately very simplistic, and only implements a very simple very small subset of HTTP/1.0. When run standalone it runs a simple webserver on port 8082, sending random files to the client. The server code isn't actually complete. Also note that this is an extremely insecure web server as a result of these two facts.

EchoProtocolComponent.py is the simplest protocol implemented, and was unsurprisingly the first one to be implemented - since it forms a simple loopback protocol useful for integration testing. In practice this component is really a data pass through component since all data recieved on its inbox is passed on directly to its outbox. As a result this component's implementation may actually migrate out to Kamaelia.Util as a simple buffer component and allow this actual protocol component to simply inherit from Kamaelia.Util.

FortuneCookieProtocol.py is one step above EchoProtocolComponent.py. It runs the fortune program on the server and sends the result to its outbox. In a networked environment what this means is the server will simply blat the fortune cookie to a client that connects to the server.

AudioCookieProtocol.py is a logical extension to FortuneCookieProtocol.py. Similar to FortuneCookieProtocol.py it too runs an external program and sends the result to its outbox. In this case however rather than a short piece of text, it is a short snippet of audio.

SimpleVideoCookieServer.py does essentially the same as FortuneCookieProtocol.py and AudioCookieProtocol.py, but this time with a short piece of video.

Utility Modules

Kamaelia.
|-- Util.
| |-- __init__.py
| |-- ToStringComponent.py
| `-- ConsoleEcho.py

This is a small collection of utility components. They're designed as simple filters for various uses - some perform simple, but useful transformations that convert data into a format suitable for other components, others are designed to be dropped in the middle of existing linkages to allow debugging and/or output.

__init__.py is as usual an empty file for the usual reasons.

ConsoleEcho.py is designed to allow the system to take any data given on its inbox and send it to stdout. It can also optionally forward the data recieved to it's outbox meaning that it can be dropped in the middle of a linkage to enable precise debugging when you wish to know what communications is taking place.

ToStringComponent.py is a simple filter component - it takes every object supplied via its inbox, calls str() on the object, and forwards the result to its outbox. This allows, for example, a user to take the output of any component, convert it to a string suitable for output to a console. This doesn't require the console component to do the conversion (since someone may wish to send the objects to a console as an easy way to check types), and doesn't require components to provide special behaviour.

Multimedia Modules

Kamaelia.
|-- AOPlayer.py deprecated
|-- vorbisDecodeComponent.py
`-- oggFilePlayComponent.py deprecated

These three files are all related to the same purpose - playback of audio, and also decoding ogg vorbis. There are some restrictions on some of these, and there is a recommendation on which ones to use. Each of these requires a set of libraries - these are covered below.

AOPlayer.py is deprecated, the actual component has moved location, and you really ought not to use this at the moment. This may change at some point in time.

• Requires libao, and the pyao python bindings.

oggFilePlayComponent.py is the oldest piece of code that can be used for decode and playback. It uses the official xiph python bindings to decode ogg vorbis. On the surface this is useful and if all you want to do is decode audio from a file, this does work.

• There is a key restriction - it MUST come from a file object that you have opened. It can't be something that looks like a file, it must be a file. The reason for this is internally the xiph bindings dive into the file object supplied, pull out the C filehandle and pass that over to libvorbisfile.
This imposes some awkward restrictions on your code. This does also mean that the operation of your codebecomes subserviant to the library, rather than the library subserviant to your code.
• Requires the following libraries: libao, libogg, libvorbis, libvorbisfile
• Requires the following python bindings: pyogg, pyvorbis, pyao

This file contains the following components:

• oggPlayer - legacy component - you used to be able to choose between AOPlayer and
• PlayerComponent - This is an abstract class that provides the main functionality of a player. Is is overridden by client subclasses. Subclasses are only expect to override the write method. The key method in the payer component is a play generator. Nastily, this generator calls VorbisFile to enable file reading, unpacking of ogg and decode.
Current concrete subclasses:
• AOPlayer - This simply provides a write method to send audio to a libao audio device.

It's worth noting that the execution here is pretty ugly, and it was the motivation for writing the next file. However, if all you want is to use libvorbisfile, oggFilePlayComponent.py can be useful.

vorbisDecodeComponent.py was written to overcome the limitations metioned above. Specifically this provides 2 components - one that will accept ogg vorbis data on an inbox and provide back raw decoded audio on its outbox, and another that takes audio on its inbox and plays the audio. This provides much cleaner separation. The testing spike included shows how to use a file reading component in conjuction with these two components to provide a file reading/decode chain.

• The two components provided are:
• VorbisDecode - takes ogg vorbis encoded audio from its inbox and provides raw decoded audio on its outbox.
  Important note: This is currently ogg vorbis - as you would read from a file, NOT just vorbis encoded data.
• AOAudioPlaybackAdaptor - takes raw audio recieved on its inbox and sends it to the audio device. Currently this may well be a blocking operation. How to do this in a non-blocking manner is under investigation. If it recieves a producerFinished message on its control inbox, this component exits.

These components rely on the following libraries being installed:

• libogg, libao, libvorbis, libvorbissimple

The python bindings required are:

• The python bindings supplied with libvorbissimple
• Pyrex is therefore also required.

RTP Related Modules

Kamaelia.
|-- bitfieldrec.py
|-- NullPayloadPreFramer.py
|-- NullPayloadRTP.py
|-- RTCPHeader.py
|-- RTPHeader.py
|-- RtpPacker.py

Bla bLa bla

• bitfieldrec.py - Module containing utility functions to allow structures with arbitrary structures for bitstreams/bitfields. Currently only supports packing. Does not support unpacking (yet).Bit Field Record Support. Usage:
  1. subclass bfrec
  2. Define a class var "fields"
  3. The value for this field should be a list of "field"s, created by calling the static method field.mkList. This takes a list of tuples, one tuple per field. (fieldname, bitwidth, None or list)
• class field(str)
• staticmethod mkList(fieldDefs)
• fieldDefs = seq of (name, size,extra)
• class bfrec(AxonObject)
• __init__(self, **args):
• structureSize(self)
• pack(self)
• serialiseable(convert,aField) - Returns an iterable collection of values. (eg list) Either an existing one, or puts scalar/singleton values into a list. Doing this removes a special case.
• Algorithm:
• This actually does need documenting properly. The algorithm was designed using test first approaches, and so is pretty solid, but documenting it would be a good idea. If only because it's actually quite nice :)
• NullPayloadPreFramer.py - Not written yet

Null Payload RTP Classes

Null Payload Pre-Framer

Null Payload RTP Packet Stuffer - Same thing.

This Null payload also assumes constant bit rate load.

Subcomponents functionality:

FileControl: - Only if RFA internal - isn't

FileReader - only if internal - isn't

FileSelector - only if internal - isn't

Format Decoding

DataFramaing

Command Interpreter (Likely to be component core code)

-----------------------------

a NullPayloadRTP.pysdsd

a RTCPHeader.pyadf

a RTPHeader.pysdsd

a RtpPacker.py adf

MIME & Disk Related modules

Kamaelia.
|-- ReadFileAdaptor.py
|-- MimeRequestComponent.py
|-- SampleMIMERequest.txt
|-- SimpleServerComponent.py
|-- requestLine.py

adf

Testing Modules

Kamaelia.
|-- runTests.sh
|-- FortuneCookie_InternetHandlingTests.py
|-- test
| |-- SynchronousLinks_SystemTest.py
| |-- debug.conf
| |-- test.log
| |-- test_ConsoleEcho.py
| |-- test_InternetConnection.py
| |-- test_MimeRequestComponent.py
| |-- test_NullPayloadRTP.py
| |-- test_RtpPacker.py
| |-- test_Selector.py
| `-- test_requestLine.py

adf

Experimentation Modules

|-- Sketch
| |-- AO_TEST.py
| |-- DONE.list
| |-- DataSteps.txt
| |-- EchoCheckerProtocolComponent.py
| |-- Forking.py
| |-- PygameTest.py
| |-- TODO.list
| |-- oggFilePlayComponent-refactor.py
| |-- oggPlayComponent.py
| |-- rgrailspuds.txt
| |-- run.log
| |-- simple_decoder.py
| `-- test.py

d

Support Files

|-- Makefile
|-- ComponentClasses.py
|-- SampleTemplateComponent.py
|-- debug.conf
|-- makeDocs.sh
|-- afortune.pl
|-- Support/ - various support files. Wav Audio comes from the OpenOffice.org suite.
|-- doc/ - Autogenerated documentation
|-- docs/ - More autogenerated docs