Kamaelia - Concurrency made useful, fun


In Kamaelia you build systems from simple components that talk to each other. This speeds development, massively aids maintenance and also means you build naturally concurrent software. It's intended to be accessible by any developer, including novices. It also makes it fun :)

What sort of systems? Network servers, clients, desktop applications, pygame based games, transcode systems and pipelines, digital TV systems, spam eradicators, teaching tools, and a fair amount more :)


News (Jul 2010): Kamaelia's License has changed to the Apache Software License 2
Read more

Use

Kamaelia based systems

 

Kamaelia Grey

Spam reduction through greylisting. Kamaelia Grey is a SMTP proxy for your inbound email, rejecting email likely to be spam. Mail likely to not be spam is forwarded to your normal mail server.

Batch Transcoder

This watches directories for new images and videos to transcode to formats suitable for the web. You can think of it as the backend needed for a youtube/flickr type site. PDF support is likely to be added soon.

Whiteboard

A collaborative whiteboard. 2 or more machines share a display you can write on - either can be a server. Whiteboards are paginated, and therefore can also be used for remote presentations. Really cool with a tablet.

ER Modeller

Built to allow modelling a database. You describe the entities and their attributes, and relationships. Also allows entities to inherit from each other. Uses a customised topology visualiser to show you your design.

Macro

Records and transcodes digital TV for later viewing (ie timeshifting). It can do this for as many channels at once as you have CPU power and tuners available for it.

Kamaelia Documentation

Build & improve Kamaelia based systems

 

Documentation

Documentation is a core asset in Kamaelia. It is continuously being extended and improved.

Tutorials

Cookbook

This section contains a number of recipes showing how to build a variety of different types of Kamaelia systems. Sections include:

  • Linking components together
  • Building Network systems, clients, servers, IM, etc
  • Building Digital TV systems
  • Building Multimedia systems
  • Using Kamaelia components & subsystems in non-kamaelia systems.

More recipes always welcome.

Component Reference

Full component reference. This is automatically generated nightly, and where tests exist, the output of the tests is parsed and included on these pages.

Axon Reference

Full reference for Axon - the core of Kamaelia. This is also automatically generated nightly and also includes test output.

Community

Kamaelia is open source, help us improve it!

 

Developer Central

Interested in helping out? You're more than welcome! In this area you'll find some pages which cover some areas of interest for ongoing dev, our general development process, project management process, guidelines on contributing (eg smart questions, through to code, and contributor agreements)

Get Help (Contact)

Sometimes the best contributions happen when someone asks the right question. Please don't be afraid to ask (or answer!) questions. We'd love to help you, and merely ask that you help others in return. Not much of a community otherwise? :) (OK, not everyone has time to do that) Find us on email, google groups, irc, etc

Summer of Code

We've been involved with GSOC now for 3 years, and it's been great. One of the less obvious things about GSOC is that it's generated a wealth of docs and ideas which can be dived into here.

Source & SVN

Kamaelia's source is hosted on code.google, and the latest release is the 1.0.12.0 release. (Y.Y.M.r). Some apps are also available in the release directory.

Wiki

This entire site is powered by, essentially, a wiki engine, if you're interested in contributing please get in contact and drop us a line. (It doesn't default to open to avoid wikispam, etc) ie: Think this site could do with improving? Please help! :)

 

A quick overview

Kamaelia is a Python library by BBC Research for concurrent programming using a simple pattern of components that send and receive data from each other. The following is an example of a system made by piping the output of one component into another:

from Kamaelia.Chassis.Pipeline import Pipeline
from Kamaelia.Util.Console import ConsoleReader, ConsoleEchoer

Pipeline(
         ConsoleReader(),
         ConsoleEchoer(),
).run()

Or maybe you want to build a presentation tool? (imports & setup excluded here - full example)

Graphline(
     CHOOSER = Chooser(items = files),
     IMAGE = Image(size=(800,600), position=(8,48)),
     NEXT = Button(caption="Next", msg="NEXT", position=(72,8)),
     PREVIOUS = Button(caption="Previous", msg="PREV",position=(8,8)),
     FIRST = Button(caption="First", msg="FIRST",position=(256,8)),
     LAST = Button(caption="Last", msg="LAST",position=(320,8)),
     linkages = {
        ("NEXT","outbox") : ("CHOOSER","inbox"),
        ("PREVIOUS","outbox") : ("CHOOSER","inbox"),
        ("FIRST","outbox") : ("CHOOSER","inbox"),
        ("LAST","outbox") : ("CHOOSER","inbox"),
        ("CHOOSER","outbox") : ("IMAGE","inbox"),
     }
).run()

That's all well and good, but how is a component written? What's inside it?

from Axon.Component import component
from Axon.Ipc import shutdownMicroprocess, producerFinished

class MyComponent(component):    
    Inboxes = {"inbox"        : "some data in",
               "control"      : "stops the component"}
    Outboxes = {"outbox"      : "some data out",
                "signal"      : "Shutdown signal"}

    def __init__(self, **argd):
        super(MyComponent, self).__init__(**argd)

    def main(self):
        while not self.doShutdown():
            if self.dataReady("inbox"):
                data = self.recv("inbox")
                # let's echo what we received...
                self.send(data, 'outbox')
                
            if not self.anyReady():
                self.pause()

            yield 1

    def doShutdown(self):
        if self.dataReady("control"):
            mes = self.recv("control")                
            if isinstance(mes, shutdownMicroprocess) or isinstance(mes, producerFinished):
                self.send(producerFinished(), "signal")
                return True
        return False

This is the simplest form a component can take. A component:

  • is a class that inherits from Axon.Component.component
  • has inboxes and outboxes

By inheriting from Axon.Component.component you make your class usable by the Axon library which is at the core of the Kamaelia library. It allows for your class to be used with other components.

Inboxes and outboxes allow your component to be linked to and from by other components.

Then your class defines a main method that simple loop until a specific kind of message is put into the "control" inbox of the component. During the looping it checks for any inboxes and process data read from them. Eventually it yields to the Axon scheduler that goes to the next available component. By using a generator we allow the shceduler to come back to the component's loop eventually.

Note that inboxes and outboxes are pure Python dictionary hence they allow for any Python objects and are not limited to strings. The component described above is simple, complex components have many inboxes and outboxes to link to and from.

Kamaelia

Kamaelia is a library of components for all kind of tasks and topics:

For example taking the previous example we could write:

from Kamaelia.Chassis.Pipeline import Pipeline
from Kamaelia.Util.Console import ConsoleReader, ConsoleEchoer

Pipeline(
         ConsoleReader(),
         MyComponent(),
         ConsoleEchoer(),
).run()

Pipeline is component that automatically links outboxes to inboxes of each provided component. The console components allow for reading and writing data from and to the command line. Because Pipeline is also a component itself it could in turns be used in another component.

Note that calling the run() method on a component blocks the process until it is killed. You can also simply activate a component which will then be in an active state but will run only when eventually run is called on another component.

Now that you have the basics of Kamaelia you should dive into the documentation and have fun with this library.

...
 

Kamaelia is an open source project originated from and guided by BBC Research. For more information browse the site or get in contact.

This is an ongoing community based development site. As a result the contents of this page is the opinions of the contributors of the pages involved not the organisations involved. Specificially, this page may contain personal views which are not the views of the BBC. (the site is powered by a wiki engine)

(C) Copyright 2008 Kamaelia Contributors, including the British Broadcasting Corporation, All Rights Reserved