Kamaelia 0.6.0 Release Announcement

With great pleasure I'd like to announce the release of Kamaelia 0.6.0 and Axon 1.6.0

Release overview

For the short of time:

Overview: library/framework for concurrency using message passing components as the concurrency metaphor. Consists of a kernel (Axon) and collection of components (Kamaelia). Support for generator, thread & process based components. Targetted towards supporting maintenance, so /accessible/ to novices, but general purpose. Uses concurrency to make life simpler, not harder. Designed as a practical toolkit.

Download:

http://www.kamaelia.org/GetKamaelia.html
http://www.kamaelia.org/release/Kamaelia-0.6.0.tar.gz
sudo easy_install Kamaelia  

Change Summary:

Major update, multicore, STM, easy concurrency, creation of Kamaelia.Apps namespace for reuse of applications, significant amounts of new functionality, major documentation improvements (including full offline reference docs), support for using Kamaelia components cleanly in non-Kamaelia apps. (ie a clean linear -> concurrent interface (Handle))

Release notes: http://www.kamaelia.org/ReleaseNotes060
Deltas: Kamaelia 0.5.0 -> 0.6.0, Axon 1.5 -> Axon 1.6.0
Last full release: October 2006

Mailing list: http://groups.google.com/group/kamaelia *CHANGED*
New website: http://www.kamaelia.org/Home.html
Reference: http://www.kamaelia.org/Components.html
Cookbook: http://www.kamaelia.org/Cookbook.html

Detailed Version...

What is Kamaelia?

Kamaelia is a library/framework for building systems from simple components that talk to each other. This is primarily targetted at aiding maintenance, but also assists in the creation of systems in the first place. It also means you build naturally concurrent software. It's intended to be powerful, but also accessible by any developer, including novices.

We also find it makes it fun to build and work with concurrent systems.

What sort of systems? Network servers, clients, desktop applications, pygame based games, transcode systems and pipelines, digital TV systems, spam eradicators, teaching tools, and a fair bit more :)

Whilst it comes out of a research project at BBC Research, it is designed as a /practical/ toolkit. This clearly affects the set of components we've created.

In order to do this, Kamaelia is divided into two main namespaces:

Axon - this provides the core component & concurrency framework.  You use to build components which communicate with one another.

Kamaelia - this is the collection of components that exist. The vast majority of these come from systems created for a variety of purposes.
As of this release, a second major carve up of name spaces has been added:

• Kamaelia.Apps - this is where components from some Kamaelia based applications reside. The purpose behind this is to provide an experimental staging ground for new components to migrate into the main Kamaelia namespace. This also means you can use components from other Kamaelia applications sooner rather than later. As a result, these components may be lacking in two main areas - documentation or generality, but putting them here allows for components to migrate to a more generally useful state.

• Kamaelia.{anything else} - this is where components will migrate to when we are happy with the fact they are sufficiently general and useful outside their original application.
  It's worth noting that the bulk of components are in this second category!
  

What's New & Changed?

Kamaelia 0.6.0 represent a update over the 0.5.0 release, and should mark the return to regular releases. (Work has been continuing constantly since the 0.5.0 release, but numbers of releases slowed)

Major to changes reflected in both Axon & Kamaelia:

New home/website :-)

• http://www.kamaelia.org/Home.html

New getting started page:

• http://www.kamaelia.org/GetKamaelia.html

Support for easy_install ...

• sudo easy_install Kamaelia

... but with caveats that you don't get the docs, tools, or examples that way...

Large scale documentation improvements

Results of nightly documentation generation now included in the tar ball.

Core autogenerated docs:

• http://www.kamaelia.org/Docs/Axon/Axon.html

• http://www.kamaelia.org/Components.html

Key changes to Axon - Kamaelia's core:

Bumped to version 1.6.0

Support for simplified software transactional memory

• http://www.kamaelia.org/STM.html

• If you've never heard/understood the term, think (a bit like) "version control for variables"

• Useful if you MUST share data between components.
  
  

Experimental multicore support

• http://www.kamaelia.org/MulticoreExample.html

• Largely boils down to put "Process" in front of "Pipeline" to make all the subcomponents of the pipeline run in seperate processes 

• Practical benefit for pygame components - it allows multiwindow pygame apps.
  
  

Inheritable default values for component initialisers.

• The core aim of this is to allow declarative config for systems rather than something less clear.

•  This allows you to turn this sort of code:

• def ReusableSocketAddrServer(port=100,
                               protocol=EchoProtocol):
      return ServerCore(protocol=protocol,
                        port=port,
                        socketOptions=(socket.SOL_SOCKET,
                                       socket.SO_REUSEADDR, 1))

• Into this:

• class ReusableSocketAttrServer(ServerCore):
      socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
  

• Full discussion here:

• 
  
  

Added in "Handle" support. This provides two key pieces of functionality:

• The ability to run Kamaelia systems in the background, via:

• from Axon.background import background
  background().start()

• The ability to wrap Kamaelia components or systems in a Handle for use in non-Kamaelia systems.

• from Axon.Handle import Handle
  from Kamaelia.Internet.TCPClient import TCPClient
  conn = Handler(TCPClient("www.kamaelia.org", 80)).activate()
  conn.put("GET / HTTP/1.0\r\n", "inbox")
  conn.put("Host: www.kamaelia.org\r\n", "inbox")
  conn.put("\r\n", "inbox")

• More detail:

• http://www.kamaelia.org/AxonHandle
  
  

Simplified system shutdown. If you want to close down an entire Kamaelia based system rapidly from inside a component, just do this:

• self.scheduler.stop()

• This puts the scheduler into a "shutting down mode" in which:

• It calls the .stop() method on all the things it's running

• It then shuts down.

• The Kamaelia.Internet components take this as an opportunity to close all the connections they have open cleanly for example.

• Example usage: http://tinyurl.com/AxonShutdown
  
  

Support for WaitComplete extended, allowing better handling of more complex protocols which are not stateless in a debuggable fashion. It also simplifies working with Pygame, etc.

• http://www.kamaelia.org/WaitComplete
  
  

As well as unpausing a component when a message is delivered tothe component, it gets unpaused when a message is taken from its outbox. This allows better synchronous behaviour for rate limited in/out-boxes.

Overview of Changes to Kamaelia itself

Key changes to Kamaelia itself:

• Creation of the Kamaelia.Apps namespace
• Shifting of the core code for Kamaelia tools into Kamaelia.Apps
• Significant numbers of new components
• Significant number of bugfixes
• SimpleServer code changed to "ServerCore", representing it's more general structure.
• ... and a lot more besides... :)

In this release there is a slew of extra components and bug fixes, a variety of new tools - from video shot change detection, through to SMTP greylisting, but also perhaps the biggest extra: Multiprocess & hence multicore support (experimental at this stage, but so far so good :) )

OK, here's the dive into the summary of changes. (Full release notes here)

New files in Kamaelia.*

This represents significant amounts of new components and new abilities added into Kamaelia.

Kamaelia/

• Chassis/Seq.py

• Codec/

• WAV.py YUV4MPEG.py

• Device/DVB/

• SoftDemux.py

• Parse/

• { lots of components for working with DVB information tables }

• Experimental/

• Chassis.py ERParsing.py

• File/

• MaxSpeedFileReader.py UnixProcess2.py

• Internet/

• TimeOutCSA.py UDP_ng.py

• Protocol/

• MimeRequestComponent.py RecoverOrder.py SDP.py

• SimpleVideoCookieServer.py (demo/example)
  

• AIM/

• AIMHarness.py ChatManager.py LoginHandler.py OSCARClient.py

• HTTP/

• HTTPRequestHandler.py
• Handlers/

• Minimal.py SessionExample.py UploadTorrents.py

• IRC/IRCClient.py          
  

• RTP/

• NullPayloadPreFramer.py NullPayloadRTP.py RTCPHeader.py RTPHeader.py RtpPacker.py RTP.py

• UI/

• Pygame/

• Text.py VideoSurface.py

• Util/

• Collate.py FirstOnly.py Max.py OneShot.py PromptedTurnstile.py RangeFilter.py RateChunker.py SequentialTransformer.py Sync.py TagWithSequenceNumber.py TwoWaySplitter.py

• Tokenisation/

•             Simple.py

• Video/

• PixFormatConversion.py DetectShotChanges.py CropAndScale.py

• Visualisation/ER/

• ERLaws.py ERVisualiserServer.py ExtraWindowFurniture.py PAttribute.py PEntity.py PISA.py PRelation.py

• XML/SimpleXMLParser.py

• Support/

• OscarUtil2.py OscarUtil.py DVB/DateTime.py Protocol/IRC.py

New files in Kamaelia.Apps

Functionality in here represents code that can be standardised for use in other apps if there's a desire to do so. Many of these components are reuseable in their current form, though may have limitations.

Kamaelia/Apps/

• Compose/

• BuildViewer.pyCodeGen.py GUI.py PipeBuild.py PipelineWriter.py

• GUI/

• ArgumentsPanel.py BuilderControlsGUI.py TextOutputGUI.py

• Games4Kids/

• BasicSprite.py MyGamesEventsComponent.py SpriteScheduler.py

• Grey/

• ConcreteMailHandler.py GreyListingPolicy.py MailHandler.py PeriodicWakeup.py Support.py WakeableIntrospector.py

• Whiteboard/

• Audio.py Canvas.py CheckpointSequencer.py CommandConsole.py Entuple.py Options.py Painter.py Palette.py Router.py Routers.py SingleShot.py TagFiltering.py Tokenisation.py TwoWaySplitter.py UI.py

• IRCLogger/Support.py

• Show/GraphSlides.py

Deleted files in Kamaelia.*

A number of files which were deprecated in the last release have been deleted from this release. (See full release notes for details)

Changed files in Kamaelia.*

Largely small improvements, changes to meta data about components, often major documentation improvements - see full release notes for details. Occasional bugfixes. Largest overall change to existing files is improvement of documentation, REsT fixes, and addition of metadata to files.

Kamaelia/

• Audio/

• Filtering.py RawAudioMixer.py

• Codec/PyMedia/

• Decoder.py Encoder.py

• PyMedia/

• Input.py Output.py Resample.py

• Chassis/

• Carousel.py ConnectedServer.py Graphline.py Pipeline.py Prefab.py

• Codec/

•         Dirac.py RawYUVFramer.py Speex.py

• Device/

• DVB/

• Core.py DemuxerService.py EIT.py NowNext.py PSITables.py Receiver.py Tuner.py

• File/

• BetterReading.py ReadFileAdaptor.py Reading.py UnixProcess.py

• Internet/

• ConnectedSocketAdapter.py Selector.py SingleServer.py TCPClient.py TCPServer.py UDP.py

• Protocol/

• EchoProtocol.py FortuneCookieProtocol.py Framing.py SimpleReliableMulticast.py

• HTTP/

• ErrorPages.py HTTPClient.py HTTPHelpers.py HTTPParser.py HTTPResourceGlue.py HTTPServer.py IcecastClient.py MimeTypes.py

• Torrent/

• TorrentClient.py TorrentMaker.py TorrentPatron.py TorrentService.py

• UI/

• GraphicDisplay.py

• OpenGL/

• Button.py Movement.py SimpleTranslationInteractor.py OpenGLComponent.py

• Pygame/

• Button.py Display.py Image.py KeyEvent.py Multiclick.py Ticker.py VideoOverlay.py

• Util/

• Backplane.py Chooser.py Chunkifier.py ChunkNamer.py Clock.py Comparator.py ConsoleEcho.py Console.py DataSource.py Fanout.py Filter.py Introspector.py MarshallComponent.py Marshalling.py NullSink.py PassThrough.py PureTransformer.py RateFilter.py Splitter.py Stringify.py UnseenOnly.py

• Visualisation/

• Axon/

• AxonVisualiserServer.py ExtraWindowFurniture.py PComponent.py

• PhysicsGraph/

• TopologyViewer.py TopologyViewerServer.py chunks_to_lines.py GridRenderer.py lines_to_tokenlists.py RenderingParticle.py

• Kamaelia/Experimental/Services.py

• Kamaelia/Automata/Behaviours.py

• Kamaelia/Support/

• Deprecate.py

• DVB/

• CRC.py Descriptors.py

• Data/

• bitfieldrec.py Experimental.py Repository.py

• Particles/

• SpatialIndexer.py

Platforms

Kamaelia has been used successfully under both Linux, Windows and Mac OS X. (mostly developed/tested under Linux & Mac OS X)

Where can I get it? & Docs?

Download:

http://www.kamaelia.org/GetKamaelia.html
http://www.kamaelia.org/release/Kamaelia-0.6.0.tar.gz

Docs:

http://www.kamaelia.org/Docs/Axon/Axon.html
http://www.kamaelia.org/Components.html
http://www.kamaelia.org/Cookbook.html
http://www.kamaelia.org/MiniAxon.html

Presentations:

http://www.slideshare.net/kamaelian

Get involved: (all locations changed since last major release)

http://www.kamaelia.org/Developers/
http://groups.google.com/group/kamaelia
http://code.google.com/p/kamaelia/

Licensing

Kamaelia is released under the Mozilla tri-license scheme (MPL1.1/GPL2.0/LGPL2.1). See http://www.kamaelia.org/Licensing

Thanks & acknowledgements

Finally, many thanks to everyone who's contributed to this release, especially including: Matt Hammond, Sylvain Hellegouarch, Jinna Lei, Jason Baker, Dave King, Patrick Thomson, Ryan Lothian

As always, Feedback, improvements, corrections & suggestions as usual, very welcome - via the mailing list :)

Michael, October 2008