April 2024 - This site, and Kamaelia are being updated. There is significant work needed, and PRs are welcome.


Reading a file as fast as possible

MaxSpeedFileReader reads a file in bytes mode as fast as it can; limited only by any size limit on the inbox it is sending the data to.

This component is therefore useful for building systems that are self rate limiting - systems that are just trying to process data as fast as they can and are limited by the speed of the slowest part of the chain.

Example Usage

Read "myfile" in in chunks of 1024 bytes. The rate is limited by the rate at which the consumer component can consume the chunks, since its inbox has a size limit of 5 items of data:

consumer = Consumer()

Pipeline( MaxSpeedFileReader("myfile", chunksize=1024),

More details

Specify a filename and chunksize and MaxSpeedFileReader will read bytes from the file in the chunksize you specified and send them out of its "outbox" outbox.

If the destination inbox it is sending chunks to is size limited, then MaxSpeedFileReader will pause until space becomes available. This is how the speed at which the file is ingested is regulated - by the rate at which it is consumed.

When the whole file has been read, this component will terminate and send a producerFinished() message out of its "signal" outbox.

If a producerFinished message is received on the "control" inbox, this component will complete sending any data that may be waiting. It will then send the producerFinished message on out of its "signal" outbox and terminate.

If a shutdownMicroprocess message is received on the "control" inbox, this component will immediately send it on out of its "signal" outbox and immediately terminate. It will not complete sending on any pending data.


class MaxSpeedFileReader(Axon.Component.component)

MaxSpeedFileReader(filename[,chunksize]) -> new MaxSpeedFileReader component.

Reads the contents of a file in bytes mode; sending it out as fast as it can in chunks from the "outbox" outbox. The rate of reading is only limited by any size limit of the destination inbox to which the data is being sent.

Keyword arguments:

  • filename -- The filename of the file to read
  • chunksize -- Optional. The maximum number of bytes in each chunk of data read from the file and sent out of the "outbox" outbox (default=32768)



Methods defined here


You should be using the inbox/outbox interface, not these methods (except construction). This documentation is designed as a roadmap as to their functionalilty for maintainers and new component developers.

__init__(self, filename[, chunksize])





waitSend(self, data, boxname)


Got a problem with the documentation? Something unclear that could be clearer? Want to help improve it? Constructive criticism is very welcome - especially if you can suggest a better rewording!

Please leave you feedback here in reply to the documentation thread in the Kamaelia blog.

-- Automatic documentation generator, 05 Jun 2009 at 03:01:38 UTC/GMT