Reading a file as fast as possible
MaxSpeedFileReader reads a file in bytes mode as fast as it can; limited only by any size limit on the inbox it is sending the data to.
This component is therefore useful for building systems that are self rate limiting - systems that are just trying to process data as fast as they can and are limited by the speed of the slowest part of the chain.
Read "myfile" in in chunks of 1024 bytes. The rate is limited by the rate at which the consumer component can consume the chunks, since its inbox has a size limit of 5 items of data:
consumer = Consumer() consumer.inboxes["inbox"].setSize(5) Pipeline( MaxSpeedFileReader("myfile", chunksize=1024), consumer, ).run()
Specify a filename and chunksize and MaxSpeedFileReader will read bytes from the file in the chunksize you specified and send them out of its "outbox" outbox.
If the destination inbox it is sending chunks to is size limited, then MaxSpeedFileReader will pause until space becomes available. This is how the speed at which the file is ingested is regulated - by the rate at which it is consumed.
When the whole file has been read, this component will terminate and send a producerFinished() message out of its "signal" outbox.
If a producerFinished message is received on the "control" inbox, this component will complete sending any data that may be waiting. It will then send the producerFinished message on out of its "signal" outbox and terminate.
If a shutdownMicroprocess message is received on the "control" inbox, this component will immediately send it on out of its "signal" outbox and immediately terminate. It will not complete sending on any pending data.
MaxSpeedFileReader(filename[,chunksize]) -> new MaxSpeedFileReader component.
Reads the contents of a file in bytes mode; sending it out as fast as it can in chunks from the "outbox" outbox. The rate of reading is only limited by any size limit of the destination inbox to which the data is being sent.
Methods defined here
Got a problem with the documentation? Something unclear that could be clearer? Want to help improve it? Constructive criticism is very welcome - especially if you can suggest a better rewording!
Please leave you feedback here in reply to the documentation thread in the Kamaelia blog.
-- Automatic documentation generator, 05 Jun 2009 at 03:01:38 UTC/GMT
This is an ongoing community based development site. As a result the contents of this page is the opinions of the contributors of the pages involved not the organisations involved. Specificially, this page may contain personal views which are not the views of the BBC. (the site is powered by a wiki engine)
(C) Copyright 2008 Kamaelia Contributors, including the British Broadcasting Corporation, All Rights Reserved.