Kamaelia.File.Reading

• prefab FixedRateControlledReusableFileReader
• component PromptedFileReader
• prefab RateControlledFileReader
• prefab RateControlledReusableFileReader
• prefab ReusableFileReader
• component SimpleReader

• Components for reading from files
  • PromptedFileReader
    • Example Usage
    • More detail
  • SimpleReader
    • Example Usage
    • More detail
  • RateControlledFileReader
    • Example Usage
    • More detail
  • ReusableFileReader
    • Example Usage
    • More detail
  • RateControlledReusableFileReader
    • Example Usage
    • More detail
  • FixedRateControlledReusableFileReader
    • Example Usage
    • More detail
  • Development history

Components for reading from files

These components provide various ways to read from files, such as manual control of the rate at which the file is read, or reusing a file reader to read from multiple files.

Key to this is a file reader component that reads data only when asked to. Control over when data flows is therefore up to another component - be that a simple component that requests data at a constant rate, or something else that only requests data when required.

PromptedFileReader

This component reads bytes or lines from the specified file when prompted.

Send the number of bytes/lines to the "inbox" inbox, and that data will be read and sent to the "outbox" outbox.

Example Usage

Reading 1000 bytes per second in 10 byte chunks from 'myfile':

Pipeline(ByteRate_RequestControl(rate=1000,chunksize=10)
         PromptedFileReader("myfile", readmode="bytes")
        ).activate()

More detail

The component will terminate if it receives a shutdownMicroprocess message on its "control" inbox. It will pass the message on out of its "signal" outbox.

If unable to read all N bytes/lines requested (perhaps because we are nearly at the end of the file) then those bytes/lines that were read successfully are still output.

When the end of the file is reached, a producerFinished message is sent to the "signal" outbox.

The file is opened only when the component is activated (enters its main loop).

The file is closed when the component shuts down.

SimpleReader

This is a simplified file reader that simply reads the given file and spits it out "outbox". It also handles maximum pipewidths, enabling rate limiting to be handled by a piped component.

Example Usage

Usage is the obvious:

from Kamaelia.Chassis.Pipeline import Pipeline
from Kamaelia.File.Reading import SimpleReader
from Kamaelia.Util.Console import ConsoleEchoer

Pipeline(
    SimpleReader("/etc/fstab"),
    ConsoleEchoer(),
).run()

More detail

This component will terminate if it receives a shutdownMicroprocess message on its "control" inbox. It will pass the message on out of its "signal" outbox.

If unable to send the message to the recipient (due to the recipient enforcing pipewidths) then the reader pauses until the recipient is ready and resends (or a shutdown message is recieved).

The file is opened only when the component is activated (enters its main loop).

The file is closed when the component shuts down.

RateControlledFileReader

This component reads bytes/lines from a file at a specified rate. It is performs the same task as the ReadFileAdapter component.

You can configure the rate, and the chunk size or frequency.

Example Usage

Read 10 lines per second, in 2 chunks of 5 lines, and output them to the console:

Pipeline(RateControlledFileReader("myfile", "lines", rate=10, chunksize=5),
         ConsoleEchoer()
        ).activate()

More detail

This component is a composition of a PromptedFileReader component and a ByteRate_RequestControl component.

The component will shut down after all data is read from the file, emitting a producerFinished message from its "signal" outbox.

The component will terminate if it receives a shutdownMicroprocess message on its "control" inbox. It will pass the message on out of its "signal" outbox.

The inbox "inbox" is not wired and therefore does nothing.

ReusableFileReader

A reusable PromptedFileReader component, based on a Carousel component. Send it a new filename and it will start reading from that file. Do this as many times as you like.

Send it the number of bytes/lines to read and it will output that much data, read from the file.

Example Usage

Read data from a sequence of files, at 1024 bytes/second in 16 byte chunks:

playlist = Chooser(["file1","file2","file3", ...]
rate = ByteRate_RequestControl(rate=1024,chunksize=16)
reader = ReusableFileReader("bytes")

playlist.link( (reader, "requestNext"), (playlist,"inbox") )
playlist.link( (playlist,"outbox"), (reader, "next") )

Pipeline(ratecontrol, reader).activate()

Or, with the Control-Signal path linked up properly, using the JoinChooserToCarousel prefab:

playlist = Chooser(["file1","file2","file3", ...]
rate = ByteRate_RequestControl(rate=1024,chunksize=16)
reader = ReusableFileReader("bytes")

playlistreader = JoinChooserToCarousel(playlist, reader)

Pipeline(ratecontrol, playlistreader).activate()

More detail

Bytes or lines are read from the file on request. Send the number of bytes/lines to the "inbox" inbox, and that data will be read and sent to the "outbox" outbox.

This component will terminate if it receives a shutdownMicroprocess or producerFinished message on its "control" inbox. The message will be passed on out of its "signal" outbox.

No producerFinished or shutdownMicroprocess type messages are sent by this component between one file and the next.

RateControlledReusableFileReader

A reusable file reader component, based on a Carousel component. Send it a filename and the rate you want it to run at, and it will start reading from that file at that rate. Do this as many times as you like.

Example Usage

Read data from a sequence of files, at different rates:

playlist = Chooser([ ("file1",{"rate":1024}),
                     ("file2",{"rate":16}), ...])
reader = RateControlledReusableFileReader("bytes")

playlist.link( (reader, "requestNext"), (playlist,"inbox") )
playlist.link( (playlist,"outbox"), (reader, "next") )

reader.activate()
playlist.activate()

Or, with the Control-Signal path linked up properly, using the JoinChooserToCarousel prefab:

playlist = Chooser([ ("file1",{"rate":1024}),
                     ("file2",{"rate":16}), ...])
reader = RateControlledReusableFileReader("bytes")

playlistreader = JoinChooserToCarousel(playlist, reader).activate()

More detail

The rate control is performed by a ByteRate_RequestControl component. The rate arguments should be those that are accepted by this component.

This component will terminate if it receives a shutdownMicroprocess or producerFinished message on its "control" inbox. The message will be passed on out of its "signal" outbox.

No producerFinished or shutdownMicroprocess type messages are sent by this component between one file and the next.

FixedRateControlledReusableFileReader

A reusable file reader component that reads data from files at a fixed rate. It is based on a Carousel component.

Send it a new filename and it will start reading from that file. Do this as many times as you like.

Example Usage

Read data from a sequence of files, at 10 lines a second:

playlist = Chooser(["file1", "file2", "file3", ... ])
reader = FixedRateControlledReusableFileReader("lines", rate=10, chunksize=1)

playlist.link( (reader, "requestNext"), (playlist,"inbox") )
playlist.link( (playlist,"outbox"), (reader, "next") )

reader.activate()
playlist.activate()

Or, with the Control-Signal path linked up properly, using the JoinChooserToCarousel prefab:

playlist = Chooser(["file1", "file2", "file3", ... ])
reader = FixedRateControlledReusableFileReader("lines", rate=10, chunksize=1)

playlistreader = JoinChooserToCarousel(playlist, reader).activate()

More detail

The rate control is performed by a ByteRate_RequestControl component. The rate arguments should be those that are accepted by this component.

This component will terminate if it receives a shutdownMicroprocess or producerFinished message on its "control" inbox. The message will be passed on out of its "signal" outbox.

No producerFinished or shutdownMicroprocess type messages are sent by this component between one file and the next.

Development history

PromptedFileReader - developed as an alternative to ReadFileAdapter - prototyped in /Sketches/filereading/ReadFileAdapter.py

---

Kamaelia.File.Reading.FixedRateControlledReusableFileReader

prefab: FixedRateControlledReusableFileReader

FixedRateControlledReusableFileReader(readmode, rateargs) -> reusable file reader component

A file reading component that can be reused. Based on a carousel - send a filename to the "next" or "inbox" inboxes to start reading from that file.

Data is read at the specified rate.

Keyword arguments: - readmode = "bytes" or "lines" - rateargs = arguments for ByteRate_RequestControl component constructor

Kamaelia.File.Reading.PromptedFileReader

class PromptedFileReader(Axon.Component.component)

PromptedFileReader(filename[,readmode]) -> file reading component

Creates a file reader component. Reads N bytes/lines from the file when N is sent to its inbox.

Keyword arguments:

• readmode -- "bytes" or "lines"

Inboxes

• control : for shutdown signalling
• inbox : requests to 'n' read bytes/lines

Outboxes

• outbox : data output
• signal : outputs 'producerFinished' after all data has been read

Methods defined here

Warning!

You should be using the inbox/outbox interface, not these methods (except construction). This documentation is designed as a roadmap as to their functionalilty for maintainers and new component developers.

__init__(self, filename[, readmode])

x.__init__(...) initializes x; see x.__class__.__doc__ for signature

closeDownComponent(self)

Closes the file handle

main(self)

Main loop

readNBytes(self, n)

readNBytes(n) -> string containing 'n' bytes read from the file.

"EOF" raised if the end of the file is reached and there is no data to return.

readNLines(self, n)

readNLines(n) -> string containing 'n' lines read from the file.

"EOF" raised if the end of the file is reached and there is no data to return.

shutdown(self)

Returns True if a shutdownMicroprocess message is received.

Also passes the message on out of the "signal" outbox.

Kamaelia.File.Reading.RateControlledFileReader

prefab: RateControlledFileReader

RateControlledFileReader(filename[,readmode][,**rateargs]) -> constant rate file reader

Creates a PromptedFileReader already linked to a ByteRate_RequestControl, to control the rate of file reading.

Keyword arguments:

• readmode -- "bytes" or "lines"
• rateargs -- arguments for ByteRate_RequestControl component constructor

Kamaelia.File.Reading.RateControlledReusableFileReader

prefab: RateControlledReusableFileReader

RateControlledReusableFileReader(readmode) -> rate controlled reusable file reader component.

A file reading component that can be reused. Based on a Carousel - send (filename, rateargs) to the "next" inbox to start reading from that file at the specified rate.

• rateargs are the arguments for a ByteRate_RequestControl component.

Keyword arguments: - readmode = "bytes" or "lines"

Kamaelia.File.Reading.ReusableFileReader

prefab: ReusableFileReader

ReusableFileReader(readmode) -> reusable file reader component.

A file reading component that can be reused. Based on a Carousel - send a filename to the "next" inbox to start reading from that file.

Must be prompted by another component - send the number of bytes/lines to read to the "inbox" inbox.

Keyword arguments: - readmode = "bytes" or "lines"

Kamaelia.File.Reading.SimpleReader

class SimpleReader(Axon.Component.component)

SimpleReader(filename[,mode][,buffering]) -> simple file reader

Creates a "SimpleReader" component.

Arguments:

• filename -- Name of the file to read
• mode -- This is the python readmode. Defaults to "r". (you may way "rb" occasionally)
• buffering -- The python buffer size. Defaults to 1. (see http://www.python.org/doc/2.5.2/lib/built-in-funcs.html)

Inboxes

Outboxes

Methods defined here

Warning!

You should be using the inbox/outbox interface, not these methods (except construction). This documentation is designed as a roadmap as to their functionalilty for maintainers and new component developers.

__init__(self, filename, **argd)

x.__init__(...) initializes x; see x.__class__.__doc__ for signature

main(self)

Main loop Simply opens the file, loops through it (using "for"), sending data to "outbox". If the recipient has a maximum pipewidth it handles that eventuallity resending by pausing and waiting for the recipient to be able to recieve.

Shutsdown on shutdownMicroprocess.

shutdown(self)

Returns True if a shutdownMicroprocess message is received.

Also passes the message on out of the "signal" outbox.