[02:20] *** eikenberry has joined #kamaelia

[04:01] *** wyleu has joined #kamaelia

[04:46] *** wyleu has joined #kamaelia

[07:21] *** tdobson has joined #kamaelia

[07:59] *** Uraeus has joined #kamaelia

[10:55] *** Uraeus has joined #kamaelia

[12:08] *** mhrd has joined #kamaelia

[12:29] *** VanL has joined #kamaelia

[12:44] *** VanL has parted #kamaelia

[17:52] *** vmlemon_ has joined #kamaelia

[17:52] < vmlemon_> Hi

[17:53] < Lawouach> hi

[18:04] *** VanL has joined #kamaelia

[18:06] < vmlemon_> Hi VanL

[18:06] < VanL> Hi

[18:06] < VanL> I am working on an IMAP server. Most times I need to buffer my input until I receive an entire line, but occasionally I need to buffer (size) bytes.

[18:06] < VanL> I wrote a BufferedReceiver (http://pastebin.com/m41eaf698) that changes its mode based on control messages

[18:07] < VanL> Any comments as to style? Is this the best way to do it? How would this be best wired up - just in a pipeline, and I can send signals to it from further down?

[18:12] *** MS- has joined #kamaelia

[18:12] < MS-> evening

[18:12] < vmlemon_> Hi MS-

[18:13] < VanL> Hi MS-

[18:13] < MS-> Hiya

[18:14] < VanL> MS-: I think I chatted with you a day or two ago about building an IMAP server in the style of the greylisting server. Is that right?

[18:14] *** MS- reading logs (& cleared pending messages on mailing list :) )

[18:14] *** MS- nods

[18:15] < VanL> oh, can you see the pastebin post above? (http://pastebin.com/m41eaf698). Either way, any comment?

[18:16] *** MS- taking a look :)

[18:16] < MS-> logs btw are here: http://www.kamaelia.org/logs/

[18:18] < MS-> Looking at it, there's somethings that I wouldn't do

[18:19] < MS-> this:

[18:19] < MS-> while self.dataReady('control'):

[18:19] < MS-> self.controlbuffer.append(self.recv('control'))

[18:19] < MS->

[18:19] < MS-> self.handle_control()

[18:19] < MS-> along with the code in handle_control()

[18:19] < MS-> is a little redundant

[18:19] < MS-> if you change:

[18:19] < MS-> for msg in self.controlbuffer:

[18:19] < MS-> to

[18:19] < MS-> for msg in self.Inbox("control"):

[18:20] < MS-> and get rid of:

[18:20] < MS-> while self.dataReady('control'):

[18:20] < MS-> self.controlbuffer.append(self.recv('control'))

[18:20] < MS->

[18:20] < MS-> Then that will perform the same way

[18:21] < MS-> Also, I'd probably write handle_data non recursively.

[18:21] < MS-> These are largely stylistic things I must admit :)

[18:22] < VanL> Thats fine, I think idiomatic code is important

[18:23] < VanL> would you keep the separate handle_control/handle_data/handle_shutdown? Or put in in main?

[18:24] < MS-> SInce you're using custom control messages (eg BufferUntil) I would keep handle_control

[18:25] < MS-> I'd do the handle_shutdown() outside the loop as well

[18:26] < MS-> Ah, it has two modes of operation as well

[18:27] < MS-> I think these days, I'd actually write that as two different components

[18:27] < MS-> one for size delimiting and one for line delimiting

[18:27] < MS-> and then have it work slightly differently, but that's something which isn't idiomatic at present

[18:28] < VanL> I have that. My concern was with extra data

[18:28] < MS-> (just something I'd do, to see if it looked better)

[18:28] < VanL> This was my second try.

[18:28] *** MS- nods

[18:29] < VanL> if we had data pending in the buffer but then needed to switch modes, I would have to send the extra data over as initial_data to the new component

[18:29] < VanL> possible, but seemed... less ideal.

[18:29] < MS-> I see

[18:31] < MS-> My personal approach was to create a "getline" generator instead here:

[18:31] < MS-> http://code.google.com/p/kamaelia/source/browse/trunk/Code/Python/Kamaelia/Kamaelia/Apps/Grey/MailHandler.py

[18:32] < MS-> The way that works is to use WaitComplete - which effectively tells the system "run this other function instead of main, and when it finishes come back here"

[18:33] < MS-> It's not as nice/composable as your component, but it can simplify code sometimes

[18:33] < VanL> My plan was actually to use it with WaitComplete

[18:33] < VanL> similarly to your code

[18:33] < MS-> That would be interesting to see

[18:34] < VanL> Updated here: http://pastebin.com/m7ff0223f

[18:36] < VanL> oops, actually http://pastebin.com/m421623f (moved handle_shutdown out of loop)

[18:38] < MS-> Ah the shutdown needs a little more as well

[18:39] < MS-> your component can shutdown and send "false" to the next component

[18:39] < VanL> I don't understand

[18:39] < MS-> Oh scratch that

[18:39] < MS-> sorry

[18:40] < MS-> missed the conditions for self.shutdown

[18:40] < MS-> if it's not set to an instance of either producerFinished, shutdownMicroprocess it doesn't get shutdown

[18:40] < MS-> which is fine

[18:41] < VanL> Are there any other shutdown conditions? Those were the only ones I saw when poking around various codebases

[18:41] < VanL> (any other standard shutdown conditions)

[18:41] < MS-> Those are the two most common

[18:41] < MS-> Over time it's become idiomatic for producerFInished() to mean

[18:42] < MS-> "I'm done, and I'm not going to send you any more data"

[18:42] < MS-> Which for many components means it's time for them to stop as well

[18:42] < VanL> That would make sense here

[18:42] < MS-> Whereas shutdownMicroprocess means "Shutdown ASAP"

[18:43] < VanL> Regarding WaitComplete - isn't an Axon component also a generator? The way IMAP works is that I receive a message crom the client saying "Expect X number of octets"; the server says, "ok to send X number of octets"; and then wait until I have received that many bytes.

[18:44] < VanL> My thought was that if I needed to do that, I could send the control message (BufferUntil(size)); and then WaitComplete on self.Inbox('inbox')

[18:46] < MS-> You could do that - I can see why you'd want to as well

[18:47] < MS-> An Axon component isn't just a generator really. The vast majority are implemented using generators, but I wouldn't say that an Axon component is also a generator

[18:50] < MS-> fwiw, it looks like the right sort of approach overall - especially given the fact you have to mode switch

[18:50] < MS-> The proof of the pudding is in the doing of course :)

[18:51] < VanL> Well, thats true. I appreciate your input and time.

[18:51] < VanL> Hmm, while I am at it, is there a good way to test these at the prompt?

[18:52] < VanL> I have been wiring up little echo servers

[18:53] < MS-> I tend to use a small pipeline or graphline and then either connect a file reader or a another component as a source (ala "cat" on unix)

[18:53] < MS-> and then dump the output through a ConsoleEchoer

[18:54] < MS-> eg http://www.kamaelia.org/Components/pydoc/Kamaelia.Util.DataSource.html

[18:54] < MS-> You can also use "Handle" from a python prompt, thinking about it

[18:55] < MS-> http://www.kamaelia.org/AxonHandle

[18:55] < MS-> doing:

[18:55] < MS-> from Axon.background import background

[18:55] < MS-> background().start()

[18:55] < MS-> starts a scheduler in the background

[18:56] < MS-> which means that works happily inside a python console

[20:14] *** mhrd-home has joined #kamaelia

[20:24] < VanL> MS-: FWIW, here is the little interact() snippet that I worked up based on Handle: http://pastebin.com/f1ecb5052

[20:25] < VanL> Thanks for the tip

[20:26] < mhrd-home> VanL: hi, reading scrollback :-)

[20:26] < mhrd-home> MS-: hi :-)

[20:28] < VanL> mhrd-home: Thanks for the help yesterday. Here is my final BufferedReceiver: http://pastebin.com/f16cb0117. Inspired by yours, but with the changes I needed for IMAP.

[20:29] < mhrd-home> hi VanL: it was a pleasure

[20:29] < mhrd-home> Stylistically, looks pretty good for a 1st/2nd attempt - has shutdown handling in it which always makes me happy :-)

[20:32] < mhrd-home> looking at what you're trying to do (switch chunking/splitting mode dynamically), and assuming I understand correctly ...

[20:33] < mhrd-home> the current design might suffer from a problem where you can't control the precise point at which it switches mode ...

[20:33] < mhrd-home> because the BufferedReceiver component could well be several lines/n-byte-chunks ahead of the next component in the pipeline which is parsing the messages and making the mode switching decisions?

[20:34] < VanL> I thought about that, but I wasn't sure I could fix it. I can receive stuff at any time.

[20:35] < VanL> Thats part of why I have the mode as a list

[20:35] < mhrd-home> aah, didn't spot that

[20:35] *** mhrd-home looks

[20:35] < VanL> so if I only want one message received the different way, I can send two successive BufferUntil messages

[20:35] *** vmlemon_ is really hating C++ right now

[20:36] < mhrd-home> the BufferUntil message gets sent to the "inbox" inbox?

[20:36] *** mhrd-home sympathises with vmlemon_

[20:36] < VanL> no, it gets sent to 'control'

[20:37] < vmlemon_> I've been trying to compile a stupid header file based on some code that I found, and there's a load of types that are used, but are undefined, and I have no idea what they're supposed to be

[20:37] *** VanL feels vmlemon_'s pain

[20:38] < mhrd-home> VanL: ah, ok, I see

[20:39] < VanL> see update_mode

[20:39] < mhrd-home> MS 's "getline generator" approach avoids the issue

[20:39] < mhrd-home> another option you can take is a similar kind of approach (albeit a bit less concise) whilst keeping it as a separate component ...

[20:41] < mhrd-home> eg. you can build a component with an additional inbox. whenever you want another chunk of data, you send a kind of "next-please" message to that additional inbox

[20:41] < mhrd-home> this component is a little like that: http://code.google.com/p/kamaelia/source/browse/trunk/Code/Python/Kamaelia/Kamaelia/Util/PromptedTurnstile.py

[20:42] < mhrd-home> so for your case you could have two kinds of "next-please" message - one is a number of bytes, the other is a delimiter

[20:43] < VanL> I see

[20:44] < VanL> I considered this sort of design as well - although this is better than what I was thinking. My concern was that normally, we didn't want to have to say "Next!" for every single command

[20:44] < mhrd-home> it is alot more verbose

[20:44] < VanL> Although in practice, it probably wouldn't be a big deal, just hidden in a function I guess

[20:45] < mhrd-home> you can use the WaitComplete approach to hide that "next-please, then wait for response" process

[20:46] < mhrd-home> I reckon the thing is to try it and see if your component does the job. if it does then there's no need to change anything

[20:49] < VanL> Yes, true.

[20:49] < VanL> Now, however, unfortunately back to less interesting billable work :(

[20:50] < VanL> bye everyone

[20:51] < mhrd-home> another component, I've just remembered, that contains something close to what you need is the WAV codec implementation I did a while ago ...

[20:51] < mhrd-home> http://code.google.com/p/kamaelia/source/browse/trunk/Code/Python/Kamaelia/Kamaelia/Codec/WAV.py

[20:52] < mhrd-home> it is done in a style similar to MS- 's getline() approach - with generators called readBytes() and readLine()

[20:52] < mhrd-home> however 1) it is more complicated that you need it to be, since I was experimenting with making it intelligently cope with outboxes which could overflow (due to a finite size limit being imposed onthem)

[20:53] < VanL> ooh, very nice

[20:53] < mhrd-home> 2) iirc WaitComplete() was broken at that time (at least that is my excuse!) so I used "for _ in self.readBytes(n): yield _"

[20:53] < mhrd-home> which kinda achieves the same effect

[20:54] < mhrd-home> hmm, I can't see why I left the readLine() generator in there - it doesn't appear to be used(!)

[20:54] < mhrd-home> ah well

[20:55] < VanL> still, this is worth knowing about, especially if my component doesn't work like I hope it will

[20:56] < VanL> ok, now really going. Thanks, mhrd-home

[20:56] *** VanL has parted #kamaelia

[20:57] < mhrd-home> np

[22:55] *** mhrd-home has parted #kamaelia