[01:15] < ms-> New likefile code merged onto private_MPS_LikeFileRewrite

[01:16] < ms-> Demos/examples changed to work

[01:16] < ms-> Now to rename to "handle"

[01:26] < j_baker> Wow. You're still up?

[01:26] < ms-> yep

[01:31] < ms-> Probably for not much longer though!

[01:32] < j_baker> I actually just got the POST methods figured out. I'll probably spend tomorrow figuring out how to get the thing to stay logged in.

[01:32] < ms-> Cool :)

[01:32] < ms-> Congrats :)

[01:32] < ms-> I've just merged onto private_MPS_LikeFileRewrite the rewrite (and rename) of LikeFile

[01:43] *** sadrul has joined #kamaelia

[07:49] *** ms- aims to work for about 3-4 hours then travel

[07:52] *** ms- notes this change on Scratch which is going onto trunk:

[07:52] < ms-> + def Inbox(self, boxname="inbox"):

[07:52] < ms-> + while self.dataReady("inbox"):

[07:52] < ms-> + yield self.recv("inbox")

[07:52] < ms-> will all this

[07:52] < ms-> for data in self.Inbox("inbox"):

[07:52] < ms-> print "data"

[07:52] < ms-> (or rather does allow this)

[07:59] < Lawouach> that's rather interesting

[07:59] < Lawouach> makes Axon look more and more pythonic in a way ;)

[08:00] < Lawouach> if you fancy the word

[08:00] < ms-> Indeed - the fact that I went the way of the application (tried to find out the appropriate idioms) rather than the pythonic way was a criticism levelled by Tommi Virtinan (sp?) a couple of years back

[08:01] < ms-> Which was valid, but I didn't want to add sugar before finding out whether the sugar was appropriate

[08:01] < Lawouach> fair point

[08:01] < Lawouach> you migh even consider adding a way for properties

[08:01] < Lawouach> so you would have:

[08:01] < Lawouach> for data in self.inbox:

[08:01] < Lawouach> print data

[08:02] < Lawouach> here inbox is not the inboxes dict

[08:02] < Lawouach> but a property like this

[08:02] < Lawouach> @property

[08:02] < Lawouach> def Inbox(self):

[08:02] < Lawouach> ...

[08:02] < ms-> Yep.

[08:02] < Lawouach> oops make the def name lower case

[08:02] < ms-> Though if I was to do that, I'd probably use Inbox not inbox and that would rile some people.

[08:03] < Lawouach> sure

[08:03] < ms-> Largely because it matches Inboxes - which is the public interface for defining them :)

[08:03] < Lawouach> but that could be done easily in the LikeHandle-whatever

[08:03] *** ms- nods

[08:03] < Lawouach> since you're already inspecting boxes

[08:03] < Lawouach> you might as well add properties for each automatically

[08:03] < ms-> LikeHandle has been merged as simply "Handle" btw

[08:03] < Lawouach> I assume it's doable in Python

[08:03] < Lawouach> alright

[08:03] < Lawouach> that's cool :)

[08:04] < Lawouach> Haven't kept track of all the changes lately

[08:04] < ms-> I've made a pre-release checkpoint BTW, so if we discover I'm merging a ton of stupid stuff it'll be easy to revert :)

[08:04] < Lawouach> :)

[08:04] < ms-> http://edit.kamaelia.org/ReleaseJune2008 - tracks the high level changes

[08:16] < ms-> Interesting - this works :

[08:16] < ms-> svn ci -m "`cat commit_message.txt`"

[08:16] < ms-> Useful

[08:16] < ms-> private_MPS_AxonCoreImprovements_mashed/ - merged

[08:23] *** Chong- has joined #kamaelia

[08:43] *** orphans has joined #kamaelia

[08:57] < ms-> biab

[08:57] *** ms- has parted #kamaelia

[09:25] *** Uraeus has joined #kamaelia

[09:30] *** vmlemon_ has joined #kamaelia

[09:32] < vmlemon_> Hi

[10:43] *** Davbo has joined #kamaelia

[10:52] *** vmlemon_ has joined #kamaelia

[11:27] *** vmlemon__ has joined #kamaelia

[11:30] *** Davb1 has joined #kamaelia

[11:33] *** Davb1 is now known as Davbo

[11:34] *** vmlemon__ is now known as vmlemon_

[12:31] < Davbo> wooo!! flood-fill tool done :D

[12:31] < Davbo> non-recursively

[12:31] *** Davbo loves open-source

[12:38] < vmlemon_> Hah, [13:09] < timsf> I take it that slide 4 that says "Flush out desktop"

[13:18] *** MS- has joined #kamaelia

[13:18] < MS-> orphans: thanks for the "up for review bit"

[13:18] < MS-> I'll probably merge but as a new namespace

[13:19] < MS-> (at least initially)

[13:19] *** MS- munches lunch

[13:20] < MS-> http://edit.kamaelia.org/release/ contains release candidates

[13:20] < MS-> highest numbered/most recent == most recent

[13:23] < Davbo> when's that mashed thing?

[13:25] < MS-> tomorrow & sunday - there's a free coach from Sheffield, Newcastle,. manchester & liverpool

[13:26] < MS-> Attendees need a laptop & a sleeping bag

[13:26] < MS-> Apparently food & drink will be available (gratis)

[13:27] < Davbo> ah

[13:27] < MS-> this is when the bus details are: http://mashed08.backnetwork.com/event/?articleid=16

[13:28] < MS-> I've been mentioning it periodically

[13:29] < MS-> if it wasn't for the release and the fact I'm talking as a result I'd blow it off and go to Digital 60 tonight - which is a thingy celebrating "the baby"

[13:31] < MS-> (cf http://radar.oreilly.com/archives/2008/06/babys-60th-birthday.html )

[13:32] < Davbo> I'd have probably tried to go to this Mashed thing, but circumstances mean it's just not possible

[13:33] < Davbo> I'm looking at going to PyCon UK though

[13:33] < MS-> i seem to remember a few weeks back you mentioning that may be the case

[13:33] < MS-> Cool

[13:33] < MS-> Come give a talk :)

[13:34] < MS-> "this is what I didn't know 6 months ago"

[13:34] < Davbo> Heh

[13:34] < MS-> ;)

[13:36] < MS-> Must admit though I'm travelling down tonight. Chances of me being even vaguely sane tomorrow if I'm on a bus at 4am are non-existent

[13:36] < MS-> Kinda useful to not talk complete twaddle

[13:37] < Davbo> Yeah it's 4:30am tomorrow for that bus from Sheffield

[13:37] < Davbo> Should I feel the urge I could be down there :p

[13:37] < MS-> Just not very plausible

[13:37] < Davbo> Hah

[13:38] < MS-> For me that's more a "time to stay up until" rather than "get up for"

[13:38] < MS-> Which probably means I'm not old yet if I still think *that* way

[13:38] < MS-> :)

[13:38] < Davbo> hehe :-)

[13:46] *** MS- has parted #kamaelia

[14:16] *** vmlemon_ has joined #kamaelia

[14:42] < Davbo> http://www.davbo.org/images/snapshot10.png woo!

[14:42] < Davbo> on that note, back later

[15:19] *** Chong- has joined #kamaelia

[15:30] *** vmlemon__ has joined #kamaelia

[15:30] *** vmlemon__ is now known as vmlemon_

[16:38] *** vmlemon_ has joined #kamaelia

[17:37] *** MS- has joined #kamaelia

[17:37] < MS-> Evening

[17:37] < vmlemon_> Hi

[17:38] *** MS- notes Davbo's flood fill appears to be working well :)

[17:38] < MS-> That's actually pretty impressive - I kinda expected that sort of thing to come later

[17:38] < MS-> since it's harder than the basics of circles, lines, dots etc

[17:39] *** MS- makes a note to show davbo how to make pygame windows which are *smaller*

[17:39] < MS-> :)

[17:41] < vmlemon_> Hmm, what was Davbo making again?

[17:41] < MS-> A multiwindow paint program using pygame

[17:41] < vmlemon_> Aah

[17:41] < MS-> I might suggest he looks at smudging next

[17:42] < MS-> (leaving what that might mean open to relative intepretation, but with the assumption its a bit like a blobby movable blur)

[17:42] < MS-> Since that'd open up interesting options

[17:42] < MS-> (especially if every window could be a pallete for setting the colour)

[17:43] < vmlemon_> An Airbrush tool might be interesting, assuming that it isn't already implemented/in progress

[17:43] *** MS- nods

[17:43] < MS-> In terms of "is it new and amazing", the answer will be a "not really", but I really want the emphasis to be on "stuff that makes it nice and easy to do two specific things"

[17:44] < MS-> One is "manual stitching together of pictures"

[17:44] < MS-> (which is something I do fairly often)

[17:44] < vmlemon_> Networked image editing?

[17:44] < MS-> The other being "easy to create new things that look nice"

[17:44] < MS-> That too

[17:44] < MS-> But that's relatively easy given its kamaelia :)

[17:45] < vmlemon_> Does it have file import and export functionality already?

[17:45] < vmlemon_> That could always come in handy

[17:45] < MS-> I doubt it.

[17:45] < MS-> It would

[17:45] < MS-> The way the whiteboard does that is it automatically saves everything in a known area (pads if you like)

[17:45] < MS-> which you can flick back and forth

[17:45] < MS-> And since that's just a directory, you can add stuff

[17:45] < MS-> but that's hardly the same as load/save

[17:47] < vmlemon_> Aah

[17:47] < MS-> Davbo's program is completely new though

[17:47] < vmlemon_> Wow, my connection is flaky as hell today

[17:47] < vmlemon_> It seems that I can download 20KB-1MB files, but I can't download small webpages without them only being half-loaded

[17:48] < MS-> The point behind it it's worth bearing in mind is that it's a demo of multiwindow stuff

[17:48] < MS-> and just intended to be fun/basically useful

[17:48] < vmlemon_> Cool

[17:48] < MS-> which is always a good starting point of course

[17:48] < MS-> But means that the focus is on fun stuff

[17:48] < MS-> (multiwindow pygame requires multiprocess due to pygame limitations)

[17:56] < vmlemon_> Hmm, the limit is due to no-one implementing support for single-process multiwindow apps? Or is it more of an architectural/legacy issue?

[17:56] < MS-> It's an SDL limitation

[17:57] < MS-> Which I think is being removed in a new version at some point

[17:57] < MS-> But until it is, it's essentially "proof" that the system is using multiple processes

[17:57] < MS-> since it simply wouldn't work otherwise

[17:57] < MS-> which is why it's a nice example

[17:57] < vmlemon_> Sounds fun

[17:58] < MS-> It also means that then the system has basic image editting available for any application

[17:59] < MS-> which is a nice side effect of it being kamaelia based :)

[18:00] < MS-> Something I really ought to do, if I don't put anything else in the release is to put the first version of multiprocess stuff in

[18:09] *** vmlemon__ has joined #kamaelia

[18:12] *** vmlemon__ is now known as vmlemon_

[18:53] < j_baker> MS-: Is there anything that a web server needs to do HTTP-wise to deal with sessions?

[18:54] < MS-> Not particularly no - HTTP is stateless - sessions are an application level concept

[18:54] < MS-> Though the web server can do interesting sessions

[18:55] < MS-> Kamaelia/Protocol/HTTP/Handlers/SessionExample.py

[18:55] < MS-> class SessionExample(component):

[18:55] < MS-> is a component that gets reactivated for the user

[18:55] < MS-> Meaning that when that while loop yields,

[18:56] < MS-> it's actually sending a page back to the user

[18:56] < MS-> It's a pity that core python doesn't support pickling of generators like Stackless Python does really

[18:56] < MS-> since that'd open up some really neat options

[18:56] < j_baker> That would be pretty interesting.

[18:57] < MS-> But as long as the server doesn't restart, it does open up some very novel applications

[18:57] < MS-> OK, Post about the new release & Mashed: http://yeoldeclue.com/cgi-bin/blog/blog.cgi?rm=viewpost&nodeid=1213988026

[18:57] < MS-> I think I'm right in saying alot is going in the new release ...

[18:58] < MS-> Doesn't even cover any of the detail, like ssl support (should add that I guess)

[18:59] *** vmlemon_ has joined #kamaelia

[19:38] < j_baker> MS-: Is the peer variable that ServerCore passes to a protocol (like the HTTPServer) equivalent to the CGI REMOTE_ADDR variable?

[19:39] < MS-> OK, let's chat about that briefly, then I'll get dinner

[19:39] < MS-> I promised to explain what's going on

[19:39] < j_baker> cool

[19:39] < MS-> because I noted you thought there was a request/response thing going on in ServerCore, and there isn't

[19:39] < MS-> So I'll go to basics if I may

[19:40] < MS-> ServerCore, starts off by creating a TCPServer component

[19:40] < MS-> This opens a port on the server. The combination of port & IP forms a socket.

[19:40] < MS-> That socket, is a LISTEN socket

[19:40] < MS-> the sort of thing that shows up like this:

[19:40] < MS-> tcp4 0 0 *.25 *.* LISTEN

[19:41] < MS-> when you type netstat -nat|grep LISTEN

[19:41] < MS-> When a client connects, the socket becomes read-ready

[19:41] < MS-> The TCPServer component goes "aha!"

[19:41] < MS-> and calls accept, creating a connected socket

[19:41] < MS-> that looks like this in netstat -nat

[19:41] < MS-> tcp4 0 0 192.168.2.9.22 84.201.167.186.43858 ESTABLISHED

[19:42] < MS-> To handle traffic (just raw bytes)

[19:42] < MS-> to/from that socket, the server creates a ConnectedSocketAdapter

[19:42] < MS-> which does just that

[19:42] < MS-> it "adapts" from in/out-boxes

[19:42] < MS-> to the socket in/out directions

[19:42] < MS-> bytes recieved on the socket get forwarded to the CSA's outbox

[19:43] < MS-> bytes recieved on the CSA's inbox get forwarded to the connected socket

[19:43] < vmlemon_> Hmm, could Kamaelia systems/apps be extended in general to work in a "network transparent" way?

[19:43] < MS-> vmlemon_: yes

[19:43] < MS-> It's worth noting that whilst this will come in chunks

[19:43] < MS-> some bug

[19:43] < MS-> big

[19:43] < MS-> some small

[19:43] < MS-> BUT, no matter the size chunks

[19:43] < MS-> The data is *not framed*

[19:43] < MS-> and *nothing can be inferred from the chunks*

[19:44] < MS-> Often you may be lucky and have them terminate on a \n though

[19:44] < MS-> :)

[19:44] < MS-> OK, so that gives you a basic "server" that provides something that will accept bytes and send them on

[19:44] < MS-> and something that can send on

[19:44] < MS-> However that's not really enough

[19:44] < MS-> So, as well as creating the CSA and activating it

[19:45] < MS-> The TCPServer component sends a copy of the CSA to anyone interested

[19:45] < MS-> In this instance the interested party is the ServerCore component

[19:45] < MS-> the message the ServerCore gets looks like this:

[19:46] < MS-> newCSAMessage.object

[19:46] < MS-> newCSAMessage rather

[19:46] < MS-> That inside it has this:

[19:46] < MS-> def __init__(self, caller, CSA, sock=None):

[19:46] < MS-> super(newCSA, self).__init__(caller, CSA)

[19:46] < MS-> self.sock = sock

[19:47] < MS-> That call to the super class means that

[19:47] < MS-> the newCSAMessage has attributes

[19:47] < MS-> sock - which is the ACTUAL connect socket the CSA manages

[19:47] < MS-> CSA - which is the CSA which manages and OWNs the connected socket

[19:47] < MS-> caller - which is the TCPServer

[19:48] < MS-> Now, the socket is provided for *information* purposes only

[19:48] < MS-> since it contains this information :

[19:48] < MS-> tcp4 0 0 192.168.2.9.22 84.201.167.186.43858 ESTABLISHED

[19:48] < MS-> ie 4 key pieces of information:

[19:49] < MS-> local address : 192.168.2.9

[19:49] < MS-> local port : 22

[19:49] < MS-> peer address : 84.201.167.186

[19:49] < MS-> peer port : 43858

[19:49] < MS-> And that's *all* the information the ServerCore has

[19:49] < MS-> But now it can make a decision

[19:50] < MS-> It's been told "this CSA is connected to someone"

[19:50] < MS-> "if you connect something else to it, that something else can send and receive bytes to/from them"

[19:50] < MS-> So it has to decide what to do - create something or not

[19:50] < MS-> It always chooses the former

[19:50] < MS-> and how to do that?

[19:51] < MS-> Well, it makes the call that you've now found

[19:51] < MS-> (self.protocol)(peer = sock_info["peer"],

[19:51] < MS-> peerport = sock_info["peerport"],

[19:51] < MS-> localip = sock_info["localip"],

[19:51] < MS-> localport = sock_info["localport"])

[19:51] < MS-> Now that *could* be written like this:

[19:52] < MS-> protocolHandlerMaker = self.protocol

[19:52] < MS-> protocolHandlerMaker ( peer = sock_info["peer"],

[19:52] < MS-> peerport = sock_info["peerport"],

[19:52] < MS-> localip = sock_info["localip"],

[19:52] < MS-> localport = sock_info["localport"])

[19:53] < MS-> But either way, it means "call this function, with all the information we have available, and ask it to create a component capable of understanding bytes in and bytes out"

[19:53] < MS-> and capable of running the protocol

[19:53] < MS-> whether it's request oriented or otherwise

[19:53] < MS-> After all, it may just choose to send out a stream of bytes

[19:54] < MS-> (like chargen for example)

[19:54] < MS-> Or it may just choose to take all the bytes and dump them to disk

[19:54] < MS-> Or ...

[19:54] < MS-> etc

[19:54] < j_baker> Interesting. So you could theoretically hook up any Kamaelia component to it.

[19:54] < MS-> But that's the core of lifecycle in a nutshell

[19:54] < MS-> Correct

[19:54] < MS-> Any

[19:54] < MS-> the output from a magnadoodle for example

[19:55] < MS-> as input to a voice recognition tool which feeds into a ticker

[19:55] < MS-> *any*

[19:56] < MS-> So, does that now make more sense?

[19:56] < MS-> This will amuse BTW

[19:56] < j_baker> So if it receives another connection on the socket before the last one completes, does it make another protocol handler and process them concurrently, or does it wait for the first one to complete?

[19:56] < j_baker> Yeah, that makes a lot more sense.

[19:56] < MS-> there's no reason why that "protocolHandlerMaker" could be a function that's a method of a class that has a wrapped LikeFile component to be used to do something interesting

[19:57] < MS-> That q goes back to the fundamentals of the system

[19:57] < MS-> think: what does the TCPServer do?

[19:57] < MS-> It creates a listener socket

[19:57] < MS-> waits for connections

[19:57] < MS-> when a connection occurs, it creates a CSA, activates it, throws it out an outbox and goes back to waiting for connection

[19:58] < MS-> The CSA and TCPServer run concurrently

[19:58] < MS-> So if there's 2 CSAs or 100 or ...

[19:58] < MS-> it makes no difference

[19:58] < MS-> Likewise the ServerCore is just waiting for "Is there a CSA? "

[19:58] < MS-> "There is?! Cool, let's wire up something to control it/talk to it"

[19:58] < MS-> "Ok, back to sleep"

[19:59] < MS-> Meaning it handles as many as it can simultaneously until you run out of resources

[19:59] < MS-> ie no limit & not sequential

[19:59] < MS-> The default is concurrent & scalable

[20:00] < MS-> The selector component is an optimisation BTW

[20:00] < MS-> You could have the same behaviour using threads

[20:00] < MS-> But without the selector

[20:00] < MS-> The selector being a *service* is another optimisation

[20:00] < j_baker> I spent a little time studying that trying to figure out why the IntelligentFileReader wouldn't close files.

[20:00] < MS-> It allows the selector to be shared between ServerCores etc

[20:01] < MS-> As well as TCPClient, Unix Process - rather than them creating their own etc

[20:02] < MS-> the interesting thing in a way, is that these components:

[20:02] < MS-> TCPServer

[20:02] < MS-> ConnectedSocketAdapter

[20:02] < MS-> Selector

[20:02] < MS-> ServerCore

[20:02] < MS-> Protocol Handlers

[20:02] < MS-> are all what you expect to see but they're rarely as clearly delineated or named

[20:02] < MS-> esp the ServerCore part.

[20:02] < MS-> Let alone the specialisations we can do

[20:03] *** vmlemon_ waits for a massively multiplayer version of the Bouncing Cat Game ;)

[20:03] < MS-> OK, if I may, I'll let you chew on that for a while since I ought to get food :-)

[20:03] < j_baker> Well, that clarifies things a little.

[20:03] < MS-> vmlemon_: Oh, you had to say it

[20:03] < MS-> It's mashed tomorrow

[20:03] < MS-> and you HAD to say it

[20:03] < vmlemon_> Hah

[20:03] < MS-> :)

[20:04] < MS-> OK, j_baker - I may have a use for the webserver

[20:04] < MS-> Like tonight/tomorrow

[20:04] < MS-> I might well just implement that for tomorrow

[20:04] < MS-> they will think I am mad

[20:04] < vmlemon_> It could be the new "World of Warcraft", or something like that ;)

[20:04] < MS-> World of Warcraft for children

[20:04] < j_baker> lol

[20:05] < MS-> :)

[20:05] < MS-> OK, I REALLY ought to get food

[20:05] < MS-> biab

[20:05] *** MS- really has to do that

[20:05] *** MS- is now known as MS-afk

[20:05] < vmlemon_> With surround sound?

[20:09] *** vmlemon_ is experimenting with D-BUS and CORBA/Bonobo over TCP/IP

[21:14] *** vmlemon__ has joined #kamaelia

[21:16] *** vmlemon__ is now known as vmlemon_

[21:29] < j_baker> Note to self: When putting debug print statements in code, make sure to make it print WHERE the debug statement is for future removal.

[21:31] < MS-afk> Commenting out is often a good idea BTW

[21:31] < MS-afk> rather than just deletion

[21:40] < MS-afk> BTW, regarding your changes in your mail...

[21:41] < MS-afk> I don't think they're sufficiently major/necessary for the immediate release, but if I get a chance I'll pull them in

[21:41] < MS-afk> (want to get multiprocess in)

[21:53] < j_baker> I understand. The only one that would be big enough to get in would be the deadlock issue.

[21:53] < j_baker> And even that isn't necessarily a huge deal.

[21:53] < MS-afk> That's the one I'll consider :)

[21:54] *** vmlemon_ has joined #kamaelia

[21:55] < j_baker> BTW, is it safe to call a component's stop() method twice in a row?

[21:58] < j_baker> Actually, I found a bug in the ServerCore. I have no idea how I didn't catch it when I merged the branch.

[21:59] < j_baker> At the end of the method is super(SimpleServer, self).stop()...

[22:01] < MS-afk> Ah thanks

[22:02] < MS-afk> fixed /merged

[22:03] < vmlemon_> o.O http://lolinator.com/lol/www.google.com/