[00:13] *** Trun has joined #kamaelia

[00:58] < Davbo> Hmm not sure I like the idea of something called Plugins in the tree

[00:59] < Davbo> i mean all Kamaelia components are designed to be "pluggable"

[06:55] < Lawouach_> morning

[07:30] < Lawouach_> Pablo did raise a good problem

[07:30] *** Lawouach_ scratches his head

[08:10] < mhrd> for ryan's basic web server, we made a subdirectory in the tree called 'handlers' - ie. Kamaelia.Protocol.HTTP.Handlers in Kamaelia.Protocol.HTTP - do something similar?

[08:10] < mhrd> plugs / modules / servlets (yuk) / wsgilets (yuk yuk!)

[08:21] < Lawouach_> ouch

[08:21] < Lawouach_> no *lets allowed!

[08:21] < mhrd> :)

[09:45] *** Chong- has joined #kamaelia

[10:17] *** Davbo has joined #kamaelia

[10:44] < Davbo> morning all

[11:12] < MS-> morning

[11:18] *** Uraeus has joined #kamaelia

[11:35] *** orphans has joined #kamaelia

[11:39] < Lawouach_> back

[11:41] < MS-> hi

[11:42] < MS-> will reply to your query shortly btw

[11:47] < Lawouach_> ta

[11:47] < Lawouach_> thanks for the feedback BTW

[11:47] < Lawouach_> Was quite informative

[12:05] < Davbo> Okay MS- starting to see where the problem is i'm having

[12:06] < Davbo> File "/usr/lib/python2.5/site-packages/Kamaelia/UI/Pygame/Display.py", line 309, in setDisplayService

[12:06] < Davbo> tracker.registerService("pygamedisplay", pygamedisplay, "notify")

[12:06] < Davbo> File "/usr/lib/python2.5/site-packages/Axon/CoordinatingAssistantTracker.py", line 205, in registerService

[12:06] < Davbo> raise ServiceAlreadyExists

[12:07] < Davbo> I'll look in Display.py, maybe I can attach an identifier to each service registered with CAT

[12:27] < MS-> Davbo: I had a look at that BTW and I came to the conclusion that you're *really* trying to do something that almost certainly won't work at presnet

[12:28] < Davbo> I bet it would work with a minor change to display.py so it doesn't always register a service by the same name

[12:28] < MS-> The fact the system is truly concurrent, rather than mainly concurrent makes a difference and there's a need to enable ordering - such that the display is created and activated *well* before anything else which is the real issu

[12:28] < MS-> There would be a huge number of knockons

[12:28] < Davbo> oh :(

[12:29] < MS-> for example - have you considered the knock on to the open gl components that register their own display for components ?

[12:30] < MS-> If you put that pygame display initialisation inside the component, you're actually really limiting your application to be 100% standalone

[12:30] < MS-> which would be a real shame

[12:30] < MS-> If you do what I was doing, then you don't

[12:30] < Davbo> yeah

[12:30] < MS-> Means for example your paint program could sit on open gl textures in a 3D space

[12:31] < MS-> Which would be an interesting alternative to different windows ...

[12:31] < MS-> :)

[12:31] < Davbo> you're right

[12:31] < Davbo> i was looking for a quick fix is all

[12:32] < Davbo> be nice if it would work as you had it

[12:34] < MS-> indeed

[12:34] < MS-> WHat didn't work the way I was doing things?

[12:35] < MS-> Ah, I guess messages weren't passing that you wanted to?

[12:35] < Davbo> Yeah

[12:35] < MS-> I guess that's because they reached the display config and went no where?

[12:35] < MS-> If that's the case you could add in a a means to copy from a to b

[12:35] < MS-> in that display config component

[12:35] < Davbo> ProcessGraphline ( Pipeline(A), Pipeline(B) )

[12:36] < Davbo> that setup you can't get from A to B

[12:36] < Davbo> if that makes sense

[12:36] < MS-> Should be able to

[12:36] < MS-> If

[12:36] < MS-> ProcessGraphline (A, B) works

[12:36] < MS-> ProcessGraphline (pipeline(A), pipeline(B) ) will work

[12:37] < MS-> Ponders if you realise something

[12:37] < MS-> Pipeline (A, B, C, D) behaves the same as Pipeline (Pipeline (A,B), Pipeline(C,D) )

[12:37] < MS-> And the same as

[12:38] < MS-> Pipeline (A, Pipeline (B,C) , D )

[12:38] < Davbo> Yeah I realise that it *should* work

[12:38] < Davbo> it's clearly the correct thing since it works with Graphline

[12:40] *** Davbo just found something really really cool

[12:40] < Davbo> If i do a Graphline and lay the Colour Selector over a paint surface it will constantly update your brush and you can draw over the colour selector

[12:41] < Davbo> Anyway, I digress. I don't see why it wont work MS-

[12:42] < Davbo> i'll put a console echo thingy in and see if i can work out what happens

[12:47] < Lawouach_> I like the fact Kamaelia.Utils always have what i need :)

[12:47] < Lawouach_> PassThrough it is today :)

[12:49] < Davbo> :-)

[12:49] < Davbo> it should be working MS-

[12:49] < Davbo> everything shows up in a console echoer

[12:52] < MS-> It does work

[12:52] < MS-> Just tested most combinations

[12:52] < MS-> https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/trunk/Sketches/MPS/Experiments/pipeline_test.py

[12:53] < MS-> Whatever you're doing is to do with your components, not to do with the Pipeline

[12:54] < Davbo> you're right it does work

[12:55] < Davbo> WINDOW1 = Pipeline(

[12:55] < Davbo> Paint(bgcolour=(100,100,172),position=(10,10), size = (500,500), transparent = True),

[12:55] < Davbo> ),

[12:55] < Davbo> works

[12:55] < Davbo> but with the DisplayConfig at the top no messages reach the Paint surface

[12:56] < Davbo> it doesn't seem to passthrough messages

[12:59] < MS-> Um, yes

[12:59] < MS-> that's because displayconfig has this definitions:

[12:59] < MS-> s/s$//

[12:59] < MS-> class DisplayConfig(Axon.Component.component):

[12:59] < MS-> width = 800

[12:59] < MS-> height = 480

[12:59] < MS-> def main(self):

[12:59] < MS-> pgd = PygameDisplay( width=self.width, height=self.height ).activate()

[12:59] < MS-> PygameDisplay.setDisplayService(pgd)

[12:59] < MS-> yield 1

[13:00] < MS-> You probably want to add some code there to pass through messages

[13:00] < MS-> eg a loop

[13:00] < MS-> something to check inbox/contol

[13:00] < MS-> something to send on

[13:00] < MS-> that sort of thing

[13:00] < MS-> details

[13:00] < MS-> ;-)

[13:00] < MS-> :-)

[13:00] < Davbo> Yeah I'm doing it atm

[13:00] < MS-> I wrote that to show how to change the window size really

[13:01] < MS-> Rather than do the whole shebang :)

[13:01] < Davbo> yeah it's my fault MS-, I got confused because it *seemed* to work with a Graphline

[13:01] < MS-> What you may want to look at is replacing

[13:01] < MS-> Pipeline with Seq actually

[13:01] < Davbo> but that was because i'd comment out the second display thinking "A regular Graphline can't do 2 windows so I don't need this"

[13:01] < MS-> http://edit.kamaelia.org/Components/pydoc/Kamaelia.Chassis.Seq.html

[13:01] < Davbo> but that was essentially solving the problem

[13:02] < Davbo> so I didn't see it

[13:02] < MS-> That may be what you want thinking about it

[13:02] < MS-> Not actually sure though :)

[13:02] < MS-> (matt wrote "seq" not me :)

[13:02] < MS-> Seq is definitely a nice possibility here though

[13:03] < Davbo> Hmm

[13:03] < MS-> Bonus points for naming the film referenced obliquely in those docs

[13:03] < Davbo> Can you pass messages to a component wrapped in a Seq?

[13:04] < Davbo> it's HAL!

[13:04] < Davbo> :-)

[13:04] < MS-> Davbo: Matt wrote it, I've not tried that

[13:04] < MS-> Maybe

[13:04] < MS-> :-)

[13:04] < Davbo> worth a try

[13:05] < Davbo> thanks for pointing me to this, got a few ideas

[13:05] < MS-> Could always be modified if necessary

[13:05] < MS-> self.link((self,"inbox"),(comp,"inbox"),passthrough=1),

[13:05] < MS-> Would imply yes

[13:06] < MS-> Doesn't pass through control/signal boxes which could be an issue

[13:06] < MS-> Not surprising given its intended usage

[13:07] *** MS- is now known as ms-afk

[13:10] < Davbo> Woo!

[13:10] < Davbo> ms-afk:

[13:10] < Davbo> it works :-)

[13:22] < ms-afk> *excellent*

[13:22] < ms-afk> That's what we like to hear :)

[13:22] < ms-afk> The sweet sound of success :)

[13:27] < mhrd> "14:05 - Davbo: it's HAL!" ... heh, someone's noticed :)

[13:28] < Lawouach_> success, luck, same difference

[13:30] < ms-afk> mhrd: I noticed first time round :)

[13:30] < mhrd> :)

[13:31] < ms-afk> But then you'd probably expect me to :)

[13:31] < mhrd> indeed :)

[13:31] < mhrd> Davbo: [each component] """is wired up so that the "inbox" inbox and "outbox" outbox are forwarded to the "inbox" inbox and "outbox" outbox of the Seq component itself.""" ... yes, you can send stuff to the seq'ed components' inboxes

[13:31] < mhrd> well, their "inbox" inbox

[13:31] < ms-afk> just not the control one

[13:31] < ms-afk> Minor missing detail :)

[13:32] < mhrd> hmm, not sure about teh control one - I can see that as rather useful, in some situations at least

[13:32] < ms-afk> (kinda surprisingly, but not too much given the context in which you wrote it)

[13:32] < ms-afk> Oh, it has a comment there:

[13:32] < ms-afk> linkages = [

[13:32] < ms-afk> self.link((comp,"outbox"),(self,"outbox"),passthrough=2),

[13:32] < ms-afk> self.link((self,"inbox"),(comp,"inbox"),passthrough=1),

[13:32] < ms-afk> # not linking signal-control, since we don't want downstream

[13:32] < ms-afk> # component to terminate prematurely

[13:32] < ms-afk> ]

[13:32] < mhrd> yeah - just spotted too

[13:33] < mhrd> but there probably is a good case for passing "control" in .... though it might lead to confusing behaviour if you treat the seq component as a black box ...

[13:33] < ms-afk> Not so sure actually.

[13:33] < mhrd> send it a shutdown message and wonder why it doesn't shutdown (because it has moved onto the next child in teh seq)

[13:34] < ms-afk> It's a chassis after all

[13:34] < ms-afk> Ah yes

[13:34] < ms-afk> That's an intriguing little issuette

[13:34] < ms-afk> But dealable with

[13:34] *** mhrd probably decided that was too difficult at the time - hence the cop out in the docs """This component ignores any messages sent to its "control" inbox"""

[13:35] < ms-afk> Indeed :)

[13:35] < ms-afk> Good copout

[13:35] < ms-afk> (ie, documented cop-out :-) )

[13:47] < Lawouach_> http://pastebin.org/47899

[13:48] < Lawouach_> Just for fun

[13:48] < Lawouach_> One way to map Qt and Kamaelia

[13:48] < Lawouach_> (similar to what Michael had done in the past I think)

[13:49] < Lawouach_> If you have PyQt4 installed you can run that code as-is, type a message in the first input box and it will be echoed back in a different input box

[13:49] < Lawouach_> Via the EchoeComponent

[13:50] < Lawouach_> Note that the signals methods are there for documentation purpose only

[13:50] < Lawouach_> PyQt doesn't actually care you have a method with that name

[13:50] < Lawouach_> signals are just string signatures

[13:51] < Lawouach_> The reason I'm doing this is because i'd like to start a Qt lib that maps headstock components

[13:51] < Lawouach_> In order to simplify the creation of XMPP-aware Qt application

[13:57] < Lawouach_> Note also that the QKamObject would be the base class of all Kam2Qt related classes.

[13:58] < Lawouach_> Currently I'm being sneaky since I create a PassThrough component to allow me to ease the delivering of data to the component associated with the Qt object.

[13:58] < Lawouach_> i.e., since I can't do: self.component.send(data, 'inbox')

[13:58] < Lawouach_> i do: somepassthrough.send(data, 'outbox')

[13:59] < Lawouach_> with its outbox being linked to the component according inbox

[13:59] < Lawouach_> (hence the loop through the linkages)

[14:00] < Lawouach_> feedback welcome :)

[14:12] < Lawouach_> It seems I scared people off :)

[14:12] *** ms-afk didn't notice the channel

[14:13] *** Lawouach_ wonders if he shouldn't use FanOut rather than several PassThrough

[14:13] < mhrd> re. cerenity/wiki : I've found tinymce works well in my favourite choice of browser :) http://tinymce.moxiecode.com/example_full.php?example=true

[14:13] *** ms-afk reads back

[14:14] < ms-afk> works here

[14:15] < ms-afk> Lawouach_: It working is always a good starting point :)

[14:15] *** ms-afk will have to take a proper link later though :)

[14:15] < Lawouach_> :)

[14:16] *** Trun has joined #kamaelia

[14:16] < Davbo> Works here too Lawouach_ looks good too

[14:18] < Lawouach_> Ultimately that could interest you guys to try out Qt in place of pygame. (I don't mean to change your project path now but later on... ;))

[14:19] < ms-afk> I quite like Qt - as mentioned before I have a wysiwyg tabbed simple html editor

[14:20] < Davbo> Qt is nice for UI elements but pygame has some nice stuff for me

[14:20] < Lawouach_> Davbo: out of curiosity, what are they?

[14:22] < Davbo> we already have some nice components for handing pygame surfaces and they're easy to draw on

[14:22] < Davbo> and are going to be the basis for my Layers which i'm working on atm

[14:22] < Davbo> :-)

[14:22] < Davbo> pygame surface == layer is how i'd like it

[14:22] < ms-afk> The transparency stuff should work particularly in your favour

[14:23] < Lawouach_> alright, sounds fair

[14:23] < Lawouach_> Though, playing devil's advocate, Qt integrates well with OpenGL, which should offer you more or less the same functionnality eventually.

[14:24] < ms-afk> I'm not so sure it'd be as simple/easy actually

[14:24] < ms-afk> But then with kamaelia it's rarely a choice of either/or

[14:25] < Lawouach_> :)

[14:25] < Lawouach_> Fair enough.

[14:25] < ms-afk> Qt would certainly be better for tools selection for Davbo really. But pygame only maximises portability. (in terms of dependencies)

[14:27] < Davbo> indeed

[14:28] < mhrd> < devils advocate> how about abstracting to run on top of either :-)

[14:28] *** ms-afk asks for moon on a sticl

[14:29] < ms-afk> s/l$/k/

[14:29] < Lawouach_> I'm not sure that's feasible. Well yes it is but it would be more or less not usable.

[14:29] < Lawouach_> pygame and Qt don't have the same scope.

[15:11] *** j_baker has joined #kamaelia

[15:40] < vmlemon> Hi

[15:47] < ms-afk> hi

[15:48] *** Lawouach_ doesn't know what to make of the fact people hack more and more into the browser what has been existing outside of the browser for years now.

[15:49] < Lawouach_> Like the orbited TCP socket hack

[15:49] < Lawouach_> I don't know if I should be excited or depressed.

[15:50] < Lawouach_> Mind you if I were to be excited by that idea I'd rather use Adobe Air or Microsoft Silverlight

[15:52] < ms-afk> orbited TCP socket ?

[15:52] *** ms-afk googles

[15:53] < Lawouach_> http://cometdaily.com/2008/07/01/sockets-in-the-browser/

[15:53] < mhrd> iiuc js library and standalone gatewaying app to provide TCP socket functionality within the browser

[15:53] < Lawouach_> which triggered http://jcalderone.livejournal.com/42060.html

[15:54] *** ms-afk was reading comet daily

[15:54] < Lawouach_> http://silverlight.idvsolutions.com/ < -- ahem.

[15:54] < Lawouach_> who talked about lock-in? :)

[15:54] < ms-afk> So it's just a proxy then really

[15:55] < Lawouach_> ms-afk: yes

[15:55] < ms-afk> So, nothing to stop you using them to "borrow" music from random other people and getting the RIAA getting evil on *their* arses then?

[15:56] < ms-afk> Or sending spam from their systems

[15:56] < Lawouach_> They use it to transport all kind of protocols (they name XMPP but I looked at their code and they support like a very small subset of the core protocol).

[15:56] < ms-afk> Sounds like they've invented a new way to be a moron

[15:56] < ms-afk> There's no way I'd provide an open proxy for arbitrary TCP Traffic

[15:57] < Lawouach_> I'm not sure they actually do but their technology would probably end up being abused that way.

[15:57] < Lawouach_> Anyhow, I'm not a big fan of Orbited personally.

[15:57] < ms-afk> It's the first thing I thought of

[15:57] < ms-afk> It's a great idea for enabling someone to seriously break your network connection.

[15:57] < Lawouach_> They brag a lot with little useful code from my own perspective.

[15:58] < ms-afk> "I know I'll use it to grab the entirety of wikipedia"

[15:58] < ms-afk> "that's a good idea"

[15:58] < ms-afk> Talk about open to DDOS

[15:59] < ms-afk> I like Brian H's comment

[15:59] < ms-afk> "Please, get over yourself."

[16:00] < ms-afk> I've not even looked at orbitd tbh

[16:01] < vmlemon> Hah, "Integrated PC Recovery Solution. Works Even When Windows Vista Doesn't"

[16:01] < ms-afk> Bragging without useful code is like driving a red sports car with a loud noisy engine that sounds like it's going at 100 mph when it's going at 35mph IMO

[16:01] < vmlemon> Hah

[16:02] < Lawouach_> j_baker: I'm going home but I'll ping you once back home to discuss your XMPP requirement

[16:02] < Lawouach_> We need to clarify certain things :)

[16:02] < j_baker> np

[16:03] < ms-afk> Oh I see

[16:03] < ms-afk> They claim "Orbited provides a pure JavaScript/HTML socket in the browser. It is a web router and firewall that allows you to integrate web applications with arbitrary back-end systems. You can implement any network protocol in the browser—without resorting to plugins."

[16:03] < ms-afk> Well, that sounds rather trivial

[16:04] < j_baker> Sounds to me like an XSSer's dream. :)

[16:05] < ms-afk> I kinda made that sort of comment above really

[16:05] < ms-afk> not quite the same comment, but similar

[16:05] < vmlemon> Urgh, http://www.phoenix.com/en/Products/Browse+by+Products/BIOS+Enhancements+and+Modules/TCSubscribe/default.htm

[16:06] < vmlemon> I'd hate to think about how a "pay as you use" computer would work

[16:06] *** ms-afk thinks he was a little more scathing

[16:07] < mhrd> just like having a phone line?

[16:07] < j_baker> Meh, I'd use it assuming it had sufficient security because I know what I'm doing. I don't think the vast majority of computer users do. :)

[16:07] < mhrd> (bill at the end of the month)

[16:07] *** j_baker works at a computer store.

[16:08] < ms-afk> Can't see any security

[16:08] *** vmlemon prefers having a machine that he knows won't magically stop working all of a sudden if recurring payments aren't made...

[16:08] < ms-afk> Ah, at least they do one thing sensible - build on top of twisted

[16:09] < ms-afk> building on our stuff would be sensible too, but at least they're not reinventing wheels

[16:09] < ms-afk> Well, not all the time anyway

[16:10] < j_baker> From what little bit of twisted I've seen, it isn't exactly the most readable code.

[16:10] < j_baker> But that seems to be common in the python world. I suppose everyone just assumes that sense python is such a readable language it absolves them of things like documentation.

[16:11] < j_baker> That's one of the big reasons why I'm avoiding using Python Paste to do more stuff with my project for example.

[16:11] < ms-afk> twisted is IMO well named

[16:11] < j_baker> lol

[16:11] < j_baker> FWIW, Kamaelia is probably the best documented python framework I've come across.

[16:12] < ms-afk> Python without docs is sometimes more readable than other languages with docs

[16:12] < ms-afk> Cheers :-)

[16:12] < ms-afk> Part of that comes from me setting one november as being a documentation month

[16:12] < ms-afk> in the spirit of nanowrimo

[16:12] < ms-afk> Matt *hugely* improved my docs that month

[16:12] < ms-afk> We did have lots of docs at that point though too :)

[16:13] < mhrd> yep

[16:13] < ms-afk> I see little point of code if people can't find docs on how to use it though

[16:13] < mhrd> mostly just went through bringing stuff up to the same level

[16:13] < mhrd> probably worth another pass at that, though will be must easier this time

[16:13] < ms-afk> Indeed. The more there is in a way the easier it gets since the patchy bits are easier to spot

[16:14] < ms-afk> aside from anything else the web access to the docs makes it easier to see

[16:14] < j_baker> It's definitely a good thing. Paste Deploy has a webpage basically saying "here's the function you should use to get at an app" and that's pretty much it. Pylons is much the same way.

[16:14] < ms-afk> That sort of thing drives me barmy

[16:14] < ms-afk> I did at one point want to try and write a cookbook entry each week really

[16:14] < mhrd> ms-afk: the autogen docs on edit.kamaelia.org - they're a snapshot, or they reflect the nightly builds?

[16:15] < ms-afk> It's a snapshot

[16:15] < ms-afk> I should integrate the copying really

[16:15] < ms-afk> since it'll be easier to do

[16:15] *** ms-afk will be much more sane after July 14th

[16:15] < ms-afk> (deadline)

[16:16] < ms-afk> (well, said deadline is the "too late" deadline)

[16:16] < ms-afk> (for practical terms the real deadline is between the 7th and 10th)

[16:17] < mhrd> ms-afk : forgot to mention - difference in axon docs - the master index is *not* autogenerated - so the STM and experimental/* docs are currently not linked to from anywhere

[16:17] < ms-afk> Ah cheer

[16:17] < ms-afk> s

[16:18] < mhrd> should be a quick'n'easy mod to __init__.py docs iirc

[16:19] *** mhrd offers to fix

[16:19] < mhrd> "likefile" is "background" now?

[16:20] < ms-afk> Handle & background

[16:20] < ms-afk> ooh cheers

[16:21] < ms-afk> https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/trunk/Code/Python/Axon/Examples/Handles/reverser.py

[16:21] < ms-afk> https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/trunk/Code/Python/Axon/Examples/Handles/TestHandle.py

[16:21] *** mhrd notes Handle.py describes itself as "experimental" ... should be in 'experimental' subdir?

[16:21] < ms-afk> Not really

[16:21] < ms-afk> It's less experimental ;)

[16:21] < ms-afk> :-D

[16:22] < mhrd> ought to not print its "polite notice" then?

[16:22] < ms-afk> (sorry)

[16:22] < mhrd> heh, np ... just asking q's :)

[16:22] < ms-afk> Not sure really

[16:22] < ms-afk> It's not as stable as I'd like

[16:22] *** ms-afk has to go afk for while NOW

[16:22] < ms-afk> biab

[16:22] < ms-afk> :)

[16:27] < j_baker> Am I the only one who has problems with google groups displaying the annoying "Discussions aren't available right now. We're sorry. Try again shortly." message?

[16:27] < j_baker> I'm guessing it's some kind of overactive flood control.

[16:28] < Lawouach> back

[16:40] < j_baker> So, what were you wanting to discuss Lawouach?

[16:41] < Lawouach> well

[16:41] < Lawouach> ServerCore(protocol=XMPPProtocol(routing), ...).run()

[16:41] < Lawouach> that won't happen :)

[16:42] < Lawouach> "Now, the idea is to have an HTTP gateway that will convert an HTTP request into XMPP somehow."

[16:42] < Lawouach> that could happen

[16:43] < j_baker> Hmmmm... do you think that would be a better place for me to start? On my schedule I've got it the other way around (work on the server first, then the gateway).

[16:43] < Lawouach> What I'd like is for you to precisely describe what you want to have at the minimum for now

[16:43] < Lawouach> So that I could help you out with headstock

[16:44] < j_baker> Well, what I would like to do is have the HTTP request in the first XMPP message the server gets.

[16:45] < j_baker> Ideally, I'd like to just pull the HTTP request out of the message, and then use the existing HTTP parsing stuff as a starting point.

[16:45] < j_baker> (that may or may not have to change once I start dealing with the body)

[16:45] < Lawouach> okay

[16:46] < Lawouach> that's doable

[16:46] < Lawouach> that being said, is there any existing client that'd be able to wrap an HTTP request in a XMPP message?

[16:47] < j_baker> I can't find anyone. But surely I can't be the first person that's thought this up. :)

[16:47] < j_baker> Perhaps I should check more on an XMPP IRC channel or mailing list.

[16:50] < Lawouach> I have a question, why not using XMPP all the way then? I haven't yet really understood why mixing HTTP and XMPP in such fashion would make things better.

[16:50] < Lawouach> Could you explain?

[16:51] < ms-afk> I personally think the best way of answering the question

[16:51] < ms-afk> Is to bear in mind that XMPP essentially is a message based system

[16:51] < j_baker> Well, that would be a good ending point, but I want to allow a person to be able to pull up the webpage in a browser without any additional software.

[16:51] < ms-afk> And messages are conveyed from a to b

[16:52] < ms-afk> and from b to a

[16:52] < ms-afk> and that as a result explaining what goes in what messages ("who says what")

[16:52] < ms-afk> would clarify things somewhat

[16:52] < Lawouach> right

[16:52] < ms-afk> After all, I have suspicions about what j_baker means, but it may not BE what j_baker means

[16:53] < Lawouach> Now from a use case point of view, how would the user interact with the application?

[16:53] < ms-afk> Paint a picture with a single example

[16:53] < Lawouach> yes please

[16:53] < j_baker> Well, the idea is that I'd be using an XMPP server to serve as the intermediary I had originally discussed.

[16:53] *** ms-afk goes back to being quiet

[16:53] < j_baker> So say user a wants to pull up a webpage that user b is hosting.

[16:53] < ms-afk> j_baker: Assume Lawouach isn't familiar with what you had originally proposed

[16:54] < j_baker> user b's webpage is hosted on the XMPP server www.foo.com and there's an HTTP gateway at http.foo.com.

[16:54] < Lawouach> hold on

[16:54] < Lawouach> user b's webpage is hosted on the XMPP server www.foo.com < -- this isn't clear for me

[16:54] < Lawouach> a webpage is served by HTTP

[16:54] < Lawouach> what is XMPP supposed to do here?

[16:55] < j_baker> Sorry, User B's webpage is hosted on his own computer and he connects to www.foo.com

[16:56] < j_baker> So then if User A wants to pull up User B's webpage, all he or she has to do is go to a url like say http.foo.com/User_B

[16:56] < j_baker> This sends the HTTP request to the HTTP gateway.

[16:56] < j_baker> The gateway puts the HTTP request into an XMPP message.

[16:57] < ms-afk> Can I but in and ask a few questions aimed at clarifying things

[16:57] < j_baker> sure

[16:57] < ms-afk> (Your statements aren't too precise)

[16:57] < ms-afk> We have 2 users alice and bob

[16:57] < ms-afk> alice is in the UK

[16:57] < ms-afk> Bob is in the USA

[16:57] < ms-afk> Both are average users who don't know about networking

[16:57] < ms-afk> They've signed up to a service "peertopublish"

[16:58] < ms-afk> and run the local kamaelia publish client

[16:58] < ms-afk> suppose

[16:58] < ms-afk> seem reasonable overlay?

[16:58] < j_baker> Well, not quite.

[16:58] < ms-afk> ish?

[16:58] < ms-afk> as a starting point?

[16:58] < j_baker> Not quite. :)

[16:59] < j_baker> Suppose that Alice wants to pull up Bob's webpage.

[16:59] < ms-afk> Hold on

[16:59] < ms-afk> stop

[16:59] < ms-afk> Let's get it clear what the actors in the system are before we talk about actions

[16:59] < ms-afk> alice and bob exist

[16:59] < ms-afk> both use a NAT ISP and neither's laptops are publically visible

[16:59] < mhrd> j_baker: "Well, not quite" applies to what precisely? :)

[17:00] < ms-afk> That's why they've signed up to a "kamaelia publish intermediary"

[17:00] < ms-afk> better ?

[17:00] < j_baker> You guys have to give me a moment to explain. :)

[17:00] < ms-afk> j_baker: You weren't being clear. (and I *think* I know where you're going)

[17:00] < ms-afk> That's why I'm asking

[17:00] < ms-afk> OK, simpler q's

[17:01] < ms-afk> Alice run's kamaelia publish, yes?

[17:01] < ms-afk> And has a normal browser ?

[17:01] < ms-afk> yes/no/ complete bollocks?

[17:01] < j_baker> Back to what I was saying, suppose Alice wants to pull up Bob's webpage (there's a reason I'm including this in the starting premise). The problem with your premise is that it assumes that Alice has to have special software to pull up Bob's webpage.

[17:01] < ms-afk> No, I'm picking a very very specific case

[17:01] < ms-afk> OK?

[17:02] < j_baker> ok

[17:02] < ms-afk> To explain what I think is going on

[17:02] < ms-afk> You have Alice running kamaelia publish, yes?

[17:02] < ms-afk> and Bob running kamaelia publish, yes ?

[17:02] < j_baker> Yes.

[17:02] < ms-afk> Neither have publically visible IP addresses

[17:02] < j_baker> Ok

[17:03] < ms-afk> They therefore BOTH have to use the intermediary

[17:03] < ms-afk> which is where the questions revolve around at the moment

[17:03] < ms-afk> Both are using a normal web browser

[17:03] < ms-afk> yes ?

[17:03] < j_baker> Yes.

[17:04] < ms-afk> That means they have to use the browser to access some third party (randomly called peertopublish.com )

[17:04] < j_baker> Ok.

[17:04] < ms-afk> in order to see any content published by either party

[17:04] < ms-afk> yes?

[17:04] < ms-afk> assume I'm a moron asking stupid questions

[17:05] < ms-afk> (the columbo principle of understanding)

[17:05] < j_baker> They don't HAVE to, but that's the best starting point I think.

[17:05] < ms-afk> k

[17:05] < Lawouach> ms-afk: finally!

[17:05] < Lawouach> ;)

[17:05] < ms-afk> :)

[17:05] < ms-afk> OK, so we have 2 people who can't actually see each other directly

[17:05] < j_baker> :P

[17:05] < ms-afk> and they therefore have a need to ask someone else to help them

[17:06] < ms-afk> OK, given the actors in the system

[17:06] < ms-afk> alice, bob, and the intermediary called peertopublish.com

[17:06] < ms-afk> what http URL does alice type in order to access bob's content?

[17:06] < ms-afk> (ignore what happens next)

[17:06] < ms-afk> And vice versa for bob

[17:07] < ms-afk> (bearing in mind the principle "the simplest possible thing that can possibly work")

[17:07] < ms-afk> just the url?

[17:07] < j_baker> Well, I don't think there would be any required URL scheme, but we'll start out with something along the lines of peertopublish.com/bob

[17:08] < Lawouach> right

[17:08] < Lawouach> what next?

[17:09] < ms-afk> So, alice goes to

[17:09] < ms-afk> http://peertopublish.com/bob

[17:10] < ms-afk> What does the intemediary actually do?

[17:10] < ms-afk> What *logical* request does that turn into

[17:10] < ms-afk> Does that *logically* turn into:

[17:10] < ms-afk> http://bobs_webserver/index.html

[17:10] < ms-afk> ?

[17:11] < j_baker> Sorry, I'm not sure quite what you mean by that. If you're asking if that's the ultimate goal, yes.

[17:11] < ms-afk> That's specifically what I'm asking yes

[17:11] < ms-afk> OK,

[17:11] < ms-afk> Now we know that's *one* side of the equation

[17:11] < ms-afk> The browser/access side

[17:12] < ms-afk> We haven't served any content, but clearly Alice needs to have told peertopublish *something*

[17:12] < ms-afk> right?

[17:12] < ms-afk> Or else peertopublish.com is stuffed

[17:12] < ms-afk> since it can't connect to her directly

[17:12] < ms-afk> correct?

[17:13] < ms-afk> (she's behind a nat, inside a company which has 3 firewalls and bullies for security)

[17:13] < j_baker> Other than an HTTP request, no (unless of course I'm misunderstanding something about XMPP).

[17:14] < j_baker> As far as Alice is concerned from a technical standpoint. peertopublish.com/bob actually IS Bob's webserver.

[17:14] < ms-afk> OK, so she is using her kamaelia publish client to connect to peertopublish.com's service

[17:14] < ms-afk> yes?

[17:14] < j_baker> Ok.

[17:14] < ms-afk> Gah sorry

[17:14] < ms-afk> Yes, I got confused

[17:14] < ms-afk> Alice is the client

[17:14] < ms-afk> Bob is the server

[17:14] < ms-afk> Alice clicks http://peertopublish.com/bob

[17:14] < ms-afk> and magic occurs

[17:15] < ms-afk> and gets his page which means http://bobs_webserver/index.html

[17:15] < j_baker> right

[17:15] < ms-afk> Peertopublish.com can't connect directly to bob, since like alice he's behind 3 natted gateways

[17:15] < ms-afk> But he can connect outwards

[17:15] < ms-afk> yes?

[17:15] < j_baker> Ok

[17:15] < ms-afk> Let's assume he's able to connect outbound using XMPP

[17:16] < ms-afk> Let's assume that peertopublish.com is running an XMPP server

[17:16] < ms-afk> is that correct?

[17:16] < j_baker> Yes.

[17:16] < ms-afk> k

[17:16] < ms-afk> So bob has a "conversation" (for want of a better phrase) open with the server at all times?

[17:16] < ms-afk> Whilst he's online?

[17:17] < j_baker> Right. XMPP is typically a persistent TCP connection.

[17:17] < ms-afk> And *locally* kamaelia publish also runs a WSGI based HTTP server

[17:17] < ms-afk> Bob's kamaelia publish this is

[17:17] < ms-afk> So he can browse his own stuff locally?

[17:17] < ms-afk> Assumption OK (for the moment) ?

[17:18] < j_baker> Ok, I don't know if that's going to be possible, but I'll go along for the sake of argument.

[17:18] < ms-afk> OK

[17:18] < ms-afk> So the question I think Lawouach has is this:

[17:18] < ms-afk> Given those pieces of code

[17:18] < ms-afk> And those actors in the system

[17:19] < ms-afk> What happens between Alice and peer, what happens between peer and bob's publish, what happens at that end point (regarding the wsgi server)

[17:19] < ms-afk> What gets sent back from bob's publish

[17:19] < ms-afk> How it's sent back from bob's publish to the peer

[17:19] < ms-afk> what the peer does when it recieves it

[17:19] < ms-afk> and what gets sent to alice

[17:19] < j_baker> By peer do you mean the XMPP server?

[17:19] < ms-afk> Lawouach Is that right ?

[17:20] < ms-afk> j_baker: peer == peertopublish.com

[17:20] < ms-afk> You need to explain any shifts between protocols

[17:20] < ms-afk> at each of those points

[17:20] < Lawouach> ms-afk: yes

[17:20] < ms-afk> I *THINK* I know what you mean/want

[17:20] < ms-afk> But I'm not certain

[17:20] < j_baker> Ok. Bob's computer sends back an XMPP message that contains the HTTP response.

[17:20] < ms-afk> hence the rather columbo style set of questions

[17:20] < ms-afk> (give examples

[17:20] < ms-afk> )

[17:21] < ms-afk> Of what get's sent

[17:21] < ms-afk> from alice, all the way through each point

[17:21] < ms-afk> it'll be much clearer

[17:21] < j_baker> Hmmm... I may have to use pastebin for something like this.

[17:21] < ms-afk> (You'll also be able to copy and paste the conversation , tart it up and call it docs then too :) )

[17:23] < j_baker> http://jason.baker.pastebin.com/d70bdc304

[17:23] < j_baker> Suppose that's the HTTP request that would normally be sent if Alice were just using normal HTTP.

[17:25] < ms-afk> That's a response

[17:25] < ms-afk> :)

[17:25] < j_baker> That's what I meant. s/request/response

[17:25] < ms-afk> No, what does alice send?

[17:25] < j_baker> Ok

[17:25] < ms-afk> Come on pretend I'm an idiot

[17:25] < ms-afk> :)

[17:25] < ms-afk> It's not that hard to imagine :)

[17:27] < j_baker> Ok, here's the request: http://jason.baker.pastebin.com/d6fd8ea80. I'm getting all this from here: http://www.jmarshall.com/easy/http/

[17:27] < j_baker> When Alice sends the request, it goes to the XMPP gateway

[17:27] < ms-afk> OK, so then what does peertopublish.com do when it get's that request?

[17:28] < ms-afk> the webserver code sends what message to what thing?

[17:28] *** mhrd is now known as hrd-afk

[17:28] *** hrd-afk is now known as mhrd-afk

[17:28] < j_baker> The gateway translates it into XMPP and then sends it to the server, which then sends it to Bob.

[17:29] < ms-afk> Assume that I know nothing about XMPP

[17:29] < ms-afk> which is pretty close to true aside from general things

[17:29] < ms-afk> I assume that XMPP messages can be (crassly) boiled down to

[17:30] < ms-afk> < xmpp_message_boiler_plate>

[17:30] < ms-afk> CONTENT

[17:30] < ms-afk> < /xmpp_message_boiler_plate>

[17:30] < ms-afk> (i did say crassly)

[17:30] < j_baker> Ok. The gateway would translate it into:

[17:30] < j_baker> < xmpp_message_boiler_plate>

[17:30] < j_baker> HTTP REQUEST

[17:31] < j_baker> < /xmpp_message_boiler_plate>

[17:31] < ms-afk> What is the CONTENT and I know that things are sent between ids, what ID is that message sent to

[17:31] < ms-afk> is HTTP REQUEST

[17:31] < ms-afk> The same as :

[17:31] < ms-afk> http://jason.baker.pastebin.com/d6fd8ea80

[17:31] < j_baker> Right.

[17:31] < ms-afk> or is it modified in anyway

[17:31] < ms-afk> ?

[17:31] < j_baker> Well, it may end up having to be modified somehow, but for now, I'm just going to go with no.

[17:31] < ms-afk> OK

[17:32] < ms-afk> So, stepping back, what ID is that message sent to?

[17:32] < j_baker> It's sent to Bob's ID.

[17:32] < ms-afk> OK

[17:32] < ms-afk> And what happens at Bob's end?

[17:33] < j_baker> Bob sends back the response in much the same fashion.

[17:33] < j_baker> < xmpp_message_boiler_plate>

[17:33] < j_baker> HTTP RESPONSE

[17:33] < j_baker> [blank line]

[17:33] < j_baker> BODY

[17:33] < j_baker> < /xmpp_message_boiler_plate>

[17:33] < j_baker> Of course sending the body may end up being more complicated.

[17:34] < ms-afk> And then what does the peertopublish.com server do?

[17:34] < j_baker> But this eventually gets sent to the gateway which removes the boiler plate and sends the HTTP response to Alice.

[17:35] < ms-afk> OK, Lawouach - there's your simple example :)

[17:36] < ms-afk> I think it should be enough for you two to be able to thrash out now :)

[17:36] < ms-afk> (I'm right in thinking that's one of the more awkward cases btw, yes?)

[17:37] < Lawouach> The gateway is peerpublish.com right?

[17:37] < ms-afk> yep

[17:37] *** ms-afk goes to cook dinner for the kids :)

[17:37] < Lawouach> so the gateway maps alice's HTTP request into a XMPP message and sends it to bob's XMPP server

[17:37] < Lawouach> that server returns the webpage wrapped into XMPP

[17:38] < Lawouach> and the gateway replies to alice with a regular HTTP response

[17:38] < Lawouach> right?

[17:38] < j_baker> Yes.

[17:38] < Lawouach> okay

[17:39] < Lawouach> Incidently bob doesn't need a XMPP server on his machine, he needs to be registered to a public server only

[17:39] < Lawouach> What bob needs on his machine is a client that understands what you're suggesting here

[17:40] < j_baker> Right. I was using server interchangably for the XMPP server and bob's server. That's probably not the best terminology.

[17:40] < Lawouach> in other words, able to extract the request from a XMPP message

[17:40] < Lawouach> that's okay now that I understand what you propose :)

[17:40] < j_baker> Awesome.

[17:41] < Lawouach> that's easy then :)

[17:41] < Lawouach> well easier anyway

[17:42] < j_baker> Yeah. I suppose now that I think about it, ServerCore could be a bad framework since it was made to have an individual connection for each request.

[17:45] < Lawouach> indeed

[17:46] < Lawouach> XMPP is not working like that as you said earlier it's a persistent connection

[17:48] < j_baker> Ok, so then if I understand headstock, ClientStream is basically where I would get the raw XMPP message?

[17:52] < Lawouach> right

[17:52] < Lawouach> let us look at the simplechat example

[17:53] < Lawouach> meh

[17:53] < Lawouach> defuze is down

[17:53] < Lawouach> right let me have diner first

[17:53] < Lawouach> i'll ping you back soon

[17:53] < j_baker> Sounds good.

[17:58] < orphans> mm, logs=interesting. I think I understand what Kamaelia Publish does a lot more now - it sounds v cool

[18:08] < Lawouach> back

[18:08] < j_baker> ty orphans.

[18:08] < Lawouach> damn defuze still down

[18:09] < j_baker> Strange. Aren't you using webfaction?

[18:09] < ms-afk> j_baker: note this:

[18:09] < ms-afk> (19:05:43) orphans: mm, logs=interesting. I think I understand what Kamaelia Publish does a lot more now - it sounds v cool

[18:09] < ms-afk> That's the key reason for me asking all those dumb q's

[18:10] < ms-afk> Once you explain all the dumb q's once everyone understands :)

[18:10] < orphans> yeah, and I'm pretty much a certified idiot :)

[18:10] < j_baker> I understand. Sometimes it's difficult for me to put nontrival technical stuff into words.

[18:11] < ms-afk> I know the feeling

[18:11] < ms-afk> For me that's actually a core part of GSOC

[18:11] < j_baker> Lawouach, if you give me a moment, I think I have a headstock repository checked out on my laptop.

[18:11] < ms-afk> helping people start doing that, and it really boils down to asking the columbo questions

[18:11] < ms-afk> Or answering them

[18:11] < ms-afk> even if it comes over as baby talk :)

[18:12] < ms-afk> And it *IS* hard :)

[18:14] < ms-afk> You should've seen some of the looks I got when I first started explaining my ideas for kamaelia at work :-D

[18:14] *** jason__ has joined #kamaelia

[18:15] *** jason__ is now known as j_baker-laptop

[18:16] < j_baker-laptop> Alright, I've got simplechat pulled up on my laptop.

[18:16] < j_baker-laptop> ms-afk: I could imagine.

[18:16] < j_baker-laptop> Kamaelia is kind of a lot to digest for someone who's used to programming a certain way.

[18:17] < Lawouach> indeed

[18:18] < Lawouach> j_baker-laptop: alright let's look at simplechat then

[18:18] < Lawouach> j_baker-laptop: do you have gmail account?

[18:18] < Lawouach> http://trac.defuze.org/wiki/headstock < -- back up

[18:19] < j_baker-laptop> I had an old one, but I'll have to see if I can remember the password.

[18:20] < Lawouach> that'd be handy because simplechat works against the google talk jabber network

[18:20] < j_baker-laptop> Hmmm... I may have to try that.

[18:20] < j_baker-laptop> I also have a webfaction hosting account. Is there any way to set up an XMPP server there?

[18:20] < ms-afk> Lawouach: does that mean if I log into gmail I should be able to talk to myself?

[18:21] < ms-afk> (so to speak)

[18:21] < Lawouach> j_baker-laptop: you'd have to install ejabberd on your server

[18:21] < Lawouach> though it's not difficult, it's much easier to use google talk for nbow

[18:21] < Lawouach> ms-afk: you can't talk to yourself with xmpp no

[18:21] < ms-afk> I was assuming 2 different user ids

[18:22] < Lawouach> but if you login to gmail you'll be able to use gtalk

[18:22] < Lawouach> yeah

[18:22] < Lawouach> totally

[18:22] < ms-afk> one in the browser, one not

[18:22] < ms-afk> cool

[18:22] < Lawouach> as long as they are contacts to each other

[18:22] < ms-afk> I'll have to have a play at some point

[18:22] < ms-afk> cool

[18:22] < Lawouach> j_baker-laptop: python simplechat.py -u gmail_user -p gmail_password -d gmail.com -a talk.google.com:5222 --usetls

[18:25] < j_baker-laptop> (just registered again for gmail, working on getting signed in)

[18:25] < Lawouach> k

[18:27] *** j_baker-laptop_ has joined #kamaelia

[18:28] < j_baker-laptop_> Ok, now I get a lot of XML. :)

[18:28] < Lawouach> ah :)

[18:29] < Lawouach> well now we're talkin'

[18:29] < j_baker-laptop_> So I'm guessing that's all the data passed back and forth between google's gmail server?

[18:29] < Lawouach> please open up the simplechat.py module

[18:29] < Lawouach> yes it is

[18:29] < Lawouach> (not everything is actually logged)

[18:30] < j_baker-laptop_> I've got simplechat up.

[18:30] < Lawouach> right

[18:30] < j_baker-laptop_> (the module that is)

[18:30] < Lawouach> so roughly what happens is this:

[18:31] < Lawouach> TCPClient < --> XMLIncrParser < --> ClientStream < --> *Dispatcher < --> *Handler

[18:31] < Lawouach> Handlers are per application

[18:31] < Lawouach> depending on what you want to do

[18:32] < j_baker-laptop_> So the handler takes care of incoming requests?

[18:32] < Lawouach> no

[18:32] < Lawouach> and you should stop thinking in terms of requests/responses when it comes to XMPP

[18:32] < Lawouach> But I guess you meant HTTP requests wrapped in an XMPP message

[18:32] < Lawouach> right?

[18:33] < j_baker-laptop_> No, I meant XMPP messages.

[18:33] < j_baker-laptop_> I'm just thinking too much in terms of HTTP. :)

[18:33] < Lawouach> :)

[18:34] < Lawouach> There is no request/response in XMPP

[18:34] < Lawouach> You open a TCP socket between two XMPP entities

[18:34] < Lawouach> and you start a stream

[18:34] < Lawouach> that stream uses XML to serialize the stanza passed back and forth

[18:35] < j_baker-laptop_> Alright.

[18:35] < Lawouach> okay

[18:35] < Lawouach> so

[18:36] < Lawouach> XMPP stanzas always have a XML namespace associated

[18:36] < Lawouach> that namespace tells you what kind of stanza you're dealing with

[18:36] < Lawouach> could be a message, a roster update, a pubsub event, etc.

[18:37] < Lawouach> what headstock does is to parse every incoming XMPP stanza into a bridge.Element instance

[18:37] < Lawouach> then ClientStream looks up for the qualified name of that parsed XML fragment (element name + element namespace)

[18:38] < Lawouach> if it finds a registered dispatcher for that combination it passes the bridge.Element instance on

[18:38] < Lawouach> which is then dealt by the dispatcher

[18:38] < Lawouach> dispatchers are components

[18:39] < j_baker-laptop_> That makes total sense to me.

[18:39] < Lawouach> Those components transform the bridge.Element instance into one of the headstock.api.* class equivalent

[18:39] < Lawouach> What is your job, as an application developer, is to link a handler component to that component dispatcher

[18:39] < Lawouach> and process the api.* instance

[18:40] < Lawouach> that's it :)

[18:40] < Lawouach> that's headstock

[18:40] < j_baker-laptop_> That clarifies things a great deal.

[18:40] < Lawouach> http://trac.defuze.org/browser/oss/headstock/headstock/example/simplechat/simplechat.py#L596

[18:40] < Lawouach> For instance here is how simplechat maps a message handler to a messgae dispatcher

[18:41] < Lawouach> http://trac.defuze.org/browser/oss/headstock/headstock/example/simplechat/simplechat.py#L132

[18:42] < Lawouach> here is the simplechat specific message handler

[18:42] < Lawouach> http://trac.defuze.org/browser/oss/headstock/headstock/api/im.py#L57

[18:42] < Lawouach> this is the instance your handler receives

[18:43] < Lawouach> if you look at the from_element static method you'll see the actual work required to map the bridge.Element instance to a headstock.api.im.Message instance

[18:43] < Lawouach> that's why headstock does it for you

[18:43] < Lawouach> it'd be painful for you otherwise

[18:43] < j_baker-laptop_> I agree.

[18:43] *** j_baker-laptop_ hates XML.

[18:43] < Lawouach> api.* is there to remove the XML verbosity off the actual data

[18:44] < Lawouach> right

[18:44] < Lawouach> so what you need then is to decide what you need from your own perspective

[18:44] < Lawouach> for instance start by mofidying the DummMessageHandler to print to the console the HTTP request you'll send within a message

[18:45] < Lawouach> then, move on to send back an HTTP response

[18:45] < Lawouach> once that works, we'll clean out what you don't need from headstock so that it matches your requirement a bit more

[18:46] < j_baker-laptop_> Alright. Headstock's pretty simple. It just seems pretty intimidating because some of the components can be somewhat complicated.

[18:47] < j_baker-laptop_> But I suppose that's what Kamaelia's for.

[18:47] < Lawouach> well headstock is intimidating because it breaks XMPP down a lot ending with lots of modules

[18:48] < j_baker-laptop_> Alright. I'll get to work on that.

[18:49] < j_baker-laptop_> One other somewhat unrelated general XMPP question:

[18:50] < j_baker-laptop_> Is a gateway basically a plugin of some kind for an XMPP server or is it totally independent of the server? (I understand that's moving further than where I am right now, but that's one thing that's been bugging me)

[18:51] < Lawouach> probably independant

[18:51] < Lawouach> The thing is a gateway is not that meanigful with XMPP

[18:52] < Lawouach> You may be connected to a server that will actually forward all your stanzas to another server rather than dealing with them

[18:52] < Lawouach> But that's probably the best I can see that'd match a gateway as you probably consider it

[18:53] < j_baker-laptop_> I see.

[18:53] < Lawouach> http://www.xmpp.org/extensions/ < -- the ONE place you always have to look first for any XMPP related questions

[18:54] < Lawouach> If there isn't an existing extension it either means you're asking the wrong question or you walking the wrong path :)

[18:54] < j_baker-laptop_> Yeah. I saw that. I saw a document there about how gateways interact with clients, but not server.

[18:54] < Lawouach> :)

[18:55] < j_baker-laptop_> Ok. I'll get to work on that today then. I'll let you know how it goes.

[18:55] < Lawouach> sure

[18:56] < Lawouach> If I'm correct, you should therefore set your environment as follow:

[18:57] < Lawouach> A local HTTP server using Kamaelia, CherryPy or whatever. That server should also run a XMPP client which has Bob in his roster list.

[18:57] < Lawouach> When Alice sends a HTTP request to that HTTP server, the server process uses its client to send a XMPP message to Bob containing the request.

[18:58] < Lawouach> The XMPP server forwards the message to Bob's XMPP client

[18:58] < Lawouach> Which returns back the webpage in response to the XMPP server.

[18:58] < Lawouach> Forwarding to the HTTP server process running the XMPP client the message containing the webpage

[18:58] < Lawouach> That HTTP server may finally respond to Alice's request

[19:00] < j_baker-laptop_> One other thing to keep in mind is that we'll also have headers to deal with at Bob's end. So it probably won't be QUITE as easy as just sending the webpage, but I see where you're coming from.

[19:00] < Lawouach> well it's your project ;)

[19:01] < j_baker-laptop_> (the headers being the ones that a WSGI app is required to send in start_response)

[19:01] < Lawouach> bera in mind also the latency induced by the XMPP roundtrip

[19:01] < Lawouach> Might bother Alice

[19:02] < j_baker-laptop_> That's one factor I hadn't considered.

[19:03] < j_baker-laptop_> The best solution is to have Alice and Bob communicate directly, but that would probably require some kind of special software on Alice's end.

[19:04] < j_baker-laptop_> But I can already hear ms-afk lecturing me on worrying about getting it working first, so I should probably just quit there. :)

[19:04] < Lawouach> ;)

[19:04] *** ms-afk suggests capturing this whole conversation into a subpage of KamaeliaPublish later on

[19:05] < ms-afk> heh, I was scrolled back reading stuff when I read that

[19:05] *** ms-afk just saw j_baker's comment

[19:05] < j_baker-laptop_> :)

[19:05] < ms-afk> Yeah, I would normally suggest something working. I'm predicatble

[19:06] < ms-afk> The latency might be an issue

[19:07] < ms-afk> but that can be rewritten using an ould word to be "could be an issue", and *ould words can often mean the opposite. I hate *ould words

[19:07] < ms-afk> eg mould

[19:07] < ms-afk> :-D

[19:07] < Lawouach> :)

[19:07] < j_baker-laptop_> Meh, we don't have to worry about that last one here in the US. :P

[19:08] < j_baker-laptop_> But I couldn't imagine it being too big an issue for just serving plain old webpages. Streaming media, bittorrent, etc may be a totally different issue though.

[19:09] < j_baker-laptop_> (the latency that is, not the mold/mould)

[19:09] < ms-afk> There really is only one way to find out

[19:09] < j_baker-laptop_> Yup.

[19:11] < ms-afk> Random q, expecting the answer to be "no, go away"

[19:12] < ms-afk> I don't suppose anyone here has written a pre-parser for python that allow you to use "end" keywords to signify blocks rather than use whitespace for indentation have they?

[19:12] < j_baker-laptop_> I thought you could already?

[19:12] < ms-afk> ?

[19:13] < ms-afk> You mean in normal python?

[19:13] < ms-afk> When did hell freeze over?

[19:13] < j_baker-laptop_> I swear I read somewhere that you can use end keywords as a response to the "whitespace is significant in python" argument.

[19:13] < ms-afk> It's not the sort of thing I'd expect in the python core

[19:13] < ms-afk> i see

[19:13] < ms-afk> That sounds wrong :)

[19:14] < ms-afk> But it'd be easy to do with a pre-parser

[19:15] < j_baker-laptop_> I could be totally wrong, because I can't find it anywhere.

[19:16] < ms-afk> http://www.secnetix.de/olli/Python/block_indentation.hawk

[19:16] < ms-afk> " Having said that, you can use keywords to indicate the end of a block (instead of indentation), such as "endif". These are not really Python keywords, but there is a tool that comes with Python which converts code using "end" keywords to correct indentation and removes those keywords. It can be used as a pre-processor to the Python compiler. However, no real Python programmer uses it, of course."

[19:16] < ms-afk> interesting

[19:16] *** ms-afk rummages

[19:17] < j_baker-laptop_> Ah. So I suppose we were both right.

[19:18] < ms-afk> :)

[19:26] < j_baker-laptop_> Did you guys see this: http://blogs.intel.com/research/2008/06/unwelcome_advice.php

[19:27] < ms-afk> it's called pindent.py btw

[19:27] *** ms-afk looks

[19:28] < ms-afk> No, I hadn't seen that

[19:28] < ms-afk> But intel were talking 100 cores about 2-3 years back at least

[19:29] < j_baker-laptop_> Part of me wants to say it's just marketing by intel.

[19:29] < ms-afk> AMD were saying similar things

[19:29] < ms-afk> The speak 'n write has 37 levels of concurrency inside it, but defaults to single core

[19:30] < ms-afk> changing all the pipelines to processpipelines would make it naturally multicore

[19:30] < ms-afk> Your code is probably even more parallel

[19:30] < ms-afk> The fun thing is we can probably go more concurrent than they're even thinking :-D

[19:31] < j_baker-laptop_> Well, as far as taking advantage of multicore processors? I thought you had to have a separate process to do that well in python?

[19:31] < j_baker-laptop_> Something about the GIL...

[19:31] < ms-afk> Yep

[19:31] < ms-afk> That's what Davbo's project is doing

[19:31] < ms-afk> It's showing how to *use* our multiprocess support :)

[19:31] < ms-afk> and working out bugs in it along the way

[19:31] < j_baker-laptop_> How is that code shaping up?

[19:31] < ms-afk> Looks quite nice

[19:32] < ms-afk> Most of the problems he's hitting are due to limitations in the stuff we've written

[19:32] < ms-afk> rather than anything fundamental in his understanding of kamaelia & the concurrency

[19:32] < ms-afk> which is *very much* the way round it should be

[19:32] < ms-afk> eg if you have a pipeline that looks like this:

[19:33] < ms-afk> Pipeline(A,B,C,D)

[19:33] < ms-afk> To make that use 5 processes you do this:

[19:33] < ms-afk> ProcessPipeline(A,B,C,D)

[19:33] < ms-afk> A,B,C & D get their own and the Pipeline gets the 5th

[19:33] < j_baker-laptop_> Interesting.

[19:34] < ms-afk> We've been designing for that scenario for a loooong while :)

[19:34] < ms-afk> I *hate* having to rewrite code simply due to something silly like a change in hardware architecture :-)

[19:35] *** orphans swears at regex very loudly

[19:35] < ms-afk> The addressing mechanism in the CELL CPU btw is either 5 or 6 bits. 6 IIRC

[19:36] < ms-afk> which means although current CELL CPU's have (essentially) 8 CPUS the basic hardware design (at least in terms of support) goes up to 64 cores

[19:36] < j_baker-laptop_> So each core gets its own address space too?

[19:37] < ms-afk> It gets some local address space

[19:37] < ms-afk> Not sure on how much

[19:38] < ms-afk> here you are - 3 years ago - the register's take on Intel's propoals for 100 core CPUs http://www.theregister.co.uk/2005/03/04/intel_100_core/

[19:39] < ms-afk> Though this could be harder to saturate:

[19:39] < ms-afk> http://www.theregister.co.uk/2007/07/26/sun_2048_thread_niagara/

[19:39] < j_baker-laptop_> I could see a hundred core CPU happening fairly quickly though. Not in the VERY near future, but somewhat soon.

[19:39] < j_baker-laptop_> At least if the number of cores ends up being a power of two like everything else in computer science.

[19:39] < ms-afk> http://techresearch.intel.com/articles/Tera-Scale/1449.htm - 80 core prototype

[19:40] < ms-afk> This incidentally is the problem I've been working towards making easy to work with

[19:40] *** orphans defeats regex. a hollow victory, but hard-fought

[19:40] < ms-afk> Or at least easy to maintain

[19:41] < ms-afk> orphans: Congratulations

[19:41] < j_baker-laptop_> Heh... isn't there a quote about regexes causing two problems?

[19:41] < ms-afk> Yeah, but it's only true if a your regex-fu is weak

[19:42] < orphans> mine is so weak it's unbelievable

[19:42] < orphans> although I can now banish revomit.py from my tree forever :)

[19:42] < ms-afk> :)

[19:46] < j_baker-laptop_> Yeah, complicated regexes and me just don't get along.

[19:52] < orphans> heh, truth be told it's a five character regex

[19:53] < orphans> straight out of how to write regex chapter 1

[19:53] < orphans> there's still a bug in it too :D

[19:53] < ms-afk> What's the regex?

[19:53] < ms-afk> what are you trying to match?

[19:53] *** ms-afk quite likes regexes

[19:53] < ms-afk> Probably why I still like perl as a language

[19:53] *** Lawouach loves the Office Space movie.

[19:54] < Lawouach> Rings so true :)

[19:54] < orphans> ok, it's basically a python version of your sdist building scripts (because I need them to do a bit more and prefer python to shell by a lot)

[19:55] < orphans> so I need to be able to match START, STARTSCRIPTS, STARTPACKAGES and STARTDATA

[19:55] < orphans> and then match the corresponding end strings

[19:56] < ms-afk> "(START|STARTSCRIPTS|STARTPACKAGES |STARTDATA) "

[19:56] < ms-afk> unless I'm missing something

[19:56] *** ms-afk notes a pesky space

[19:56] < orphans> if only it was that simple :)

[19:56] < orphans> because a lot of them are of the form:

[19:56] < orphans> "Kamaelia.Blah" #START

[19:57] < orphans> and I still need the Kamaelia.Blah

[19:57] < orphans> plus I need them all matching separately

[19:57] < orphans> hang on, I'll pastebin what I have so far

[19:57] < ms-afk> So you've wondering how you do this:

[19:57] < ms-afk> egrep -B1000 "# REPLACE" setup.py.src |grep -v "# REPLACE" > ../setup.py

[19:57] < ms-afk> egrep -A 1000 START ../../../Axon/setup.py|egrep -B1000 LAST >> ../setup.py

[19:57] < ms-afk> egrep -A 1000 START ../../../Kamaelia/setup.py|egrep -B1000 LAST >> ../setup.py

[19:57] < ms-afk> egrep -A1000 "# REPLACE" setup.py.src |grep -v "# REPLACE" >> ../setup.py

[19:57] < ms-afk> in python?

[19:58] < orphans> http://orphans.pastebin.com/d5c1d1138

[19:58] < j_baker> Lawouach, that's one of my favorite movies too.

[19:58] *** ms-afk has never heard of that film

[19:58] < orphans> ms-afk, yeah but allowing me to specify things like extra data, scripts etc

[19:59] < ms-afk> I see

[19:59] < ms-afk> If I was doing it python, I don't think I'd do it quite the same way

[19:59] < ms-afk> For starters the REPLACEME's would all get changed to something like

[19:59] < ms-afk> packages = [\

[20:00] < ms-afk> %(packages)s

[20:00] < ms-afk> ""],

[20:00] < orphans> yeah, I'm doing that

[20:00] < ms-afk> k

[20:00] < ms-afk> and then I'd just loop through the file

[20:00] < ms-afk> probably

[20:01] < orphans> mm, I was going to do that. the code seemed like it'd be pretty ugly though

[20:01] < orphans> not that what I just wrote isn't ugly...

[20:02] < orphans> anyway, it works well enough for the moment

[20:02] < Lawouach> ms-afk: You might want to watch it then :)

[20:03] < ms-afk> package_lines = []

[20:03] < ms-afk> doingpackages = False

[20:03] < ms-afk> for line in file("some_setup.pyfile"):

[20:03] < ms-afk> if not doingpackages:

[20:03] < ms-afk> if "STARTPACKAGES" in line:

[20:03] < ms-afk> package_lines.append(line)

[20:03] < ms-afk> doingpackages = True

[20:03] < ms-afk> else:

[20:03] < ms-afk> package_lines.append(line)

[20:03] < ms-afk> if "ENDPACKAGES" in line:

[20:03] < ms-afk> doingpackages = False

[20:03] < ms-afk> Lawouach: will do if it comes on :)

[20:04] *** ms-afk finds two films call that

[20:05] < ms-afk> Guesses it's not the 2 minute version

[20:05] < orphans> ms-afk, yeah, just thought it might end up looking like a massive switch statement. This was a while ago though, I'm pretty sure you're right looking at that :D

[20:05] < ms-afk> I have a tendency to go for simplicity

[20:05] < ms-afk> (says the man who decided that concurrency was the way to simplify his code...)

[20:06] < orphans> :)

[20:07] < orphans> I had some ill-conceived idea that I could do it elegantly using regex, forgetting my incredible lack of skill at using it :)

[20:15] < ms-afk> heh

[20:15] < ms-afk> You probably could FWIW

[20:15] < ms-afk> But I suspect it'd be quite painful

[20:16] < orphans> yeah

[20:17] < orphans> they'll never make a perl hacker out of me

[20:47] *** MS- has joined #kamaelia

[20:50] < j_baker> Hey MS-: perhaps you can enlighten me. Is there any big reason why HTTP uses CRLF to terminate each line instead of just a LF?

[20:52] < j_baker> I'm just curious because it would be easier for me to work with copying and pasting a request in a jabber client if I didn't have to worry about what the line ending was.

[20:59] < MS-> It's because CRLF is a network end of line in general really

[20:59] < MS-> Historical thing

[20:59] < MS-> My guess (without checking) is due to telnet

[20:59] < MS-> and telnet would do that due to teletypes

[21:00] < MS-> where carriage return/line feed themselves map back to what you get with typewriters

[21:01] < MS-> Where pressing the lever to the left gives you a line feed and then pushing it moves it back to the beginning of the line (being the carriage return)

[21:01] < MS-> It's a bit like "why are trains (in the UK for example) the width they are?"

[21:02] < MS-> And that goes back to "well, that's because the rails are that far apart"

[21:02] < j_baker> The only other thing I could think of is if you were to have a multi-line header. But I've yet to see that (in my limited experience).

[21:02] < MS-> And the reason they're that far apart is because they're the same width that wagons' wheels were nromally once upon a time

[21:03] < MS-> (You get multiline headers Mime, and you could do the same with HTTP, though you normally repeat the header)

[21:03] < MS-> the wagons' wheels were that far apart in the UK due to being pulled by horses 2 abreast

[21:03] < MS-> hence the width

[21:04] < MS-> Has knockons for road sizes etc

[21:04] < j_baker> I'd heard that. It even affects the size of rockets in the US.

[21:04] < MS-> If you do elsewhere - eg russia - you find a different gauge for trains -

[21:04] < MS-> the reason is because they used to do 3 horses abreast

[21:04] < MS-> It's the same sort of historical accident

[21:05] < j_baker> Alright. That's what I wanted to know.

[21:06] < MS-> But the key aspect to bear in mind is probably to enable telnet based testing

[21:06] < MS-> at a guess :)

[21:09] *** Trun has joined #kamaelia

[21:15] < orphans> *sigh* MS- this way of packaging is fiendish

[21:16] < MS-> ?

[21:16] < MS-> What's up?

[21:16] *** MS- isn't sure that's good or bad :-)

[21:17] < orphans> having to a) match combinations of #END and just END (if they're on the same line), b) match things on the same line, c) match STARTSCRIPTS and START as different things

[21:18] < orphans> makes the code ugly as sin

[21:18] *** MS- suspects you're making life more complex for yourself than you have to

[21:18] < orphans> heh, I'm 100% sure I am :)

[21:19] < j_baker> Are you just storing configuration data or is this some other kind of data?

[21:19] < MS-> Want to check in the current example?

[21:19] < orphans> mhm, will do

[21:19] < MS-> That you're fighting this is :)

[21:19] < orphans> actually, can I pastebin it - I need to fiddle around a bit with svn there

[21:19] < MS-> My primary reason for taking that approach is just to create a unified tarfile

[21:19] < MS-> pastebin is fine

[21:20] < orphans> http://orphans.pastebin.com/dd293d02

[21:20] < orphans> there's just a couple of corner cases which make it difficult

[21:21] < MS-> Ah no, I meant an example of the data you want to parse

[21:21] < orphans> oh, right, sorry

[21:21] < MS-> np

[21:22] < orphans> http://kamaelia.svn.sourceforge.net/viewvc/kamaelia/trunk/Sketches/JT/Jam/library/trunk/setup.py?revision=4613&view=markup

[21:22] < MS-> btw, you know that you can type

[21:22] < MS-> svn info ?

[21:22] < orphans> http://kamaelia.svn.sourceforge.net/viewvc/kamaelia/trunk/Sketches/JT/Jam/application/trunk/setup.py?view=markup

[21:22] < orphans> nope, didn't know that

[21:22] < MS-> I didn't either for a loooong time

[21:23] < orphans> at least a minute saved a day there :D

[21:23] < MS-> isn't the same thing

[21:23] < MS-> but useful

[21:23] < MS-> scripts = [\

[21:23] < MS-> "jam", #STARTSCRIPTS LASTSCRIPTS

[21:23] < MS-> ],

[21:23] < MS-> data_files = [("share/kamaelia",

[21:23] < MS-> [\

[21:23] < MS-> "PD/PureJam.pd", #STARTDATA LASTDATA

[21:23] < MS-> ]

[21:23] < MS-> )

[21:23] < MS-> ?

[21:24] < orphans> it's basically my setup file from the app so I can test out installing just the app without the library and vice-versa

[21:24] < MS-> scripts = [\

[21:24] < MS-> nb, the \ isn't needed there

[21:24] < orphans> ah, ok

[21:24] *** MS- nods

[21:25] < MS-> So

[21:25] < MS-> https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/trunk/Sketches/JT/Jam/application/trunk/setup.py

[21:25] < MS-> and

[21:25] < MS-> https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/trunk/Sketches/JT/Jam/library/trunk/setup.py

[21:26] < MS-> k

[21:26] < orphans> yeah, that and the axon and kamaelia setup files

[21:26] < orphans> basically it's a bit aggregator of setup files into one setup file

[21:27] *** MS- nods

[21:27] < orphans> I thought if I could get it working any way sensibly it might be a useful addition to your version, allowing you to use a bit more of distutils rather than just the modules stuff

[21:29] < MS-> So, you're looking for

[21:29] < MS-> scripts

[21:29] < MS-> data

[21:29] < MS-> packages

[21:29] < orphans> yup

[21:30] < orphans> it nearly gets it right now, it's just that START keeps matching positive for STARTPACKAGES

[21:30] < orphans> and the same with END and ENDPACKAGES

[21:31] < orphans> the simple fix is to split the line, and match word by word, but then you have to take into account the hash

[21:33] < MS-> or just flip the words

[21:36] < orphans> ahh

[21:45] < MS-> http://kamaelian.pastebin.com/m6cdbb533

[21:45] < MS-> Doesn't handle the Axon lines yet btw

[21:49] < MS-> There you go:

[21:49] < MS-> http://kamaelian.pastebin.com/m32ebb20c

[21:50] < MS-> That's actually quite nice

[21:52] < MS-> matcher == a thing that returns the lines between two regexes (which were originally just tokens)

[21:52] *** orphans bows to your infinitely superior hacking skills

[21:52] < orphans> that's amazing, cheers MS-

[21:53] < MS-> heh

[21:53] < MS-> yw

[21:54] < orphans> right, with my build system no longer making me cry tears of pure pain I think that'll do me for this evening :)

[21:55] < j_baker> Lawouach: ping? You still up?

[21:58] < orphans> night all

[22:08] *** bcarlyon|laptop has joined #kamaelia

[22:12] < MS-> as proof that you can make unreadable code in python:

[22:13] < MS-> http://kamaelian.pastebin.com/m3d337b95

[22:14] < MS-> Actually that's probably got a bug

[22:14] < MS-> http://kamaelian.pastebin.com/m5b47b631

[22:14] < MS-> is probably not buggy

[22:14] < MS-> actually no, it probably is

[22:16] < MS-> There, that won't be : http://kamaelian.pastebin.com/m7ab2acc0

[22:17] < j_baker> Dumb question: Wouldn't lines 8-12 cause an infinite loop?

[22:18] < MS-> Nope, because there's a yield there

[22:18] < MS-> The generators are run by this line:

[22:18] < MS-> [ X.next() for X in GS for line in file(filename)]

[22:19] < j_baker> Oh, ok. I get it.

[22:19] < MS-> It's not something I'd recommend in real code

[22:19] < j_baker> How haskell-esque of you. :)

[22:20] < MS-> Largely because whilst it works and correct, it's not obvious to a maintainer, and has no real benefit that being kinda fun to see how short I can make it

[22:20] < MS-> :)

[22:20] < MS-> I've written a several thousand line piece of code in SML in the past

[22:20] < MS-> (SML is another functional language)

[22:20] < MS-> So some habits claw through sometimes

[22:21] < j_baker> You know, the more I personally use some of python's functional tools, the more I think I'd be a Lisp programmer if the syntax wasn't so hideous.

[22:21] < MS-> Lisp is conceptually a nice language

[22:22] < MS-> I'd never use it though because there's just too many goddamn brackets

[22:22] < MS-> I mean, why do I want to write in an AST

[22:22] < MS-> when computers are very good at going from something readable to an AST

[22:23] < j_baker> Agreed. All the people that I've talked to who programmed in Lisp and most other functional languages always seem to not be very concerned about whether anyone else can read it.

[22:23] < j_baker> So it seems to be a good language for the "let me write my code and leave me alone" group of people.

[22:24] < MS-> maybe

[22:41] *** MS- sleeps

[22:41] < MS-> cya

[22:41] *** MS- has parted #kamaelia

[23:56] *** bcarlyon|laptop is now known as BCarlyon|Laptop