"Last few weeks I was busy implementing CGL Map Reduce a streaming based map reduce implementation that uses a content dissemination network for all its communication. Our main objective behind this implementation is to avoid the overhead imposed by the technique adopted by both Google and Hadoop in their map-reduce implementations, that is communicate data via files." - Vlado Handziski
via Bookmarklet
This is good but I wish they'd made Hadoop use a pub-sub system so that they would be comparing pub-sub versus using a distributed file system. As it is they're comparing a different model (pub-sub) written in C++ with Hadoop's Java distributed file system. It's hard to tell if the performance differences they're seeing mean anything. - Adewale Oshineye
"The SWARM of autonomous beings by their very nature will have emergent and complex behavior. They will flock , flirt, dance and interact, and their actions will surprise and astonish even us, their creators. They are simple, but together they will behave in ways more complex than we can predict. A lot like LIFE ." - Vlado Handziski
via Bookmarklet
"Resource-constrained sensor gateways can use the WebSphere MQ Telemetry Transport (MQtt) protocol to communicate with the WebSphere MQ brokers. It is an open and lightweight publish/subscribe protocol designed specifically for remote telemetry applications and optimized for Networkings over low-bandwidth, high-cost networks. A Java implementation of the client side of the MQtt protocol can be downloaded from this location." - Vlado Handziski
via Bookmarklet
"In virtually every field of endeavor, most people learn quickly at first, then more slowly and then stop developing completely. Yet a few do improve for years and even decades, and go on to greatness." - Adewale Oshineye
Great tips. I hope they find their way in the next edition of the patterns book :) - Vlado Handziski
At some point in the next month I have to change: http://softwarecraftsmanship.o... to make more explicit reference to Ericsson's notions of "deliberate practice" and possibly the, out of print, book called Etudes for Programmers - Adewale Oshineye
Cool. I did not know that you have some of the content online! - Vlado Handziski
The plan is to put everything online once the book's published. - Adewale Oshineye
:( I hope we get some feedback from the FF guys on what happened... - Vlado Handziski
my guess is too many people. Scoble seems to think this is a better infrastructure or some bullshit like that... but my guess is that all these nets have the same problem of popularity stresses. we'll see now won't we? - AnotherⓃⓄⒶⒽ
it will be really interesting to see if FF tell us all what happend - h1ro
"these are the main reasons why we chose XMPP as the native message passing protocol for the Evergreen backend. It's very low overhead, extremely reliable and because it is connection oriented it has much lower latency and startup cost than HTTP for high-traffic environments"
More examples undermining the Restafarian Jihad - Adewale Oshineye
Wait, XMPP is *low overhead*? Sigh, what have we come to. - ⓞnor
XMPP is the "silver bullet" layer. It's complex enough that few people understand it, and so they can push all of their problem down to XMPP and imagine that it solves them :) - Paul Buchheit
Don't worry, the XMPP BOSH people will "solve" that problem also :) We keep piling levels of abstraction and hope that the architecture is going to hold. Until now the technology was always one step ahead, so we were never cornered into a position where we have to stop and rethink the basics. But those times are over. That is why all the next-Internet research is so important. That's why GENI is important, so we can test new ideas without the burdens of backwards compatibility... - Vlado Handziski
For a taste of these architectural explorations, google the following papers: "Making the World (of Communications) a Different Place", "A Data-Oriented (and Beyond) Network Architecture", "Towards a Modern Communications API", etc. - Vlado Handziski
Wait. You're tired of the layering in the XMPP stack, so the answer is to reinvent the entire Internet protocol? Anyway, isn't XMPP just packets, only slower? It's not like it's reliable or anything. - ⓞnor
What is the sudden interest in XMPP pubsub, AMQP, telling us? People find the interaction pattern and its semantics useful for their applications. They are data-oriented, they can benefit from the identity and time decoupling... So why are we still using the BSD socket as the de-facto API between the APP and the networking stack and build all these abstractions on the top. Maybe the time has come to re-base and promote pubsub as the basic transport level API? We in academia have to earn our bread somehow :) - Vlado Handziski
Isn't that called IP multicast? Nobody wanted it. - ⓞnor
In IP-multicast one needs to explicitly specify the network identities of the parties to communicate with (via the multicast address). The whole idea of pubsub is exactly not having to know the peer identity. i.e. one subscribes to a particular topic, or places a filter on the content of the notifications, etc. The optimal network layer implementation of this semantics would skip the service lookup step that would be needed if one tries to implement pubsub on top of raw IP multicast. - Vlado Handziski
Well, it's all totally moot, because it's clear that nobody's going to be changing IP any time soon. In any case, it seems like we should figure out what we would want at the application layer before anyone goes nuts redesigning TCP/IP. Sure, people are talking about pub/sub - in fact, people have been talking about pub/sub for approximately as long as there has been pub and/or sub - but it's not like we have a clear model that everyone loves. - ⓞnor
XMPP has a rather large weakness: there are very few implementations of many of the critical XEPs but it's proponents are fond of handwaving away problems by saying that there's a XEP that covers that. For instance this discussion: http://ralphm.jaiku.com/presen... with @BobWyman revealed that no-one seems to know where you can get hold of an open source XEP-0060 implementation. Since that's the pub-sub XEP I'm a little concerned. - Adewale Oshineye
@nor, we can decouple the interface and to some extent its semantics from the implementation, so we can use the existing protocol stack. All the MOMs have been doing this for a long time. Building directly on TCP like XMPP does, instead of tunneling everything over HTTP is a good first step towards reducing this overhead in the Web world. But down the road we will have to solve the problems we are facing today in a more systematic way. - Vlado Handziski
This Twisted-based code has not been touched for a long time, but it claims to have interoperated with ejabbered pub-sub at some time: http://code.google.com/p/pytho... . Still a long way to XEP0060 compliance, I guess... - Vlado Handziski
BTW, what is the status of Wokkel, ralphm's own implementation effort? - Vlado Handziski
Some people prefer simple solutions that work. Some people prefer theoretically ideal solutions. In general, the simple solutions are the ones that we actually use today, but the theoretically ideal solutions are expected to replace them as soon as we work out a few of the unexpected complications :) - Paul Buchheit
food for thought: Can SMTP be used for these scenarios? What are enabling features in XMPP that are not present in SMTP? - Gary Burd
@Vlado -- I'm skeptical of the argument that "building directly on TCP like XMPP does, instead of tunneling everything over HTTP is a good first step towards reducing this overhead in the Web world." The reason I'm skeptical is that URL-addressed HTTP/1.1 with Keep-Alive runs just about everywhere today, works well across existing infrastructure like firewalls and proxies and cache servers, and can be made to be more efficient on the server (comet, eventlets, greenlets). - DeWitt Clinton
DeWitt, as long as performance / scale is not a problem we can continue building our infrastructure on top of HTTP. After ajax and comet, now we even have TCP sockets in JavaScript (http://cometdaily.com/2008/07/...). What's next? Building web servers in our browsers? At some point this will have to stop. If sending XML over TCP can do the job, why go over HTTP? - Vlado Handziski
@Vlado - why go over HTTP? Because it is already supported by every server and client in the world, it provides well-known and well understood semantics for resource addressing (URLs), caching (If-Modified, Expires, Etags, etc), proxying (Location, X-Forwarded-For), authentication (Basic, Digest, OAuth), persistence (Keep-Alive), state (Cookies), representational transfer (GET, POST, PUT, DELETE, etc). Reinventing and redeploying that all over again seems like the wrong boulder to push up the hill. - DeWitt Clinton
@DeWitt: I am far from denying HTTP's success. It is currently our only viable workaround around several systemic shortcomings in the existing network architecture.But things would look different if we can redesign from scratch :) URLs are a good resource addressing scheme, so why are we not directly routing by them (yes, string matching can be made almost as efficient as IP matching)? Many applications want SUBSCRIBE/UNSUBSCRIBE, PUBLISH so why shoestring over GET/POST/PUT/DELETE semantics, etc... - Vlado Handziski
"As yet unseen footage of the first stable version of levelHead, an augmented-reality spatial-memory game by Julian Oliver.
Warning! This is a spoiler for the first 3 cubes: the easy Red Cube, the tricky Green Cube and the ridiculously difficult Orange Cube." - Vlado Handziski
via Bookmarklet
"Except Google might well be moving away from XMPP to a binary protocol for Google Talk on mobile devices, in Android, to reduce the communication overhead. While XMPP support will probably get back into Android, maybe a binary protocol (not this one) is better suited to high-volume notifications?" - Vlado Handziski
via Bookmarklet
""touched echo" is a minimal medial intervention in public space. The visitors of the Brühl's Terrace (Dresden, Germany) are taken back in time to the night of the terrible air raid on 13th February 1945. In their role as a performer they put themselves into the place of the people who shut their ears away from the noise of the explosions. While leaning on the balustrade the sound of airplanes and explosions is transmitted from the swinging balustrade through their arm directly into into the inner ear (bone conduction)." - Vlado Handziski
via Bookmarklet
"Smart people generally get very educated, and higher education (in the US at least) teaches only critical thinking. Smart people spend all their formative years getting rewarded for finding problems, for focusing on the negative. They leave school thinking that the way to be useful and show your smarts is to point out why things won’t work, rather than using some of those smart to find a way forward" - Adewale Oshineye
"So here's my first piece of advice: if you care about open source for the cloud, build on services that are designed to be federated rather than centralized. Architecture trumps licensing any time." -Tim O'Reilly. Just one snippet of wisdom included in what may be remembered as one of the most important pieces about the future of the web. Read this post twice. - DeWitt Clinton
via Bookmarklet
"The HDMI Extender includes a set of HDMI Transmitter and a HDMI Receiver and allows transmission of 1080P HD over 150ft over Ethernet cables. CAT 6 cable is recommended for 1080P and best performance." - Chris White
via Bookmarklet
I must be psychic, since I wired my house with 2 cat6 cables a couple years ago. :) - Chris White
It's just bits, right? Why do we keep inventing different, incompatible ways to move bits through wires? - ⓞnor
My electrician's A/V guy told me he usually wires high-end homes with component cable throughout the house. This seems archaic to me, and also requires a really thick conduit. Cat6, on the other hand, can be used for ethernet, phone, remote control, and now HD video. I'm really excited about this product. - Chris White
interesting - but I wonder - how far away exactly is most people's dvd players from their televisions to where this would be needed? And if so, seems like the wireless option will have more appeal http://news.cnet.com/8301-1078... - Jason Kaneshiro
:) Archaic is probably what I did 3 years ago when I used a combination of RG6 and Cat6 runs to all sockets. I had to build a custom patch panel to terminate all the video cables (the commercial solutions from the EU equivalents of Leviton, Lagrand were way too expensive). All cabling fits nicely in a 11U rack, but I use the RG6 only for distributing a satellite TV signal to all the rooms. I guess I should have used 2xCat6 instead :( - Vlado Handziski
It's not for DVD players, it's so you can have a single centralized media center PC with your movies and stuff on hard disk, and you don't have to try to make your PC run silent and look pretty under the TV, and you can drive all your TVs from a single box in the basement or a closet somewhere. - ⓞnor
@nor. Yes, exactly. I've built a media closet with all the wiring in the house feeding into it. There's a bundled cable you can pull to get 2 cat6 and 2 coax: http://www.broadbandutopia.com.... I also have all 20 speakers in the house terminating in the closet. - Chris White
I already do streaming from a computer to the entertainment system via WiFi. And also, don't have ethernet ports anywhere near the entertainment area. Running wires all over the place is very do not want. - Jason Kaneshiro
Jason, I'm trying to build a system that allows me to watch the highest quality HD video (1080p) in different rooms, including the kitchen, bedroom, and living room. I don't want different boxes in all those locations. The Belkin product looks interesting, I'll be interested in seeing if it can handle wireless interference and still deliver 1080p HD. - Chris White
However, I'm writing software to handle all this, and I want to do some reasonably sophisticated home automation stuff, so off the shelf solutions might not work well for me. That's why I'm excited about this extender. - Chris White
"What’s intriguing is the notion that we can learn about Ultra LARGE Scale issues by working at any extreme scale, where the extremes “break” the received software engineering wisdom." - Vlado Handziski
It is nice to see that people are starting to understand that the large scale Internet/Web and the Wireless Sensor Networks communities have a lot to learn from each other. - Vlado Handziski
"The changeable British weather may stop us from being outside as often as we might like, but it has been a spur to turning our homes into lightfilled spaces that capture a sense of outdoors inside." - RAPatton
via Bookmarklet
Very cool approach to modern architecture...I love natural light!! - Susan Beebe
I like this a lot. Our kitchen opens out on to a back deck which extends our living space into the backyard. - Mel "de Silentio" McB
What speaks against a simple IM setup using any of the Jabber clients out there? - Vlado Handziski
Nothing in particular - but is there a servce that will give me free IM to SMS? The keys here are the ability to subscribe to a specific stream, the fact that is has RSS output, and the fact that it can be delivered via SMS (and this can be controlled independently of the subscription). Anything else that can do those (and is an open platform as well) would be better than using Twitter. - Cameron Neylon
If free SMS is a requirement then there are not too many options. For the ones willing to pay, there are several commercial Jabber2SMS bridges: http://www.jabber.org/node/465 - Vlado Handziski
I have to admit the free SMS from Twitter is appealing - just because it means I don't have to spend money. That said it costs money to send SMS so you get what you pay for. Its important to have the messaging work when we're not necessarily at a computer. Ultimately building something using Jabber or just RSS that does exactly what we want will become appealling. - Cameron Neylon
"Dave Munger over at Cognitive Daily just wrote this post about people's lack of understanding of error bars in the graphical representation of data. The post is very interesting and I encourage people to take the quiz that he has posted on the correct interpretation of error bars." - Vlado Handziski
via Bookmarklet
"After many months in the making, we ‘re announcing the Mobile Industry Atlas. The Atlas is a visual map of who’s who in the mobile handset industry, available in glossy A2 wallchart and PDF format. This is a comprehensive map showcasing 400+ leading companies across 30 market sectors, spanning all major players involved from handset design through retailing including development and delivery of hardware, software, SIM cards, services and content." - Vlado Handziski
via Bookmarklet
"There is a personal motivation in proving that java technology can help solve many of the technical problems of the field. The HA field is very fragmented many many vendors, many standards, few dominant, a lot of proprietary standards. Using the modularity of JBoss we plan on supporting various such protocols out of the box." - Vlado Handziski
via Bookmarklet
I think that it should be clear by now that different applications will have different requirements in terms of the naming scheme as well as the underlying notification and subscription routing. The trade-off between the expressiveness of the naming model and the complexity of the matching, routing and forwarding will have to be selected in each case separately. - Vlado Handziski
"Kemp's model, developed with Massachusetts Institute of Technology cognitive scientist Josh Tenenbaum, runs data through an assortment of potential maps -- trees, linear orders, multidimensional spaces, rings, dominance hierarchies, cliques and so on -- and decides which type of relationship best fits the data." - Vlado Handziski
