Congratulations, Tudor and Jeanette! You are both certainly invited to use my just-for-fun http://bebepool.com site if you were planning on having friends/family prognosticate about babby's arrival :) Cheers!
- Micah Wittman
I kind of wondered about that at the time -- there were some pretty silly assumptions in the original paper that violated the "assume the designers of the system you're attacking wouldn't do incredibly stupid things" principle.
- Joel Webber
Well, the authors of the original paper were mostly looking at commercial database systems, so that principle might not apply.
- Jim Norris
Fair point, Jim :) Although one of the authors is apparently involved in a company *making* a commercial database, so I guess that implies that he assumes himself to be unreasonable?
- Joel Webber
what original paper are you guys talking about? the original map reduce paper?
- Neha Narula
oh, the blog post. stonebraker is very entertaining :) i think that whole back-and-forth was a confusion over what the goals of map reduce are, and how they differ from the goals and uses of RDBMSs.
- Neha Narula
I think there's just some confusion over how much RDBMSs suck when you want to do something innovative. They want to control everything, and so of course you can say that that black box "could" be better than MapReduce in some theoretical way, but MapReduce scales from both a data and an engineering standpoint in ways that cause the more intricate and complex foundations of RDBMSs to collapse and beg for mercy.
- Jim Norris
that's because traditional RDBMSs do waaay more. Like joins. and most of SQL. I think old-school db researchers get annoyed because they thought map-reduce like things way back in the 80s, but didn't build them out, and forged ahead to make what you describe as these black boxes instead.
- Neha Narula
Neha, that's fair. What's not is attacking new systems (with pseudo ad-hominems to boot) without putting enough of an effort to understand why they might be useful - perhaps in under different environments than what the old systems were suitable for.
- Ashwin Bharambe
What seems strange to me is that, given how hard you have to squint to make an analysis tool like MR look like a database, that they would bother to attack it at all. Unless they truly believe that data should only ever be analyzed in an RDBMS, which is a pretty absurd stance to take.
- Joel Webber
from BuddyFeed
That's why I say the conflict is really confusion over goals. I definitely have heard people talk about Hadoop replacing RDBMSs, and I'm not even sure what that means. They are two very different tools. I think the authors of the blog post were starting from the premise that people thought that map reduce could *replace* a traditional RDBMS, and they were pointing out why it couldn't....
more...
- Neha Narula
Amazon Web Services Developer Community : Announcing Beta Support for Versioning in US West (Northern California) Region - http://developer.amazonwebserv...
"We are pleased to announce the availability of the Versioning feature for beta use with buckets in our Northern California Region. Versioning allows you to preserve, retrieve, and restore every version of every object in an Amazon S3 bucket. Once enabled for a bucket, Amazon S3 preserves existing objects anytime you perform a PUT, POST, COPY, or DELETE operation on them. By default, GET requests will retrieve the most recently written version. Older versions of an overwritten or deleted object can be retrieved by specifying a version in the request. "
- DeWitt Clinton
from Bookmarklet
"What I notice is that my peers are progressing to more and more complicated and convoluted designs. They are impressed with the flashiest APIs, the biggest buzzwords, and the most intricate of useless features. They are more than happy to write endless unit tests to test their endless refactoring all the while claiming that they follow XP’s “the simplest thing that works” mantra. I’ve actually seen a guy take a single class that did nothing more than encapsulate the addition of two strings, and somehow “refactor” it to be four classes and two interfaces. How is this improving things? How can more somehow equal simpler? This should never be the case. These are the actions of an expert. These experts are very smart, capable, and skilled, but they are too busy impressing everyone to realize that their actions are only making things worse for themselves. In the end all of their impressive designs are doing nothing but making more work for themselves and everyone around them. It’s as if...
more...
- Paul Buchheit
from Bookmarklet
This applies to experts in any field.
- WorldofHiglet
It takes smart people to make complicated things simple.
- imabonehead
Is it possible he's talking about Java programmers?
- Gabe
i really liked this post (it resonated with me) until the end, at which point i felt alienated.
- Neha Narula
What alienated you, Neha? To me, it seemed valid enough but a bit overwrought and trite. I know plenty of experienced, skilled working programmers who value just-get-it-done simplicity -- the "professional master" doesn't seem that elusive.
- ⓞnor
from Android
I'm a big fan of keeping it simple, but some problems do require a thorough approach.
- Andrew C
"In contrast there are masters in the martial arts who learned their art as a means of survival and became masters in a realistic and hostile environment. We don't have anyone like this in the programming profession, " ... what about Carmack and Abrash & co?
- Andrew C
BTW, I dunno if this is what put Neha off, but it almost sounds like Shaw wants to deny the reality of a nice O(n log n) solution beating out an O(n^2) solution (assuming small k, whatever) on a problem of decent size.
- Andrew C
I mean, the stories of the martial arts masters may involve simple-looking moves, but they are also (in the stories) _perfectly_ executed, the product of careful observation of one's opponent and expert timing and precise angles. You might be able to pare down a simple linked list to the bare essentials, but I don't think it's quite analogous to not using a more complex structure _where appropriate_.
- Andrew C
Nice... "The main thing I noticed about the experts I’ve encountered is they are into impressing you with their abilities. They are usually incredibly good, but their need for recognition gets in the way of mastery. Everything they do is an attempt to prove themselves and in order to do this they must perform like an actor on stage. There’s nothing wrong with this, and I don’t think the...
more...
- Ken Sheppardson
Andrew: Maybe the point was that an Expert would say "Aha! You need to keep these items in order, so a self-balancing tree is the perfect solution.", while a Master would say "Ah, but you never have more than 5 items, so a linked-list will always be faster!"
- Gabe
this part, so much guy/son stuff! i dislike superfluous interfaces as much as anybody else: “There was this guy I worked with who once optimized a complicated red- black tree getting 300% performance boost. I was baffled and ask, 'How’d you do that? That’s impossible.’ To which he responded…” “'That’s my linked list my son.’”
- Neha Narula
This is the kind of crap that gives java such a bad image. It used to be that people used it for what it was -- a simple OO language with garbage collection and a fast VM. Now you have architecture astronauts going off the deep end and making everyone assume the language has to be that way. I believe this disease stems from people who focus more on the process than on the product of their work. That's a recipe for disaster in my book.
- Joel Webber
from BuddyFeed
Neha: So lt's the fact that the language is male?
- ⓞnor
from Android
The impulse is good, but people have such different senses of what is simple, what has quality, what flows with the Tao. It's like beauty that way. What the story doesn't say is the 300% performance boost was on a limited test data set, in the real world it performed 3x worse and all the complexity had a reason that made sense once you "know." :-)
- Todd Hoff
Complexity that's "there for a reason" is the worst kind. But who even talks about red-black trees vs linked lists? TreeMap vs LinkedList isn't the issue, interface swaddling and hyperfine dependency injection is the issue. Thing is, fights are decisively won, but code maintainability is much harder to measure, and even the importance of performance can be disputed.
- ⓞnor
from Android
I find it funny how the article, while praising simple approach, suffers from superfluity of language.
- andrei_c
Neha, I thought the final "That's my linked list my son" was to make clear the parallel with the earlier quote "That was my foot my son" from Mestre Bimba.
- Ruchira S. Datta
Todd: Imagine the situation where you are storing data for the US Census, and need to keep track of the people in a household by age. Since it's sorted and unbounded (there's no maximum number of children a family can have), you can easily think that a nice O(n lg n) algorithm that keeps a balanced binary tree is the right way to go. However, if you bother to look at the data, you'd see...
more...
- Gabe
I wish I could "Like" this article again :)
- scott willeke
might have created a "MEGA-liked" button:)
- alex melnikov
It's a great analogy, but in reality, the martial arts stuff is mythology. Wing Chun proponents often talk about simplicity of the art, but they'd get their butts kicked in a sloppy street fight because invariably, most real world fights are messy, quickly go to the ground, and result in grappling and choking and eye gouging. Bullshido has lots of examples of this. The 80 year old guy...
more...
- Ray Cromwell
Har har.... I assume there is ZERO chance of this ever happening.
- Jay
@Jay - I suspect Paul wouldn't ask if he thought there was zero chance. Last time I remember Paul asking for something from Google to be open sourced it was their JS compiler. That took a while, but http://code.google.com/closure...
- Nick Lothian
@Jay: Remember that Paul's referring to (relatively) generic infrastructure here, not search ranking code. But I think Daniel's right that it would be a *lot* of work, since most Google infrastructure is not "productized" and easy to wrap up in a bow for public release. Like any company with a lot of infrastructure, there are a lot of interdependencies that would be difficult to untangle. I think it would probably be better to simply publish papers on how it works, as with GFS, BigTable, etc.
- Joel Webber
Boy, that would be a bold move Paul. Agreed that it would help out many though!
- manielse (Mark Nielsen)
How about just dumping the source to the web without all the dependencies, even if it doesn't even compile? If it looks useful enough there's a good chance someone would adopt it.
- Jim Norris
I suspect you're right about that, Jim. But Google would probably catch more crap about a "throwing it over the wall and letting it stagnate" open-sourcing than it's worth. But maybe I'm just down on it because Google catches crap no matter what these days...
- Joel Webber
It's hard to open source distributed algorithms -- there's no obvious public standard to use, and the reasonable choices (TCP sockets? MPI?) are nothing like Google's internal infrastructure. I think a paper would be more useful than source code, the way MapReduce papers lead to Hadoop. Paul, have you looked at Vowpal Wabbit (http://hunch.net/~vw/)? It has experimental support for cluster parallelism, and I hear good things about it.
- ⓞnor
Well, it doesn't have to be an either/or issue.
- Jim Norris
If the code is too hard to separate from the infrastructure, then maybe a compute service like EC2 that provides an application interface specifically for solving problems with SETI could be good for both the world and good for the Google.
- Bill Strathearn
@Bill: Now *that* sounds like a good idea to me, especially if accompanied by a paper describing the algorithms in use.
- Joel Webber
Although technically really interesting and usefully, I do think that Google will not open source or even give inside information about such a key differentiating technology in the hands of their competitors. But I agree that it would be really great for the world.
- yusuf arslan
The value of "differentiating technology" is not in novel algorithms, but in the thousands of places where implementations of these algorithms have been fixed and customized and tuned to solve the problem at hand -- which wouldn't have to be described in a whitepaper.
- Tudor Bosman
@Tudor Bosman: I do not agree that the competitive advantage is the knowledge of fine tuning and implementing the algorithms. The concept/design of Google's machine learning infrastructure is very important. Don't get me wrong, I do think that Google SHOULD open source this. But I think they WILL not because of business considerations.
- yusuf arslan
I think even just a paper would be very useful and fruitful.
- Ruchira S. Datta
Ruchira, do you have particular biological datasets in mind? Are they too large even for a single machine version of Vowpal Wabbit to comfortably cope with (say, several tens or hundreds of billions of labeled instances)?
- Simon
Simon, my comment was just general. Now that you mention it, I could use Vowpal Wabbit on some of my projects, which are not anywhere close to that big. Thanks for the tip!
- Ruchira S. Datta
"... If this maintenance was not done quickly, our operations team feared that the service might go down for days. With the whole team on edge, my colleague Jessica Verrilli and I called our head of operations to convince him to do what was deemed extremely difficult if not impossible — reschedule the maintenance."
- Louis Gray
from Bookmarklet
It's pretty crazy. I hadn't checked Google Wave for a 5-6 weeks and now when I logged back in pretty much everyone I know seems to have an account. Not much activity from any of them though :( However I did get a couple of waves/messages(?) but didn't know about them since the wave did not send me email :)
Same here, nobody seems to be interested except about giving invite to others. Google wave has failed to live up to its hype for now.
- ashish
from iPhone
When people first started to use email, did they expect a letter through the post or a telephone call to tell them they had something in their inbox?
- Tony Ruscoe
from fftogo
Tony, I am not sure Wave aspires to replace email ... does it? If so, I am not really clear why/how? Email follows a simple universal protocol and pretty much everyone today can send and receive email. Wave has to come up with something much more compelling for people to switch. Even for one person to use Wave as an email client - wave needs to make sure everyone else the person will...
more...
- Bindu Reddy
No, much like email didn't replace the postal service or telephone calls. Email assumes that anyone you want to contact by email has an email address. That's commonplace these days but wasn't when email was first introduced. You're right that it would probably help to have email alerts, but people using Wave regularly don't need that. Personally, I think a common interface for Gmail and Google Wave would help increase its usage and awareness.
- Tony Ruscoe
from fftogo
There are people in my life who call me to tell me that they've sent me an email. And there are people who I email to tell them that I've updated their wave.
- Amit Patel
i think they should integrate wave with gmail - if you have gmail account, you are a waveist. ..otherwise, i wonder how is it going to catch on ... like friend connect? - almost nobody uses it.
- Petr Buben
Actually I think Google’s Wave project came out so the developers could get some feedback from it’s users and now they’re off correcting the bugs and implementing some of the many suggestions users gave them. I expect an updated, improved and much more friendly to come out of the labs soon.
- Tom Awtry
"We don't know where this is coming from so take it with a grain of salt, but rumor has it from AndroidSPIN that the progenitor Android device, the T-Mobile G1, is destined to officially get an over-the-air update for an Eclair build, Android 2-point-something. It's not specified which version -- whether it's 2.0 or the Nexus One-tastic 2.1 -- will be coming out, but if we had our say, always shoot for the latest and greatest."
- Cristo
from Bookmarklet
"For thousands of years astrologers have claimed that disasters are foretold in the movement of the planets.Now a new study suggests they might be right. Scientists have discovered that the faint gravitational tug of the sun and moon can set off tremors deep underground in one of the world's most dangerous earthquake zones. Although the pull of planetary objects is too weak to set off a full blown quake, the findings suggest that they could set in motion a chain of events, leading to devastation on the surface. The findings come from a study of the San Andreas Fault - the infamous crack in the ground which triggered the 1906 San Francisco earthquake and the deadly fires that followed. The fault marks the boundary of the Pacific and North America tectonic plates and runs 800 miles from the southern California desert to northern California. American earthquake experts compared records of 2,000 small tremors in the Parkfield region 170 miles north west of Los Angeles with the movement of...
more...
- RAPatton
from Bookmarklet
Sure they can cause earthquakes, but that pales in comparison to making Eddie sparkle and Jake transform
- RAPatton
from iPhone
Unveiled: China's 245mph train service is the world's fastest... and it was completed in just FOUR years | Mail Online - http://www.dailymail.co.uk/news...
"In the week that Britain's high speed rail link closed down because the wrong sort of snow interfered with the engine's electronics, China unveiled the world's fastest train service on one of the coldest days of the year.Days after thousands of passengers were left stranded when Eurostar services were cancelled, China's new system connects the modern cities of Guangzhou and Wuhan at an average speed of 217mph - and it took just four years to build.The super-high-speed train reduces the 664-mile journey to just a three-hour ride and cuts the previous journey time by more than seven-and-a-half hours, the official Xinhua news agency said.Work on the project began in 2005 as part of plans to expand a high-speed network aimed at eventually linking Guangzhou, a business hub in southern China near Hong Kong, with the capital Beijing, Xinhua added. newsHome News Sport TV&Showbiz Femail Health Science&Tech Money Debate Coffee Break Property Motoring Travel News Home World news Headlines...
more...
- RAPatton
from Bookmarklet
"Inventor Le Trung spent Christmas Day with the most important woman in his life - his robot Aiko. The science genius enjoyed a festive dinner with his mum, dad and his £30,000 fembot which he designed and built by hand. Le, 34, from Brampton, Ontario, Canada, even bought gifts for his dream girl, who is so lifelike she speaks fluent English and Japanese, helped cook the turkey and hang up decorations. 'Aiko is like any woman, she enjoys getting new clothes,' he said. 'I loved buying them for her too.'"
- RAPatton
from Bookmarklet
"Le has built up huge debts working on his fembot and is still trying to find a technology company to sponsor his research. But in the meantime he has had to move back in with his parents and they will all be spending the festive period and New Year together. 'My family found it a bit odd at first, but now they all love Aiko,' said Le. 'My mum and dad chat away to her. It helps by...
more...
- RAPatton
She doesn't look happy to me; has anyone projected their own feelings and agendas on the fembot and argued what is best for her?
- RAPatton
I would find it very disconcerting if someone showed up for dinner with a robot as a date.
- John (a.k.a. dendroica)
"The production of new proteins can only occur when the RNA that will make the required proteins is turned on. Until then, the RNA is "locked up" by a silencing molecule, which is a micro RNA. The RNA and micro RNA are part of a package that includes several other proteins...When synapses got activated, one of the proteins wrapped around that silencing complex gets degraded." When the signal comes in, the wrapping protein degrades or gets fragmented. Then the RNA is suddenly free to synthesize a new protein."
- bob
from Bookmarklet
"Barry Schwartz makes a passionate call for "practical wisdom" as an antidote to a society gone mad with bureaucracy. He argues powerfully that rules often fail us, incentives often backfire, and practical, everyday wisdom will help rebuild our world."
- Paul Buchheit
from Bookmarklet
Rational, Twitter usage is flattening, people are finally figuring out that Twitter sucks for having a conversation. People though Friendfeed was hard to use, try having to install several software 'crutches' to be able to use Twitter productively without even half the functionality as Freindfeed.
- Jeff P. Henderson
Either this is tounge in cheek or Robert really hasn't gotten over our breakup... Robert, I hope we can still be friends but we have to move on and grow... FriendFeed will always love you, but we need some 'us' time to find out who 'we' are... *cues The Bodygaurd soundtrack*... *walks out into the rain, adjusts collar, walks off down the road*... *fade to black, credits*
- Johnny Worthington
from iPhone
Comment bait. Even Robert can't generate interest these days.
- Russellreno
I agree. Friendfeed is for those who need more from a 2d interface. Until there spatial interface is there, Friendfeed is on top of the pyramid (especially when one knows how to use it best).
- Kirill Bolgarov
Lets hope so ... Miss some of my old peeps :)
- Charlie Anzman
I'm surprised Robert made this statement but I find it very interesting in the change of view and wonder what has changed his mind or if he knows more info.
- manielse (Mark Nielsen)
The unpredictably erratic Scobot is at it again...
- Ciro
Well, after glancing at his Twitter timeline, he's just making a bunch of joke predictions. I'd like to think that there's some truth in this one though...
- manielse (Mark Nielsen)
Sniffs... Do you smell that? It's what we used to call a troll. Robert is getting dangerously close to becoming irrelevent.
- Jason Williams
from iPhone
Robert is waiting until 2010 to respond to this thread. :)
- Louis Gray
Scoble - I disagree ONLY because I think that FriendFeed activity is already migrating to Facebook via deeper integration over there but I'm wiling to hear you out (as are the other 40+ people commenting here). Do share why you think this is true?
- Aaron Strout
I wrote this tweet for Twitter, not for FriendFeed. But nice to see you all! :-)
- Robert Scoble
Jason: I'm definitely irrelevant if the people calling me irrelevant don't even have 1,000 subscribers. Sigh.
- Robert Scoble
I actually see FriendFeed growing instead of slowing and there will also be new enhancements. A testing ground for Facebook. Just to expand, Robert's still the man!
- amarquart
Akiva wins the internets with this mathematical formula for spotting comment bait.
- Nicholas Kreidberg
You think we're fighting, I think we're finally talking! -all in one page
- Tim Jones
Funny, when I write, I write for everyone, everywhere
- Johnny Worthington
@Jason Williams, wow that's the pot calling the kettle black, doncha think? Robert's not a troll.
- Jason Huebel
@Kol, somebody mentioned that those stats often only consider US visitors (why that would be, I don't know). FF's non-US contingent has grown tremendously, so that graph may not show the whole picture.
- Jason Huebel
I'm finding that FriendFeed is fiendishly sticky. On Twitter lists do help, they make twitter better and all but but but-but I think a lot of users are getting more sophisticated in their web usage faster than Twitter can evolve their tech. Louis Grey recently postulated that even if there is no interaction on FriendFeed it makes complete sense to stay riding this horse. That got me...
more...
- JSLeFanu
Dude..... I totally hope you're right, but not too sure about this! LOL Explain your rationale please :)
- Susan Beebe
i agree with the original prediction. by the way, what's a resurgence? is it different than insurgents? :-)
- Morgan Haley
Morgan - the insurgents never left ... ;-p
- Robyn Hawk
I'll stay one faithful user, for those "outside the US" stats. Still feature plenty, still enjoyable. Still effective. Thanks for rallying the troops again Robert. Bring the hopes, and forget the ropes! 8)
- ElijahBailey-Zu of FF <0,
I like this whole thread. ++Akiva, Johnny, JSLeFanu, Jason Heubel, Mike Chelen . . . Robert, Twitter may have geeks and famous people, but you can actually do things here. Oh, wait - I'm not here. I'm in a widget on a blog in someone's Posterous whom I don't even know, that I'm searching through to find FF peeps to subscribe to over here. Now, that's just mind-blowing!
- MaryB, BrandingBroadOfFF
"Apple has reserved space in late January at a venue in San Francisco in advance of a planned product announcement, the Financial Times reported on its blog. The company has rented a stage at the Yerba Buena Center for the Arts and is expected to make a "major product announcement" on Tuesday, January 26, the report said, citing people familiar with the plans. Apple declined to comment. The Yerba Buena Center could not immediately be reached for comment The report comes amid mounting excitement about Apple's rumored plans to release a tablet computer. Apple has never confirmed the existence of the device - which is said to resemble a larger iPhone or iPod touch - but speculation has been rampant for months."
- RAPatton
from Bookmarklet
What does it say about my sons that their block buster Christmas gift, the one they will be most excited about, this year is a collection of books? - http://search.barnesandnoble.com/The-Com...
The Complete Calvin and Hobbes is their big gift, and I think they will be very excited. It is just interesting that they are at an age where most of what they will recieve is either a book or a game. I think they've read most of Calvin and Hobbes already, but this edition will make sure they haven't missed anything
- RAPatton
I wonder if I should put some of my book tokens towards that...
- Pete
Funny, my wish list is all books and games and I'm 31!
- Kyle Hebert
Exactly, Kyle; they are growing up.
- RAPatton
from iPhone
My kids LOVE Calvin and Hobbes. LOVE. They sit there reading them for hours. Actually, the voice that my oldest son uses to read the part of Calvin is quite annoying. :-D
- Jason Huebel
Cause these books are AWESOME and they have good taste.
- Uncle CW™
It means your sons are the big awesome! LOVE!!!!
- Mary Carmen
I'd really like to get my hands on the Peanuts collections. The early stuff is so different than the late stuff.
- Spidra Webster
That they are as awesome as my girls who read the Calvin & Hobbes books reapeatedly for several years. And that they will have an enhanced vocabulary as a result.
- Rochelle Rochelle
My wife's been buying me these kinds of collections for Christmas for a few years now. I've got this one, The Far Side and a good chunk of the early Peanuts.
- J Wynia
I was always most excited about the books that I received at the holidays. Good for you and them.
- Mathew A. Koeneker