On Thursday one of FriendFeed's co-founders, Paul Bucheit, will be on a FastCompanyTV webinar with me along with Matt Mullenweg and Nat Brown, the CTO of iLike to talk about how to make a system that stays up and responds quickly, even with millions of users. I can't wait, but I'm also terrified and need your help.
- Robert Scoble
A link from work (Not sponsored, not an ad) for anyone interested. It's related to Robert's topic: http://www.bluearc.com/dataflo... (And don't shoot me for blending the work life with FriendFeed)
- Louis Gray
There is usually a session at mix with the myspace crew. It's Microsofy centric bit might help.
- Roberto Bonini
Robert, I left you a comment on your blog post - but maybe it'd easier to catch you here. Come on up into the city anytime this week before your interview, and it'd be my honor to help answer your questions, and take you through what we did at Technorati (and now at Offbeat Guides) with regards to building scalable systems - including all the mistakes we made, some of them not so obvious! Give me a ring at 415 846-0232 or DM me or whatever, we'll set it up...
- David Sifry
from twhirl
Glen, I guess many of us are interested so if you can share them we would be very grateful!
- Andrés David Aparicio
from twhirl
These guys have all answered these questions before and know how to talk about the topic. So I don't think you are on the hook for deep insightful questions. More of just getting the conversation started. Maybe take it from an uber user point of view and ask for stories about how they've scaled, made performant, and made reliable some of your favorite features with special attention to challenges and how they were solved. The proud parents will do the rest.
- Todd Hoff
All this time I thought Glen was just this funny guy on Friendfeed :) Love the anonymity!
- Charlie Anzman
Scalability is about identifying bottlenecks that keep the system as a whole from achieving the goal of linear scalability (adding 2x as many boxes gives 2x as much throughput). Ask about what typical bottlenecks they avoid, and which bottlenecks they're still fighting. Also, the largest cluster-size they've run, and where the tipping points are for when certain bottlenecks become a problem.
- Jason Carreira
Anytime Glen. Creative blending of obsfucation and bloviation is a talent :-)
- Todd Hoff
I think a good question to ask would be what assumptions each of them made that ended up being wrong and the most expensive to fix? All developers/architects make assumptions ... about how much traffic they'll see, what kind of traffic they will see, what the most important performance metric will be (the responsiveness vs. throughput question, which anyone building a scalable application needs to answer for their domain) ... Did any of their assumptions in these or other areas lead to big problems?
- J. McConnell
Glen, maybe the reason they're up-and-coming is that they're thinking in terms of value to the user, and not thinking, e.g., "how much harder will it be to shard our data if we implement this feature?"
- Bruce Lewis
from fftogo
Mr. Scoble - If I had one question to ask it would be "How do you BUILD for an predictable spike?" Say you're WalMart.com and you KNOW your biggest day of the year is going to be day after Thanksgiving. Walk us through preparing for surviving the spike and servicing all the transactions. If I had time for a second question I'd ask about the current state of cloud computing (namely Amazon's EC2)? And, a third question would be framework related (ala Ginger or MaryAnn?).... Rails or C?
- Gerald Buckley
I'd suggest that database sharding is a bandaid solution for the wrong up-front architecture.
- Jason Carreira
Scalability is all about CAPACITY PLANNING. Numerous aspects of systems design (and potential failure) must be examined and well planned in order to avert potential future failures. In an ideal world, you'd know the limits your system will need to scale to. But we don't live in a perfect world now do we. So product managers must work with IT architects to build a system that can "scale" to meet burgeoning demands as their product "takes off". Get your crystal ball out to start forecasting demand.
- Susan Beebe
Cont'd: CAPACITY PLANNING involves all 7 layers of the OSI model. GO from the electrical plug in the wall to the "presentation" layer (screen) of the customer's view of your product. Now consider all the potential SPF (single points of failure) along the way; whether that be core infrastructure issues like bandwidth, load balancing (web servers handling the load), memory caching, DNS, optimized O/S, application design, database design, well structured queries, smart technologies like Java and AJAX
- Susan Beebe
SCALABILITY actually refers to the concept of a Production Application running withing acceptable SLA limits without downtime or performance degradation, which can be caused by a slew of offending issues. Volumes have been written about scalability. Focus on key bottlenecks: bandwidth, infrastructure hardware, application design, data storage, especially as it pertains to database / query design. Sometimes large queries, reports, sprocs, batch jobs can hose a good app (needs smart devs to optimize).
- Susan Beebe
Where are your users and where are your servers? Scaling globally. When did you move away from managed hosting and run your own racks? What do you consider successful site responsiveness? Have you noticed positive correlation between really quick page loads and user engagement?
- Niall Kennedy