On Thursday one of FriendFeed's co-founders, Paul Bucheit, will be on a FastCompanyTV webinar with me along with Matt Mullenweg and Nat Brown, the CTO of iLike to talk about how to make a system that stays up and responds quickly, even with millions of users. I can't wait, but I'm also terrified and need your help.
- Robert Scoble
A link from work (Not sponsored, not an ad) for anyone interested. It's related to Robert's topic: http://www.bluearc.com/dataflo... (And don't shoot me for blending the work life with FriendFeed)
- Louis Gray
There is usually a session at mix with the myspace crew. It's Microsofy centric bit might help.
- Roberto Bonini
Robert, I left you a comment on your blog post - but maybe it'd easier to catch you here. Come on up into the city anytime this week before your interview, and it'd be my honor to help answer your questions, and take you through what we did at Technorati (and now at Offbeat Guides) with regards to building scalable systems - including all the mistakes we made, some of them not so obvious! Give me a ring at 415 846-0232 or DM me or whatever, we'll set it up...
- David Sifry
from twhirl
I'd ask a simple one about the fact that traffic spikes can happen at any point, day or night, how do they deal it? Do they have round-the-clock staff or any type of early warning system?
- Steven Cains
I've built several sites for Yahoo! and give conference presentations on website scalability. Would you like to chat before then?
- Glen Campbell
I have a bunch of conference presentations on scalability; let me know if you're interested (might be too technical for your audience)
- Glen Campbell
Glen, I guess many of us are interested so if you can share them we would be very grateful!
- Andrés David Aparicio
from twhirl
These guys have all answered these questions before and know how to talk about the topic. So I don't think you are on the hook for deep insightful questions. More of just getting the conversation started. Maybe take it from an uber user point of view and ask for stories about how they've scaled, made performant, and made reliable some of your favorite features with special attention to challenges and how they were solved. The proud parents will do the rest.
- Todd Hoff
All this time I thought Glen was just this funny guy on Friendfeed :) Love the anonymity!
- Charlie Anzman
http://files.broadpool.com/ has directories for various things. "dcphp2007" was a presentation entitled "Drinking from the Firehose"; "phpconference2008" was on service-oriented architectures; "webinale2008" was on the best ways to bring down your website (and how to avoid them).
- Glen Campbell
Scalability is about identifying bottlenecks that keep the system as a whole from achieving the goal of linear scalability (adding 2x as many boxes gives 2x as much throughput). Ask about what typical bottlenecks they avoid, and which bottlenecks they're still fighting. Also, the largest cluster-size they've run, and where the tipping points are for when certain bottlenecks become a problem.
- Jason Carreira
todd, -10 for using the word "performant" :-)
- Glen Campbell
Anytime Glen. Creative blending of obsfucation and bloviation is a talent :-)
- Todd Hoff
I think a good question to ask would be what assumptions each of them made that ended up being wrong and the most expensive to fix? All developers/architects make assumptions ... about how much traffic they'll see, what kind of traffic they will see, what the most important performance metric will be (the responsiveness vs. throughput question, which anyone building a scalable application needs to answer for their domain) ... Did any of their assumptions in these or other areas lead to big problems?
- J. McConnell
IMHO the biggest problems I've seen with up-and-coming web apps is that they tend to think of scalability as something you "add on" instead of something you build in. If you've designed the architecture properly, you can increase the scale by simply adding hardware; any solutions that require additional coding or extra software (i.e., "just recode that using memcached") simply are not properly scalable.
- Glen Campbell
Glen, maybe the reason they're up-and-coming is that they're thinking in terms of value to the user, and not thinking, e.g., "how much harder will it be to shard our data if we implement this feature?"
- Bruce Lewis
from fftogo
Mr. Scoble - If I had one question to ask it would be "How do you BUILD for an predictable spike?" Say you're WalMart.com and you KNOW your biggest day of the year is going to be day after Thanksgiving. Walk us through preparing for surviving the spike and servicing all the transactions. If I had time for a second question I'd ask about the current state of cloud computing (namely Amazon's EC2)? And, a third question would be framework related (ala Ginger or MaryAnn?).... Rails or C?
- Gerald Buckley
Bruce, absolutely true; however, it's actually cheaper to build scalability in at the beginning than to add it on later. And it doesn't have to be more difficult (sharding data? no one at Yahoo! does that, so why would a smaller company need to?), just a different architecture.
- Glen Campbell
I'd suggest that database sharding is a bandaid solution for the wrong up-front architecture.
- Jason Carreira
My main interest in scalability (and the most problematic in my 10 years of experience) is always the database. I'd ask if there's a rule of thumb for the design of databases to support a truly scalable application.
- Jorge Escobar
Scalability is all about CAPACITY PLANNING. Numerous aspects of systems design (and potential failure) must be examined and well planned in order to avert potential future failures. In an ideal world, you'd know the limits your system will need to scale to. But we don't live in a perfect world now do we. So product managers must work with IT architects to build a system that can "scale" to meet burgeoning demands as their product "takes off". Get your crystal ball out to start forecasting demand.
- Susan Beebe
Cont'd: CAPACITY PLANNING involves all 7 layers of the OSI model. GO from the electrical plug in the wall to the "presentation" layer (screen) of the customer's view of your product. Now consider all the potential SPF (single points of failure) along the way; whether that be core infrastructure issues like bandwidth, load balancing (web servers handling the load), memory caching, DNS, optimized O/S, application design, database design, well structured queries, smart technologies like Java and AJAX
- Susan Beebe
SCALABILITY actually refers to the concept of a Production Application running withing acceptable SLA limits without downtime or performance degradation, which can be caused by a slew of offending issues. Volumes have been written about scalability. Focus on key bottlenecks: bandwidth, infrastructure hardware, application design, data storage, especially as it pertains to database / query design. Sometimes large queries, reports, sprocs, batch jobs can hose a good app (needs smart devs to optimize).
- Susan Beebe
Where are your users and where are your servers? Scaling globally. When did you move away from managed hosting and run your own racks? What do you consider successful site responsiveness? Have you noticed positive correlation between really quick page loads and user engagement?
- Niall Kennedy