 |
|
| FriendFeed is the easiest way to share online. Learn more » |
Jonathan Wells › Likes
A quick difference between Twitter's architecture and that of FriendFeed pointed out by @biz's post from today...
Jimminy, CoG of FF, minitoka, Mohammad Abdurraafay and 101 other people liked this
Biz writes, "Every time someone wrote a reply Twitter had to check and see what each of their followers' reply setting was and then manifest that tweet accordingly in their timeline—this was the most expensive work the database was doing and it was causing other features to degrade."
- Louis Gray
But on FriendFeed, every time you see an update, FriendFeed checks if you subscribe to someone, and if you have hidden any service, or even any specific blog from any service.
- Louis Gray
FriendFeed knows when you post an update who can see it and who can't see it, based on who is following, and what services they have opted into or out of. And when you think of how many different ways the database has to be set, the unlimited power is very impressive.
- Louis Gray
FriendFeed checks by list, by service, by name of service, and can even check by if something is liked or not, how many times, has comments, and how many, and who from, if you use the search. There is a ton of data behind the data.
- Louis Gray
Math is hard </barbie>. Sorry, but this seems like a serious cop out to me. If FF can design the DB to function as quickly as it does with as many selection parameters, surely Twitter could do the same (if, that is, they have the proper backend platform to do so, and it doesn't look like they do).
- FFing Enigma
And yet, FriendFeed continues to add features and have near-flawless uptime without slowness.
- Louis Gray
I agree...it is a major copout. We have been using that technique in online chat systems since the early 1980's. It is not difficult and does not take as much power as @biz seems to be implying.
- Semipro
I am serious. They had downtime last week, and that was a blip. And what slowness, Glen? (Referring to FriendFeed)
- Louis Gray
Twitter's was. Not FriendFeed's.
- Louis Gray
My comments should have run together. Tina helpfully interrupted. :)
- Louis Gray
Yeah, the "And yet, they continue..." sorta started down the road to Ambiguity.
- Ken Sheppardson
(Edited for clarity) and Glen, look forward to that happening.
- Louis Gray
@ in the middle must be a lot less frequent than @ at the beginning thus saving processing power
- Matsis
But a beginning @ is index 0, there's no seek time compared to an any position @.
- Micah
also I believe a fair comparison would be twitter removing the @ replies to all (lame) with FF removing FoaF entries from lists other than the home feed. And twitter search is real time - that's probably eating up processing power
- Matsis
Good point re: Friend of a Friend, Matsis. FriendFeed already has to check if you have it enabled or not, and then if you have it enabled for a specific friend or not, and then, execute. I neglected to mention that.
- Louis Gray
Leather: When the "see all @ replies" was an option, Twitter had to decide how to handle @ replies (whether to propagate the tweet to you or to hide it, depending on your setting), which is expensive to the database. So, either they make everybody see all @ replies (which changes the setting of 98% of the users), or they sacrifice the 2%
- Andre P. Siregar
Micah I think twitter's point was that they didn't want to go check the setting (too much work for them) so they removed it. We b*itched and moaned so they gave back something less than what was. Or in other words, if the feature is available, and Oprah has a back and forth with aplusk (say replies 10-15 times), twitter dies. Also what Andre said :)
- Matsis
This is a good lesson for all comp sci students. The @-reply was not a feature Twitter had when they first launched. It's sort of a hack from the community that later got incorporated. But the architecture didn't anticipate this feature. And now it's hard to change the foundation when the house has been built
- Andre P. Siregar
Andre, I also think it's fair to say (many have stated this many times) that the whole of twitter's original architecture wasn't built with the anticipation of its current use.
- Micah
I think it all has to do with the schema-less design on top of MySQL FriendFeed is using. They're simply making smarter choices ahead of time, rather than reactive choices.
- Jesse Stay
So, by the presumed logic, no one's going to a see any new features that involves a follower dependent setting regarding anything tweet in stream. Well, ok then.
- Micah
Jesse, I have no doubt Friendfeed made smarter design decisions. When Twitter started the whole micro-blogging thing nobody really knew back then how it was going to grow and be used (now Twitter is much more than just about "I'm having lunch"). So when Friendfeed started, I'm sure they took some lessons learned from Twitter and others
- Andre P. Siregar
Micah, yes regarding Twitter's original architecture. IIRC, Digg went through the same re-architecting exercise a few years back when their users started to skyrocket.
- Andre P. Siregar
It's typical to have to re-architect a system once you realize how much the userbase is skyrocketing. Ideally you've constructed it in such a way so that you can rip out modular pieces and replace them as necessary. It seems to me that the whole of Twitter's backend is a much more serially constructed system, which is why they have to have complete system outages still now at three...
more...
- guruvan (Rob Nelson)
Leather, here is what Bret talked about for their MySQL implementation: http://bret.appspot.com/entry...
- Louis Gray
This thread is fascinating. I'm learning.
- joey
And though I wasn't on Twitter to see it, Robert suggests that they've been seeing these problems since he had about 1500 followers.
- guruvan (Rob Nelson)
Rob: that's absolutely true. Twitter was crashing even back in 2006. The architecture they chose was always horrid. It's getting better, actually, but still is having sizeable problems. Friendfeed has been down twice, once for an hour, and once for more than an hour because of a power failure that took down the entire data center. (At least that I remember). Twitter seems like it was down every day in 2008 and even this week has been down more than friendfeed has ever been down.
- Robert Scoble
It's hard to compare the two systems DB functionality because of the lower traffic FriendFeed has compared to Twitter.
- Rick Cogley
Rick, you can compare them based on the time of their lifespan though, and FriendFeed is doing much, much better. See the interview Scoble and I did with them last year when they had a very similar user-base to FriendFeed's.
- Jesse Stay
Rick: absolutely NOT true. Friendfeed today has MUCH higher traffic than Twitter did two years ago.
- Robert Scoble
It is easy to look at the total user base and make excuses for Twitter, but what I'm talking about is the fact that FriendFeed is accomplishing much more complex tasks, and not missing a beat. Even when they are 10 times bigger than they are today, I expect they will scale smoothly, because of their talent and their track record.
- Louis Gray
Leather: I'm assuming the difference is with the amount of WRITE operation in the database (which is much more expensive than parsing)
- Andre P. Siregar
I would love to read how friendfeed has architected itself to avoid the problems twitter is having.
- ian kennedy
Traffic may be one thing, but I would venture to guess the activity in the database is higher in FF than in Twitter for the same number of users
- Andre P. Siregar
ian: on friendfeed there is more metadata to study and filter with. I can say "show me all items that mention Obama" on both services, but only on friendfeed can I also say "and display only those items that have a like or more." That is how friendfeed can remove a major amount of spam (and I use it all the time). Spam doesn't get likes. Another way?
- Robert Scoble
On friendfeed I can delete any comment underneath one of my items. So I can help keep my corner of friendfeed clean. And if I see any underneath, say, your items, I can let you know about it. But what about on Twitter? We can't do shit about spam except report it and hope that @biz or someone gets to it.
- Robert Scoble
Leather: Twitter is more than three years old. Friendfeed is just about 1.5 years old. You aren't factoring that in. Go and compare the two services all along at the same age. I don't compare my 1.5 year old to a three year old, and neither should you. That's a HUGE mistake you are making.
- Robert Scoble
When Twitter was 1.5 years old it had fewer users than friendfeed and was FAR LESS COMPLEX yet it was going down about 100 times more often than friendfeed is today. Twitter's architecture was NEVER thought out very well and we're still seeing the artifacts of that.
- Robert Scoble
Leather: I'm comparing friendfeed today to where Twitter was 1.5 years ago.
- Robert Scoble
Leather: true, but friendfeed is a LOT more complex than Twitter ever was and it also has more users than Twitter did when Twitter was 1.5 years old.
- Robert Scoble
Leather: then what are you arguing with me about? Heheh.
- Robert Scoble
Scoble, yes the filtering on advanced searches is impressive (it even supports simple exclusions). I am impressed that they can keep it all running with every page updating in*real*time. Is each page an open session?
- ian kennedy
ian: I believe it is. Leather: go to compete.com or any traffic site and compare. But adjust the dates.
- Robert Scoble
Leather: keep in mind I've been on Twitter 907 days and I'm not going to do your homework for you. :-)
- Robert Scoble
Louis, you're absolutely correct about FF accomplishing much more complex tasks much more effortlessly. Overall the leaders of this team are more effective. We had a saying back when I was consulting ISP startups "The network IS the business plan" In an online business that depends entirely on its systems for potential revenues, the architecture of the network becomes a model for the...
more...
- guruvan (Rob Nelson)
Leather: believe me, Twitter didn't grow that fast until the last year. The first two years Twitter grew slower than friendfeed did and had a TON more problems (and was even less complex).
- Robert Scoble
Leather: 900 != 700. Heheh. That's a huge number of days in this world. And, anyway, both traffic and complexity are higher on friendfeed and amount of time down is MUCH lower here. So, by either metric friendfeed is winning HUGE.
- Robert Scoble
Robert, here's the screen capture from Compete.com: http://twitpic.com/57bgl Y'all seem to be forgetting about non-Twitter-account web traffic. FF is flat; Twitter skyrockets. And yes, FF is "younger" but is built on top of Twitter so it did not start from the same "zero" point -- nor does it have as many API calls -- nor is it as accessible off-the-laptop (iPhones excepted).
- Kathy E Gill
Kathy: you have NOT shifted the graph to show equivilent times. At 4/2009 Friendfeed is STILL far younger than Twitter was at 4/2008. Shift the graphs and come back. Also, at 1.5 years into its life Twitter didn't have that many apps either and Twitter's API is far more simplistic than friendfeed's is. Twitter didn't take off until after it was two years old. Friendfeed is already growing faster. And who cares what it is built on top of? Twitter was built on top of blogs and we don't hold that against it.
- Robert Scoble
Kathy: Benjamin Golub of FF wrote fftogo.com. FriendFeed on any phone.
- Johnny
Twitter started March 21, 2006. Friendfeed started October 1, 2007. You must shift the charts to compare apples to apples.
- Robert Scoble
I just don't understand why it's so hard to compare apples to apples. If you want to compare the two companies, you have to compare them at similar stages in their lifecycles. it's really simple.
- guruvan (Rob Nelson)
Rob: because our tools make it hard to do the comparison. But I was there and saw how Twitter grew. Twitter was crashing before it even got to the famous SXSW and I only had something like 1000 followers then (and that was in March of 2007).
- Robert Scoble
I use fftogo almost every day. and I use the IM on the other days (twitter isn't really even sophisticated enough to keep their IM service running)
- guruvan (Rob Nelson)
Not to mention email
- Johnny
I don't know about that Robert. We can grok how to compare money from today to money in 1920. We should be able to adjust stats about companies in our brains. We don't compare the revenues of a 3yo company to the revenues of a 30yo company, right (unless the 3yo is performing Outrageously). I haven't heard a single person disagree with what you've said about "the old days" of twitter either. So I can only assume that it must be true. These problems don't sneak up on a company.
- guruvan (Rob Nelson)
Rob: Twitter's management will admit they've always had technical problems. Remember, their architect was fired (Blair) and @ev admitted that it was a test project that they never expected to get so big.
- Robert Scoble
Ok, well the first step to getting well is admitting you have a problem, right? Never expecting a thing to get so big is a lack of vision, and not a very good quality for a management team IMO. Since they never expected it, and now that it is, they should be stepping aside as they are certainly no longer qualified to lead it. And THAT is the true sign of success of management teams....
more...
- guruvan (Rob Nelson)
Choose Success, not failure. But, it's interesting to note that "fail whale" and twitter are synonymous. Fail is practically in the name. And that needs to be corrected. It affects team spirit and morale, and causes it to become a vicious cycle.
- guruvan (Rob Nelson)
Rob: you have to go back in @ev's history, though. Remember when he started Pyra? They made Blogger. Remember how often it was down? A lot, just like Twitter. How slow it was? Very. How much spam it had? A lot, just like Twitter. Yet it made Ev a lot of money, just like Twitter will. So, since he's been down this path before, why not repeat? It actually doesn't make sense to over...
more...
- Robert Scoble
And that's where I come back to the lack of vision. If you "aren't sure" then you don't have the vision. You're right that it doesn't make sense to over engineer something if you have doubts, but if you have doubts, why jack with it? But in some sense it does make sense to over engineer things, because in a pinch you can often sell the company for the technology that's running it to someone who has a different vision for it, someone who knows what to do with what you've built.
- guruvan (Rob Nelson)
Robert, I was talking about now, but I see your point about comparing two years ago. However, it's still hard to compare two different systems, different db's, different programming languages, different ways of indexing data, etc, in my opinion. It's pretty apparent that FF has been very stable, comparatively, however.
- Rick Cogley
lots of testosterone on this thread, did we at least all agree that twitter's "organic architecture" sucks now just like it did from inception & friendfeeds "designed architecture" should scale nicely just like it has from inception?
- mike "glemak" dunn
Mike, yes, agreed on that. ;-)
- Rick Cogley
FriendFeed enthusiasts in Scoble's network agree that it's better than Twitter! Film at 11.
- Christian Crumlish
What would Internet Marketing without the hype and BS look like? I intend to find out.
Jonathan Wells liked this
Recession Is Over According to Financial Experts - ABC News - http://abcnews.go.com/Busines...
Jonathan Wells and Roney Smith liked this
www.ministrymarketingsolutionsblog.com
- Pam Perry, PR Coach
“I cannot always control what goes on outside.
But I can always control what goes on inside.”
— Dr. Wayne Dyer - http://denisegriffitts.tumblr.com/post...
Jonathan Wells liked this
Whenever you feel sad or lonely, identified 1 thing U can do that will contribute to someone else and do it. Guaranteed U will feel better.
Jonathan Wells liked this
Other ways to read this feed:
