Dave Senior › Likes

Matt Frog
"Mojo is music sharing done right.  With just two clicks, you are ready to browse, select, and download music from other Mojo users.  Plus, subscribe to the playlists of other users, and Mojo will update those playlists anytime they are online.  Finally, all downloaded songs will automatically be added to your iTunes library." - Matt Frog from Bookmarklet
If you sign up and install, please do share your username if you'd like to connect! (Abba fans need not worry themselves ;-) ). I am mattuk@deusty.com - Matt Frog
Just set this up, after hoping I could get some of my mates to install it. neurario@deusty.com - Neurario
just signed up. colerice04@deusty.com - cole
If you still have a working copy, add me - briantorok@deusty.com - Brian Torok
Srikanth AD
Yes, all this hosting and file serving is free to provide anyway. They must just be trying to really screw us by raking in even MORE money with those ads. How evil of them. - David Chartier from iPhone
Awesome share :) TIM J. @ WWW.MHWD.ORG - Tim Tunnicliff from email
skipscreen was working before but it stopped working for rapidshare recently. If you are downloading lot of stuff it makes sense to actually pay for rapidshare month by month, with no yearly commitments, mind you. - TrafficBug
Scott Beale
Jason Calacanis
This Week in Startups about to startup!!!! join us live right now! http://thisweekinstartups.com/ PLEASE RETWEET r/t!
Dobromir Hadzhiev
Jesse Stay
Bwana ☠
Nmap 5.00 Released - biggest release since 1997 - http://nmap.org/5/
Nmap 5.00 Released - biggest release since 1997
Nmap 5.00 Released - biggest release since 1997
Show all
Nmap 5.00 Released July 16, 2009 -- Insecure.Org is pleased to announce the immediate, free availability of the Nmap Security Scanner version 5.00 from http://nmap.org/. This is the first stable release since 4.76 (last September), and the first major release since the 4.50 release in 2007. Dozens of development releases led up to this. Considering all the changes, we consider this the most important Nmap release since 1997, and we recommend that all current users upgrade. About Nmap Nmap ("Network Mapper") is a free and open source (license) utility for network exploration or security auditing. Many systems and network administrators also find it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running,... - Bwana ☠
Hawtness - Scoble, Alex Scoble
Jake Marsh
Screen-Scraping With CSS Selectors in PHP - http://thejakemarsh.com/1982/
When it comes to programming techniques, screen-scraping can be a complicated and annoying thing to deal with. I’ve dealt with it quite a bit in various projects, most recently my TV Library plugin for boxee. I use PHP to accomplish most of my screen scraping and its got a pretty great arsenal. However, if ease [...] - Jake Marsh
ianf ⌘
This being the only way to post an entry, a picture AND the first comment at the same time, I propose that the bookmarklet, when invoked over a FF page, automatically links to the topmost story on that page [here: http://ff.im/4O5GX], rather than to the feed, or search result itself. How's that for a simple enhancement? - ianf ⌘ from Bookmarklet
That's a parallel possibility. Another would be to have the bookmarklet stub code return a dialog with a different "Post New" button, or an additional checkbox "Unlink this new post" to decouple it from underlying base page. - ianf ⌘
Orli Yakuel
Visitor Contact: http://www.go2web20.net/app... a sticky contact button that can be used in websites. Simple and useful.
Patrick Jordan
Coming Soon: Command & Conquer Red Alert for iPhone — Just Another iPhone Blog - http://justanotheriphoneblog.com/wordpre...
Coming Soon: Command & Conquer Red Alert for iPhone — Just Another iPhone Blog
Sweet! - TheHenry
Why Quicksilver is still the greatest Mac app of all time. - http://thenextweb.com/2009...
Why Quicksilver is still the greatest Mac app of all time. - http://thenextweb.com/2009/05/04/quicksilver-mac-greatest-app-time/
Why Quicksilver is still the greatest Mac app of all time. - http://thenextweb.com/2009/05/04/quicksilver-mac-greatest-app-time/
can't agree more. - vijay
I dunno, I've been using Launchbar (the beta) for a while and I prefer it. - Jalada
i dont mean to sound aggressive Jalada, but I'll put my house on it that Quicksilver can do everything and more better than Launchbar - Zee.
Oh yea zee. Qs can do a lot more than Launchbar. I do prefer Launchbar anyway :) - zio bodhisattva
I couldn't use QS for more than search & running apps. And for that the Spotlight works well, which made me remove QS. BUT, knowing it now posts to friendfeed directly, I guess I'll try it again... - arnaldostream
After reading this review I guess I should install this as the next level of **Pimping my Mac** - Jonathan.Rivera
I'm a long time fan of QS, but -for the sake of infosharing- you can do quick unit conversions using the Google Quick Search Box as well. Simply double-tap the command key and when the search box comes up type, for instance: 100 miles into km. - The first choice offered should be "Search Google..." - Hit that, and Google will do the unit conversion in your browser. - Carlos Granier-Phelps
ça me plait - andré
I'm sure Quicksilver can do it all, but I always found Quicksilver's configuration to be cumbersome and complicated. Maybe I was just doing it wrong :P - Jalada
I used QS for a long time and love it but found it ate a lot of resources on my old powerbook so I stopped using it. I tried Butler and although very different metaphor it did everything QS did but again found the same thing with resources. Now I use the google quick search which is nowhere as hungry but doesn't really cut it yet. As I'm doing more and more in firefox now I'm using ubiquity more and more and that is rocking. - Brett Jackson
Shedali: as far as I know, there is still some active development ongoing by members of the quicksilver community. However, Alcor, the creator and lead developer of Quicksilver, is now heading up a team at Google developing the Google Quick Search Box: http://code.google.com/p... - Brandon Titus
I would love to have a Quicksilver guru help me use Q more effectively. Seem like I'm mostly using it as an app launcher. Any tips INS goodsite for this? - JR Holmes from Nambu
I was a satisfied user of Quicksilver until OS X 10.5 came out. Quicksilver became unstable on it and some plugins didn't work anymore. I now use Google Quick Search Box which looks more stable and promising than QS. - Brome
I dumped it last year - Quicksliver is too complicated and it's balky on Leopard. - Bill Sodeman
I can't live without QS and will be heartbroken if it stops being compatible with the upcoming os releases. - wynlim
Quicksilver tutorial: http://bit.ly/kGNY Manual: http://bit.ly/Fa7FC QuickRef: http://bit.ly/11d3vT - Howard Melman
I can't live without Quicksilver, first thing I install on a Mac - Simon Tracey
My problem with Quicksilver is that I become SO dependent on it on my own mac that I can't seem to do anything when I get on a Mac that doesn't have it. - Jeff Smith from twhirl
+1 Jeff lol... - Zee.
JR Holmes, I'm no expert, but: http://friendfeed.com/micahwi... - Micah
Recently I haven't been using it that much, but that's because I've been spending less time at the computer. - Richard A.
Quicksilver was indeed ahead of it's time ESP for those us us who prefered not to use mice - before spotlight using the GUI on a Mac wasn't my favorite - thinfilms from Nambu
I wish I had a nickel for every second of time QS has saved me...It's a must have. - Chris Rossini
Major Nelson (Larry)
Want to be one of the first to play 1 vs 100 on Xbox LIVE? If you are in Canada, you're in luck. Detals on teh beta here: http://bit.l ...
Robert Scoble
I am trying to build a bunch of developer and geeky-focused searches on real-time-web sites. Post here your favorite geeky search:
Hadoop, geek, Heroku, PHP are some of mine. - Robert Scoble
site:php.net "Really is" - Leonid S. Knyshov
geek search? hm... what about searching for drivers for your ubuntu installation or linux how-to's and guides. uber geeknes ftw. =) - Apostolos Papadopoulos
jquery, ruby, drizzle, php, MVC - mjc
Apostolos: no, I'm talking about searches for Twitter Search http://search.twitter.com or friendfeed search, like I'm posting here. One thing, if they are too noisy I just make them display things that have a "like." - Robert Scoble
Robert: yes just saw that. sorry ^_^ - Apostolos Papadopoulos
Spam filter for all searches, use service:internal, http://friendfeed.com/search... (because there are many unnecessary facebook, ping.fm, blog entries) - Alp
drizzle: http://friendfeed.com/search... (a great one, but brings up weather info too). - Robert Scoble
site:stackoverflow.com "C#" OR "Java" OR "PHP" OR "Python" OR "Ruby" - Varun Mahajan
vmware, iscsi, NAS, SAN, storage, - rob
also "service:delicious" filter brings really good stuff. - Alp
xkcd - Varun Mahajan
How many "OR's" can you do on friendfeed? We should build bundles for people and put those into a URL. - Robert Scoble
realtime OR real-time OR "real time" - Varun Mahajan
scoble - Chris Poulson
#zf OR zend OR #php -stackalert -fresh_projects -freelance_jobs -lampjobs -flashability -hashphp -hashwp -hashwordpress -phplance http://www.twazzup.com/search... - arjo
arjo: that's cool AND geeky! - Robert Scoble
Brian: here's that search for friendfeed: http://friendfeed.com/search... - Robert Scoble
date next girl geek dinner Mountain View - don manu
don: that's funny! - Robert Scoble
;-) - don manu
It may be the keyword technology, but that search resulted in pretty bad results! Try MySQL - Chris Frost
I modified Brian's search a little to come up with something I like (it obviously needs to be tailored to your interests): http://friendfeed.com/search... - Brandon Titus
(to much) potentially interesting PHP blogs http://friendfeed.com/phpc... - arjo
MID's,Atom,Tablet PC,Snapdragon,UMPC,Moorestown: http://friendfeed.com/search... - Rodfather
iPhone,Palm Pre,Android FF rooms: http://friendfeed.com/search... - Rodfather
github - very under-hyped outside of the rails crowd. More important imo than git itself. - Capn' One Eye - adrift
Cyril: I don't like Twazzup. Harder to filter. Here's same search on friendfeed: http://friendfeed.com/search... - Robert Scoble
busybox, embedded - Kevin Johnson
Robert: I'm a nerd and I really like friendfeed too. Twazzup might integrate with friendfeed in the future, but right now most of the real-time-web content is on twitter. If there are things we can do with twazzup to make it easier to filter, I would love to hear your thoughts. - Cyril Moutran
Cyril: I'm using Twazzup too. You're onto something. The money is in search, I hope you succeed! Where are you located, by the way? - Robert Scoble
django - Eric Florenzano
I really like Twazzup more then Twitter search. New entries come in real time.No need to refresh the page. Slowly the web is moving into real time updates. - Keven
roku AND (netflix OR amazon) - Kevin Kuphal
ruby on rails, ajax, palm pre, cloud computing, the cloud - Angus Burton
what do rails, django piston, web2py and the Twitter and Facebook APIs have in common? REST. it's a style of programming described in a dissertation by http://twitter.com/fielding and it's something i'm always looking to learn more about - Brian Hendrickson
thanks Robert! thanks Keven! - we're located downtown SF - Cyril Moutran
Jason Wehmhoener
qsb-mac - Google Code - http://code.google.com/p... (via http://friendfeed.com/e...) This has replaced both Quicksilver and Launch Bar for me. I love this little app, it works perfectly.
qsb-mac - Google Code - http://code.google.com/p/qsb-mac/ (via http://ff.im/2pr1e) This has replaced both Quicksilver and Launch Bar for me. I love this little app, it works perfectly.
Interesting. I'll have to give it a try. - Beau Liening
I tried it when it was first released and it was too buggy and slow for me to use as a quicksilver replacement. I'll have to take another look at it. - Phil Maxwell
It even let's you send updates to Twitter: "To tweet using QSB first set up your Twitter account (as documented above). Once that is setup you are ready to tweet. To actually tweet, you need to know the secret of creating a text item in QSB. Text items are things that start with a space (i.e. hit the space bar and then start typing). So to tweet you must activate the QSB, hit the space... more... - Peter van Teeseling
same guy developed it, if i got ir correctly - Naor Mark
There's a Mac version of Desktop too. I've used both versions. QSB is quite a bit more useful. - Jason Wehmhoener
I'm going to have to give this another look. It looks like they've made some more improvements. @Naor yes, Alcor, the developer of Quicksilver is the lead dev at Google on this project now. - Adam Turetzky
I've been using it since the first release. It's replaced Quicksilver for me although there are still some things that I miss. It's quite stable. - Brandon Titus
same here, it's a fantastic app I can't live without now. - Jim Carter
@Adarm, that's a good reason to follow this one, should be interesting to see how it'll eveolve, it has to go some way before replacing QS, although it does feel great, wonder if they'll let plug-ins to be developed - Naor Mark
yup, +1 Naor - Zee.
This looks interesting. I never really got into Quicksilver, I'll give this a try instead. - DGentry
Chris Messina
Stephen Wolfram discusses Wolfram|Alpha: Computational Knowledge Engine - http://www.youtube.com/watch...
Stephen Wolfram discusses Wolfram|Alpha: Computational Knowledge Engine
Leo Laporte
Live now: net@night with Amber MacArthur and Sarah Lane, Guest Phil Libin, CEO of Evernote. Discuss here...
I am loving Evernote, great service that continues to get better. - Dave Senior
Yeah, I'm a OneNote -> Evernote convert. Doesn't seem like we really have Window client parity with Mac yet. - Ken Sheppardson
I use Evernote all the time to keep track of things, love it - Kim Landwehr
Question for Phil: When will us Windows users have all the same functionality Mac users have? The "Notes by title" thumbnail view comes to mind. Seems to be in the online and mac versions, not on Windows. - Ken Sheppardson
Are they developing a Palm Pre, webOS client? - Ken Sheppardson
I just downloaded it, i love it so far! - Callum
Would like to see plain old Jabber/XMPP/Google Talk integration, so I could just IM stuff into a notebook - Ken Sheppardson
I love EverNote. I use it across Mac, PC and iPhone. - Michael
just got my cigar aficionado magazine nice review of the palm pre don't know how they got one but great review - Gary Prechtel
I just use the browser plugin rather than the software installed version - Greg
I'm sorta confused whether Phil was making a Palm Pre joke or a dis... - Ken Sheppardson
Evernote rocks! I wish I worked for them. They really have something to be proud of there. Ken: I think Phil was being coy because he couldn't talk about what they're doing with Evernote on the Pre yet. - Joe
Knew about EverNote for a while but am just trying it now and LOVE IT, its going to rock during school with long science class's and lots of writing on whiteboards! - Dakota O'Neill
OK, I think one of the terms of the Palm Pre early access dev program is you can't say you're part of the early access dev program... so now I feel better :-) - Ken Sheppardson
Sorry if this was asked before..will there be Blackberry Bold support? - Jeff Gignac
@Jeff Gignac Like he said they are working on a lot more phones. But you can always just use the email way. Which is just as good! - Dakota O'Neill
what's the discount code for squarespace? - Karma Martell
"night" - Leo Laporte
Leo, I thought we were doing a Gillmor Gang tonight. Is that still on? - Robert Scoble
Yes, he said he is just about to do it! - Dakota O'Neill
leo, in Skype video, right click on the video and click turn off return video or something to take off the PIP in Skype-Please - Andrew Martin
If you can tell me how to turn off the PIP without turning off video for our callers I will. Bugs me too, but our hosts want to see the same thing you're seeing. - Leo Laporte
Scott Beale
the Mac version of @tweetie is coming out this Monday, April 20th, here's a preview http://www.atebits.com/tweetie...
Its going to be hard to decide between tweetie for mac and Nambu - Dickie Maxx
Is it free? - Richard A.
I hope! - Michael Forian
TechCrunch article says free ad-based version of @tweetie, and pay version at $15 (although first comment says no official price yet). http://www.techcrunch.com/2009... - Niklas Morberg
Absolutely can't wait. Tweetie for the iPhone has everything you'd ever want in a Twitter client, IMHO. While Tweetdeck has a load of features, it's not Tweetie. Looking forward to this and will happily drop some coin for the app. - Anthony Baker
Easy choice, Nambu, free across platforms :-) - Richard A.
I like the Seesmic client very much but a real application not Air based is much more interesting. - Donald Townsend
Looks good from the video. I'm guessing it'll be a paid app, just like the iPhone version. - Dom Barnes
I hope it's free - Outsanity
What is it with people and free? I understand that the economy is in the shitter (boy do I ever) but if you can't afford a $15 app maybe Twitter isn't the best way to be spending your time. Tweetie is developed by one guy just trying to make a living. Sheesh. - jakebf
Michael Forian
CANADA! Doesn't our flag kick ass?
Mohomed, didn't know you were Canadian - Shey
hehe ok :) - Shey
yes, indeed! - Mat Wiemann
Robert Scoble
I'm building my own server in the cloud, thanks to Dave Winer: http://www.scripting.com/stories... but right now that will have to wait cause Maryam wants me to come downstairs and stop playing on my computer.
Are you using Rackspace? - Jesse Stay
I'm not sure I'd recommend EC2 for beginners, even with a HOWTO. There are much cheaper cloud-based alternatives (like Rackspace's Slicehost offering). "Turning it off when you're not using it" isn't a solution. This isn't 1970. - Mark Trapp
Jesse: no, for now I'm just following along Dave's podcast. - Robert Scoble
After I get it running on Amazon I'll get it running on Mosso. Gotta run, Maryam's getting mad I'm still playing on the computer. - Robert Scoble
Is s3 mounted as a filesystem on EC2 much faster than s3fs elsewhere? I'm doing rsync with s3fs on a linode server and it's taking all weekend. - Bruce Lewis
Bruce: how big is your diff? That doesn't seem right. - Mark Trapp
Amazon has Windows servers... which Dave uses in his example. Mosso Cloud Servers and Slicehost look like they are Linux-only for now. EDIT: GoGrid has Windows servers as well, which are even easier to bring up than they are at Amazon. - Karim
I know there is a project that can use EC2 and S3 rattling around my brain, I just haven't been able to find it in there yet. - Mike Flynn
I've fallen in love with all of amazon web services... hard. I have trouble imaging building scalable projects without it now. My current project uses ec2, s3, cloudfront, elastic ips, elastic block storage, simple queue service and simple db. - Ted Roden
Karim: Mosso offers a Windows server (http://www.mosso.com/cloud.jsp) but it's a bit more expensive than leaving an EC2 instance running 24/7. It's hell of a lot easier to set up, though. - Mark Trapp
Great idea Robert. Once you try those, I'm very interested on what I'd get from RackSpace that I wouldn't get from Amazon, or more importantly, what of all those you try out you think is best. I keep coming back to Amazon due to price and flexibility, but I'm always open to new things if they make sense. - Jesse Stay
Mark, I thought those Mosso Windows offerings were Cloud *Sites* not servers -- i.e. basically a webroot where you get to upload stuff? From what I read, the advantage is that they automagically scale, but since it's not a server of your own, you can't log in as Administrator & install stuff. - Karim
Ted, what do you do for load balancing on EC2? - Karim
Karim: and what, exactly, is a beginner non-developer/non-IT/non-sysadmin going to be doing that requires a full server? I don't think people who require full control over the inner workings of a server are the people being targeted, here. - Mark Trapp
"BOBBY!... TIME FOR DINNER DONCHA KNOW" - rob from twhirl
Karim: On http://enjoysthin.gs I'm running one web front end so I don't need a proper load balancer. I split up some of the backend work to separate servers... same with the DB. But the app handles all that. In the past I've used both HAProxy and simple round robin dns. - Ted Roden
Mark, let me ask Dave Winer why poets need servers and get back to you. lol - Karim
Ted, thanks. Roll-your-own HAProxy seems to be pretty popular with the EC2 set. Amazon has threatened to come out with a load balacing solution at some point... - Karim
Karim: they will. Don't get to comfy with any load balancing solution, amazon will come up with something that beats it. - Ted Roden
Do what 'the boss' says ... or you're doomed (and she's right :) - Charlie Anzman
I get that a lot from Amanda. Good to know I'm not alone:-) - B.J. Mendelson
Mark Trapp - Sorry I fell asleep. The diff is less than 10GB. It looks like it might be almost finished now. - Bruce Lewis
Read the comments. It's a pricy proposition, the more you use it. There are cheaper solutions, no? - Ernie Oporto from Nambu
I've been playing with EC2 for a few months now, including creating my own linux images that weren't (aren't?) included "out of the box". This also gives you the legroom when you're going to mount your S3 buckets for your own files. It's a simple and painless task if you know your way around linux/unix systems... - Holger Eilhard
RackSpace has more instance types available than EC2. The lower end ones have less memory so they can end up being a lot cheaper. It may be worth checking out depending on your usage. - Todd Hoff
Martin Bryant
Create podcasts in an instant with AudioBoo - http://14sandwiches.com/2009...
Create podcasts in an instant with AudioBoo
"Attempting to inject some fresh thinking into podcasting is AudioBoo, a new service from the team at Best Before. Currently restricted to the iPhone, their mobile app allows users to record instant podcasts on the go. Forget having to deal with post-production tasks like editing and compression, AudioBoo podcasts (or ‘Boos’ as they are called) are rough-and-ready, spur-of-the-moment recordings." - Martin Bryant from Bookmarklet
Guy Kawasaki
Best tutorial videos I've ever seen: http://adjix.com/5vrt
NBA Basketball
LeBron James With the One-Handed Power-Slam - http://www.youtube.com/watch...
LeBron James With the One-Handed Power-Slam
Steve Gillmor
The Gillmor Gang - Jason Calacanis, Marc Canter, Leo Laporte, Dan Farber, Robert Scoble, Andrew Keen, Loren Feldman, and Doc Searls. Recorded Saturday, February 28, 2009. - Steve Gillmor
Pass the tea and Gillmor Gang. - Denise Howell
Downloaded it to my iPhone with RssPlayer while on 3G. Will listen when able. Got to love good ol timeshifting. - Christian Burns
what is rssplayer? - Steve Gillmor
I believe its a Podcatcher for the iPhone. I believe its the one they banned that has been renamed. - Phillip Molly Malone
Marc Canter was certainly in a funk. Didn't blame Jason Calacanis for biting back hard. Canter went a bit too far. - Karoli
Oh my, this was a feisty one. - Mike Doeff
MC was baiting, but he and the others, including JC, knew it was driving the show forward. My read was that the back and forth never lost its equilibrium. The gang dynamic was in full swing, and the venting well represents pent up feelings in the wider social/economic climate. Good stuff. - Micah
Great Gang... I think Leo's influence + Gillmor Gang madness = almost tolerable show. :-) - Jason Calacanis
@stevegillmor RssPlayer is the podcatcher for the iPhone. It's so much beter than iTunes at this point. And you can email feedback to the developer. It's only getting better. - Christian Burns
I agree with Jason. I have now started listening and enjoying this show since Leo joined. It's also nice that he manages everyone getting mic'd properly unlike some other shows where it was painful to listen to Steve do it :) - Mark Krynsky
Does Leo dominate the show like on his TWIT podcasts? Or is it more of a shared effort? - ComicList
I don't think Leo dominates HIS podcasts. I love hearing Leo's voice. And I haven't heard Gillmor Gang yet, but from what I saw from the Live stream, it was an interesting show. - Molly Song ;)
I enjoy Leo Laporte's participation on GG. HIs followers? Not so much. The rudeness and self-indulgent flaming in those chat rooms is nearly intolerable. - Karoli
Seth Greenblatt
YTPodcaster - Convert Youtube User Videos to Video Podcasts for iPod/iTunes, Zune and more #web - http://www.ytpodcaster.com/
YTPodcaster - Convert Youtube User Videos to Video Podcasts for iPod/iTunes, Zune and more #web
this is something I've been wondering why it didn't exist already - alphaxion
Now that YTpodcaster is dead, RSShandler seems to be an OK alternative. Not as userfriendly as YTpodcaster, though. I'm sure it will die soon too, due to bandwidth issues - so is there any sort of open-source (or even commercial) alternative? - Ingo
Sounds like some online video company should acquire ytpodcaster. If funding is the issue surely Boxee/Hulu/Youtube would be interested. - Keith - @tsudo
If you don't have to use YouTube, http://blip.tv/ has podcast features built in. - Daniel Sims
Like the idea, I would hope that Google or Mogulus would eat this up. - CLC Radio
#web siteTweet | Real-time Twitter metrics for your site alert tool - http://sitetweet.net/
#web siteTweet |  Real-time Twitter metrics for your site alert tool
"sitetweet allows webmasters to post immediate messages on Twitter about specific behaviors on their site. Usually installed on strategic pages, you can configure sitetweet to send a special message when a user performs a certain action on your site. " - Zee. from Bookmarklet
Why a site needs Tweet? That's interesting. - K.D.
no no, you can set it up to tweet you when a visitor performs a certain action. So if a visitor visits your "about" page - it can tweet you - Zee.
that will be overwhelming - thats what analytics is for - TrafficBug
Bret Taylor
We just launched a new backend system for FriendFeed that should make things a lot faster. Let us know if you see any problems or unusual changes in your feed. Thanks!
even faster? :) - directeur
Even faster :) - Bret Taylor
is it like the backend of the new mac mini -- 80 USB ports? - MG Siegler
Increasing the number of USB ports was vital to this upgrade :) - Bret Taylor
Nice... you were doing what I was doing today... see you can make major upgrades WITHOUT down time. Nice Job FF! - Brian Roy
Ok! Being behind a proxy makes my way somehow difficult but not that bad, I'll comment on that with stats if I see changes. Thanks Bret! - Zu from AOD
I will write something up at some point in the near term. Maybe after it has been running for a couple weeks so we know it works like we think it will :) - Bret Taylor
Hopefully this helps with the lag time that I'm seeing from Twitter. I like using FF as a Twitter client; it's been tough lately with tweets taking over a half hour to get here. Nonetheless, well-done, guys. :) - Jared Smith
faster in what respect Bret? - Zee.
Sweet bejeesus, you guys rock! I like new features without taking down the whole site. This is efficiency baby. - Scott Jarkoff
Zee: loading your home feed, especially if you have many subscriptions, will be a lot faster. In general, viewing feeds with lots of people in them is much faster. - Bret Taylor
Bret, what about the RSS machinery? Will it fetch more feeds more often? - directeur
That is somewhat independent of this change, directeur. However, it should improve feed publishing a bit over the course of the next fews weeks as we completely decommission the old system - it will improve some aspects of our publishing throughput. However, most of the feed fetching improvements are coming as sites like BrightKite and Identi.ca adopt SUP (http://simpleupdateprotocol.go...) - Bret Taylor
one of the reasons I use ff so much is that I can often open a new tab, load friend feed, and read a couple of entries in the time it takes for some other site to load; nice to see that speed remains a priority :-) - Karl Rosaen
Thanks for the details Bret, I like the SUP concept and I actually use it on my Jazz blog and on SocialWhois too :) - directeur
Bret great to hear:) Keep up the great work. - Rob Cairns
Fast is better than slow, after all :) - Joel Webber
Bret - good stuff. Any effect on the delayed Twitter feeds? Or is that out of FriendFeed's hands? - Hutch Carpenter
Yeah, we need way more USB ports! LOL :) - Susan Beebe
i have not found FF faster at China - 阿石
It was quite fast to start off with, didn't notice any big differences. (Good nor bad) - Umit Namli
2Bret: According to Alexa traffic graph, FF doubled its traffic in this week. Is it real and if its real what was the reason for that? If its not confidential, May be you can share with us :) - Ömer Faruk Kurt
Ömer: Alexa is like a Ford T, it's a bit old. Don't know what to make of that service. - Umit Namli
Damn, now it's _too_ fast. - Robert Konigsberg
Now FF is so fast I'm seeing posts from next friday. - dario
FF is Fantastic. - Russellreno
♥FF! - Mattb4rd
I still seem to get the FF is unavailable screen quite a bit. I'd say about 50% of the time when trying to do searches, filter by service (especially FF itself) or look at best of day. Usually after getting the unavailable screen if I refresh one to five more time eventually the page loads. Annoying though. - Thomas Hawk
What? was FF slow?? - Andru Edwards
Thomas: yah, your account has been a stress test for our system for a variety of reasons. We hope to make progress on the Best of Day and search ASAP, and I will let you know. Sorry for the trouble. - Bret Taylor
w00t! - mike fabio from twhirl
Great work. I second the fact that it is only second to Google homepage in load-time - Varun Mahajan
Very cool, kudos! - Dave Martin
Woo-hoo! - Joey Gibson
@Bret: is this update related to the stuff mentioned in your 'Schema-less MySQL' post? - MikeAmundsen
Mike: yes, the new backend uses the approach described in the schema-less MySQL post - Benjamin Golub
Woot! (Does that explain my excitement well?) - Danielle Closs
Friendfeed gets fast, Twitter gets slow; all on the same day. - Eric - Final Countdown
really? dosen't feel any differences. - Lira.H.C
Does anyone know how to turn/convert a regular RSS feed into a Media RSS feed? I once found a site which would take your feed and spit it out so that Friendfeed would import the entries with their photos or other media.
Hoping to find it again... - Mitch
Where is the feed from? A WP blog? - coldbrew
If it is from a WP blog, there is some discussion about that here: https://friendfeed.com/e... - coldbrew
From Google Reader shared items or a shared tag - Mitch
I'm not sure, but I think that Greader shared items produce ATOM feeds. I've never seen Greader shares with multimedia here on FF and I don't think the required elements are included in the feed, so it doesn't make sense to me that there could be a service to add them in to the feed. Hopefully, someone more knowledgeable can chime in here. - coldbrew
Colbrew - The reader items would have to be added as a blog. - Mitch
I just looked at someone's Greader shared-feed, and it is an ATOM feed, and it does not appear to include any media enclosures. Check out: http://en.wikipedia.org/wiki... - coldbrew
I was just wondering the other day what the characteristics of an ATOM feed were. Sounds like media is precluded when using that format. - Laura Norvig
ATOM does support non-textual content. I would guess it is not included in this instance to encourage people to go to the originating site. - coldbrew
Media really isn't default in RSS either though is it? You need to start identifying it with MediaRSS tags which is what I need to have "applied" to my GReader feed. - Mitch
so coldbrew, sorry for my ignorance but what is the difference between a regular RSS feed and ATOM? - Laura Norvig
ATOM is the IETF backed format (standard), while RSS (now in v2) was the original, and is kind of a de facto standard. On the surface there are no real differences. If you like to read technical content, check out: http://www.intertwingly.net/wiki... - coldbrew
BTW, Atom isn't an acronym; It's just "Atom". - Ken Sheppardson
I made a Yahoo Pipe that contained my Flickr feed plus others. When I run the pipe I then get the choice of Image or List - when I get the code for a badge it seems to include all the media contained in the other feeds. - Nick B.
Nick - can you post a link to your pipe? - Nathan Rein
thanks! - Nathan Rein
Bret Taylor
How FriendFeed uses MySQL to store schema-less data - http://bret.appspot.com/entry...
This is a technical overview of the new backend system we launched a couple of days ago. - Bret Taylor
First question, why MySQL instead of a straight key/value/attribute store, like a bdb? (Amazon's SimpleDB works along similar principles, btw.) - DeWitt Clinton
Cool stuff Bret. Love to see this kind of stuff. Would love that chat about the Ajax side of things ;) - Dion Almaer
DeWitt: Historical reasons; we have some operational experience maintaining MySQL servers, and MySQL is very popular so there's a lot of support available. - Tudor Bosman
Oh right, none of you are ex-Amazon. (Any Amazonian reading this will understand...) - DeWitt Clinton
How small do you keep the shards? Can you keep the entire index tables in RAM? Are you backed with SSD? - DeWitt Clinton
Not all of it is RAM. We use normal hard drives for this system, and we have enough RAM such that the working set fits in memory. Most of the oldest stuff on FriendFeed is not accessed frequently, and that data is not typically resident in memory. - Bret Taylor
Thanks Bret, nice post. - Ömer Faruk Kurt
So for something like 'user_id_index.get_all' you'd trust that InnoDB has that user in RAM, and that you have only a small set of active users at any given time (for web requests). Same for the relevant entities you'd "join" against. Makes sense. (Sorry, having a hard time typing coherently tonight.) - DeWitt Clinton
Yah, those indexes are typically in RAM. Our indexes contain a timestamp as well (since they are ordered reverse chronologically), so the oldest entries referenced by that index would likely not be in RAM depending on how many pages back they are on most feeds. - Bret Taylor
BTW, did you guys look at any other serialization mechanism other than pickle? Any pros/cons for cpickle? - Arvind Sundararajan
cPickle is the same format as pickle, just implemented in C instead of Python. We use cPickle. marshal is faster, but according to docs may change from version to version of Python, so we didn't want to use it in our DB. - Bret Taylor
@Bret "250 million entries", is it possible to give the size of the DB? and what is your opinion about when will you need to change this system again or is it strong enough for years Meybe you will only need hardware instead of structure changes? - Ömer Faruk Kurt
Ömer: I don't have an exact number off the top of my head, but a lot of data is the indexes themselves, so adding indexes is almost as significant as adding entries. - Bret Taylor
This sounds exceptionally similar to how the AppEngine datastore does things, except that it's MySQL and not BigTable. - Alex Power
Aren't commercial RDBMSes (like Oracle, MSSQL) designed to support the sorts of things you're looking for, like online reindexing and joining across shards? Why not use them? Lack of experience? NIH? Too expensive? - Gabe
"We like MySQL for storage, just not RDBMS usage patterns" Excellent post Bret, thanks for sharing! - Mahesh CR
Gabe: "designed to support" and actually working are two different things. I think that is why companies like ours generally don't use commercial software and choose to use open source software. If it doesn't work, we can fix it. In my experience, most of those features don't work at the scale consumer web sites need, and it certainly wasn't worth the cost and time to find out from our perspective. - Bret Taylor
Non-RDBMS-patterns may work when there is no great needs for processing the actual data (like reports etc). In LOB-apps this would be a big no-no, but in large-scale services like FF alternative methods are almost necessary... - Jemm
Thanks for the writeup! Is there a reason you don't declare the added_id as an UNSIGNED INT? - Roger
Roger: no, no reason. - Bret Taylor
@Bret, this is a pretty cool write u!. Might I suggest as a subject for future write ups, how the realtime update system works (e.g. how it knows what updates it needs to push), and perhaps how the feed fetching system works and how these might inter-relate? - Ray Cromwell
Ray: good ideas. We will do more of these. We want to in theory - they just take a bit of time to write up, and we like writing code more than blog posts :) - Bret Taylor
Bret, do you really think that guys like Oracle and MS create features for enterprise-level databases that don't work or scale to meet the needs of those who buy them? Of course you may be right, but the TPC tests generally get results in the hundreds and thousands of transactions per second. In reality, though, what can you really fix if something doesn't work? Are you going to debug MySQL if it starts corrupting data or optimize it if it's too slow? - Gabe
Gabe: yes, we have already debugged MySQL when it has crashed. It would have been impossible without the debug symbols and source code. - Bret Taylor
Gabe: I've been doing a fair amount of tracing through MySQL core dumps over the past week, and just having the ability to look at the code, plus the size of the community working with the same code base and debugging problems, seems to make it worth the cost at this point. We haven't changed any of the MySQL code yet, but Google and others certainly have. - Jim Norris
This is the best thing I read today. - pastas9
I'm skeptical of Oracle and other commercial pre-packaged systems because the companies are focused on extracting revenue via software sales and their products are highly optimized for a certain problem domain that doesn't really fit our experience very well. - Jim Norris
So I guess the downside is that if you change your indexes you need to change the code, too. How do you manage the table creation, code migration and data sharding all at once? - Nick Lothian
Nick: It is actually not bad: make the tables, update the code, start writing to the indexes for new entities, then run the "Cleaner" to fill in the indexes for the older entities. When the cleaner is done, you can start using the indexes for features. - Bret Taylor
The most important decision about your design- and why not going with databases RDBMS etc. to manage the indices - is that your users don't really look up the old stuff, so keeping the old entries in RAM/live indices isn't worth the time hit it would take to do huge indexes on long tables. This seems like a "running index"- that is, it's populated by the application in many places at one time, instead of in one place, that the DB then optimizes over the entire history. - anna sauce
Reminds me of some multi-user java apps back 10 years or so ago that had to manage simultaneous users on live systems. great post, Bret, and fun to see what's working behind the scenes. - anna sauce
"if you change your indexes you need to change the code" is not a problem unique to this system. You always have to update your code along with your schema. You have to write code to read from the tables even if you use a typical RDBMS, and you can't do that without an index, so this staged process exists in some form no matter what your storage scheme looks like. - Bret Taylor
Bret, it sounds like MySQL crashes a lot. Are you assuming that commercial products will crash just as often and the vendors won't debug it? - Gabe
Gabe: Every piece of software crashes, whether it is written by Oracle or by open source developers. The difference with open source is that we are not dependent on someone else to diagnose and fix the problems. This is not a unique sentiment. Google, Yahoo, Facebook, FriendFeed, Twitter, et al, have all chosen open source infrastructure for this and a variety of other compelling reasons we have already discussed. - Bret Taylor
Brett, I agree that not being dependent on somebody else is great. In fact, the product I work on is probably 99% open source or developed in-house. I even replaced a relational DB with a pickled Python object store a year or two ago. However, I'm also not considering writing my own transaction protocol, indexing, and query engine because the open source DB I use doesn't support... more... - Gabe
Re: "change your indexes you need to change the code" - yes, conventionally you need to change your code if you change your tables. But you do get some opportunities for runtime optimisations by changing your indexes (although in practice this can have availability costs as you note) - Nick Lothian
OK I'm going to put this out there: sounds like you're re-inventing the wheel. - anna sauce
Being able to open something up to understand, diagnose, and fix things yourself is underrated. - Amit Patel
DeWitt, part of the reason we didn't use a simpler store is that we're using mysql replication. We'd have to replace that too. Probably not hard, but it's working well for us. Also sometimes you do want transactions for performance within a single DB (each xsaction is a single log write, vs multiple little writes). - Private Sanjeev
re: SSDs and mysql. I can't speak for all SSDs, but the Intel X25-M gets internally fragmented very quickly, reducing write performance by 10-20x. Things that do large sequential writes like bigtable or lucene are a better fit. - Private Sanjeev
Part of the purpose behind traditional schema driven designs is to 'protect' the data from the programmers and the applications - which is not insane given the level of commitment people have in some kinds of organization. One of the statements that the FF people are making here is that they have trust in themselves and each other and they care about what they are doing. - Robin Barooah
You guys rock so hard it HURTS! THANK YOU, FRIENDFEED! - Josh Haley
Just logged in... this sounds really cool.... hmmm :) - Susan Beebe
great post -- liked the discussion in the comments too! some people thinking relational databases can do everything; other people not understanding rationale behind key/value store; other people mad you've reinvented it using MySQL... :-D favorite comment from guy who is having a heart attack because anyone might think it's a good idea: "it is like using a database to solve the problem... more... - Karim
what is the correct name for this? "schema-less data?" "entity attribute/value (EAV) store?" "key/value store?" personally, i like "property bag" :-D also, have you considered Amazon SimpleDB, or Microsoft SQL Data Services for hosted services, since they are based on similar ideas, just not on MySQL? - Karim
Sanjeev, I heard rumors about the X25-M performance running down after fragmentation, but last I heard was Intel was unable to replicate & "looking into it." 10-20x should not be hard to spot. :-) Are you in touch with Intel...? Also, have you seen similar problems on SSDs with Samsung controllers? (I went with the Corsair...) - Karim
Karim, it may not get as bad with other SSDs, but they start off with much higher write latency for small random writes than the X25-M. - Private Sanjeev
The Samsung, you mean? All I was trying to do was *not* get the JMicron controller and *afford* it :-D - Karim
but why are the ssIDs varchars, is the question (from @eonarts comment) - anna sauce
anna: what are ssIDs? If you mean the UUIDs, they are just 16 byte binary strings. - Bret Taylor
Just look how many comments on this begin with someone's name. "anna:" "Karim," "Sanjeev,". Somehow there needs to be a reply-to-comment mechanism. Fortunately FriendFeed uses json blobs in their database so this won't require a painful schema change ;-) - Kevin
Bret: you're right, I re-read the post and couldn't find evidence of the Primary Key Erin mentions in her comment, as varchar. Am I missing it? - anna sauce
schema changes aren't painful, and I have a question for more current DB/programmatic people (than me): why is this called schema-less? There is a schema, it's just that the database application isn't being used to index, Bret's crew are doing it manually. Blobs in columns doesn't mean it's schema-free, that's been going on for 15 years. I think it's just that he's cracking the indexing methodology and offroading. I'd like to hear from some PostGres people on this- Disqus is on PostGres aren't they? - anna sauce
Anna: that person is talking about the URL index. What we are doing is correct for this application; we want the index to be in that order on disk, not in a different order. - Bret Taylor
Bret: ah, ok - anna sauce
I just remembered that one of my client's DBs is actually Btrieve, which is nothing more than a way of storing binary blobs with certain byte ranges indexed. (They are still using a DOS app from the '80s, with the Btrieve running on a Novell server.) The only real difference is that they have fixed-length keys and records, while the MySQL data is variable-length. - Gabe
Bret: Thanks a lot for the informative post! Any thoughts on MySQL vs Postgres? We found tables crashed with too many simultaneous writers in MySQL, but your post says it "doesn't corrupt data", so I guess you haven't had such issues. - Ruchira S. Datta
Bret: Do you apply the same approach to perform searches on your stored text data? (i.e. inside the data storing the BLOB) - Roger
Delivr: Share mobile friendly links. - http://delivr.com/
Picture 35.png
All they are doing is giving you the google mobile version with extra junk you don't want or need, which you can do it yourself by prefixing the url with this: http://www.google.com/gwt... - April Russo (FForever!)
@April Russo http:// option is just one of several options provided. Each option providing a mobile-friendly page with various enhancements. ...and do you really see the masses prefixing URLs in front of other URLs on their mobile devices? - David Harper
Other ways to read this feed:Feed readerFacebook