Dave Winer
A more low-tech approach to ping hubs - http://www.scripting.com/stories...
The changes.xml approach is basically what is codified in the Simple Update Protocol that FriendFeed has been developing: http://code.google.com/p.... It's still polling, but it is a more efficient polling mechanism for large numbers of feeds, and it can be published either by an individual site or an aggregator. RFC here: http://simpleupdateprotocol.go... - DeWitt Clinton
How could it be changes.xml when it's not even XML? - Dave Winer
When you guys design these things, do you think a few steps into the future, when there are N ways to do something instead of 1. The confusion about whether your format will gain traction or the other one cause people to freeze, so neither one gets uptake. I cannot for the life of me figure out why you wouldn't just use what's already there. - Dave Winer
Did you read the spec? : ) - DeWitt Clinton
I didn't read the whole thing -- I got as far as seeing that it's JSON and realized there was no point continuing. - Dave Winer
Please read further if you have a sec. I'd love to hear your feedback -- there's still plenty of time for comments (it's still in pre-review). SUP adds a) opaque tokens for private resources, b) the ability to work with arbitrary addressable resources, not just feeds, c) poll and update intervals, and d) a link and HTTP-based discovery mechanism. Regarding the JSON format instead of XML, I'm agnostic -- I don't think it matters much. - DeWitt Clinton
The discussion room for SUP is here: http://friendfeed.com/simple-.... If people have comments, that's the best place to post feedback about SUP itself, I think, as Paul, Gary, Ben, Ade, me, etc., are following that. - DeWitt Clinton
The only reason I looked is because of the first sentence of your first comment. I got as far as seeing that it wasn't changes.xml. It is what I thought it was when I first looked a few months back. - Dave Winer
Feedback: I make the same suggestion to you that I made to the IETF guys when they were embarking on Atom. Start with changes.xml, and then change whatever it is you feel you can't live with, and document your rationales. That way what you end up with will be minimally different from what's already out there, and future implementers won't curse us for not having the sense to have one way to do things. - Dave Winer
Analog: When I travel to Europe, I wonder why they couldn't just do electric plugs the same way we do in the US. That way I wouldn't have to carry an adapter with me. I wish their cell phone systems worked the same way ours did (I gather they do now, somewhat) and that they billing worked the same (I'll let you know when the bill from my June trip arrives). When I travel to London I wish they had the good sense to drive on the correct side of the road. Each of these inconveniences was caused by engineers thinking they didn't "have to" worry about the way things were done before. They were right, they didn't have to, and all future users paid for their insistence. - Dave Winer
Heh. Is this where someone is supposed to say the secret word ('Standardization') and the duck flies down?? - ɐ ɯıʞ sıɹɥɔ
@Dave - thanks for the feedback. Can you point me to the changes.xml specification? - DeWitt Clinton
@DeWitt: You'll find a link in this piece which I just wrote: http://www.scripting.com/stories... - Dave Winer
BTW, I just re-read it. The Notes describe the way it was used in 2001, a long time ago, and before the weblogs world grew at a huge pace. We eventually had a shortChanges.xml feed that had to be polled much more frequently than once an hour as we started getting a million then two million pings a day. I have no idea what it's at now. - Dave Winer
Yeah, scale becomes an issue. For example, Google Blog Search publishes a changes.xml for every changed feed it discovers (http://blogsearch.google.com/changes...), and it weighs in at a whopping 11MB for just the last 10 minutes. I have no idea whether this is even comprehensive. - DeWitt Clinton
I think its cool that Google are bending over backwards to get your input on all this Dave, and seem seriously interested in your advice. - Mark
Mark, what matters is what they do, not who they stroke. - Dave Winer
Remember when Microsoft came into RSS? They did a lot of stroking, but in the end, not enough listening. And the net-result was a big zero. They did nothing to improve things, and they really could have. - Dave Winer
We're in an equivalent situation now, with Twitter rolling up the entire pubsub space with virtually no opposition. We should have a small-pieces-loosely-joined alternative, it wouldn't take very much, but Google has to get their shit together here and lead. Add what they can add, and don't try to re-solve problems that have already been solved. - Dave Winer
@Dewitt, that's why I said in my essay today that if there are parts you must change, that you can't live with, then change them, and document your rationale (and also put that up for review). - Dave Winer
I remember watching this video in 2005 on the subject of Microsoft and RSS. The Tee Shirts really sold me http://channel9.msdn.com/posts... - Mark
From the Microsoft video on RSS: "Today Microsoft announced the addition of several new RSS features in the next version of Windows, code-named Longhorn. The Longhorn Browsing and RSS team (the one we interviewed here) is also are announcing a new RSS extension, to be released into Creative Commons, that lets you do lists in subscriptions." - Mark
Agreed, Dave. Though I should point out that FriendFeed authored and invented SUP -- I'm merely helping edit the RFC. I started doing so because there was interest within Google for it and no one wanted to implement SUP without a stable specification. So we were following a herd here; it was just Paul's herd, not a different one. You'd have to ask him why he chose JSON and whatnot. I just wrote down what he was doing on FriendFeed. - DeWitt Clinton
To start -- I want to be able to ping FF saying that my RSS feed has updated and have them read it right now. It's got to be that simple. And then why not just use the same endpoints that weblogs.com uses so that any blogging software can do it without code updates. - Dave Winer
code updates are sometimes necessary and/or unavoidable - and then again sometimes not - Chris Heath
It seems to me that SUP and changes.xml are aimed at slightly different use-cases: changes.xml is mainly aimed at the individual instance of a blog notifying a centralized service, whereas SUP seems aimed at centralized services notifying each other via a firehose. I do understand that these use-cases overlap, but do I have the distinction right? - Michael R. Bernstein
Regarding what I think we'd want to see in a refresh of changes.xml -- ignoring the distinction between JSON and XML for a moment -- the list is the same as above. First, I'd want it to be non-feed specific, and work with any addressable resource. This is the substantive terminology change I made when drafting the SUP spec. The second is a discovery mechanism, like we formalized using mnot's spec for HTTP link headers. - DeWitt Clinton
The third is the capacity for opaque resource tokens, which is necessary to support private feeds. And last, configuration/visibility of the aggregated intervals. - DeWitt Clinton
Dave: you can use public SUP (http://friendfeed.com/api...) to ping FriendFeed. It's as simple as adding a HTTP header or link tag to your feed and then pinging Friendfeed. - Benjamin Golub
And you can see the code in action on my blog: http://github.com/bgolub... it's only a few lines of code to be able to ping FriendFeed. - Benjamin Golub
Also, if you only want to ping FriendFeed (and don't care about the info being available to anyone else), then you don't even need to include the HTTP header or link tag -- you can simply ping FriendFeed with the url of the feed. We've also been consuming the weblogs.com and google blog search changes.xml for about 1.5 years now. In short, we take every available source of info -- we're not forcing any one solution on anyone. - Paul Buchheit
@Benjamin: Let me play that back to see if I understand. If I do an HTTP GET on this URL: http://friendfeed.com/api... that will tell FF that the scripting.com feed has updated. It will then fetch the feed and if there are any changes that will be reflected on FF. Correct? - Dave Winer
My response to you dave: http://rbonini.wordpress.com/2009... (and you are welcome to tell me how wrong I am - :) ) - Roberto Bonini
Roberto -- yes that could work. But we already reinvented RSS when we came up with changes.xml -- a format that basically does what you're saying RSS could be used for (which it could). You could argue that we shouldn't have reinvented way back when, but the politics of the software world were different then and what's done is done. To use RSS now for that purpose would itself be reinvention, and I wouldn't advocate it. - Dave Winer
I just tried a test, and did a GET on the URL included above, and nothing showed up here. So I guess that isn't the correct protocol. I'll wait for feedback. - Dave Winer
It showed up after 3 minutes. Not sure if that was due to polling or because I pinged. - Dave Winer
Understood Dave. - Roberto Bonini
That's correct Dave, the public-sup ping will work even without the tags (the tags are necessary for discovery, but internally we can connect the ping and the feed without them). We should recrawl the feed within a couple of seconds. If it's not working, double check that the url you are pinging is exactly the same url as the imported rss feed. If it still doesn't work, let me know and I'll try to debug what's going on. - Paul Buchheit
Dave. I looked into our system and we have http://www.scripting.com/rss... as your feed (note the www). Did you try pinging with www? - Benjamin Golub
Yes. - Dave Winer
Dave, I like your thinking - but it seems laced with nationalist pride. The English have superior electrical plugs (with safety features that mean that they are less likely to electrocute people) and of course, they are the original seat of western civilisation, so it is the Americans that stubbornly failed to follow tradition by spelling things all wrong and driving on the wrong side of the road ;-) - Tim Tyler
i get the impression that a sup document must enumerate the set of changed resources. is this correct? if so, this will not work for systems with unconstrained sets of resources, such as a search engine rss feed or a tag intersection feed on say delicious. this seems problematic. - joshua schachter