Log in

No account? Create an account
On PubSubHubbub and rssCloud ... - brad's life — LiveJournal [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

On PubSubHubbub and rssCloud ... [Sep. 15th, 2009|11:17 am]
Brad Fitzpatrick

Are you confused about the difference between PubSubHubbub and rssCloud?  You're not alone.

Here's how the confusion came about:

Dave Winer invented rssCloud way back in the day.  It only distributed lite pings, the callback endpoint was the IP address that you subscribed from, and nobody really ever implemented it, so you probably never heard of it.  We sure hadn't.

Fast forward 5 or 6 years.  Brett Slatkin and I want to fix the polling problem and we're annoyed that all companies seem to have an internal pubsub system, but none of them work on the Internet, and XEP-0060 just isn't getting adopted, probably because XMPP weirds people out.  We start sketching out PubSubHubbub.  From day 1 we do all development in the open, on pubsubhubbub.googlecode.com.  Initially we only target Atom for simplicity until we got a prototype working.  We work on PSHB for about a year, during which time we hear about rssCloud and are impressed at the foresight but reject it as not satisfying our design goals.   After a year of working on PSHB, we demo it at the "TechCrunch Real-time Stream Crunch-Up" event.  We add RSS support a few days before the event because it's trivial at that point.  It's unfortunate that both Atom and RSS exist, but that's reality, so we support both.

Right after we present, we get a request to call Dave Winer.  He wants to "do voice", so Brett and I hop on our cellphones and the three of us have a conference call, with Brett and I outside on the street trying to find places to stand with less road noise.  Over the next 15-30 minutes, we slowly walk him through PubSubHubbub, repeatedly, explaining why webhooks and fat pings are important (no thundering herds DoSing publishers!), explain all of our design goals (pushing complexity to the hub, keeping publishing simple, decentralized, using HTTP, etc, etc...)

Dave Winer writes an article praising PubSubHubbub.  Great!

Dave Winer reads the PSHB spec and notices it still says "Atom only, not RSS".  Shit.  We forgot to update the spec after we added RSS support.

Perhaps due to our RSS documentation omission, or perhaps because he realized pubsub was finally in vogue, he's now gone and dusted-off and augmented his old rssCloud protocol that's RSS-centric.

The arguments in favor of rssCloud go something like this:  'we can't have BigCo control the spec.  We should have an independent spec!'  Or, in his words: "Google sux".  To reply to that specifically: This isn't even a Google-initiated project --- it's Brett & my 20% (or 5%?) time project, trying to fix something we find annoying on the web. We've been transparently working on this in the open from day 1.  Yes, we happen to be Google employees.  We have no internal docs or project plan on this.  If Dave wants something different, he's just as welcome on our mailing list as everybody else (many individuals and companies, working together to build consensus....).  Instead, he's heavily promoting the largely-unchanged rssCloud and not wanting feedback. Seems silly, but that's that.

Unlike rssCloud, which Winer says is frozen and a "done deal", the Hubbub protocol isn't frozen.  It's in development so far as we'll make changes and additions that are good and useful, and try hard not to break backwards compatibility (especially on pinging).  We have a few major things yet mostly untackled (including distribution of private content).  The rssCloud mailing list says "This is a mail list not a standards body".  If you'd like to work on a standard, join the PSHB mailing list.

Some of the good articles on the technical differences between the two protocols:
Anyway, I apologize for all the confusion.  I feel like had we only promoted RSS more heavily in the 0.1 draft of the spec, I wouldn't be writing this blog post today.

Hopefully this is the last I'm going to say on this topic.  Back to doing productive things....

From: (Anonymous)
2009-09-15 07:05 pm (UTC)

RSS's cloud was in fact implemented

Hi Brad -- Dave Winer here.

We implemented the cloud element in Radio and Manila, two leading blog platforms, in 2001 and 2002. So it's not true that no one implemented it. And it wasn't hiding, it was right there in the RSS 2.0 spec, which is a very widely implemented format.
(Reply) (Thread)
[User Picture]From: brad
2009-09-15 07:10 pm (UTC)

Re: RSS's cloud was in fact implemented

Sorry, I meant widely implemented. 2 implementations isn't very many.
(Reply) (Parent) (Thread) (Expand)
(Deleted comment)
From: (Anonymous)
2009-09-15 08:53 pm (UTC)


Brad. Kudos for Pubsubhubbub: this is a key piece of technology. I hope that you and Dave can found a middle ground: the competition is not an alternative spec but the proprietary micro messaging platforms. May be the solution is to: get pubsubhubbub, make RSS a first class citizen, add a section explaining how it is an evolution of RSS Cloud) and even use the RSS cloud name (in my opinion pubsubhubbub is too complex of name for it to become mainstream).

Can we try to set up a time for you and Dave to sit down and discuss this?

-Edwin Khodabakchian (edwink@devhd.com, co-founder of feedly)
(Reply) (Thread)
From: (Anonymous)
2009-09-15 10:21 pm (UTC)

Re: Fusion

Edwin: Could you elaborate on what you mean by first-class citizen? RSS is fully supported (0.92, 1.0, 2.0) by the reference implementation and documented here (http://code.google.com/p/pubsubhubbub/wiki/RssFeeds).
(Reply) (Parent) (Thread)
From: rcade [typekey.com]
2009-09-15 09:36 pm (UTC)

I'm Glad PSHB has a Public Process

You're right to tout the benefits of how PubSubHubbub is being created.

As a member of the RSS Advisory Board, I can't stress enough how important it is for specs to be written as a public process. The RSSCloud Interface should not be controlled by one person, as I argue today on my blog:

http://workbench.cadenhead.org/news/3559 (http://workbench.cadenhead.org/news/3559)
(Reply) (Thread)
[User Picture]From: kragen
2009-09-16 01:49 am (UTC)
As usual, interacting with Dave Winer leads to no good. You should have known better than to try to engage with him. Best just to ignore him, stay far, far away, and don't let him get involved in anything you care about.
(Reply) (Thread)
[User Picture]From: anildash
2009-09-16 04:26 am (UTC)
Kragen, fwiw, I've been interacting a lot with Dave lately, and a bit with Brad and Brett on this, to try and help explain the benefits of all this work to outsiders. Here's the real-world perspective: All three are on the side of the angels. They're doing good work, iterating in their own way, and are gonna get good results. And frankly, they've all been magnanimous and open to feedback, even though they're obvious coming at this from slightly divergent points of origin.

Do I wish things were a little less complicated? Sure, but ultimately, the distinctions between these systems will filter out, and we'll have one or two (but less than three!) systems that enable a valuable new ecosystem. Let the shippers ship, and don't be one of those guys who's flaming one side or the other. If you have an opinion, then commit code or make something useful, but I care too much about the overall success of the effort to have decentralized realtime infrastructure to let this turn into another stupid internet shitstorm that distracts from the work at hand.

In short, don't be That Guy. I don't think you wanna be That Guy. :)
(Reply) (Parent) (Thread) (Expand)
(Deleted comment)
[User Picture]From: mart
2009-09-16 06:43 am (UTC)

This reminds me of the early days of OpenID when there were all the other random OpenID-like systems that did mostly the same thing but with different bits on the wire. I think just as with OpenID vs. TypeKey (for example) it won't be too hard to map between these things so that one can be implemented in terms of the other.

While we're on the subject of bridging things, it'd be neat to have a hubbub hub that can use weblogs.com-style pings as an optional second notification mechanism so that folks running software that already supports such pings don't need to install anything new to get started.

(Reply) (Thread)
From: ext_207224
2009-09-16 07:17 am (UTC)
I already did this in July, but we haven't turned it on:


The issue is that XML-RPC pings only take the channel URL, so you have to parse HTML to rediscover the actual feed URLs, or maintain a relation of feeds to channel links (the latter is easier, actually).

Otherwise, I still think that light pinging is going to hinder everyone long-term. The performance characteristics are bad. I'm starting to expound on why here:


Feedback would be appreciated!
(Reply) (Parent) (Thread)
From: evan
2009-09-16 07:53 am (UTC)
Comments on some posts I read indicate they think only Google can run a hub, which is where the "bigco sux" stuff comes from. Can you preemptively respond to that? (Perhaps by amending your post in the paragraph where you talk about 20% time.)
(Reply) (Thread)
From: evan
2009-09-16 07:56 am (UTC)
PS they just checked libjingle into Chrome since it's used for bookmarks sync. If you have any grand plans involving XMPP bridges into JS I'd be interested in hacking them out.
(Reply) (Parent) (Thread)
From: ext_207224
2009-09-16 02:44 pm (UTC)
So far, I did a presentation on how to build a hub here:

But more documentation (and a more thorough compliance test suite than we already have) would be great.
(Reply) (Parent) (Thread)
[User Picture]From: mirukux
2009-09-16 11:20 am (UTC)
and XEP-0060 just isn't getting adopted, probably because XMPP weirds people out

what about GTalk and Wave? and how will PubSubHubbub communicate with these? XMPP-0060, some other gateway or, ugh, a bot?

what do you think of the Superfeedr architecture?

why can't gateways and clients start using XEP-0004 more like XEP-0060 asks? what if Google had bridged this chicken-and-egg gap.. (possibly with the help their own handrolled protocol a la VOIP *wonders how long until someone implements Jingle in Wave..*)?
(Reply) (Thread)
(Deleted comment)
[User Picture]From: kragen
2009-09-16 04:35 pm (UTC)
That's completely unfair. Ted Nelson actually invented a lot of cool stuff, wrote a couple of important books, and funded a lot of research.
(Reply) (Parent) (Thread) (Expand)