brad's life - Never ending feed of Atom feeds [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Never ending feed of Atom feeds [Aug. 16th, 2005|12:58 pm]
Previous Entry Share Next Entry
[Tags|, , , ]

An increasing number of companies (large and small) are really insistent that we ping them with all blog updates, for reasons I won't rant about.

Just to prove a point, I flooded a couple of them and found that sure enough, nobody can really keep up. It's even more annoying when they don't even support persistent HTTP connections.

So --- I decided to turn things on their head and make them get data from us. If they can't keep up, it's their loss.

Prototype: (not its final home)

$ telnet danga.com 8081
GET /atom-stream.xml HTTP/1.0<enter>
<enter>


And enjoy the never ending XML stream of Atom feeds, each containing one entry. And if you get more than 256k behind (not including your TCP window size), then we start dropping entries to you and you see:

<sorryTooSlow youMissed="23" />

I think soon we'll get TypePad and perhaps MovableType blogs all being sent through this. The final home will probably be on a subdomain of sixapart.com somewhere, including documentation better than this blog entry.

And yes, I'm sure my Atom syntax is bogus or something. I spent a good 2 minutes on that part of it.
LinkReply

Comments:
[User Picture]From: kragen
2005-08-17 11:35 pm (UTC)

"bogus or something"

(Link)

According to the current Atom spec, the <feed> is supposed to contain an <id> with the URI of the feed (rather than just a <link>) and an <updated> as well; these two, plus the <title>, are the only mandatory bits of <feed>.

The same three elements are the only mandatory bits of <entry>. The standard LJ .../data/atom Atom feeds seem to use some hokey URN for the entry id, but there's no need for that --- LJ posts have perfectly valid, dereferenceable perma-URLs, which are presently in the <link> elements in this stream.

I'm no Atom expert, so I hope this is helpful. Presumably either you will change the stream format to be valid Atom, or people who want to feed these things to their current Atom-consuming software will have to transform your current format into valid Atom --- what do you plan to do? I'd hate to put effort into doing the mapping myself if you're about to fix it.

[User Picture]From: brad
2005-08-17 11:37 pm (UTC)

Re: "bogus or something"

(Link)

The Atom LJ is injecting is total absolute crap. It's just a demo.

TypePad's injecting real Atom and LJ's about to start.