brad's life - Friends page in JavaScript [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Friends page in JavaScript [Sep. 12th, 2005|10:49 pm]
Previous Entry Add to Memories Tell a Friend Next Entry
[Tags|, , ]

In my copious free time, I've "ported" the LiveJournal friend's page algorithm from Perl (running on the server), to JavaScript (running in the browser).

So instead of making one huge friends page request and getting back a rendered document with 95% duplicate stuff that your client has already downloaded 15 seconds ago when you habitually hammered refresh, your client instead makes dozens of tiny requests, easily cachable on both the client and server, all the way up to the BIG-IP where I can later just instruct it to 304 any If-Modified-Since request on certain immutable URLs. (ala /userpic/ requests)

Plus it can have a background thread polling the last updated times of your friends page, deciding when to refresh itself, just doing the minimum work necessary to update fully. And even if you hit shift-reload to cache-bust, that only cache-busts the pages necessary to onLoad() the document. After that, javascript takes over so the browser goes back to caching. (Had to do some an old-school work-around to make Firefox cache, but IE behaves.... I wish XmlHttpRequest's caching behavior was specified and consistent.)

Anyway, works in IE/Safari/Firefox. Been reading my friend's page in it the past couple days. Pretty addictive.

I'll post a URL when it's beautified a bit and it's not lacking 75% of its future feature set.
LinkReply

Comments:
From: [info]sherm
2005-09-13 05:55 am (UTC)

(Link)

?mode=live lives again!!!!!!! ;)
[User Picture]From: [info]brad
2005-09-13 05:57 am (UTC)

(Link)

?mode=live is currently live on the site, but I doubt it works. I removed it from cvs the other day, knowing this was on its way, and to make the code cleaner.
[User Picture]From: [info]midendian
2005-09-13 06:00 am (UTC)

(Link)

Neat.
[User Picture]From: [info]bigjimsjazz
2005-09-13 06:04 am (UTC)

(Link)

Sounds cool.
[User Picture]From: [info]zetawoof
2005-09-13 06:04 am (UTC)

(Link)

This does deal with edited entries correctly, right?
[User Picture]From: [info]brad
2005-09-13 07:05 am (UTC)

(Link)

It's not even released yet. So relax.
[User Picture]From: [info]grumpy_sysadmin
2005-09-13 01:09 pm (UTC)

(Link)

So that's a "no" then. ;^>
[User Picture]From: [info]erik
2005-09-13 04:09 pm (UTC)

(Link)

I'm so going to start randomly bolding text in my posts now. It's on like donkey kong.
[User Picture]From: [info]avva
2005-09-13 06:06 am (UTC)

(Link)

Wow.

I've been waiting for you to do this eventually, with your enthusiasm for
XmlHttpRequest... but, Wow.
[User Picture]From: [info]mcfnord
2005-09-13 06:11 am (UTC)

(Link)

you're a clever cat.
[User Picture]From: [info]matthew
2005-09-13 06:25 am (UTC)

(Link)

Rock on. Anything that reduces load makes me a happy admin :)

Why aren't we 304'ing If-Modified-Since requests on those immutable URLs already? It would save some requests from googlebot.
[User Picture]From: [info]evan
2005-09-13 06:50 am (UTC)

(Link)

for the current entry pages on lj, they can change -- you can edit the post, or get another comment, etc. i think brad constructs unique urls each time the page changes so a given url is always static.
[User Picture]From: [info]brad
2005-09-13 07:06 am (UTC)

(Link)

There aren't many immutable URLs right now. Just /userpic/* really.
[User Picture]From: [info]mart
2005-09-13 07:01 am (UTC)

(Link)

How are you doing the page construction? Is it client-side S2, or have you come up with something better?

[User Picture]From: [info]brad
2005-09-13 07:04 am (UTC)

(Link)

It won't be user-customizable. Likely/maybe CSS for font/colors/etc, but not S2, and not S1. There might not even be user CSS, just a selection of a few themes users can select. I want to control the DOM and classnames and ids so I can do things like commentmanage.js easily, without each and every fucking style having to advertise its DOM structure, and none working perfectly.

[User Picture]From: [info]mart
2005-09-13 07:11 am (UTC)

(Link)

Well, once you've got it exporting data as JavaScript I'll probably go ahead and wire it up to client-side S2 just for my own amusement. I'm not as fanatical about cute DOM/AJAX tricks as you are. ;)

[User Picture]From: [info]revmischa
2005-09-13 07:26 am (UTC)

(Link)

this sounds familiar...
[User Picture]From: [info]brad
2005-09-13 07:40 am (UTC)

(Link)

Yeah, we give the hard projects to the interns. :-)
[User Picture]From: [info]torgo_x
2005-09-16 10:55 am (UTC)

S2 In The CSS Dimension

(Link)

Speaking of S2 and CSS and customizability, this S2 thingy I made might interest you. I heard rumblings at some point from someone there at Danga who was going to make it a system style, but I think that that idea got mentally lost in the move down to SF.
[User Picture]From: [info]taral
2005-09-13 07:09 am (UTC)

(Link)

So how's that ?start= for friends pages coming, hm? You know, the one that was in the works YEARS AGO? :P
[User Picture]From: [info]brad
2005-09-13 07:16 am (UTC)

(Link)

Not gonna happen. Doesn't scale. The friends page algorithm is finishes in 's' steps, where 's' is the number of items to view. The ?start= would require way more indexes (beyond being worth it) and/or an algorithm that runs in 'f' steps, where 'f' is your number of friend, which also won't happen.

However, bookmarkable URLs are totally doable, encoding in them the content of the page, and where you're at in time. Then the previous/back links can be optionally shown, if the time you're viewing the page is within the 2 week scrollback period.

As for solving the "I was away on vacation for 3 weeks" problem where you want to catch up going forward in time, that's more of the always-talked-about-but-never-done ESN system, which is actually now on our roadmap after Scrapbook improvements, but since I'm not so involved with Scrapbook, I might work on ESN a bit earlier.

In fact --- much of my work on this javascript friends page is to make ESN integration a lot easier in the future.
[User Picture]From: [info]taral
2005-09-13 07:24 am (UTC)

(Link)

Okay, bookmarkable URLs is all I wanted to start with. :)

What is ESN?
[User Picture]From: [info]mart
2005-09-13 07:27 am (UTC)

(Link)

Event Subscription/Notification. Everything triggers an event, and you can subscribe to given events with given notification. Possible notifications could be your new friends page or email, IM, etc.

[User Picture]From: [info]nothings
2005-09-13 09:19 am (UTC)

(Link)

Ok, I can't quite infer what the ?start= would be, but I guess bookmarkable URLs would address the "going forward in time, you'll miss entries if anyone updates" issue?

Topic #2, nothing to do with the above: isn't the fact that shift-reload is only going to reload the javascript loader code actually a bad thing, not a good thing, since it breaks the actual point of shift-reload (which, for me, at least, is "hey browser, some of this stuff didn't load right for some reason, uncache it and try again")? I'm not clear what the use case is for where you want shift-reload to do less (well, from a user's standpoint, it's obvious from an LJ-load standpoint).
[User Picture]From: [info]alohawolf
2005-09-13 07:16 am (UTC)

(Link)

presumably when you put this into place, it would auto detect that your browser could support, and do it the old way, if the browser didn't support Javascript fully?
[User Picture]From: [info]brad
2005-09-13 07:18 am (UTC)

(Link)

Yeah. Or more likely tell you to go use the friends page, since this system is at a different URL.

There will be a new option to let you disable your friends page... either completely, or just the link to it from your journal.

So for a couple years (or likely forever), the friends page will still exist for those that don't want to use the fancy version.
[User Picture]From: [info]alohawolf
2005-09-13 08:23 am (UTC)

(Link)

see.. once it was working, I would default it to the new system, and then allow a user to select much like they can select a page style for LJ, select which manner they wish to use.
[User Picture]From: [info]brad
2005-09-13 04:04 pm (UTC)

(Link)

Naah, it'll be so different, it'll be opt-in. I won't make it the default.
[User Picture]From: [info]subbes
2005-09-13 02:29 pm (UTC)

(Link)

Or more likely tell you to go use the friends page, since this system is at a different URL.

Could those with no javascript get bounced to a (possibly re-written, since it apparently wasn't 'clean code') version of mode=live? That would allow javascript-less people to enjoy the live view.
[User Picture]From: [info]brad
2005-09-13 04:05 pm (UTC)

(Link)

Not going to happen.
[User Picture]From: [info]subbes
2005-09-13 04:08 pm (UTC)

(Link)

Ah, well, thanks for the answer.
[User Picture]From: [info]taral
2005-09-13 07:53 am (UTC)

(Link)

Wait. This means you have machine-readable interfaces for (a) retrieving entries and (b) retrieving entry links for the friends list. Are these going to be published interfaces?
[User Picture]From: [info]andrewducker
2005-09-13 01:18 pm (UTC)

(Link)

That was my first thought - this could all become client-based! Oh, to have a nice news-reader type interface for LJ.
[User Picture]From: [info]brad
2005-09-13 04:07 pm (UTC)

(Link)

Yup. They exist and they're even live, albeit at a "secret" URL. I doubt they'll become published or "stable" interfaces, though, since the product is sure to change rapidly over the next several months.

But I suppose another version can be made stable.
[User Picture]From: [info]taral
2005-09-13 07:47 pm (UTC)

(Link)

Oh, yes please. :)
[User Picture]From: [info]d4b
2005-09-13 01:26 pm (UTC)

(Link)

Such a shame, all of those unused CPU cycles out there... I'm speaking of those who are repeatedly hitting their refresh button in the first place.

Sure wish I could show them ads while they were waiting. :-)
[User Picture]From: [info]brad
2005-09-13 04:05 pm (UTC)

(Link)

Go away. :P
[User Picture]From: [info]dakus
2005-09-13 04:12 pm (UTC)

(Link)

Can we see it yet?

I'm imagining mouseover comment# links having a popup list of usernames that have commented and other cool stuff...
[User Picture]From: [info]lisa
2005-09-13 04:27 pm (UTC)

(Link)

bossome.
[User Picture]From: [info]brad
2005-09-13 09:37 pm (UTC)

(Link)

Is that when the boss does something awesome?
[User Picture]From: [info]jay
2005-09-13 07:50 pm (UTC)

slick

(Link)

I hope you can get before/after stats of what you attribute to load contribution of serial reloaders.

It seems like it would go a long way to removing one of my bad user habits.
[User Picture]From: [info]torgo_x
2005-09-16 10:59 am (UTC)

Higher-Order JavaScript

(Link)

Speaking of JavaScript, you might like this stuff I've been writing lately.