brad's life - Thoughts on the Social Graph [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Thoughts on the Social Graph [Aug. 17th, 2007|02:34 pm]
Previous Entry Add to Memories Share Next Entry
[Tags|, , ]

I wrote a huge post that grew too long for a blog post, and put it here:

http://bradfitz.com/social-graph-problem/

Comments welcome!
LinkReply

Comments:
Page 1 of 2
<<[1] [2] >>
From: evan
2007-08-17 09:50 pm (UTC)

(Link)

Should spell-check. Bunch of typos.
[User Picture]From: brad
2007-08-17 09:57 pm (UTC)

(Link)

Whoops, fixed.
[User Picture]From: jsmarr
2007-08-17 10:03 pm (UTC)

This is awesome work

(Link)

Brad-congratz on getting this all written down, and great job. I think this is just the spark needed to really ignite the open social web blaze to come. :) I'll try to post some more thoughts from my/Plaxo's side, but really excited to see you get this out there! See you at BarCampBlock. js
[User Picture]From: jsmarr
2007-08-18 12:32 am (UTC)

Re: This is awesome work

(Link)

Here are some more thoughts on the subject from our vantage point at Plaxo: http://blog.plaxo.com/archives/2007/08/more_on_social.html
[User Picture]From: drmellow
2007-08-17 10:15 pm (UTC)

(Link)

No comments other than it's cool, and I really hope it takes off.
[User Picture]From: grumbeld
2007-08-17 10:41 pm (UTC)

(Link)

I didn't even know that such a thing was being considered, but I think that it's both brilliant and necessary. I'm looking forward to seeing this come about.
[User Picture]From: girl_on_a_stick
2007-08-17 10:46 pm (UTC)

Two thoughts

(Link)

1. Wow, very cool project

2. Needs more Venn Diagrams
[User Picture]From: obra
2007-08-17 11:03 pm (UTC)

Thank God!

(Link)

This will make my life so much better :)
[User Picture]From: jproulx
2007-08-17 11:40 pm (UTC)

(Link)

Needs more code.
[User Picture]From: brad
2007-08-17 11:58 pm (UTC)

(Link)

I just didn't link the svn repo. There is code.
[User Picture]From: erik
2007-08-18 12:08 am (UTC)

(Link)

It seems like what this is (and I don't mean this in a disparaging way at all - just trying to relate and visualize it) is a glorified Trillian or Adium. It collects various accounts and aggregates them. The question seems to be, how much can be grabbed/aggregated automatically, and how much requires intervention/maintenance by the end-user to keep it functional/correct.

Is it ideal to simply present a user with a page that has check boxes available for every social networking site they use, along with fields to enter their user/pass, and from there some code runs which grabs all of their friend info? Or are you trying to automate it even past this point?

P.S. I guess this is why you were all up on everybody being your friend on Facebook awhile back? I feel duped. Here I thought it was just cause were chums. :-P

P.P.S. Despite Evan's comment and your reply, still shitloads of typos. Mostly grammatical errors. Go look again.
[User Picture]From: brad
2007-08-18 01:32 am (UTC)

(Link)

It's way more than Adium.
[User Picture]From: scosol
2007-08-18 12:41 am (UTC)

(Link)

> Comments welcome!

You asked for it ;)
This is eBay->Y2K all over again-
A social networking site and an auction site both have the same trait of their worth being solely dependant on the size of their userbase.
For about a year, the auction sites were sorting themselves and actual businesses were built around tracking them all and aggregating all the information from them in to one spot.
Then eBay got a bit of a lead, took over, and you've never heard of any of the other competitors or the aggregators since...

Same thing will happen here- one service will take over and everything else will become irrelevant.
(see Google)
My money's on facebook- once they add the luser profile-pimping wizard things, it's all over.
[User Picture]From: memegarden
2007-08-18 04:25 pm (UTC)

(Link)

Facebook is run by the CIA. Some of us will never use it.
From: Judielaine [claimid.com]
2007-08-18 12:52 am (UTC)

Applause

(Link)

I've been a fan of FoaF for ages and was tickled when (i realized) LJ exposed the friend network in that format. I can imagine the problems of just putting data out for other systems to harvest; to make things interesting a harvester/indexer needs to exist. I've been watching sites like ClaimID and peopleaggregator.net and ex.plode.us and Spock as well as Plaxo's new release with fascinating and frustration. Nothing looks scalable yet across countless small specialized social network tools.

Here's a lovely dream: think of how pleasant it would be if un/conferences helped you with your networking -- perhaps sending you a report a week before the event which listed your friends and 2nd order connections who would also be at the event.

good luck!
[User Picture]From: mtbg
2007-08-18 12:56 am (UTC)

(Link)

First, let me just say that this is really cool stuff, and I wish you the best of luck with it. I'd like to say that I'd help hack on it, but realistically, that's just not going to happen :/

A few comments/questions:

The words "claim" and "is" figure large in your slides, but you don't really define them in the essay.

What are your goals (if any) for maintaining currency?

Is the glue bottle supposed to be look like it's ejaculating everywhere?
From: selfmade
2007-08-18 02:19 am (UTC)

(Link)

Good approach.
Gonna publish link to svn repository?
[User Picture]From: iamo
2007-08-18 02:51 am (UTC)

(Link)

As a development manager at a regional social network, this is all pretty interesting. I'll have to keep an eye on it. There's not a lot of activity on the google group, though.
[User Picture]From: ibneko
2007-08-18 03:48 am (UTC)

(Link)

Ah, so interested. Now to figure out how to balance school while getting involved as much as possible.
[User Picture]From: adamthebastard
2007-08-18 05:08 am (UTC)

(Link)

Wouldn't it be nice if everyone would drop their users table and used OpenID to provide authentication. Setup the OpenID providers to track relations to other OpenIDs. The user could define groups of friends (school, work, random internet persons, stalkers, etc.). Then let the user specify ACLs for each social networking site.

If a 'friend' uses their OpenID on a new networking site, when your OpenID provider next scraps your friend's data your provider will notify you and ask if your want to allow the new site your friend is using read access to your relationship.

One identity, one place to maintain relationships. Unfortunately that also means one SPOF, unless someone smarter than me can fix that little problem.
[User Picture]From: skreyola
2007-08-18 04:29 pm (UTC)

(Link)

I think OpenID is a great thing, but I wouldn't be comfortable using OpenID to login to LiveJournal. I like having a separate login here.
[User Picture]From: fallenpegasus
2007-08-18 06:05 am (UTC)

(Link)

I came home from FOOcamp with my mind buzzing with something similar. It was in the zeitgeist, I guess.

Between being annoyed at some of 6As recent actions, and annoyed that if I left, I would lose a lot of valuable social network information. And then the sessions on distributed social networks, openid, oauth. And playing with the crowdvine social network software that had been set up for FOO.

And then I added in my own current hot interest, massive utility grid computing, and the attendent fall in price and ease of access for users to buy really cheap really transient processors.

And I had a vision.

Combine the emergence of "next gen FOAF" data, like you have just excellently described. And take the concept of the small start-your-own social network app, like CrowdVine. And the concept of buying for uber cheap, transient processing with persistant storage.

Everyone could run their very own "social networking site", that would be all about just them. And that it could interop with all the other "sites of one", and also interop with all/most/some/any of the current and future Big Systems.

A person could have something like a LJ/MySpace/Facebook/Vox/etc "account", but it would be easily hosted, using both the current model of renting a cheap hosting provider, like Linode or something, or the next generation of transient computing, like the successor or evolution of EC2.

Probably the basic "system of one" would have the basic core to handle I/O, the netgen-distributed-FOAF stuff, and access (I'm thinking something like gdata, Atom over HTTP with TLS).

And then common plugins for blogging, images, IM, presence, additional access (email from phones, SMS), and "hot what" (think distributed Twitter / Dodgeball / etc).

There doesnt and wont be only one implementation, either. Many many can be written and deployed. As long as they mostly possess as sufficently overlapping set of mutually understood protocols...

Because each node has processing available, it can "do stuff", it's not just a static set of files and of database table rows. It can respond to and explore the Social Graph on it's own, on behalf of it's owner and for people who make both autheticated claims and provisional "are you so-and-so?" claims from it's owners friends.

When I described it to a friend of mine, he suggested I call the concept "Manfred", from the Charlie Stross story.

I think that this is more or less the way things are going to go. The current silos who provide just internal social graphs, blogging, and images, are going to be as snowballs in water brought to a boil.

Even the more "open" systems that join in with the federated distributed social networking protocols, are going to have a hard time going if that's all they offer.

There is going to be no money in running a "basic" social network site at all. About the only user-benift such will be able to offer will be "end user support", and attempts at uptime guarantees, and slashdot effect resistance.

There will surely be lots of money to be made. But it will be via applications that havnt been realized yet.

The online support of the social graph will stop being something that people make money with, and become something that people make money because of.
[User Picture]From: makomk
2007-08-19 02:17 pm (UTC)

(Link)

That's an interesting vision - and one that I broadly agree with - but none of the social networking sites would go for it. Brad's idea seems to be designed not to decentralize social networking in the sense you're suggesting, but instead to make it easier to have accounts on multiple social networking sites. (Like other people have said, it's the equivalent of a multi-protocol IM client, not Jabber.)

The only way it'd mitigate things like 6A's recent actions would be by making it slightly easier to migrate to a different site. There'd still be the problem of moving content, persuading friends to move, etc.
From: ifacethoughts.net
2007-08-18 06:58 am (UTC)

We need jabber for social networks.

(Link)

A big step, but it is still closer to a multiple messenger tool. We need a jabber for social networks (http://ifacethoughts.net/2007/08/18/proposals-for-social-network-interoperability/) or they should be more email like. It should eliminate the need to create accounts in multiple social network applications. A good and detailed document though, a big step towards interoperability.
From: dannyayers.com
2007-08-18 07:31 am (UTC)

FOAF and Linking Open Data

(Link)

+1 to the idea of using FOAF, its web-oriented graph model is the right structure and the simple extensibility of RDF can easily cover all the per-social network extra bits.

re. making the social graph a community asset - there's also a very quick route to this using FOAF, check out LinkingOpenData (http://esw.w3.org/topic/SweoIG/TaskForces/CommunityProjects/LinkingOpenData) (scroll to the diagram - 2 billion interlinked *open* triples). Let us know when you have a graph (mailing list (http://simile.mit.edu/mailman/listinfo/linking-open-data)) and we'll have it interlinked and with tools hooked on in no time.
[User Picture]From: 2xlp
2007-08-18 04:34 pm (UTC)

Welcome to the club

(Link)

Nice to see you joining in on this, Brad.

Its crazy to see how crowded this space is getting -- we've been working to address this exact problem on FindMeOn.com since early 2006.  Back then, it was pretty much ClaimID and us.  Now there are 30+ options?

Anyways, the main difference between our approach and yours is that we've been focusing on the idea of a "lens" of privacy at each network.   We love OpenID, but are scared to death of it-- the problem we encountered was with the misuse of OpenID being an be-all solution.  With everything resolves to the same endpoint, your work and social lives start to overlap -- especially when you don't want them to.  Our solution was to use OpenID as a login mechanism... but use abstracted endpoints ( along with digital sigs for fallback verification ). 

Simplest put -- I want to port my friends and profiles around, but I don't want VCs I'm talking with to see drunken photos of me at a Skeeball match.

With that in mind, we launched our system last August ( our 1 yr bday was 2 weeks ago! ) which would allow people to import friends and to cross-site relationship derivation -- but control visibility on a very granular level.

We've got a really lengthy FAQ section that often reads much like your manifesto.  It was trimmed down last October, I'd be glad to send you some of the old evangelical copy for review.

When it got to the point of dealing with all the different profiles, we tried using a ton of existing standards... but after surveying 300+ proprietary and open profile formats, we just went with an intermediary that we call OpenSN.  OpenSN just wraps every damn field we've run into, so we can quickly convert a myspace profile to OpenSN, then that OpenSN doc to anything else we support under the sun.

We released our identity system as an Open Standard in september at findmeon.org

And we released our profile format as an Open Standard in october - thats at OpenSN.org

If any of that interests you and you want to talk more, please get in touch -- http://findmeon.com/user/jvanasco
[User Picture]From: glyphard
2007-08-18 05:34 pm (UTC)

(Link)

reminded me of jerry mcguire a little bit for some reason.
[User Picture]From: ydna
2007-08-18 09:45 pm (UTC)

On copyrights

(Link)

In Goals 1a:

Establish a non-profit and open source software</a> (with copyrights held by the non-profit)...

The copyright is for the software then, right? What about the data for the nodes and edges? It'll be all over the place. Once my data gets out there, do I retain control of it? I'm assuming control is retained for making updates and corrections, naturally. But I can't really control how others might use it. Would I even care? I don't know. Yeah, it's a ways down the road I guess.
[User Picture]From: ydna
2007-08-18 09:45 pm (UTC)

Re: On copyrights

(Link)

Ooo, nice job on teh hmtl, doofus.
From: karmaweb.wordpress.com
2007-08-19 01:33 am (UTC)

What about context specific relationships

(Link)

The social graph is very dependent on the context...E.g. I might like somebody in a particular professional area but would have not desire to have that person as a family friend...There contexts are typically provided by the application one is dealing with...How can a global social map handle these context specific relationships?
[User Picture]From: harryh
2007-08-19 05:22 pm (UTC)

(Link)

You also have to have usernames, passwords (or hopefully you use OpenID instead), a way to invite friends, add/remove friends, and the list goes on. So generally you have to ask for email addresses too, requiring you to send out address verification emails, etc.

It is not clear to me from your document how what you are proposing solves many of these problems. If I am running MySocialNetwork.com and I use your tool to tell user JoeBob "here is a list of freinds you have on other sites, who do you want to be friends with here?" I still have to do many (all?) of the above things.

I can avoid doing some of them if I use OpenID instead of my own account system, but that's a separate problem.
From: (Anonymous)
2007-08-19 05:28 pm (UTC)

Network metrics

(Link)

Hi,

Tennessee Leeuwenburg here, also Editor-In-Chief of "The Python Papers" (http://pythonpapers.org)

I came to your post researching network metrics. I had this )(http://wiki.python.org/moin/LinkedInPythonNetwork) idea recently. One thing which is majorly holding back the coolness factor is the ability to measure how this effort is going. I'd love to be able to calculate the following metrics:
* Average connectedness of all PyLinkedIn members
* Total size of PyLinkedIn group
* Depth of PyLinkedIn tree

It was then that I realised that LinkedIn had no leet API that I could use to calculate this. Facebook has an API and some others must also. LinkedIn apparently is coming.

So, I realised I want to be able to do stuff with, as you say, the social tree. I realised that a Facebook app could scrape UID information which could be linked to IDs for other systems. If this could be fully automated, so much the better.

Your effort seems to rely on social nets publishing APIs that allow external apps to scrape public data. Do you think this will be successful? Would it be easier to write an external site that you could register with.

LinkedIn will let you register your Google ID (requires password, just don't think about security implications ) and will scrape your contacts.

Anyway, fascinating post. Have joined the Google Group.

Thanks,
-Tennessee
[User Picture]From: loganb
2007-08-19 08:24 pm (UTC)

(Link)

+1 on all the kudos; it was only a matter of time before the smaller social networks would start to cooperate on this stuff and it's nice to see good folks are on it.

Post OpenID, I'm a little curious what you advocate for a central authority. Have you considered a more decentralized approach, using URLs to represent nodes with content that asserts the existence of edges? There are lots of really boring business and financial complications with having a neutral 3rd party that ties sites together.

I'm totally hand-waving here, but suppose user identity on each site is simply a unique URL. Then, links are formed by making a forward assertion by listing another URL. The assertion can be verified by interrogating the remote URL and asking if the forward assertion is valid (perhaps with some cryptographic glue to prevent dictionary attacks/extractions).

It's FOAF-ish, but rather than being a static document it is a living URL. Associations don't have to be published, they can be interrogated. Also unlike FOAF, it would be necessary for resource URLs to have a mutual way to cross-authenticate so that private data could be exchanged. Auto-linking by proving knowledge of a shared secret (such as the user's password or mutual auth from the same OpenID) could also play a big part.
From: (Anonymous)
2007-08-19 11:17 pm (UTC)

Google

(Link)

The social graph is being constructed by Google - it's known as a Google Account. Or even a Yahoo Account

Furthermore, there are real privacy issues.

And if Facebook gets bought by Yahoo or Google it'll just be added in to the information those Search Engines hold.
Community Assets - (Anonymous) Expand
Page 1 of 2
<<[1] [2] >>