brad's life - AddressBooker & exporting my Facebook Phonebook [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

AddressBooker & exporting my Facebook Phonebook [Nov. 30th, 2008|09:25 pm]
Previous Entry Add to Memories Share Next Entry
[Tags|, , , ]

This 4-day weekend was awesome for catching up on personal hacking projects. In addition to adding IPv6 support to Perlbal and hacking on my interactive shadow/art wall more, I also worked on a little address book management tool, AddressBooker [source here].

Basically AddressBooker takes a POST of contacts data in JSON form, and does stuff with it, where "stuff" is currently limited to merging it into your Google Contacts. (GMail, Android, etc) This was my experiment in learning GData, AuthSub, and App Engine a bit more.

Anyway, I then wanted to get my Facebook Phonebook exported to my Google Contacts, so it'd sync to my Android phone. I didn't see an export option in Facebook (maybe I missed it?), so I wrote a little GreaseMonkey script instead to automate the whole process:

http://bradfitz.com/greasemonkey/facebook_phonebook_export.user.js

If you have Firefox and GreaseMonkey, then click the above link and it'll ask if you want to install it. Install it, then go to your Facebook Phonebook (sorry, no permalink to it), then go into Tools > GreaseMonkey > User Script Commands... > and you'll see Export Facebook Phonebook. That'll then page through your phone book (you should probably start on page 1: it's kinda flaky) and extract the data, and then POST it to AddressBooker for you, which will then guide you through merging it into your Google Contacts.

Enjoy!

(And keep in mind I barely know browser stuff or Greasemonkey or Python or App Engine or GData, so patches welcome!... brad@danga.com, or Github)

Update 2009-05-09: Updated the JavaScript to work with Facebook's new layout.

Update 2010-06-06: Facebook's markup changed again, but I'm too lazy to go update the script. Patches welcome.
Link

Comments:
Page 1 of 2
<<[1] [2] >>
[User Picture]From: daveman692
2008-12-01 05:49 am (UTC)

(Link)

Cool! Been wanting this for awhile, I might just need to write the vCard export so I can get these into Address Book and onto my iPhone.

First time I ran the script got a Greasemonkey error, though disabling some add-ons and restarting FF fixed that. Not sure which one did it.

On merging, seems it's getting stuck on my 101st friend. Can send you the error from Python.
(Replies frozen) (Thread)
[User Picture]From: daveman692
2008-12-01 06:38 am (UTC)

(Link)

Here's some dead simple vCard download code. Would be nice if a contact knew it's first name from last since right now I'm just sticking the entire name in FN which AddressBook treats as their last name. Haven't bothered embedding a photo, but that shouldn't be too hard either.

Sorry it's not a real patch (don't have Git installed) and it's my first bit of Python so pick away.
class DownloadVcard(webapp.RequestHandler):
  """Download a vCard of the contacts for a given handle."""

  def get(self):
    key = self.request.get('key')
    if not key:
      raise "Missing argument 'key'"
    post_dump = models.PostDump.get(db.Key(key))
    if not post_dump:
      raise "State lost?  Um, do it again."

    contacts = simplejson.loads(post_dump.json)
    self.response.headers['Content-Type'] = "text/x-vcard"
    self.response.headers['Content-Disposition'] = "attachment; filename=\"addressbooker.vcf\""
    for contact in contacts:
      self.response.out.write("BEGIN:VCARD\n")
      self.response.out.write("VERSION:3.0\n")
      self.response.out.write("FN:" + contact["name"] + "\n")

      for number in contact["numbers"]:
        phone = "TEL;type="
        if number["type"] == "Mobile:":
            phone += "CELL"
        elif number["type"] == "Other:":
            phone += "HOME"

        phone += ":" + number["number"]
        self.response.out.write(phone + "\n")

      self.response.out.write("END:VCARD\n")
in main():
    ('/download/vcard', DownloadVcard),
(Replies frozen) (Parent) (Thread) (Expand)
[User Picture]From: marcusramberg
2008-12-01 08:55 am (UTC)

Guess this won't get the email address?

(Link)

Seeing as it's presented as a picture in facebook.
(Replies frozen) (Thread)
[User Picture]From: dan_lane
2008-12-01 11:33 am (UTC)

(Link)

Isn't this against the Facebook terms of service? There was a very useful Mac Address Book sync tool but it was shutdown because exporting contact details from Facebook is, apparently, against their TOS :(

Didn't Plaxo get a few users in trouble when they launched their Facebook sync tool as well?
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-01 04:51 pm (UTC)

(Link)

Isn't this against the Facebook terms of service?

I don't know.

There was a very useful Mac Address Book sync tool but it was shutdown because exporting contact details from Facebook is, apparently, against their TOS :(

Well, this isn't a Facebook App, so not sure what would be shut down.

Didn't Plaxo get a few users in trouble when they launched their Facebook sync tool as well?

That tool slammed the hell out of Facebok's servers, so people using it (Scoble) had their accounts suspended for rate limit abuses. The Greasemonkey script I wrote does just a couple HTTP requests (one per 50 friends w/ phone numbers), spaced ~1 second apart... very human-speed.

(Replies frozen) (Parent) (Thread) (Expand)
[User Picture]From: davidglasser
2008-12-01 06:38 pm (UTC)

(Link)

Oh, man, I wanted this just last week.

Hmm, does it do any sort of search-and-merge with the Google contacts, or does it always create new ones?
(Replies frozen) (Thread)
[User Picture]From: obra
2008-12-01 06:43 pm (UTC)

(Link)

It did some search-and-merge for me.
(Replies frozen) (Parent) (Thread)
Too many contacts - (Anonymous) Expand
ditto - (Anonymous) Expand
[User Picture]From: obra
2008-12-01 06:43 pm (UTC)

(Link)

This rocks. You rock.
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-01 07:23 pm (UTC)

(Link)

So I assume it worked for you, no issues?

Keep up the good work on K-9 Mail, btw. I use it and love it.

(Replies frozen) (Parent) (Thread)
From: pathawks.com
2008-12-02 08:44 am (UTC)

Thanks so much

(Link)

Worked without a hitch.
As a side note, this is what caused me to finally break down and install GreaseMonkey. Congratulations, I guess.

I really hope that someday I can buy you a beer.
(Replies frozen) (Thread)
[User Picture]From: patschmitz
2008-12-03 12:07 am (UTC)

(Link)

Would it be possible to add more data than just phone number (email, aim sn) and including all friends even those without listed phone numbers?

Thanks for the tool, worked great!
(Replies frozen) (Thread)
From: hmcnally.myvidoop.com
2008-12-12 04:01 pm (UTC)

Thanks dude

(Link)

My theorem of great software ideas, "if you think of it, it must already exist," is borne out again. FWIW, the vCard export, at least for me, put the whole name into the last name, so when I merged the new stuff with my existing (OS X 10.5) address book, I got entries like "Robert Robert Smith." A small price to pay, though!
(Replies frozen) (Parent) (Thread)
From: pickle800
2008-12-13 03:14 am (UTC)

(Link)

The merge worked fine for me, but was it supposed to import the facebook default pictures into gmail? I got all of my phone numbers/names/emails merged, but no photos? When addressbooker popped up asking if the merge was correct it showed each picture...
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-13 07:53 pm (UTC)

(Link)

Yeah, haven't done that yet.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - (Anonymous) Expand
From: bewlay_brother
2008-12-13 07:53 pm (UTC)

ToS violation

(Link)

Having spent a lot of time in the FB world, this is probably against the Terms of Use. http://www.facebook.com/terms.php

Specifically, the following clauses:
"All content on the Site and available through the Service, including designs, text, graphics, pictures, video, information, applications, software, music, sound and other files, and their selection and arrangement (the "Site Content"), are the proprietary property of the Company, its users or its licensors with all rights reserved. No Site Content may be modified, copied, distributed, framed, reproduced, republished, downloaded, scraped, displayed, posted, transmitted, or sold in any form or by any means, in whole or in part, without the Company's prior written permission, except that the foregoing does not apply to your own User Content (as defined below) that you legally post on the Site."

And
"In addition, you agree not to use the Service or the Site to: ... use automated scripts to collect information from or otherwise interact with the Service or the Site"

On top of the legalese, the spirit of the Facebook Terms of Use is that people should have control over their privacy through FB, no matter where that information resides. Facebook would say, "What if someone changes their privacy preferences so that you're no longer allowed access to their phone number? Or what if they change numbers? Your data will at best be out of date and at worst violate their privacy."

Since Facebook offers no controlled mechanism for accessing phone numbers I think it's a safe bet that they don't want users exporting their friends phone numbers, no matter what.

YMMV, I guess, since Facebook is notoriously inconsistent in enforcing their terms of service.
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-13 07:56 pm (UTC)

Re: ToS violation

(Link)

Use at your own risk, then. However, the script does do 1 second pauses between flipping pages, and HTTP requests come from your own browser's user-agent and cookies, so it'd be a little hard to fingerprint.
(Replies frozen) (Parent) (Thread)
From: (Anonymous)
2008-12-13 08:16 pm (UTC)

Here's a permalink

(Link)

http://www.facebook.com/mobile/?phonebook
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-13 08:25 pm (UTC)

Re: Here's a permalink

(Link)

thanks!
(Replies frozen) (Parent) (Thread) (Expand)
From: (Anonymous)
2008-12-14 03:50 pm (UTC)

Done before.

(Link)

Facebook Scavenger was a Firefox extension that allowedyou to export your friends' contact info, including e-mail addresses: http://www.chrisfinke.com/addons/facebook-scavenger/

Facebook will most certainly send you takedown notice as well.
(Replies frozen) (Thread)
From: (Anonymous)
2008-12-15 02:41 pm (UTC)

Re: Done before.

(Link)

Thanks for the great hack! I was just looking for a nice way to update information in my contacts list for the G1, and this seems to do the trick (at least for phone numbers). Addresses would be great too. :)

It does look like you are out of your AppEngine URL fetching quota though:

raise self.exception
OverQuotaError: The API call urlfetch.Fetch() required more quota than is available.

(Replies frozen) (Parent) (Thread)
From: http://claimid.com/smenashy
2008-12-27 01:45 am (UTC)

(Link)

The GreaseMonkey script worked perfectly for me over four pages, thanks very much!

Now to merge with phonebook, WAB, Thunderbird and Google Contacts and remove duplicates - best solution I've found so far is Zyb (http://zyb.com) but it's still not ideal...
(Replies frozen) (Thread)
From: (Anonymous)
2009-01-05 04:56 pm (UTC)

Problem with exporting facebook friends

(Link)

Sorry but i speak not very good english but i've a problem with Facebook:
I went into User Script Commands when i was on this page http://www.facebook.com/mobile/?phonebook but after Facebook said :

(look at the image)
http://nsa03.casimages.com/img/2009/01/05/090105060020573077.png
(Replies frozen) (Thread)
[User Picture]From: brad
2009-01-05 05:17 pm (UTC)

Re: Problem with exporting facebook friends

(Link)

It only works in English mode.
(Replies frozen) (Parent) (Thread)
From: startuplife.ca
2009-01-07 10:28 pm (UTC)

Best. App. Ever.

(Link)

You've managed to do what I've spent months looking for. Thank you!
(Replies frozen) (Thread)
From: (Anonymous)
2009-01-28 05:16 am (UTC)

Great app!

(Link)

This app is great! I can finally get the phone numbers of my contacts into a vCard! It would be better if the first/last names were separated though...
(Replies frozen) (Thread)
[User Picture]From: neried7
2009-02-19 06:28 pm (UTC)

Thanks!

(Link)

I just got my G1 and this is exactly what I was looking for, it works like a charm. Good job!
(Replies frozen) (Thread)
[User Picture]From: jacebenson
2009-03-06 02:59 pm (UTC)

This rocks!

(Link)

Worked well for me. Exported 29 contacts.

I wish it would grab the non-number people as well. I have some people's number in my google address book but not their email address. It would be nice to see it as an option.

Thanks for making painful work painless.
(Replies frozen) (Thread)
[User Picture]From: scottcrawford
2009-03-08 09:30 am (UTC)

(Link)

This is mighty awesome, Brad. Now buy back LJ from the Russians already! ;D
(Replies frozen) (Thread)
[User Picture]From: thefayth
2009-03-30 05:02 am (UTC)

(Link)

Worked for me after a bit of tweaking. Size of FB contact list doesn't seem = fail, it's if gmail has a huge contact list. I used a 2ndary gmail with no contacts and things went smoothly. FB phonebook section can be entered via events, mobile and friends but if you visit /friends and go to phonebook from there, Greasemonkey works fine.
(Replies frozen) (Thread)
[User Picture]From: noodlesandbeef
2009-04-14 09:03 pm (UTC)

(Link)

This worked to perfection--thanks for creating it!
(Replies frozen) (Thread)
[User Picture]From: topbit
2009-04-25 02:40 pm (UTC)

(Link)

Thanks Brad, it's very useful. I've just run it, without issues. You're a good part (not all, but significant) of the reason I've just gotten myself a G1, with the various things you've been doing with it.
(Replies frozen) (Thread)
[User Picture]From: sjfootman
2009-05-01 05:57 pm (UTC)

Probably bad timing

(Link)

Did LJ just change their friends page in a way that breaks this script? Figures that it would happen just before I needed it. Ah well.

(Replies frozen) (Thread)
[User Picture]From: brad
2009-05-09 11:31 pm (UTC)

Re: Probably bad timing

(Link)

I updated the script. Try again?
(Replies frozen) (Parent) (Thread)
From: knif.blogspot.com
2009-05-06 06:42 am (UTC)

This is awesome, would love to use it...

(Link)

... but it's giving me an error;

Required element 'summary' not found; aborting.

I disabled all my other Add-Ons and whatnot, so I dunno. I'm sure this is just something I'm not aware of.
(Replies frozen) (Thread)
[User Picture]From: brad
2009-05-09 10:34 pm (UTC)

Re: This is awesome, would love to use it...

(Link)

Facebook's layout/HTML change broke it. I've now fixed the script. Try again.
(Replies frozen) (Parent) (Thread)
From: (Anonymous)
2009-05-07 10:28 am (UTC)

Error

(Link)

Hey,

Thanks for coming up with this, its a great idea and really useful. But whenever I run the GreaseMonkey script I get the message "Required element 'summary' not found; aborting."

Any idea whats gone wrong?

Thanks!
(Replies frozen) (Thread)
[User Picture]From: brad
2009-05-09 10:35 pm (UTC)

Re: Error

(Link)

It's now fixed.
(Replies frozen) (Parent) (Thread)
From: limi.net
2009-05-08 07:41 pm (UTC)

Facebook just changed their markup

(Link)

Of course, the day I find this, Facebook updates their address book code, so the script doesn't work anymore:
Required element 'summary' not found; aborting.

I'll check back later — would submit a patch if I knew more Javascript. :)

Thanks for creating this!
(Replies frozen) (Thread)
[User Picture]From: brad
2009-05-09 10:35 pm (UTC)

Re: Facebook just changed their markup

(Link)

I've now fixed it. Try again!
(Replies frozen) (Parent) (Thread) (Expand)
Page 1 of 2
<<[1] [2] >>