?

Log in

No account? Create an account
AddressBooker & exporting my Facebook Phonebook - brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

AddressBooker & exporting my Facebook Phonebook [Nov. 30th, 2008|09:25 pm]
Brad Fitzpatrick
[Tags|, , , ]

This 4-day weekend was awesome for catching up on personal hacking projects. In addition to adding IPv6 support to Perlbal and hacking on my interactive shadow/art wall more, I also worked on a little address book management tool, AddressBooker [source here].

Basically AddressBooker takes a POST of contacts data in JSON form, and does stuff with it, where "stuff" is currently limited to merging it into your Google Contacts. (GMail, Android, etc) This was my experiment in learning GData, AuthSub, and App Engine a bit more.

Anyway, I then wanted to get my Facebook Phonebook exported to my Google Contacts, so it'd sync to my Android phone. I didn't see an export option in Facebook (maybe I missed it?), so I wrote a little GreaseMonkey script instead to automate the whole process:

http://bradfitz.com/greasemonkey/facebook_phonebook_export.user.js

If you have Firefox and GreaseMonkey, then click the above link and it'll ask if you want to install it. Install it, then go to your Facebook Phonebook (sorry, no permalink to it), then go into Tools > GreaseMonkey > User Script Commands... > and you'll see Export Facebook Phonebook. That'll then page through your phone book (you should probably start on page 1: it's kinda flaky) and extract the data, and then POST it to AddressBooker for you, which will then guide you through merging it into your Google Contacts.

Enjoy!

(And keep in mind I barely know browser stuff or Greasemonkey or Python or App Engine or GData, so patches welcome!... brad@danga.com, or Github)

Update 2009-05-09: Updated the JavaScript to work with Facebook's new layout.

Update 2010-06-06: Facebook's markup changed again, but I'm too lazy to go update the script. Patches welcome.
Link

Comments:
Page 1 of 2
<<[1] [2] >>
[User Picture]From: daveman692
2008-12-01 05:49 am (UTC)
Cool! Been wanting this for awhile, I might just need to write the vCard export so I can get these into Address Book and onto my iPhone.

First time I ran the script got a Greasemonkey error, though disabling some add-ons and restarting FF fixed that. Not sure which one did it.

On merging, seems it's getting stuck on my 101st friend. Can send you the error from Python.
(Replies frozen) (Thread)
[User Picture]From: daveman692
2008-12-01 06:38 am (UTC)
Here's some dead simple vCard download code. Would be nice if a contact knew it's first name from last since right now I'm just sticking the entire name in FN which AddressBook treats as their last name. Haven't bothered embedding a photo, but that shouldn't be too hard either.

Sorry it's not a real patch (don't have Git installed) and it's my first bit of Python so pick away.
class DownloadVcard(webapp.RequestHandler):
  """Download a vCard of the contacts for a given handle."""

  def get(self):
    key = self.request.get('key')
    if not key:
      raise "Missing argument 'key'"
    post_dump = models.PostDump.get(db.Key(key))
    if not post_dump:
      raise "State lost?  Um, do it again."

    contacts = simplejson.loads(post_dump.json)
    self.response.headers['Content-Type'] = "text/x-vcard"
    self.response.headers['Content-Disposition'] = "attachment; filename=\"addressbooker.vcf\""
    for contact in contacts:
      self.response.out.write("BEGIN:VCARD\n")
      self.response.out.write("VERSION:3.0\n")
      self.response.out.write("FN:" + contact["name"] + "\n")

      for number in contact["numbers"]:
        phone = "TEL;type="
        if number["type"] == "Mobile:":
            phone += "CELL"
        elif number["type"] == "Other:":
            phone += "HOME"

        phone += ":" + number["number"]
        self.response.out.write(phone + "\n")

      self.response.out.write("END:VCARD\n")
in main():
    ('/download/vcard', DownloadVcard),
(Replies frozen) (Parent) (Thread) (Expand)
[User Picture]From: marcusramberg
2008-12-01 08:55 am (UTC)

Guess this won't get the email address?

Seeing as it's presented as a picture in facebook.
(Replies frozen) (Thread)
[User Picture]From: dan_lane
2008-12-01 11:33 am (UTC)
Isn't this against the Facebook terms of service? There was a very useful Mac Address Book sync tool but it was shutdown because exporting contact details from Facebook is, apparently, against their TOS :(

Didn't Plaxo get a few users in trouble when they launched their Facebook sync tool as well?
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-01 04:51 pm (UTC)
Isn't this against the Facebook terms of service?

I don't know.

There was a very useful Mac Address Book sync tool but it was shutdown because exporting contact details from Facebook is, apparently, against their TOS :(

Well, this isn't a Facebook App, so not sure what would be shut down.

Didn't Plaxo get a few users in trouble when they launched their Facebook sync tool as well?

That tool slammed the hell out of Facebok's servers, so people using it (Scoble) had their accounts suspended for rate limit abuses. The Greasemonkey script I wrote does just a couple HTTP requests (one per 50 friends w/ phone numbers), spaced ~1 second apart... very human-speed.

(Replies frozen) (Parent) (Thread) (Expand)
[User Picture]From: davidglasser
2008-12-01 06:38 pm (UTC)
Oh, man, I wanted this just last week.

Hmm, does it do any sort of search-and-merge with the Google contacts, or does it always create new ones?
(Replies frozen) (Thread)
[User Picture]From: obra
2008-12-01 06:43 pm (UTC)
It did some search-and-merge for me.
(Replies frozen) (Parent) (Thread)
Too many contacts - (Anonymous) Expand
ditto - (Anonymous) Expand
[User Picture]From: obra
2008-12-01 06:43 pm (UTC)
This rocks. You rock.
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-01 07:23 pm (UTC)
So I assume it worked for you, no issues?

Keep up the good work on K-9 Mail, btw. I use it and love it.

(Replies frozen) (Parent) (Thread)
From: pathawks.com
2008-12-02 08:44 am (UTC)

Thanks so much

Worked without a hitch.
As a side note, this is what caused me to finally break down and install GreaseMonkey. Congratulations, I guess.

I really hope that someday I can buy you a beer.
(Replies frozen) (Thread)
[User Picture]From: patschmitz
2008-12-03 12:07 am (UTC)
Would it be possible to add more data than just phone number (email, aim sn) and including all friends even those without listed phone numbers?

Thanks for the tool, worked great!
(Replies frozen) (Thread)
From: ext_137855
2008-12-12 04:01 pm (UTC)

Thanks dude

My theorem of great software ideas, "if you think of it, it must already exist," is borne out again. FWIW, the vCard export, at least for me, put the whole name into the last name, so when I merged the new stuff with my existing (OS X 10.5) address book, I got entries like "Robert Robert Smith." A small price to pay, though!
(Replies frozen) (Parent) (Thread)
From: pickle800
2008-12-13 03:14 am (UTC)
The merge worked fine for me, but was it supposed to import the facebook default pictures into gmail? I got all of my phone numbers/names/emails merged, but no photos? When addressbooker popped up asking if the merge was correct it showed each picture...
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-13 07:53 pm (UTC)
Yeah, haven't done that yet.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - (Anonymous) Expand
From: bewlay_brother
2008-12-13 07:53 pm (UTC)

ToS violation

Having spent a lot of time in the FB world, this is probably against the Terms of Use. http://www.facebook.com/terms.php

Specifically, the following clauses:
"All content on the Site and available through the Service, including designs, text, graphics, pictures, video, information, applications, software, music, sound and other files, and their selection and arrangement (the "Site Content"), are the proprietary property of the Company, its users or its licensors with all rights reserved. No Site Content may be modified, copied, distributed, framed, reproduced, republished, downloaded, scraped, displayed, posted, transmitted, or sold in any form or by any means, in whole or in part, without the Company's prior written permission, except that the foregoing does not apply to your own User Content (as defined below) that you legally post on the Site."

And
"In addition, you agree not to use the Service or the Site to: ... use automated scripts to collect information from or otherwise interact with the Service or the Site"

On top of the legalese, the spirit of the Facebook Terms of Use is that people should have control over their privacy through FB, no matter where that information resides. Facebook would say, "What if someone changes their privacy preferences so that you're no longer allowed access to their phone number? Or what if they change numbers? Your data will at best be out of date and at worst violate their privacy."

Since Facebook offers no controlled mechanism for accessing phone numbers I think it's a safe bet that they don't want users exporting their friends phone numbers, no matter what.

YMMV, I guess, since Facebook is notoriously inconsistent in enforcing their terms of service.
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-13 07:56 pm (UTC)

Re: ToS violation

Use at your own risk, then. However, the script does do 1 second pauses between flipping pages, and HTTP requests come from your own browser's user-agent and cookies, so it'd be a little hard to fingerprint.
(Replies frozen) (Parent) (Thread)
From: (Anonymous)
2008-12-13 08:16 pm (UTC)

Here's a permalink

http://www.facebook.com/mobile/?phonebook
(Replies frozen) (Thread)
[User Picture]From: brad
2008-12-13 08:25 pm (UTC)

Re: Here's a permalink

thanks!
(Replies frozen) (Parent) (Thread) (Expand)
From: (Anonymous)
2008-12-14 03:50 pm (UTC)

Done before.

Facebook Scavenger was a Firefox extension that allowedyou to export your friends' contact info, including e-mail addresses: http://www.chrisfinke.com/addons/facebook-scavenger/

Facebook will most certainly send you takedown notice as well.
(Replies frozen) (Thread)
From: (Anonymous)
2008-12-15 02:41 pm (UTC)

Re: Done before.

Thanks for the great hack! I was just looking for a nice way to update information in my contacts list for the G1, and this seems to do the trick (at least for phone numbers). Addresses would be great too. :)

It does look like you are out of your AppEngine URL fetching quota though:

raise self.exception
OverQuotaError: The API call urlfetch.Fetch() required more quota than is available.

(Replies frozen) (Parent) (Thread)
From: ext_140004
2008-12-27 01:45 am (UTC)
The GreaseMonkey script worked perfectly for me over four pages, thanks very much!

Now to merge with phonebook, WAB, Thunderbird and Google Contacts and remove duplicates - best solution I've found so far is Zyb (http://zyb.com) but it's still not ideal...
(Replies frozen) (Thread)
From: (Anonymous)
2009-01-05 04:56 pm (UTC)

Problem with exporting facebook friends

Sorry but i speak not very good english but i've a problem with Facebook:
I went into User Script Commands when i was on this page http://www.facebook.com/mobile/?phonebook but after Facebook said :

(look at the image)
http://nsa03.casimages.com/img/2009/01/05/090105060020573077.png
(Replies frozen) (Thread)
[User Picture]From: brad
2009-01-05 05:17 pm (UTC)

Re: Problem with exporting facebook friends

It only works in English mode.
(Replies frozen) (Parent) (Thread)
From: ext_141703
2009-01-07 10:28 pm (UTC)

Best. App. Ever.

You've managed to do what I've spent months looking for. Thank you!
(Replies frozen) (Thread)
From: (Anonymous)
2009-01-28 05:16 am (UTC)

Great app!

This app is great! I can finally get the phone numbers of my contacts into a vCard! It would be better if the first/last names were separated though...
(Replies frozen) (Thread)
Page 1 of 2
<<[1] [2] >>