brad's life - Contributing to Open Source projects [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Contributing to Open Source projects [Mar. 20th, 2010|06:12 pm]
Previous Entry Share Next Entry
[Tags|, , , ]

Prior to joining Google I always joked that Google was the black hole that swallowed up open source programmers. I'd see awesome, productive hackers join Google and then hear little to nothing from them afterwards. When I joined I decided I'd solve this mystery and post about it but it's been over 2.5 years and I've been busy and somewhat forgot. Fortunately a discussion at work last week reminded me of this again, and a bunch of us got to talking about the phenomenon.

Just as there are rarely absolutes in anything, there are no absolutes about open source programmers' activities after joining Google. The main reasons for them sometimes disappearing, as far as I can tell, are:
  • Many open source programmers are just programmers. They like working on fun, hard problems, whether on open source or otherwise.
  • They're busy. Google seems to suck everybody's free time, and then some. It's not that Google is forcing them to work all the time, but they are anyway because there are so many cool things that can be done. I often joke that I have seven 20% projects.
  • The Google development environment is so nice. The source control, build system, code review tools, debuggers, profilers, submit queues, continuous builds, test bots, documentation, and all associated machinery and processes are incredibly well done. It's very easy to hack on anything, anywhere and submit patches to anybody, and notably: to find who or what list to submit patches to. Generally submitting a patch is the best way to even start a discussion about a feature, showing that you're serious, even if your patch is wrong.
Personally, my increased involvement with Google side-projects and decreased involvement with public open source projects is a bit of all three of those bullets.

Notably, though, I want to discuss the last bullet.

It's pretty difficult to figure out how to contribute in the open source community. Given some package on your system or some tarball you downloaded, it's not always obvious what the right process is for that community to get patches upstream. It's often a research project just to find the upstream version control system, or bug tracker, or the mailing list to send patches to. CONTRIBUTING files in tarballs, if present at all, are often out of date.

When you're used to this, perhaps it's not so bad, but inside a company with a very consistent and easy-to-hack-hack-hack environment, this can be daunting. I'm not just talking about Google here. I'm sure most companies have more internal consistency in tools & processes than the collective open source community.

My request:
So here's my request to the open source community: make a webpage for your project that summarizes your community's development resources & process. And then link the hell out of it. Link it from all over your project's documentation. Make sure you have a CONTRIBUTING file, but don't put the current information in the file.... it'll just get stale. Instead, put your contributing documentation URL in your CONTRIBUTING file. Tools and processes change, but tarballs get old, and distros are rarely bleeding edge.

Good examples of people doing this already (from a quick search) include Django, Mono, and MySQL.

If your project doesn't already do this, as most of mine haven't, or haven't well enough, I made a website to make this easy:

Contributing: http://contributing.appspot.com/

Anybody can (and should!) use that for their project to create a project page with a stable URL listing their project's resources and quick summary of the project's development workflow. Where's your source, bug tracker, code review tool, style guide, mailing list, etc?

I've been creating project pages for all projects I'd started in the past, and making sure to update all their docs and websites with links to the Contributing page.

Here are some of mine:

http://contributing.appspot.com/memcached
http://contributing.appspot.com/perlbal
http://contributing.appspot.com/sgnodemapper
http://contributing.appspot.com/contributing
http://contributing.appspot.com/djabberd
....

Still creating them, but afterwards I hope to be able to filter more of my mailing list subscriptions and not feel guilty about people having out-of-date information and emailing me directly.

From now on I will never either a) fail to document the contribution process for a new project I start, or b) document that sending me patches directly is the answer. That may be true for a bit, but projects often change hands, and stale documentation sucks.
LinkReply

Comments:
[User Picture]From: bluesmoon
2010-03-21 03:55 am (UTC)

(Link)

I started writing opensource code in 1999, and I wrote a lot of it. I was also a very heavy contributor to various lugs in my area and on irc. Then in 2004 I joined Yahoo! and the bulk of my contributions stopped. Over the following months I almost completely disappeared from the mailing lists I used to frequent.

The biggest reason for this though, was that my primary opensource projects competed with a Yahoo! product, so I couldn't ethically keep working on it. My participation dropped to commenting on coding style and giving people commit access. Secondly, Yahoo! has an awesome internal developer community from which I could get my daily "fix" of technical discussions, rants and flame wars. I no longer felt the need to drop in to an external mailing list and the only times I did were to say hello to all the people I'd met over the years.

Contributing was never a problem. If something didn't work, I'd search for the source website, download the latest tarball and submit a patch on the mailing list. If they didn't want it, I didn't care, I'd just put it up on my web page and let others download it from there. As long as it solved my own problem, I was happy. In one instance though, my patch wasn't just accepted, but I was instantly made "lead volunteer" of the module I'd just patched since no one else was doing it.