?

Log in

No account? Create an account
brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

fd passing [Jan. 27th, 2005|02:39 pm]
Brad Fitzpatrick
[Tags|, ]

Today's goal was to learn how file descriptor passing works in Unix.

I couldn't find a Perl module to do it so I set off writing it in C, with the hope to understand it well enough to implement it in Perl. So I wrote it in C, got it working, and started to implement it in Perl, quickly finding:

http://search.cpan.org/~addi/File-FDpasser-0.09/FDpasser.pm

... which is exactly what I need. And reasonably portable. The act of writing it in C just led me down the road of knowing the right keywords to look for (namely "sendmsg"), which made me find this module, which led me to FDpasser.

The next goal is ripping out all the non-portable Linux::AIO code in Perlbal to instead do sync operations in child threads using fd passing.
LinkReply

Comments:
[User Picture]From: lithiana
2005-01-27 11:02 pm (UTC)
Is there any reason you're using child processes rather than pthreads?
(Reply) (Thread)
[User Picture]From: brad
2005-01-27 11:14 pm (UTC)
You know I'm using Perl, right? You mean Perl's ithreads?

I hate Perl and threading. I mostly don't know enough about it, but I also don't care. The few times I've tried to use it it's proven too painful and not worth it.
(Reply) (Parent) (Thread)
[User Picture]From: lithiana
2005-01-27 11:43 pm (UTC)
Yes, I should've been more clear there, I meant pthreads as opposed to 'threading' using processes.

Fair enough about threads - doing anything non-trivial can be a pain. Depends what you are doing though, if it's something pretty unconnected to the main logic (i.e. spawn thread, wait for result) it's not so bad.
(Reply) (Parent) (Thread)
[User Picture]From: mart
2005-01-27 11:59 pm (UTC)

I generally tend to go for processes rather than threads in UNIX myself. I can't really explain why, except perhaps that it allows for more lazy programming… but that's a poor excuse.

Spawning processes in Linux is pretty inexpensive anyway. I don't know much about how other UNIX-like systems do it, but I have heard that process spawing in Windows is pretty expensive. For this reason, I've never used multiple processes in a program in Windows, and just endure the extra pain that multithreaded programming in C results in. I've never written a multithreaded program in Perl because the Perl I use is too old to support threading properly.

(Reply) (Parent) (Thread)
[User Picture]From: brentdax
2005-01-28 12:16 am (UTC)
Windows doesn't have a fork() call, so there's really no way to spawn a process the way you can on Unix. Even if you start a process from the beginning, it's really slow.

Actually, on Windows they invented something even lighter-weight than threads called "fibers". I have no idea what they actually do, and don't know anybody else who does either, including my father and all of the cow-orkers at his Windows-only shop.
(Reply) (Parent) (Thread)
[User Picture]From: rooneg
2005-01-28 03:10 am (UTC)
Fibers are basically threads that you have to schedule manually.

You create a bunch of them, then say "run this fiber", it runs for a while, then says "ok, i'm done" and you say "run this other fiber", and so on. The fiber that's being run borrows the stack of the thread that starts it, if I recall correctly, so it's very very light weight (the stack is most of the overhead of creating a thread), all that's required to create a fiber is the memory to store the processor registers.

It's been a while since I read about them though, so I'm probably getting some details wrong.

They let you be very specific about how things get scheduled, but is very hard to do right and if you think you need them you're probably wrong.
(Reply) (Parent) (Thread)
[User Picture]From: youngoat
2005-01-28 03:24 am (UTC)

Fibers

I believe they're like threads except you schedule them yourself rather than the OS doing it. The idea is you can do quick lightweight context switche within your thread rather than a heavy-duty OS-level one. I guess SQL Server uses them to crank out a bit extra performance...

If you're interested, here's a pretty good blog post: Why does Win32 even have Fibers?
(Reply) (Parent) (Thread)
[User Picture]From: mart
2005-01-28 10:18 am (UTC)

If you've done computer science, it might help to be told that Win32 Fibers are basically just coroutines. However, they are a bitch to use and so people don't really use them very often. They also cause some nasty problems because the Win32 API has lots of items which are attached to a given thread, and when you start using fibers you find that things like window handles and so forth continue to be thread-related, not fiber-related.

I should stop pretending I know what I'm talking about, though. I only played with them once and it all went horribly wrong. ;)

(Reply) (Parent) (Thread)
From: evan
2005-01-27 11:21 pm (UTC)
Forgive my ignorance, but aren't file descriptors integers? And aren't they shared across forks? (What piece of this picture am I missing?)
(Reply) (Thread)
[User Picture]From: brad
2005-01-27 11:36 pm (UTC)
An integer is just your handle to an fd. Think:

class process {
....
file* fd[MAX_FDS];
....
}

You can dup() your handles and get two handles (integers) to the same in-kernel object... sharing the same file position and everything.

Similarly you can pass an fd between arbitrary programs with sendmsg() and unix sockets (and the Perl modules above). You may send fd 5 and it might show up as fd 8 in the other process, but they point to the same in-kernel file object, just as if they were dup'ed.

Yes, forking can share fds, but what I care about is the parent/child making new fds after the fork and sharing them, which requires passing them over a unix socket.

The Linux::AIO module for Perl gets around that by using the Linux clone() syscall with the CLONE_FILES flag, which means they always have the same fd numbers. But that's not portable between Unices.
(Reply) (Parent) (Thread)
From: evan
2005-01-27 11:49 pm (UTC)
what I care about is the parent/child making new fds after the fork and sharing them
Ah, that's what I was missing. :)
Thanks!
(Reply) (Parent) (Thread)
[User Picture]From: ydna
2005-01-27 11:48 pm (UTC)
I love the name heading in that module:

File::FDpasser - Perl extension for blah blah blah
(Reply) (Thread)
[User Picture]From: brentdax
2005-01-28 12:21 am (UTC)
That's the default created by h2xs. At least he bothered to change the module author field from A.U. Thor...
(Reply) (Parent) (Thread)
[User Picture]From: ydna
2005-01-30 03:01 am (UTC)
Sorry. I forgot the <sarcasm> block.
(Reply) (Parent) (Thread)
[User Picture]From: torgo_x
2005-01-28 12:31 am (UTC)

Addi's FDPasser

Addi is a cool guy. He works at Google now, but I knew him when!
(Reply) (Thread)
[User Picture]From: iamo
2005-01-28 01:22 am (UTC)
I was, a while ago, actually looking into using this sort of thing to write an apache2 mpm for multiuser use (where requests happen under a configured userid) that actually works (there are actually two in the distribution, but they're both somewhere between busted and dysfunctional). I never got far because I had trouble finding documentation on this feature and its portability.

This might come in handy, as example code of doing it portably, if I ever give that a go again :)
(Reply) (Thread)
[User Picture]From: scosol
2005-01-28 01:22 am (UTC)
Are you switching some systems to a BSD derivative or something?
(Reply) (Thread)
[User Picture]From: mart
2005-01-28 02:06 am (UTC)

Perlbal is general (non-lj-specific) software, so I assume there are people who want to (or will in future want to) run it on non-Linux systems, even if LiveJournal has no direct use for it.

(Reply) (Parent) (Thread)
[User Picture]From: brad
2005-01-28 02:50 am (UTC)
Correct.

And also because one of the MogileFS components (mogstored) depends on Perlbal.pm (because mogstored is just a webserver) and thus MogileFS is Linux-only.

We don't want to cut out the BSD and OS X users.
(Reply) (Parent) (Thread)
[User Picture]From: toast0
2005-01-28 05:05 am (UTC)
You know... this will feed the Yahoo! LJ rumors... afterall, Y! is a large user of FreeBSD...
(Reply) (Parent) (Thread)
[User Picture]From: brad
2005-01-28 05:36 am (UTC)
*yawn*

Just go look at the mailing list archives where BSD people are asking why it doesn't work.
(Reply) (Parent) (Thread)
[User Picture]From: gaal
2005-01-28 05:43 am (UTC)
Hey, that's way cool. Too bad it won't work with Perl-buffered IO though :)
(Reply) (Thread)
From: pos_le_terrible
2005-01-28 09:32 am (UTC)

Socket-PassAccessRights

There is also the Socket-PassAccessRights module on CPAN which does the same thing.
(Reply) (Thread)