Log in

No account? Create an account
brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Bike Ride [Jul. 13th, 2004|07:13 pm]
Brad Fitzpatrick
[Tags|, , ]

I was feeling lethargic and hating databases more than usual today so I bailed on work and went on a 30+ mile bike ride in the heat.

It sucked for the first 10 minutes or so, then it felt really good.

Along the way did a lot of work-related thinking and wrote up a big lj_core post (protected, so no link), so it feels justified.

And from what I hear from Whitaker just now (who just stopped by and left with Frank and Tiffany), things were productive at work today, so tomorrow should be cool.

MogileFS without NFS is coming along swimmingly. Go Junior! Considering the NFS bits were the hardest to setup and maintain, I can't wait to release and document this and let everybody have a simplifed, easy-to-use distributed file system without shared storage.

I have a Perlbal presentation tomorrow tonight at the Portland Perl Mongers. That should be fun. I should make up better slides than I had last time.

[User Picture]From: marksmith
2004-07-13 07:43 pm (UTC)
Hey, I got replication over HTTP working today!

Not sure I like how the design is, though, from a "setting up Perlbal" point of view. We can talk about it tomorrow or whenever.
(Reply) (Thread)
[User Picture]From: brad
2004-07-13 07:50 pm (UTC)
We can make a front-end script to invoke the Perlbal classes with the correct configuration instead of using the normal "perlbal". So instead we make "mognoded" which is really just the web server that does GET/PUT on that node's devices. Then there's no configuration.

That your qualm?
(Reply) (Parent) (Thread)
[User Picture]From: marksmith
2004-07-13 08:10 pm (UTC)
The way I had it configured, I have one port per Mogile device:

host_a:7001 = /mnt/mogilefs/host_a/dev1/
host_a:7002 = /mnt/mogilefs/host_a/dev2/

Then, I modified the device table to store a port. Since we already have an IP address for the Mogile node, we just connect to ip:port and upload the file to the raw path.

This kinda sucked because then we have N ports, one for every device. So what I was thinking is just use one standard port, and have the URI include the device name.

Either can be done with "mognoded", I think the latter is cleaner--having only one port open. But then the file URI includes the device name, which may or may not be desirable?
(Reply) (Parent) (Thread)
[User Picture]From: brad
2004-07-13 08:34 pm (UTC)
Port per device? I was thinking port per host.

sto1:7009 = /var/mogdata/

And on each storage node, /var/mogdata/ has directories for each device:

/var/mogdata/dev1 -- ext2
/var/mogdata/dev2 -- ext3
/var/mogdata/dev3 -- reiser
/var/mogdata/dev4 -- whatever

I'm trying to think what you got confused onto the port-per-device way of thinking. Probably something about how the tables are laid out or the old code was structured?
(Reply) (Parent) (Thread)
[User Picture]From: marksmith
2004-07-13 09:01 pm (UTC)
Yeah. I was thinking that we didn't want to expose device names in the file URIs. Not sure why. However, that's really easy to change.

Anyway, port per host? If we run one mognoded on each host we can use the same port across all of them, right? Would make configuration a lot easier and make it so we don't have to change the schema at all.
(Reply) (Parent) (Thread)
[User Picture]From: brad
2004-07-13 09:26 pm (UTC)
One new config option we should add is "min_required_put_directory" or something, so if we do a:

PUT /dev17/foo/bar/2343.fid HTTP/1.0

And that host doesn't have 'dev17', we don't store it on that machine's /var filesystem and end up mktree'ing /var/mogdata/dev17

(Reply) (Parent) (Thread)
[User Picture]From: marksmith
2004-07-13 09:41 pm (UTC)
Yep, good idea. I'll get that added in!

As an aside, this thread has demonstrated one of LiveJournal's weaknesses -- conversations. It starts to look weird when you reply back and forth, back and forth.

Might be interesting to investigate a 'no threading' mode/option or something at some point.
(Reply) (Parent) (Thread)
[User Picture]From: xotiffany
2004-07-13 10:37 pm (UTC)
We decided to pass on the bubble bobble tonight, but maybe I can take a raincheck?

Anytime you feel like skipping out on a few miles and want to park at my place, just let me know.
(Reply) (Thread)
From: freshfossil
2004-07-13 11:41 pm (UTC)
I love the random silly icon pic. ;o)
(Reply) (Thread)
[User Picture]From: scsi
2004-07-14 12:56 am (UTC)
Jr = friggin hard core.
(Reply) (Thread)