Log in

No account? Create an account
Xen, again - brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Xen, again [Dec. 4th, 2005|01:01 am]
Brad Fitzpatrick
[Tags|, ]

I can't stop talking about how cool Xen is.

Try out a new program? Use a whole new operating system... a new operating system (Debian sarge) only takes 114 MB, so why not?

Paranoid some program might have an exploit? It probably does... new operating system!
bepo:~# xm list
Name              Id  Mem(MB)  CPU  State  Time(s)  Console
Domain-0           0      507    0  r----   3069.9        
danga             11      511    1  -b---   2899.0    9611
danga-cvs         10      255    3  -b---   3133.8    9610
mail              13      511    3  -b---  14273.3    9613
personal_web      16      255    3  -b---      3.8    9616
rtzilla           14      255    1  -b---    260.7    9614
I wrote this fun tool:

# xen-create --name='personal_web' --size=10G --ip= --mem=256

Which does exactly what you'd think. Creates a new LVM2 LV, makes a filesystems, rsyncs from my base system's skeleton, sets up hostname, network interfaces, unmounts, and starts up xen (after putting conf in /etc/xen/auto/, so it auto-starts on boot).

About 30 seconds after running that and a new Debian sarge machine (with latest security updates) is pinging on the net, and I can ssh in to it at root, since I initialize the machine's authorized_keys file to include my own.

And each machine has its own ssh host keys, thanks to this one-liner, which runs once on boot, right before ssh starts:

bepo:~# cat /var/xen-skel/etc/rc2.d/S15ssh-setup 
# one-time configuration of ssh:

/var/lib/dpkg/info/ssh.postinst configure && rm /etc/rc2.d/S15ssh-setup
Fun stuff.

[User Picture]From: chrisj04
2005-12-04 09:36 am (UTC)
would i be able to get a copy of that script from you, coz i'm too lazy to write my own?
(Reply) (Thread)
[User Picture]From: vanbeast
2005-12-04 09:40 am (UTC)
Is there a good "getting started" type howto out there? More and more I need something like xen but I haven't been able to get it running on my debian testing machine.
(Reply) (Thread)
[User Picture]From: topher
2005-12-04 11:33 am (UTC)

Xen rocks!

Without a doubt, I completely agree.

About six months back, I needed to test out some stuff at work, and I only had one machine available to me. I'd been hearing some neat stuff about this "Xen" thing, and decided to give it a shot. Not only did it work perfectly, but word got around, and within a week it proved itself so well that management approved a new server to act as a dedicated Xen server, with a crapload of RAM (4x our "standard" order) in order to run multiple instances for the developers to test on.

I've since moved a couple of my home machines to Xen setups, too. It's just too cool, and *so* useful!

I really hope Red Hat's (and others) push to get it into the kernel proper pays off, and it gets integrated soon. Either way, with the latest Fedora Core supporting it, I'm guessing the next RHEL will include full support for it. (I'm only mentioning Red Hat as much as I am because it's our standard platform at work, and despite my best efforts, will remain so.)
(Reply) (Thread)
[User Picture]From: meowpurrr
2005-12-04 02:17 pm (UTC)
it sounds really cool, but i really can't think of a use for it for me.

at work we're about to upgrade our servers to multiple cpus and many gigs of ram, they'll be needed by mysql/whatever, rather than separating things into virtual machines..

then again, i don't use linux on workstations, and tend to avoid installing things that aren't in yum, which is probably where the "i want to avoid hosing my system" would come into it.
(Reply) (Thread)
[User Picture]From: taral
2005-12-04 07:27 pm (UTC)
Have you tried setting up read-only mounts to share the OS across VMs?
(Reply) (Thread)
[User Picture]From: brad
2005-12-04 07:32 pm (UTC)
I hate that. We do that with our web farm, PXE booting with a shared NFS root. Each machine has its own /etc, /var. But if a Debian package is installed on the NFS server's root (in a chroot), each actual web node doesn't get the postinst configuration.... management hell, sometimes. Certain packages work, but others which need to touch /etc and /var don't.
(Reply) (Parent) (Thread)
[User Picture]From: taral
2005-12-04 08:00 pm (UTC)
Sounds like a problem that needs a clever solution. :)
(Reply) (Parent) (Thread)
[User Picture]From: scosol
2005-12-06 03:03 am (UTC)
arrgh- now i'm reading the Xen docs (again) and wheels are turning-

anyway- from the docs: "Note that the Linux NFS root implementation is known to have stability problems under high load (this is not a Xen-specific problem), so this configuration may not be appropriate for critical servers."

I understand you don't like it in that pkg-mgmt sense, but stability-wise, no problems related to NFS-root? I wonder what is meant exactly by "high load"-
(Reply) (Parent) (Thread)
[User Picture]From: tijuanacartel
2005-12-06 08:59 pm (UTC)
If you export as read-only there's very little that can go wrong, except for the NFS server blocking.
(Reply) (Parent) (Thread)
[User Picture]From: tijuanacartel
2005-12-06 08:58 pm (UTC)


Oracle Cluster Filesystem has this totally rad thing called Context Dependent Symbolic Links. Basically it means that you can have a shared root but certain individual files/directories can be mapped out to unique/local versions for each box in the cluster. It's designed for shared root Oracle installations, so it's fast as. It's a nice alternative to NFS that still allows you to do live Xen migrations. You've read about that i'm sure.

Also, unique var/etc is masochism. You only really ever need a few files/directories in etc and var to be unique. You can have these in a /local tree, then link to them from your shared /etc and /var.
(Reply) (Parent) (Thread)
[User Picture]From: dossy
2005-12-05 01:54 am (UTC)
I can't stop talking about how cool Xen is.

Try out a new program? Use a whole new operating system... a new operating system (Debian sarge) only takes 114 MB, so why not?

Why not? Doesn't Xen require modifications to the OS (unless you're running an Intel chip with VT)?

Needing modified guest OS'es or special hardware makes Xen a cool, but unsuitable replacement for VMware. It'd be cool if VMware came out with a VMware-based hypervisor product: boot into VMware, run unmodified guest OS'es underneath it, on any hardware.

I can dream, can't I? :-)
(Reply) (Thread)
[User Picture]From: brad
2005-12-05 02:07 am (UTC)
Xen 2.x requires a paravirtualized kernel, but not userland. And since I almost always run with a hand-made kernel anyway, typing ARCH=xen when building isn't a big setback.

Xen 3.x can run unmodified guests with VT chips.

VMWare is making new versions that use the VT stuff, I believe.
(Reply) (Parent) (Thread)
From: jmason
2005-12-05 07:13 am (UTC)

kool-aid = drunk

yep, that's cool.

Someone needs to spend time making a xen-oriented linux distro, built around concepts (and scripts) like that. Xenbuntu!
(Reply) (Thread)
[User Picture]From: tijuanacartel
2005-12-06 11:11 am (UTC)
Heh, I have a script like that... my secret weapon! I'm currently writing an app that lets you take stuff you try out in a Xen instance and make it into a template you can later run basically by replaying your (edited, and documented) changes. For instance, set up a Hula instance by providing a variable for hulasetup. My idea is for it to make build testing insanely easy, and also yeild unit tests that can automatically determine if a server is doing its designated function properly.

I'm in the midst of making my CV hardcore for scrutiny by your crack team of HR people. Oh yeah, CV means Resume. Having never applied for a job in America before, I'm not really sure what you guys expect. Basically I am just writing about all the cool stuff I've been doing, what else should I put in?
(Reply) (Thread)
[User Picture]From: decklin
2005-12-07 10:00 pm (UTC)
Not at all related to this post, but *someone* who uses Xen needs to read it:

(Reply) (Thread)