?

Log in

No account? Create an account
code push party; load balancing; garbage collection - brad's life — LiveJournal [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

code push party; load balancing; garbage collection [Feb. 23rd, 2004|09:26 pm]
Brad Fitzpatrick
Everybody's meeting at the office at 10 pm for pizza and pushing weeks of new code live. Fun fun.

Somebody asked how late we'll be there. I said, "It depends on how bad your code is." :P

I worked most the day on monobal (my C# load balancer). I also reinvestigated Pound the other day, but wasn't impressed. The SSL functionality is neat, but the load balancing isn't. For example, if a backend node is found dead, the client gets a 503 error... rather than the load balancer using a new node? I read that in the manpage. Could that be? Or does it differentiate between TCP failures and HTTP response timeouts. Monobal doesn't handle HTTP response timeouts (yet?) because I've never had a problem with that.

I think I'll be copying Pound's config file format, though, at least for the simple load balancing method. (Monobal lets you select between load balancing policies...)

Mono currently uses the Boehm GC, but I don't trust it. At least not for 24/7 usage allocating and losing pointers to tons of small objects without big GC interruptions. So, I made monobal use freelists for its rapidly created/discarded objects so I make the GC work less.

I'm getting anxious about actually testing monobal in production... I keep finding something else to improve first, but I'm about done I think. If/when it proves itself in production I want to start polishing it up (make the generic round-robin policies/etc) and give it a website.
LinkReply

Comments:
[User Picture]From: scosol
2004-02-23 09:35 pm (UTC)
*shudder* external garbage collection outside of an interpreter scares me
i imagine things running rampant...
i know you're a PERL guy, but Python will show you gods you havent seen :P
(Reply) (Thread)
[User Picture]From: denshi
2004-02-23 09:45 pm (UTC)

Re:

What's the state of Python GC these days?
(Reply) (Parent) (Thread)
[User Picture]From: scosol
2004-02-23 11:46 pm (UTC)

Re:

With 2.3 there were some weird takedown issues, but nothing that should affect the day-to-day hacker- so- solid as always :)
(Reply) (Parent) (Thread)
[User Picture]From: cjcollier
2004-02-23 10:13 pm (UTC)
When I see comments like this, I get curious whether those making them are familliar with Perl... Most who I know well don't know Perl. The fact that you've capitalized PERL makes me wonder ;)

Mayhap I should learn Python right now and find out for myself :)
(Reply) (Parent) (Thread)
[User Picture]From: scosol
2004-02-23 11:45 pm (UTC)

Re:

Yes- I'm familiar with PERL- judging by productivity output, it is in fact my language of choice- but with a higher calling I've embraced Python- ESR said it correctly in "you can contain it entirely inside your mindspace" (paraphrased)- in PERL there are just too many ways to do things- in Python there is only "The Way" :P
And Capitalization?
Pattern Extraction and R(shit- cant remember) Language
(Reply) (Parent) (Thread)
From: insom
2004-02-24 01:03 am (UTC)

Re:

a) R = Reporting
b) "Python fits your brain."
(Reply) (Parent) (Thread)
[User Picture]From: deveiant
2004-02-24 01:30 am (UTC)

Re:

Practical Extraction and Report Language. And it's a unix command, so it really should be perl(1), no? =:)
(Reply) (Parent) (Thread)
[User Picture]From: cjcollier
2004-02-24 09:18 am (UTC)
That's what they call a "backronym." People kept asking what Perl stood for (it didn't :), so Larry made up a couple of acronyms:

Practical Extraction and Report Language
or
Pathologically Eclectic Rubbish Lister

(Reply) (Parent) (Thread)
[User Picture]From: scosol
2004-02-24 10:19 am (UTC)
Hmmm as someone else commented, it's apparently a "backronym" hahah:

People kept asking what Perl stood for (it didn't
:), so
Larry made up a couple of acronyms:

Practical Extraction and Report Language or Pathologically Eclectic
Rubbish Lister


Here's the official blurb from Larry:

You may or may not choose to follow this usage. For example, parallelism means "awk and perl" and "Python and Perl" look OK, while "awk and Perl" and "Python and perl" do not. But never write "PERL", because perl isn't really an acronym, apocryphal folklore and post-facto expansions notwithstanding.

So Perl it is then :)
(Reply) (Parent) (Thread)
[User Picture]From: faepuck
2004-02-23 09:53 pm (UTC)

Yay code, but yayer pizza. Now I'm all hungry...
(Reply) (Thread)
[User Picture]From: ghewgill
2004-02-23 10:01 pm (UTC)
It's not necessarily a safe assumption that GC is unreliable or that it will make your program slower. The Boehm GC is an incremental collector that won't interrupt your program. In particular, see the Allocation and GC Myths for a set of slides outlining some of the common misconceptions about memory allocation. The only way to be sure about performance is to measure it.
(Reply) (Thread)
[User Picture]From: brad
2004-02-23 10:25 pm (UTC)

Re:

At least how I have it now, the allocation/freeing is centralized, so I can easily benchmark:

* pure GC
* my maybe-ghetto freelists
* a C library to do just those allocations (if it comes to that)
(Reply) (Parent) (Thread)
[User Picture]From: ch
2004-02-23 10:11 pm (UTC)
Careful. Most GC algorithms have a time complexity of
O(amount of active storage). Freelists of objects that require tracing will increase the amount of work for the garbage collector.
(Reply) (Thread)
From: sheehan
2004-02-23 11:17 pm (UTC)

metacomp

looks nifty... but the first "NULL" "bug in linux" that i looked at doesn't seem like a bug... complaining that
priv		    = drm_alloc(sizeof(*priv), DRM_MEM_FILES);

is accessing uninitialized memory... but sizeof is purely a compile time operator as far as i know (is there some gcc extension that is at work here?)... and most of the NULL bugs looked like that...

the other categories of bugs look pretty interesting... ie blocking issues... although i can't comment on the severity (if any) of most of them...

--scott
(Reply) (Thread)
[User Picture]From: mulix
2004-02-24 03:15 am (UTC)

slab caches

You might want to consider slab caches, which are data structures that are specifically designed to be used in cases where you have lots of small objects that are allocated/freed rapidly, especially if some state can be retained in the object between allocation/freeing (your object has three fields, but typically only one changes between uses).

See BONWICK, J. The slab allocator: An object-caching kernel memory allocator. In USENIX Summer 1994 Technical Conference (1994).

And there's a follow up paper I can't find at the moment for SMP'ing slab caches. Neat stuff. And hey, Linux uses it, so it must be great.

(Reply) (Thread)
[User Picture]From: brad
2004-02-24 12:18 pm (UTC)

Re: slab caches

Yeah, memcached uses a slab allocator.

It doesn't really work in C# because you can't allocate arbitrary chunks of memory and cast them as whatever. (C#'s all safe, ya know?)

Hence the freelists. (which make more sense because I'm only allocating two types of objects often...)
(Reply) (Parent) (Thread)
[User Picture]From: mulix
2004-02-25 12:04 am (UTC)

Re: slab caches

Why do you need arbitrary chunks of memory? a simplistic slab is just an array of objects _of a given type_. Dynamically allocate the slab, and then allocate/free objects from that slab. Surely you have some semblance of dynamic allocation in C#?
(Reply) (Parent) (Thread)
[User Picture]From: scsi
2004-02-24 08:11 am (UTC)
Hmm, I should get pizza for my code upgrading party.. But its not much of a party when there is only one of you..
(Reply) (Thread)