? ?
brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

class UsefulStream { ... } [Dec. 21st, 2003|10:10 pm]
Brad Fitzpatrick
[Tags|, ]

jwz comments:
The fact that I wasn't able to use the various String classes, and had to write my own "network buffer" class to do protocol-ish stuff was one of my earliest gripes with Java, too.
And sure enough, that turned out to be my solution in C# as well.

Made a class "UsefulStream" which has an async method "BeginGetLine" taking the .NET-standard AsyncCallback delegate. The class then sees if it has a line. If not, begins an async read with an internal callback. Keeps checking for lines or eof, reading more as necessary. Eventually calls back into the caller's callback with the useless IAsyncResult value (I used null). Then caller invokes EndGetLine() on its UsefulStream instance, which returns the String, or null if EOF.

This actually makes things pretty easy now. I can centralize timeouts inside that class. I'll probably mimic Apache with both soft and hard timeouts (time since last read, time since first read).

UsefulStream always assumes lines are ASCII. (as protocol headers tend to be... except memcached's text protocol, which supports object key names with 8bit, I believe... feh)

And next I plan to add normal "BeginRead/EndRead" to UsefulStream, which pretty much wrap the normal NetworkStream, but also return the left-over crap that was read in while we were reading lines. (it already works the other way around: switching from getting byte chunks to reading lines...)

But yeah, hacky.

And of course this is all 2 fucking lines in Perl:

$line = <F>
read(F, $dst_scalar, $n_bytes_to_read);

(and you can even change the line separator with $/ !)

I love you Perl.

[User Picture]From: jwz
2003-12-21 10:22 pm (UTC)
It's only 2 lines in Perl once you've figured out how to break the legs off of its Unicode support so that it doesn't do Bad Craziness to your octet-stream on the way in!
(Reply) (Thread)
[User Picture]From: brad
2003-12-21 10:53 pm (UTC)
Yeah. :-(

Which is why I use Perl 5.6 still in production.

But the misfeature you're refering to is actually fixed in 5.8.1: when your terminal supported UTF-8 (which RedHat did by default, which you used) then stdin was assumed to be in utf-8.

But I think I gave Perl too much credit in another regard: those two lines block. The little UsefulStream class I did is all async, which is damn near impossible in Perl. Perl has the incredibly obtuse and slow POE.pm, but that's about it.

I think Python and its twisted library wins for this proboem.
(Reply) (Parent) (Thread)
[User Picture]From: taral
2003-12-22 09:04 am (UTC)
I don't get it. Why are you doing async IO instead of sync IO? The perl code you posted is certainly synchronous.
(Reply) (Thread)