getting unacked packet count, current seq number, etc....

brad's life

Brad Fitzpatrick

[	website	\|	bradfitz.com	]
[	userinfo	\|	livejournal userinfo	]
[	archive	\|	journal archive	]

[Jul. 5th, 2006|08:39 pm]

Brad Fitzpatrick

[

Tags

lazyweb, tech, unix

]

Dear Lazyweb,

I'm interesting in obtaining, in userspace, the number of unacked sequence numbers (and ideally the last/last acked sequence numbers themselves) for a given (TCP) socket.

With Linux I can do this, partially at least (only the unacked count), using getsockopt TCP_INFO and getting back a struct tcp_info from /usr/include/linux/net.h.

Couple questions:

1) is there another way on Linux that might give me more info, short of libpcap'ing my own connection? Ideally I'd like the sequence numbers themselves, not just the unacked count.

2) is there a library to do this portable on different operating systems?

3) if not 2, how do I do this on $YOUR_OS so I can write a library.

Thanks!

P.S. In Perl, it looks like:

sub unacked {
    my ($sock) = @_;
    my $IPPROTO_TCP = 6;
    my $TCP_INFO    = 11;
    my $data =  getsockopt($sock, $IPPROTO_TCP, $TCP_INFO);
    die "no data" unless defined $data;
    # tcpi_unacked (u32) from struct tcp_info
    return unpack("L", substr($data, 24, 4));
}

Link

Comments:

From:

taral
2006-07-06 04:35 am (UTC)

Why?

(Reply) (Thread)

From:

brad
2006-07-06 04:46 am (UTC)

XMPP doesn't have a protocol-level way of saying "thanks, I got your message you just pushed to me". That is, there's no protocol-level ack, only the TCP-level acks.

It'd be nice to be able to distinguish between sending them a <message> XML stanza and knowing they received it (yeahyeahproxies... rare), vs writing to a socket that's still open to a laptop whose lid just closed after a meeting.

Then if I detect an unacceptable delay acking the TCP packets, I boot them and send a message delivery error to the sender.

(Reply) (Parent) (Thread)

From:

taral
2006-07-06 05:16 am (UTC)

YUK.

*sigh* The pain of working with a badly designed protocol.

(Reply) (Parent) (Thread)

From:

brad
2006-07-06 05:31 am (UTC)

I wouldn't say it's that bad. There are other modes that do always have a response, so another way to do guaranteed delivery of messages is to follow each message with an "IQ" packet, which always has a reply....

S: message
S: IQ 'get'
C: IQ 'result.

And then that IQ 'result' is the ACK of the first two packets.

And I could do that, and probably not even generate any more IP packets than with the normal ack approach (well, depends on the client), but it's definitely more CPU parsing XML replies. I'd rather just use the built-in ack mechanism in TCP.

I'd say it's more of a YUK that the data isn't better exposed. (yes, but admitted TCP acks aren't end-to-end)

In any case, I have another need for this data outside of xmpp where it makes a lot more sense. Topic for another post, though.

(Reply) (Parent) (Thread)

From:

taral
2006-07-06 06:03 am (UTC)

I worked on a redesign of Jabber IM at one point -- it was a datagram-based protocol with transactional semantics. Using the TCP ACK data is a result of using a stream protocol for fundamentally non-stream data.

(Reply) (Parent) (Thread)

From: (Anonymous)
2006-07-06 05:32 am (UTC)

Or the pain of universally-bad socket libraries, that don't expose perfectly good information that the local machine has access to. TCP over TCP would tend to take the I out of IM.

Try using the SIOCOUTQ ioctl on Linux.

(Reply) (Parent) (Thread)

From:

brad
2006-07-06 05:36 am (UTC)

"TCP over TCP would tend to take the I out of IM."

Heh.

(Reply) (Parent) (Thread)

From:

taral
2006-07-06 06:03 am (UTC)

Ha, yes.

(Reply) (Parent) (Thread)

(Deleted comment)

From:

brad
2006-07-06 07:53 am (UTC)

Most the time I want SO_SNDBUF pretty big, though. Definitely not a few bytes.

I'd prefer not play games with constantly changing the size of SO_SNDBUF depending on if I'm sending regular data or probe data.

(Reply) (Parent) (Thread)

(Deleted comment)

From:

brad
2006-07-06 04:12 pm (UTC)

Hah. No, I hadn't seen those.

(Reply) (Parent) (Thread)

From:

dormando
2006-07-06 06:21 am (UTC)

Ugh. Isn't one of the ways of doing that ensuring that your tcp buffers aren't huge for low-bandwidth connections, then internally mimicking window scaling when writing to a socket nets EWOULDBLOCK or similar? They seem to do this really well for World of Warcraft. A single TCP connection with certain packet types being lossy. The new gaming engine at work is being designed to do similar, mainly to help scale down to cell phones over time.

Maybe windows has the exact interface you're looking for :\ Too bad more shit isn't OSS.

(Reply) (Thread)

From:

dormando
2006-07-06 06:22 am (UTC)

By "aren't huge for low bandwidth connections" - I mean not setting 512k tcp buffers when the box is going to primarily run IM and not burst out web traffic :P

(Reply) (Parent) (Thread)

From:

cjensen
2006-07-06 06:47 am (UTC)

I don't know, but I have a stupid suggestion.

Can you set SO_SNDLOWAT to the same value as SO_SNDBUF so that the select() only reports that the socket is writable when the TCP output queue is entirely empty? Or change it dynamically depending on whether you need to know if some stuff can be written vs. needing to know about an empty output queue.

Then you could just sit back and wait for the first "it doesn't work on my very special network stack" bug report.

(Reply) (Thread)

From:

brad
2006-07-06 07:52 am (UTC)

SO_SNDLOWAT is always 1 byte on Linux, and you can't change it.

And I don't want SO_SNDBUF to be really low: I'd like to be able to write a lot of data to sockets most the time. It's just when I haven't heard from them in awhile, I'd like to check if they're alive by writing a byte and seeing if they got it. I could move the SO_SNDBUF really small and then large again, but that's kinda gross too.

(Reply) (Parent) (Thread)

From:

boggyb
2006-07-06 12:52 pm (UTC)

TCP_INFO isn't there under Windows 2000. Nor for that matter are SO_RCVLOWAT, SO_RCVTIMEO, SO_SNDLOWAT, SO_SNDTIMEO or TCP_MAXSEG. Your best bet under Windows is probably to use libpcap or SOCK_RAW, but those may require you to be running as an administrator.

I think there's another networking library in Windows XP, but not sure if it's any use.

(Reply) (Thread)

Log in