?

Log in

No account? Create an account
The Internet sucks this weekend - brad's life — LiveJournal [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

The Internet sucks this weekend [May. 23rd, 2004|12:02 pm]
Brad Fitzpatrick
Yesterday it was the Alter.net backbone sucking, and today sprintlink.net.

Lame.

If I got cable as well as DSL, could I make TCP connections that were atop two independent IP source addresses, assuming I had a cooperating stack/daemon on the other end?

I'd like it setup so TCP uses one IP address (the DSL one, say) all the time, until it realizes a reply didn't come back in time, at which time it sends a retransmission out the other interface. (but not delayed... no exponential backoff until both interfaces don't reply) The retransmission's IP header would the cable modem's src IP, and it'd be up to the multi-homed device on my end to process the reply when it comes back and realize it's part of the same connection.

Anybody know of a project to do stuff like that?
LinkReply

Comments:
[User Picture]From: mart
2004-05-23 12:17 pm (UTC)

If you change source address mid-connection, you've basically got a new socket (TCB) at the other end, right? My understanding that a unique connection is described by the key of Source Address, Source Port, Dest Address, Dest Port.

The initial SYN could do this, theoretically, but I don't think you could do it with an already-established connection unless TCP has some trick I don't know about.

(Reply) (Thread)
[User Picture]From: brad
2004-05-23 12:34 pm (UTC)
My understanding that a unique connection is described by the key of Source Address, Source Port, Dest Address, Dest Port.

Yes. But my post was explaining how I want to change that. I want multiple source addresses to share the source port space, and some cooperating system on the destination side (where I ssh to) to recognize that.

Alternatively, something like a VPN where some network (say, 10.99.*) exists both at destination and source, and the VPN is kept alive by two separate real TCP connections via different providers, routes.
(Reply) (Parent) (Thread)
[User Picture]From: mart
2004-05-23 04:54 pm (UTC)

If you're only talking about connecting to one or two places (The office, Marklar?) you could just have a couple of tunnels and then SNAT into the target network so all of the hosts there see one address. You still need some trickery client-side to make it transparent, but then you were going to have to do that anyway.

I suppose if you're going to invent something new you might as well make something new for the remote host as well, though.

Whenever something goes awry with our cable, I usually just shift to dialup for a few hours. Much cheaper than paying monthly for a separate service we'd hardly ever use… SSH is a bit hellish over dialup, though, I admit.

(Reply) (Parent) (Thread)
[User Picture]From: warrend
2004-05-23 12:24 pm (UTC)
Short of writing some crazy stack implementation yourself, you could always just use lvs/ipvs. It might be a bit more overhead than you need, though.

I know connection sync works between two load balancers, but you might be able to get it to work for two VIP sources.

Then you could just point all outbound traffic at the VIP, and it would fail over and sync the traffic to the cable modem if a failure occured.

Like I said, not an elegant solution.
(Reply) (Thread)
[User Picture]From: brad
2004-05-23 12:38 pm (UTC)
I've never got into LVS as much as I'd like, but from what I've read, I thought it's just for load balancing web farms and stuff. How would multi-route stateful connection failover work without the other side knowing what's going on?
(Reply) (Parent) (Thread)
[User Picture]From: warrend
2004-05-23 01:00 pm (UTC)
It is mainly for load balancing, but you can do some interesting stuff with it.

Now that I think about it, you might be right. I have a few ideas on how to get it working, but there might be too many failure points to be worth it.

Let me think about it a little bit.
(Reply) (Parent) (Thread)
[User Picture]From: brad
2004-05-23 01:01 pm (UTC)
See below.
(Reply) (Parent) (Thread)
(Deleted comment)
[User Picture]From: brad
2004-05-23 12:39 pm (UTC)
There are more backbones than just MCI.
(Reply) (Parent) (Thread)
[User Picture]From: caladri
2004-05-23 12:57 pm (UTC)
Give or take the extra state, I don't see why you couldn't do this. Probably easiest using UDP across the internet as a layer 2 link, and have it do the determination of where to send the next packet, and so on, and use that as a virtual device, and do tcp, etc., across that.

Just a thought.
(Reply) (Thread)
[User Picture]From: brad
2004-05-23 01:00 pm (UTC)
Kinda what I'm thinking.

Using the userspace tun/tap stuff or something.
(Reply) (Parent) (Thread)
[User Picture]From: caladri
2004-05-23 01:02 pm (UTC)
There you go. I did something which used a common ground to switch the packets (in this case, IRC, cause I'm sick...) in userspace... It'd be pretty easy to just have two connections to some central point, and do duplicate elimination. Bonus points if you just use two multicast tunnels.
(Reply) (Parent) (Thread)
[User Picture]From: brad
2004-05-23 01:05 pm (UTC)
You are officially my LJ friend. (at least so you can comment unscreened... it remains to be seen whether I'll read or skim your entries.... :-))
(Reply) (Parent) (Thread)
[User Picture]From: brad
2004-05-23 01:14 pm (UTC)
Why do duplicate elimination? TCP atop TUN will just do that anyway.

The trick is to not send duplicates in the first place, which involves depending on one interface most the time, but watching replies, parsing the TCP headers encapsulated in the UDP packets between both hosts, and sending retransmissions if necessary, quicker than TCP would on top.

So basically the daemon will have to know TCP a bit, but only enough.
(Reply) (Parent) (Thread)
[User Picture]From: caladri
2004-05-23 08:50 pm (UTC)
Not a bad approach to it. I do ethernet evil for a living, so I'm used to holding heavily to the mantra of "let the upper layer protocols do their job, they're smarter than me," but this is not always true when you control the individual application. Man is this ever not true. :)
(Reply) (Parent) (Thread)
[User Picture]From: ch
2004-05-23 01:30 pm (UTC)
MobileIP is pretty close to this.

We have some connection diversity work at HP Labs that can transpartently move IP connections.

One more thing come to mind: there was also a reliable socks implementation out there (don't have a pointer off hand) that would fool (socks-aware) applications as to the real state of layer 3, restablishing the TCP/IP connections whenever they failed.

If you can control routing, there are other possibilies, but that's not possible with residential cable or dsl.
(Reply) (Thread)
[User Picture]From: brad
2004-05-23 01:45 pm (UTC)
MobileIP... interesting. Thanks for the info!
(Reply) (Parent) (Thread)
[User Picture]From: taral
2004-05-23 01:43 pm (UTC)
You could use multipath IPIP tunnels.
(Reply) (Thread)
[User Picture]From: edm
2004-05-23 03:04 pm (UTC)
Last time I was asked to plan something like this (remote office with two "domestic" connections -- cable and ADSL -- wanting to connect to a (set of) central servers), my plan was basically to build two tunnels (GRE, IPSec, IPIP, whatever), and then run a routing protocol through the tunnels, so that if one of the tunnels went down (or depending on your routing protocol/policy, got really icky to use by, eg, touring the world) then the traffic would use the other tunnel.

Conceptually it seems like it should work and that all the parts to do it exist. I haven't, however, actually deployed it as the office in question decided to move their remote office to someone on the metro fibre ring and got a fibre connection instead, and hence the problem went away.

It wouldn't surprise me if someone had implemented something like this already, but I didn't come across it at the time.

Ewen
(Reply) (Thread)
[User Picture]From: iamo
2004-05-23 03:29 pm (UTC)
I would think from the userland end of it, a pair of proxies (either something like socks or something like ssh tunneling) that act together so that connection failures are dealt with between the proxies rather than between the client and the server. You'd have proxy A running on your LAN, in front of the modems, and it'd be aware of the two available connections, then on the other end you have proxy B that is just waiting for connections.

At the lower level, I'm sure you could do something with the existing NAT implementations and only one proxy on the other end.

If you're in control of the protocol, then I'd definitely have to say that using UDP and doing your own reliability layer would be the best way to go.
(Reply) (Thread)
From: ex_ff928
2004-05-24 11:36 pm (UTC)

Multi-homed, multi-path L4 protocol

Take a look at SCTP.

Hasn't gained wide acceptance yet, but adoption's beginning to pick up, especially in the wireless market.

(Reply) (Thread)
[User Picture]From: brad
2004-05-24 11:43 pm (UTC)

Re: Multi-homed, multi-path L4 protocol

Oh, that's perfect! I just read/skimmed 3286. That's exactly what I was looking for.

Thanks for the link.
(Reply) (Parent) (Thread)