Brad Fitzpatrick (brad) wrote,
Brad Fitzpatrick
brad

Powerloss

So I guess we lost an entire cabinet (one of our 4) at Internap because power cut out. 20 servers down, but the site's still running. (Except for users on one cluster, because both masters were in the same cluster.... need to fix that!)

I just got back from dinner and Lisa's on it, but I think we lost:

2 storage machines (not yet used)
2 databases
16 web nodes

Dear Internap, not a good gesture right when we're back in the middle of contract negotiations.

So I guess it's our fault since we tripped a circuit breaker and weren't tracking how much power we were using. But how should we know? We had no means of monitoring it, and servers don't exactly say anywhere how much they're using. Guess this was our lesson learned. Time to buy some power meters and stuff.
Subscribe
  • Post a new comment

    Error

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 12 comments