?

Log in

No account? Create an account
Power Outage - brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Power Outage [Nov. 4th, 2006|03:16 pm]
Brad Fitzpatrick
[Tags|, ]

I think it's a LiveJournal tradition to lose power at the data center once a year or so. Wtf?
LinkReply

Comments:
Page 1 of 2
<<[1] [2] >>
[User Picture]From: burr86
2006-11-04 11:17 pm (UTC)
Every time we sell the userbase, we lose power two weeks aftewards. ;P
(Reply) (Thread)
[User Picture]From: brad
2006-11-04 11:20 pm (UTC)
Dude, stop adding fuel to the fire .... Russia's already convinced that the downtime was us "moving" the site to Russia. It was in newspapers or something I heard.
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: lisa
2006-11-04 11:17 pm (UTC)
next time i'd like to celebrate it in the country.
(Reply) (Thread)
[User Picture]From: brad
2006-11-04 11:21 pm (UTC)
Next time I'd like to be woken up 7 hours earlier.

I'd like to think it was careful sleep management on somebody's part, but I've been overly optimistic many times before.
(Reply) (Parent) (Thread)
[User Picture]From: smirnoff
2006-11-04 11:17 pm (UTC)
woo-hoo! lj's back?
(Reply) (Thread)
[User Picture]From: brad
2006-11-04 11:19 pm (UTC)
We're watching it closely, but I hope so.
(Reply) (Parent) (Thread)
[User Picture]From: substitute
2006-11-04 11:21 pm (UTC)

unplugged, ug!

What a mess. I hope you guys can get some kind of multiple data center/failover thing going. Power problems are the worst.

Especially when the data center claims their own redundance and that turns out to be a fib.
(Reply) (Thread)
[User Picture]From: forgers
2006-11-04 11:22 pm (UTC)

tinfoil hat tales

You should see all the conspiracy theories in the Russian LJ about it -- it's related to a "Russian March" of nationalists which was dispersed today in Moscow subway, where LJ was used by the organizers (while Nossik escaped to Budapest); and a known (but very silly) economist and LJ user blamed SUP for it. State-owned TV chose not to talk about the march, and they classified LJ shutdown as KGB work, too! (Time to check datacenter employees? Or the power company ones?:) Now California energy policy has Moscow repercussions -- all thanks to the power of the 'net! We live in a wonderful age.:)
(Reply) (Thread)
[User Picture]From: brad
2006-11-05 12:16 am (UTC)

Re: tinfoil hat tales

hah. thanks for the update.
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: hughe
2006-11-04 11:27 pm (UTC)
disk complications again?
(Reply) (Thread)
[User Picture]From: brad
2006-11-05 12:17 am (UTC)
Fortunately, no. At least not as much as last time. I think all the details need their own write-up.... it's pretty amusing in a painful sort of way.
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: funjon
2006-11-05 12:15 am (UTC)
There are distinct disadvantages, then, to having your two servers colo'd in the same facility as LiveJournal, and on (one of) the same backbone. :(

*recovers lost data*
(Reply) (Thread)
[User Picture]From: dan_lane
2006-11-05 01:26 am (UTC)
So, nobody told you that LJ consistently choose the WORST datacentres?
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: lisa
2006-11-05 12:23 am (UTC)
is that part of the same week where we celebrate ISAM files?
(Reply) (Thread)
[User Picture]From: funjon
2006-11-05 12:25 am (UTC)
What, you say it like ISAM tables are a bad thing!

*runs away*
(Reply) (Parent) (Thread) (Expand)
(Deleted comment)
[User Picture]From: eu_shestakov
2006-11-05 12:26 am (UTC)
Shit happens. But the show goes on:)
(Reply) (Thread)
[User Picture]From: brad
2006-11-05 12:29 am (UTC)
It's an annual show. We crash once a year for everybody's amusement.
(Reply) (Parent) (Thread) (Expand)
(Deleted comment)
[User Picture]From: brad
2006-11-05 12:35 am (UTC)

Re: plan for next year...

I hope that catches on. But that also means replacing all our servers.
(Reply) (Parent) (Thread)
(Deleted comment)
[User Picture]From: wilmygirl
2006-11-05 12:38 am (UTC)
I've had an awesome time. Party on! :)
(Reply) (Thread)
[User Picture]From: alohawolf
2006-11-05 12:39 am (UTC)
its a tradition, like fruitcake, or eggnog, or spending hours with relitives you would rather stab then spend time with.
(Reply) (Thread)
[User Picture]From: flashfire
2006-11-05 01:29 am (UTC)
I wonder who hit THE RED BUTTON this time.

I'll bet someone's already made a "I survived the LiveJournal Blackout of 2006" userpic, too.
(Reply) (Thread)
[User Picture]From: kvance
2006-11-05 01:36 am (UTC)
I thought you guys got pwned this morning. My traceroute to www.livejournal.com went to NY, then the Netherlands, then stopped in Germany with an !X.
(Reply) (Thread)
[User Picture]From: brad
2006-11-05 01:38 am (UTC)
Hah ... I guess that's what happens when you stop advertising routes for awhile? Never knew that.

I didn't know our routers also went down.
(Reply) (Parent) (Thread)
[User Picture]From: supersat
2006-11-05 01:43 am (UTC)
Weren't you guys going to invest in your own UPS units after the incident in 2001?
(Reply) (Thread)
[User Picture]From: brad
2006-11-05 02:37 am (UTC)
Something about them not being allowed due to fire code or something. I forget.
(Reply) (Parent) (Thread) (Expand)
Page 1 of 2
<<[1] [2] >>