?

Log in

No account? Create an account
Googlebotted - brad's life — LiveJournal [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Googlebotted [Oct. 26th, 2003|05:22 pm]
Brad Fitzpatrick
[Tags|, ]

I am the 24th most popular LiveJournal user! I beat evan by 4 places! But damn avva in 14th... *grumble* :-)

Check out stats here:
http://www.livejournal.com/community/lj_nifty/87082.html

Stats like that are so easy to do when you're logging to a database. Note to anybody thinking about doing logging to MySQL: no indexes, and INSERT DELAYED. Then you're golden. Maybe one index (time), but what I prefer to do is make a new table every hour, then I can summarize/compress/discard old hours easily.
LinkReply

Comments:
[User Picture]From: stalk_her
2003-10-26 06:17 pm (UTC)
weird. I thought you'd be #1
(Reply) (Thread)
[User Picture]From: chrishaas
2003-10-26 07:22 pm (UTC)

Congratulations on your popularity

(Reply) (Thread)
[User Picture]From: jamiemccarthy
2003-10-26 11:32 pm (UTC)

Replication

Write to a machine with no indexes except a numeric primary key, and INSERT DELAYED. Replicate that to a slave with all the indexes you need. Run your stats on the slave, who cares if it gets behind, it'll catch up in the middle of the night.

You will have to DELETE FROM WHERE id < x on the master every so often. That can take a while but inserts continue while it runs.

So tell me how you're switching tables out from underneath your code without them noticing? Does it really work? Nothing complains? What happens if the process doing that chokes up and stops doing it for a few hours, would a table 5x the expected size quickly grind the machine to a halt?

(Reply) (Thread)
[User Picture]From: avva
2003-10-27 03:30 am (UTC)
*demonic laughter*
(Reply) (Thread)
[User Picture]From: taral
2003-10-27 09:02 am (UTC)
For logging to a database: use a real database :) PostgreSQL is quite effective for this, even with indexes.
(Reply) (Thread)