But what if you did this all in software, over multiple redundant machines?
The pieces are almost all available:
Follow the layers:
-- database journal (innodb logs) or filesystem journal, on:
-- Linux iscsi initator (presents block device over iSCSI), talking to:
-- iscsi target (local machine?), talking to:
-- 2x or 4x: MySQL Cluster
CREATE TABLE sector (
devid .... SMALLINT UNSIGNED NOT NULL,
sector ... INT UNSIGNED NOT NULL,
PRIMARY KEY (devid, sector),
All we'd have to write is the iscsi target, which would be userspace.
Would it be as fast as a SCSI or FiberChannel SSD device? Probably not.
Would it be faster than a disk that had to physically spin to fsync or seek to do random I/O? Maybe.
It's that Maybe that inspires me.
If anything, it's a dorky project to learn the iSCSI protocol better. Who knows what the MySQL Cluster internal wastage is. I know memcached is about 33% internal fragmentation. (not that I'd use memcached ... the point is reliability, and MySQL cluster is designed for that, whereas memcached is cache.)
Also, clarification: I wouldn't actually use SQL or the MySQL protocol. I'd do this as an NDB client. (NDB = MySQL cluster)
Update: To add confusion with my talk of NDB above, I'd like to add that it'd probably be easier to prototype this all with NBD (Network Block Device) first, since the protocol is so much simpler. That's probably what I'll do....