Brad Fitzpatrick (brad) wrote,
Brad Fitzpatrick
brad

latest crazy idea: iscsi on mysql cluster

So these things are like $4k for a 1GB scsi disk, or $7k for 4GB.

But what if you did this all in software, over multiple redundant machines?

The pieces are almost all available:

Follow the layers:

-- database journal (innodb logs) or filesystem journal, on:
-- Linux iscsi initator (presents block device over iSCSI), talking to:
-- iscsi target (local machine?), talking to:
-- 2x or 4x: MySQL Cluster

Schema:

CREATE TABLE sector (
devid .... SMALLINT UNSIGNED NOT NULL,
sector ... INT UNSIGNED NOT NULL,
PRIMARY KEY (devid, sector),
data CHAR(512),
);

All we'd have to write is the iscsi target, which would be userspace.

Would it be as fast as a SCSI or FiberChannel SSD device? Probably not.

Would it be faster than a disk that had to physically spin to fsync or seek to do random I/O? Maybe.

It's that Maybe that inspires me.

If anything, it's a dorky project to learn the iSCSI protocol better. Who knows what the MySQL Cluster internal wastage is. I know memcached is about 33% internal fragmentation. (not that I'd use memcached ... the point is reliability, and MySQL cluster is designed for that, whereas memcached is cache.)

Also, clarification: I wouldn't actually use SQL or the MySQL protocol. I'd do this as an NDB client. (NDB = MySQL cluster)

Update: To add confusion with my talk of NDB above, I'd like to add that it'd probably be easier to prototype this all with NBD (
Network Block Device) first, since the protocol is so much simpler. That's probably what I'll do....
Tags: mysql, tech
Subscribe
  • Post a new comment

    Error

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 6 comments