I now know the painstaking details of RGB (additive), CYMK (subtractive), HSV (cone, good for modifying saturation, V of 1.0 = full brightness), HSL (double cone, good for modifying lightness, L (luminance) of 1.0 = full lightness (white), full brightness at middle of two cones...). Also read up on the NTSC color model and grey scale conversions.
Also discovered that every damn sample algorithm on the web is missing a few important details. And a Perl library I won't name should be taken out behind the CPAN and shot.
But after a good battle, we now have some functions which convert back and forth between RGB/HSV/HSL without any loss in quality whatsoever. (tested on 25^3 colors or so....)