I've never heard of this happening, and I know people who wear/have worn colored contacts. Maybe they just didn't say anything, though.
Calling the eye doctor and making sure the lenses are the right size/prescription couldn't hurt--in fact, that's probably what I'd do. Other than that, I'm not sure...aside from just sticking to clear contacts from now on. That's a bummer, though, because I've always wanted to try the colored ones, too.