I like Apalachicola, FL - not too far from Tallahassee, if you wanted to go on a little day trip.
In Atlanta, I second the aquarium. The MLK Nat'l Historic Site is always good. Haven't made it to the new & improved Coke Museum yet, but I liked the old one, so I would imagine the new one can only be better
I always like to go to the High Museum of Art. Not really sure what kind of things you're into, but I would recommend checking out the Virginia Highlands area (
http://www.virginiahighland.com/) - lots of great local places.