Stay out of midtown, and stay in the Village. Much more character.
Also, a good museum to visit, on the Upper East Side, is the
Frick Collection, because its not to big. While you're there, the
Polo store is a few blocks away, and unlike anything that you've ever seen.