I just pulled out my
Eleusius (Chi O magazine) and we have quite a few chapters with above a 3.5, my chapter is one of them

It actually looks like it isn't too uncommon to have GPAs in the 3.5-3.7 range. 3.8 or above, now that's really impressive (or your school has crazy high grade inflation). Along those same lines, if you go to a school where 4.0s are unheard of, a 3.3 average could be really really impressive! It's tough to compare raw numbers alone.
ETA: It's interesting that sorority averages tend to be so much higher than fraternity averages...