I've always wondered... what is so bad about Ma'am?

I'm 24 and I get called ma'am all the time... I don't mind... what is it about that word?

I'd rather be called that than "Hey, lady" or "Hey, You"