Shani, I was just like you because I was not raised to say "Yes ma'am/sir" to any and everybody. It was mostly used when I got in trouble and I had to respond to Mom and Dad. (IE "Are you going to stop hitting your brother?" "Yes, ma'am.")
I prefer to show courtesy and respect by using ma'am and sir when I talk to anyone, especially my elders. It automatically grants you respect in return. It's sad that some of our children aren't being taught to say ma'am and sir. I know several times, I have to correct teenagers who respond to me with, "Yeah" or "Uh huh". Don't let someone try to get my attention with, "Hey."
That's when I get into regulator mode, and the children always learn.
Generally, it seems quite regional. Being southern bred, it happens more often than not.