My sorority doesn't allow anyone but initiated sisters to wear or display the Greek letters - whether they are stitched-on, screenprinted, lavaliers, keychains, anything. New members and non-members can wear the letters spelled out, either "Alpha Epsilon Phi" or "AEPhi".
If I saw a woman wearing letters, I'd assume she's a sister, and I'd be annoyed if I found out otherwise. If I saw a man wearing letters, I'd want to know why, and explain that they are the letters of a national sorority and only initiated members should be wearing them.
If I saw anyone (male or female) wearing the letters spelled out, I'd want to find out what their relationship to AEPhi is... if it's a woman, maybe she's a new member or a sister, or a mother of a sister... if it's a man, maybe his wife, girlfriend, daughter, etc. is an AEPhi. But I would not be annoyed. Same thing if I saw anyone (male or female) with an AEPhi keychain, car sticker, etc. even with the Greek letters... after all, you're not going to take the AEPhi sticker off your car when your husband wants to drive it
However, I would be upset if I saw anyone wearing or displaying anything associated with AEPhi while drinking, doing drugs, etc.