Well where I'm from it's ok for sorority girls to where their letters to fraternity houses but it is considered disrespectful for guys to wear their letters to another house.
We usually don't wear our letters to parties because if we are wearing letters then we can't have any alcoholic beverages.
I would check with people first, get some opinions from people on your campus, don't mess up Greek relations because of letters.
If you can't wear them at keggers, where them everywhere else. If you are trying to make your presence known pick a day and have everyone in your organization wear a letter shirt.
Have a nice day