I'm with Theilgirlie - it depends on usage, but the term "frat boy" was a HUGE no-no on my college campus.
I guess it varies school to school, however - last thursday I was out with my boyfriend (who is a Pike...I just keep coming back to those Pike guys

), my best friend (who is an independent), and his girlfriend, who is in a to-remain-nameless NPC sorority at a different school. She and I were talking about greek stereotypes and whether we are shown respect, etc. My bf said something, and she turned to him and said:
"Oh, so you're a frat boy too, hey?"
The two of us just stared at her, and the best part is - it was my best friend (her bf and the only independent among us!) who calmly said to her "I believe the correct word is "fraternity"." Heh heh heh.