One bad member most definitely can kill your rush- and the bad member's attitude tends to be the ONE that PNM's remember. My chapter completely folded due to two bad rush years. We went from 20 women my freshman year to a very shaky 6 by my senior year-- and only one membership class in two years (for one member). A huge problem that we had was lack of commitment from two of the six members-- they didn't want to come to events, didn't take meetings seriously, etc.
The key is to ask yourselves what you, as potential new members, would find unappealing or appealing about your chapter as it stands. Fix the problems, highlight the good parts. Be tight-knit as brothers, but not so much that you appear to be cliquish and unwelcoming to "outsiders."
Have all sorts of different events-- celebrate the diversity of interests that are already in your chapter in the members, and attract others who share those interests. Make sure to do things that encourage interaction (i.e. not movie nights, since you'll be sitting there watching the movie and not really talking).
I think GeekyPenguin's idea is a great one-- encourage your members to join another group on campus and get to know the guys there.
The more diverse, the better chance you have of attracting members from all segments of the campus to strengthen both your numbers AND the quality of experience within the brotherhood!
~ Mel.