Do you have national fraternities at your campus or are they locals (meaning they are only at that campus.) You said your campus is smaller so I am wondering if it's the latter (or a mix).
I would give it another try honestly. Obviously you have some experience going through the recruitment process. Technically all campuses should have a dry rush period (ie no alcohol) and if the group doesn't follow that rule, that should be a red flag for sure.
Go with who you feel comfortable with, not where ALL the guys are going. And go in with an open mind. I have known guys join up with a fraternity that was "the" fraternity on campus. They didn't last very long.
But have some confidence!
ETA: nevermind the first paragraph. i did a little research and you have:
Alpha Gamma Rho
Alpha Gamma Sigma
Alpha Kappa Lambda
Alpha Phi Alpha
Delta Chi
Iota Phi Theta
Kappa Alpha
Kappa Alpha Psi
Kappa Sigma
Phi Beta Sigma
Phi Delta Theta
Sigma Chi
Sigma Phi Epsilon
Sigma Pi
Theta Chi
on your campus.