Ryan - I had 4 others in my pledge class. There was only one who wasn't certifiable!!
Don't worry if you didn't click with them, a lot of times it doesn't happen that way at all.
If that is who you got stuck rooming with, let the fraternity know that one of your conditions of staying is not living with them. But seriously, I hope you stick it out!! You seem like such a sweetie and so dedicated to what SAE is supposed to be. Plus - if these guys are that brain dead, they might flunk out over the summer and solve the problem.
If you still have to do it in the fall do it, but for now I would let it ride. Good luck!