Speaking as someone who is out in the working world, I'd approach it like a job that didn't work out.
Go find something that's a better fit, but don't burn any bridges or trash your former organization (although they sound as if they deserve it). As someone suggested "It didn't work out" or "the chemistry just wasn't right" is a good neutral way of dealing with questions. There's nothing that sounds worse than someone bad-mouthing their former organization, even if it was deserved. And if the guys are jerks, people will already know this, and admire your restraint.
The fact that you aren't listing where you're from or naming names tells me that you have already figured this out.
It's a shame you had time to develop a fondness for the national organization, but there are plenty of good, strong fraternities out there. Greek life ends up paying dividends long after college is over, so I hope you're able to find a new home. Good luck.
__________________
A∑A
|