Dexter, I commend you for both taking up for your younger frat brother AND walking away. I totally agree with everyone else when they said that you should not feel bad about what you did. Walking away CERTAINLY showed a HIGH level of maturity AND wisdom on YOUR part. Think of it THIS way. I'm willing to bet that the guys that you walked away from were embarassed and probably confused when you did that. You know, people usually aren't going to be intimidated or scared if you choose to follow them up. It's when you're quiet and you try to avoid physical and/or verbal confrontation that puts people on edge because they don't know WHAT you may be thinking or WHAT you may do.
I don't know the sex of your child. But it appears to me that you are setting a great example to him or her (specifically if your child is a boy). You're older, educated, and, most importantly, you respect YOURSELF. You did a very UNSELFISH act. You thought about the safety of you and your frat. And you thought about your family. That's something to be VERY proud of. And guess what? I'M PROUD OF YOU TOO!!!!!