Tell your mom that you won't be able to attend as you have plans for a SERIOUS hangover.
Seriously, just don't go. If you feel comfortable doing so, tell your mother that you have not enjoyed the previous brunches and that's why you've chosen not to attend. When she says, "Everyone loves the New Year's Day brunch!" just respond, "I don't."