I think it's alright to wear it.
I remember my mother worse a white dress to my cousin's wedding a few years ago. Well it was actually more cream than white, but it hade black trim and she work black shoes and carried a black bag with it.
It was fine. Besides, and I don't mean to be rude, no one is going to be looking at you...they're all going to be focusing on the bride (or how ugly the bridesmaids dresses are
)