Need Quick Help about Dinner Invite!

[sj05]

My freshman year I ended up pledging for a very small, professional sorority, specifically for my major, but the time commitment was far too overwhelming for me and other pledges as health science majors. (in fact it went from a 7 to 2 people pledge class).

I made the decision after that, that Greek life would most likely be too intense for me.

However, I ended up going with a friend to a 1 rush event for a similar, co-ed professional fraternity just to accompany her. I assumed because I just went to 1 they would forget about me.

But (without thinking) I ended up going to one of their parties and they gave me an invite to their private dinner.

But from what I remember, these dinners for people who are very interested and are the time when the members contemplate bids.

So would it be rude to not go? Or should I be tactful and show up and then tell them later on that I'm not interested?

This may be a stupid question, but I need to RSVP by tonight. What is the most polite thing to do?

EDIT: Also, if I decided not to go does this mean it would be really taboo/awkward if I ever went to one of their parties again?