No you are not being ridiculous, Panhellenic at my school did this, I was actually surprised it isn't common at other schools.
All groups have the option of going back and inviting someone they released earlier. And they all usually know who these women are. It's not a big deal anywhere I've worked with and I'm talking mostly SEC and southern schools.