I'm not a big fan of crush parties, but we held one last year & this year.
How we went about it was to write down the names & contact info. of 2 or 3 different guys, and then our social chair contacted the guys. You wouldn't know who your crush was going to be until you got there, unless it was your long-term boyfriend. From what I can tell, this year's went off without a hitch!
Threaded Mode