It does get very frustrating to say the least.
But, the only thing you can do is keep trying!
Keep planning events and then give a follow up note to remind them.
Do you have a good email listing first off. If a so, keep building on it as much as possible to reach more people.
After events, then give a little note to highlight who was there and what went on. Try to keep it as positive as possible. If some think others are having fun and see names they know, they may opt to come the next time.
Good luck and hang in there!