Sounds like you're being territorial when you're not supposed to. You seem to relish in the fact that you're the common link between these two groups, and you need to get rid of this complex before you find yourself friendless. Seriously, if you're fully aware of this, what makes you think that they aren't too?

Start sharing the spotlight. You'll be happier that way.