I am an explorer traveler. If I am in the city, that means I am walking down as many streets as I can, just to see what I see. If I am in the country, that means I am hiking, climbing waterfalls, riding horses, whatever.
I don't think that makes me difficult to travel with, it just means that I have to be selective about with whom I travel. I have a group that doesn't like to do any of those things, which is ok. That just means I know that I am signing up for when I decide to vacation with them. Luckily, I also have a group that LOVES that stuff, so I can travel with them when I want to too.
To be honest, there are times when I just want to go by myself. If you are by yourself, your agenda is your own. You don't have to compromise on what you want to do or see. There are also times when I have to go by myself. I am not going to miss my opportunity to go somewhere while I am waiting for other folks to get their act together.