No your chapter isn't way off. My undergrad chapter is in the North (outside of Philly) and we also didn't claim anyone as our brothers, but when the Kappas and SGRho's got together, holding/twirling eachother's canes was just no big deal. I think that on some campuses where we don't acknowledge one another as brother and sister, there is still a mutual respect for our relationship regarding the canes. We both know that the reason SGRho's carry canes and why some step and twirl with them is because of our long standing close relationship with Kappas on some campuses (Who do you think first taught SGRho's to twirl?). So I think because of that mutual understanding and respect, exchanging canes is just no big deal on some campuses. Keep in mind though that some SGRho and Kappa chapters know more than others about the connection between our organizations and the ones that know the most or that were "made" by members who had a high respect and knowledge of our sister/brother relationship (where it exists) will usually have more respect for the connection and thus not mind the exchanging of canes.
I say if you have a problem with someone else touching your cane, just kindly let them know that you don't do that if they ask for it. There is no reason; however, to be nasty and rude, especially when you know that on some campuses exchanging canes is perfectly normal.