Having lived in a college town all my life, I've found this stuff to be pretty common. A lot of families love the fact that a college town brings lots of cultural opportunities/"enlightened attitudes"/etc. into their city without bringing the crime and pollution and all that that would come with living in a bigger city. Most of these families wouldn't even BE living in Madison (or insert the name of the college town of your choice) if the university wasn't here. And yet they complain about the students constantly! I've heard people who live right next to the football stadium (which is one of the most heavily populated-with-students areas there is) complaining about the noise on football Saturdays -- give me a break. If you don't like it, don't live there. The stadium's been there since 1917. Don't try to act like you didn't know what you were getting into.
Madison might be a little different because most of the students live in neighborhoods that are almost entirely student-occupied, so I think there's less of this kind of thing. The student neighborhoods are pretty well staked out, so if a family decides to move into a house on Mifflin Street (a student-dominated area), they know what they're getting into, and if they start complaining about all the students near them they need to just be quiet. The situation isn't like this in all college towns so it might be a little different . . . but still. College towns have college students in them. I don't see why this is so difficult for people to understand.
Of course, in twenty years I'll be on the other side of this issue, so . . .
