I think it really depends on the kind of place you're getting to-go from. I worked at a restaurant as a hostess/to-go person/occasional busboy and I would just take the orders over the phone, and package them up for the customer.. I never thought I should get a tip because it was part of my job. While at some places I guess there is more involved (although unless you're cooking the food too, I'm not sure what else you'd be doing other than taking the order, boxing it up, and ringing up the customer), at a lot of places there isn't, so people don't tip.
About the baby thing: Even if your baby is the most well behaved child in the world, the server has probably seen 15 other demon babies who have screamed and cried the entire meal before your party came in. It's not an excuse to be rude by any means, but when most of the babies you've dealt with have been terrors (and if you're a teenager and aren't that fond of babies yet), it's not your first reaction to make sure you say hi to the baby. Just giving the other perspective on it