Hey, I'm not Christian, but I'll exchange gifts with my family and friends if I darn well please! After all, presents are what the winter holidays are all about (at least, according to South Park...)
As for Xmas being an appropriate abbreviation, that's the explanation I always heard (X = Chi = Christ). It still looks strange to me, unless I can see a good reason why it was shortened (e.g. someone decorating their house and writing "Merry Xmas" in Christmas lights - easier to do an X than spell it out).