I bothers me because it is incorrect, and there are terms that are more appropriate.
How is it incorrect? Informal, sure, but alum (which is in the dictionary as an informal form of alumnus/i/a/ae) is no more incorrect than dorm, fridge or grad student.