I think it's if you're carrying unhealthy, or potentially unhealthy weight. If you're endangering yourself with your weight, or your weight could impede recovery in case of an injury, yeah, you're "fat". But, if your doctor says you're healthy, or if it's genetic (and no one in your family has died due to heart disease/diabetes/other problems aggravated by weight), you're fine.
I too, get a lot of comments about how much I eat, and how skinny I am. Sometimes, I even get them when I'm at home!

I know I have a fast metabolism, and now that my suitemates realize I don't run to the bathroom after every meal, things should be just fine.