You must be quite young, surrounded by kind peers, or maybe you're taller than you think you are to be noticing this now. It's height discrimination.
It's just a fact of life: those who don't respect you will make it a point to knock you down a few notches by pointing out a flaw or something that they're better at than you. It's a way to feel good about themselves, satisfying their need to feel less insecure.
Women are vicious to each other about what they wear, who they're with, what someone said, etc. Similarly men put each other down, and the low-hanging fruit is height. You'll notice that if a small guy works out a lot some people will say that he's compensating for his height, while average or taller men won't get negative comments.
As I said, it's about respect. The older you get the less you'll hear these putdowns in the open, because it's immature to do so. However, they might be talking behind your back or base decisions on how they perceive you without your input. For example, taller men (and thinner women) earn more money than their peers.
And let's be clear that this isn't restricted to height, it's just another flavor of discrimination. It can be your weight, age, disability, education, gender, nationality, race, religion, etc. We all make conscious and unconscious decisions based on these facets no matter how unbiased you think you are, some are just much worse about it. That being said, many seem to deny the existence of height discrimination, which is probably why short stature is still openly mocked.
