When I was two years old.  Doubling a 2 year old's height gets you very close to the final adult height.  When my 6'1 father heard I was going to be 5'7 he said "that's too short!"

Luckily I didn't know or care about stuff like that at the time.  My height didn't really start to affect me until middle school (age 13) when almost all of my classmates were taller than me.  I remember almost getting into a fight with another small kid, and one of the "cool" (i.e. tall) kids sarcastically said "this is going to be a real clash of the titans."