AITA for saying my nephew is getting big?

Over the weekend, I attended a family gathering and ran into one of my nephews whom I haven't seen in three years. I noticed that he had grown significantly taller since the last time we met, and I commented on it, saying that he had gotten bigger. However, it seems that my nephew misinterpreted my remark, as he later told his mom that I was making fun of him. My sister then confronted me, demanding an apology for hurting her son's feelings. I noticed that he was also crying as well.

I tried to explain that I didn't mean any harm and that I was only commenting on his height, not his weight. Despite my efforts to clarify the situation, my sister still wanted me to apologize, so I did to keep the peace.

Later on, my mom chimed in, advising me not to comment on people's bodies in the future to avoid any misunderstandings. However, I don't feel like I did anything wrong here, and I'm still a bit confused about how my comment could have been misinterpreted.

Was I in the wrong for what I said?