It is very easy to dislocate the shoulders of little kids, that is the reason I have been given for not picking them up by their arms.

You probably haven't done any damage so far, but best not to do it in future.