I would feel threatened if a stranger felt they could ignore my wishes as to where my child was allowed to sit in a public place. I would feel threatened if a stranger directly against my wishes physically moved my child. Actually even if I didn't express a wish as to what my child was allowed to do I would expect another adult to request my permission to move my child particularly if my child was in no immediate danger and I was in close proximity. If I then let them know I preferred they didn't touch my child and there was no danger to anyone I would expect them to follow my wishes.
So yes the man was threatening. He directly ignored the wishes of a woman regarding her child. He made it clear by his actions that he would not follow her wishes.........threatening behaviour in my world........what other wishes would he ignore?