DS absolutely hated it with a passion. So i didnt bother with tummy time for him. He is no and has been no different to any other baby of child his age. He met everyone of his milestones, some even earlier then expected. DD i didnt really bother too much either. She was a little happier to do it so i would give her a little time on her tummy, but only when i remembered. There is no difference between the two of them, in fact DS actually crawled earlier than DD.
So really honey if its stressing both you and your DS out that much, dont bother. You can maybe give it a go with time to come but honestly, in my eyes, it really doesnt make a difference if they do it or not.