It must be extra hard with 2!
In my experience, yes, that is entirely normal. The only way to survive this is to bed-share and get whatever rest you can at other times - calling in any help you possibly can.
Both my kids woke 6+ times a night pretty much every night till they were over 18 months old. But I only had one at a time.

For reference, one-third of 4 year olds wake in the night and need help from a parent to go back to sleep.