yes, it's fine. The only issue during pregnancy is overheating - so make sure it's not too hot. If you're sweating, and or looking lobsterish time to get out.
Enjoy!

BTW - I was never told that during IVF. Sounds suss to me