Chrissy the issue with cat's poo is a real scare for toxoplasmosis. You can get tested with pre natal testing to see if you are okay. I am a vet nurse and have had the test in the early stages of pregnancy. One thing is to have disposable gloves on hand and pick the poo up with them & throw it out straight away and then wash your hands. Not that you would normally pick poo up with bare hands, but the real fear is that you would get a bit of poo into your mouth and you cat happens to have toxoplasmosis at that time. If you have been exposed to toxoplasmosis prior to getting pregnant, you don't have to worry too much about it but do have to continue clean sanitary practices when emptying the litter tray.

I would say alleviate your fears & have the test done (btw its cheaper for you to be tested than to have your cat tested). And to be safe, avoid it as much as possible, but if you do have to do it use gloves & wash your hands afterwards. HTH!