No, I wouldn't be worried, as long as you gave it a wash straight away and put some disinfectant/alcohol on it to kill any nasties... I had two cats while pg with DD (now I have 3!!) and was forever getting scratched and bitten when bathing them - I never had an issue. As Kelly says, you have to be pretty filthy to get toxo or anything, if you're worried or it gets infected a quick visit to your GP to see what can be done should put your mind at ease.
HTH