Trust your instincts and listen to your body, recognising your own abilities. My last horse ride was at 27 wks. I know my own horse, I rode in a fenced arena, stopped jumping, no paddock rides (cos less controlled environment) and only stopped then because I then diagnosed SPD - mounting and dismounting became untenable, but had I been lifted on I would have been fine! My belly wasn't very big and never got in the way.
My point is that on some level you ought to know when you've reached your limits for you and the baby.
I copped a lot of flak for turning out to a car accident with my fire brigade because I was about 7 months pregnant. I had stayed home for other jobs and figured that if there weren't enough members there to crew a truck (it was Easter Sunday) then I'd get on and be of some sort of use. I actually did a couple of things at the scene that were non-physical and often overlooked aspects of car accidents (offering counselling to witnesses), and the knockers were all people who weren't at the scene to judge my suitability for the job anyway. The crew on the truck all said that they weren't worried because they figured if I felt up for it, who were they to judge unless I was being reckless?
Listen to yourself - and trust what you hear!