This happened to my SIL as well - her grandfather died when she was 39 weeks, and funeral was a three hour drive. She went to a chapel in town instead, and lit a candle for him. I went with her and it was really special.

I think it's a good choice not to go - even if you have your peanut in your arms by then, sitting in a car for that long right after your birth is probably not the best start for either of you.