I'm glad you've had the courage to come on here and say it like it is. This is exactly what I am fearing and now I don't feel so bad about it.

I would say it's worse at night because:
1. You're tired after a long day
2. You've been in a routine and had the night to yourself for so long that it's a big adjustment
3. I've always looked forward to my 'work day' finishing at 7pm (when DS) goes to bed, and that won't be the case anymore.
I have no doubt it'll get better. Once the little one is sleeping through and you can have you and DH time back, you'll be feeling much better - it just takes time, as we all learnt from baby #1.