Anna, I also was going to suggest he may be having some problems with the types of food you are feeding him?
Other than that, Xander is always going through phases of being quite unsettled - though at the moment it is because of teething. If Jett is teething, you could try panadol or a cold wet flannel as he is still a bit young for Bonjela etc.

No, you are not a bad mother for passing him to your DH when you have had enough... actually, I think that is the best thing you could do as we all need a break sometimes; even if only for 5-10mins whilst we step away and compose ourselves again. I'd go nuts if I didn't do this.