I'm in a bit of a similar situation - in that I was brought up as a Christian (pentecostal) and DH doesn't believe in God - as child he occasionally went to church for Christmas or Easter, usually to an Anglican church.

I'm not a member of any church but I really wanted Isabelle to be dedicated (sort of like a naming but with a religious focus). DH is happy with whatever, and so what we have decided to do is really compromise. We are having the service in a park, my BIL is going to run the dedication and its going to be a really personal reflection of who we are as parents, plus with Al giving a blessing (although he just sees it as a kind of "what /we want for Izzy" ) This way we have been able to make him comfortable, my parents who still attend church regularly are happy, my IL's dont really get it but are happy with the decision we have made and we are both happy.

Long and rambling but I guess what I'm getting at is that you and your partner need to decide what is important for the two of you in relation to bubby and maybe have a compromise based on what you both want. It is your decision and your baby, don't do something to make others happy (easy to say I know) but do what the 2 of you feel is right for you. If that means a Baptism - then although your partner doesn't believe in it - he can still be involved as its not really about him, its about bub IYKWIM I know its hard and good luck - not long to go now!