I have my bub in bed with me most of the time, but put him in the bassinette if i wake up and he's asleep. He doesn't mind being moved. The co-sleeping is more for my benefit as I got a lot less sleep with my first bub when I sat up for his feeds (and found myself falling asleep falling up - not safe for baby!) This time I attach him & go back to sleep.

If it weren't for your man smoking I'd say just put bubs in the bassinette for intimate time & bring baby back to bed next time s/he wakes. But 'cause of the smoking I'd say just keep bubs in the bassinette when your man is home and co-sleep while he's away if you want.
HTH