It's not dumb hehehehe
I have the FB settings set up to automatically post my blog to my FB profile and using that I cant pick and choose who see's it, it just posts it to my wall so I'd say the 365 site is the same. However what you could do is to paste the link/url to each days photo directly into facebook and then click on the padlock and select your 365 list that way. A bit more work - but you would have control that way - Hope that makes sense :)

