Do your kids go to daycare/kindy/occasional care?
DD1 goes to DC, DD2 is at home, I work from home.

How old were they when they started?
DD1 started at 8 months of age, when I was having trouble getting work done at home with her and not achieving much when I went into the office

Am I being irrational/selfish?
I don't think you are being selfish or irrational, I think it is pretty normal to not want to part with your child at what ever age, we don't have the option of any family to watch our DD's instead of going to care and FDC didn't have any openings, so we really didn't have much of a choice. But DD thrives in DC, she is coming along in leaps and bounds, and would not have been able to provide as much social interaction at home due to work, needed to be home more....

Maybe you can use that time as "your time" doing something special that you can't easily do when she is home, make it a positive experience for both of you and see how it all goes