Yes baby oil is simply mineral oil and does no good for the skin. It can't sink into the pores and I have heard it could be toxic.
I am not a fan of J&J - they are very chemical based.

Cold pressed olive oil or sweet almond oil is the way to go if you want all natural skin care for bubs.