I've made it using rice milk - turns out the same but tastes like rice milk not creamy. When I was making it for my DD I didn't usually add sugar, because the rice milk is sweet already - but i'm not sure if oat milk is sweet in the same way? I would puree some fruit or use the ready pureed tubs (goulburn valley) and she ate the lot every time LOL.

Depending on how old your baby is, you can make it using custard powder, or just cornflour, or just egg/egg yolks and add a couple of drops of (real) vanilla. I also would do it like a baked custard sometimes too. Just experiment, I only ever made tiny batches, like a cup of milk at a time. If you want to introduce some texture you could try doing semolina or rice puddings as well.