I'm more inclined to boil the water first too. Boiling kills any nasty germs/bacteria that might be lurking in the water.

We also have spring water here at home (Neverfail) and I've given him that without boiling it hehe. So I dunno really I just go by what I would drink. I dont' like drinking straight out of the tap, so I won't give bub water straight out of the tap, unless we're desperate