Think about what your baby needs to do: eat, sleep, and be clothed and warm. You can do that quite simply, really.
Keep the outfits simple, as the PP have said. (I loved ones that snapped between the legs all the way down. Very easy for changing.)
Breastfeed, if at all possible.
And don't go crazy on bedding for the crib. Blankets and bumper pads aren't really recommended anyways. A nice set of sheets (I'd recommend not going too cheap here, 'cause otherwise the sides come untucked all the time from crummy elastics!) and a "sleeping bag" are fine. (The sleeping bag things, like grobag, can be made yourself pretty easily. There are patterns available at fabric stores and online.) A couple of stretchy blankets are nice, for bundling a newborn, if you have a cooler weather baby.
Many women love their carriers, but shop around online and second hand. You don't have to buy one new! I just got one myself, and while I was looking at the hug a bub,($110) but I found a very similar one (Moby wrap - $50) for about half the price. You can also make one.
Cloth nappies are money savers (although you might want to have both, since disposables can be easier if you are out.)
For toys - don't worry. You're bound to get lots as gifts, plus babies and young toddlers like to play with "non-toys" just as well. A frozen washcloth is a great teething toy. Pots, pans, and a spoon are great for banging. A jar with some beads (tightly closed!) is a fine rattle. Sing and read to your baby (the library is free!). He'll be just as happy as the baby next door with the designer rattle, and perhaps more so, because your baby has your attention.
Above all else: SHOP SECOND HAND!!! Here in Canada we have some great children's second hand stores, but there are always garage sales, etc. The Parents of Multiples Association here has a big sale twice a year where they get rid of the extra stuff they needed for their twins, triplets, etc, and it's a great place for bargains. Perhaps there is something similar near you.
All the best!