My son loved the ball too.
She's so very very young (still a newborn), she's still adjusting to the outside world. It's ok to use your breasts, or the fitball, slings or whatever helps her. Why make life harder for yourself? She will change, and very quickly at this age, just try to ride it out.
I also found just jiggling/bobbing around with DS in a wrap also helped a lot - so you can kind of bounce without the ball.

You're not 'giving in' to her when she cries, you're meeting her needs. She obviously finds bouncing very soothing! And you didn't create a problem here, you very cleverly figured out something that works very well for her.