I think it would be perfectly fine to put on the invites that if people are bringing a gift, a contribution to his library would be appreciated

While some people might be fine giving money or giftcards, I think an equal number would be offended (right or wrong - that's another tangent ...)