YES! It is my pet peeve at the moment. I'm getting married next week and we had to chase up about 10 RSVP's. During the week I had to help my bridesmaids chase up RSVP's for the hens party on Saturday. We invited 30 people and only 5 gave us an RSVP. I was sooo peeved cause the girls had booked catering and needed to know numbers. I sent a rather blunt text message message saying "I need your RSVP TODAY for the hens party for catering". Got quite a few rsvps straight after that...hehe

But IMO it's rude full stop not to rsvp whether you are going or not. A simple yes or no via text, email or phone is easy to do and does not take much effort or time at all.