In my honest opinion, so many people don't even bother to RSVP if they are or aren't coming that they will be glad to know that one person is a definite no.

It is an ethical issue but I wouldn't consider it the best time to discuss ethics so I would probably decline but give a different excuse. If you feel bad about that, makes plans for that day so you aren't actually lying.