A little random insight on men I have discovered. They will only accuse their partner of cheating under one of three circumstances.

1) You talk about your good friend incessantly. Become cheerier at the mention of their name or are much happier when you run into them than you are when he gets home
2) You tell him you are going out shopping with the girls and either he or a mate of his drives by a cafe and sees you sat down to lunch with a guy (or similar. Caught out in a lie basically).
3) If they do not believe they are worthy of you. They have their own massive self esteem issue that leads them to believe you simply have to be having an affair.