You might need some legal advice. If he was smoking something you can't buy at the shops, you might have some trouble. Check your lease, but most leases say that any illegal activity is grounds for eviction.

Check your lease, and get some legal advice about whether the inspection was unlawful - and if it was, whether they are still able to use what they saw in the inspection to evict you.