Me too! At 18-20 weeks, I had a bulge, but there was nothing really there that made me look pg.
I think at about 25-28 weeks you could start to tell I was pregnant, but I was still getting small comments up until about 38 weeks. Its only been the last 4 weeks (I can't believe I can say 4 weeks after the 38th!!) that I have really filled out. I'm so happy that happened now, cause life is a whole heap more uncomfy for me being bigger.
Enjoy being on the small side while you can! (Although it is totally frustrating)
Fi