Just wanted to let you know you are not alone. DS1 went through this from about 1.5 to 2+yrs old. Nothing worked. don't really know what to tell you, just letting you know that you are not alone. Even now he is not a great sleeper, waking up regularly - but atleast he doesn't wake the neighbours screaming inconsolably anymore. I am not sure why he suddenly improved (it has only been a month or so, hope I'm not jinxing us). But he was always worse when he was ill (ear infections, etc).

good luck.