Don't ask me about windows... can't see out of them LOL, but I clean my oven by sprinkling bi-carb all over then splash with white vinegar and scrub with a plastic brush... you can also use this method on a cloth if needed. You may have to do tough areas twice, but it is not difficult. Then rinse/wipe out with clean water.

My oven was pretty bad and this worked a treat!