It may not help but then again it may so I will share.
When I went through driving lessons I had the worst teacher. He was so awful and treated me quite nastily. Anyway just as I was going to quit he quit instead and I got transferred to a new teacher. He really built up my confidence again. One thing he told me that struck me and has stayed with me, was "Driving a car isn't rocket science. Look how many people drive. If that many people can learn to drive then you can too".
Just remember driving is a skill. It is simply something that is learnt and practiced, not a genetic gift from your parents or luck. Just take it at your own pace and you will be fine.