Hun, it will dry up eventually because you are not expresssing it all, so the amount being re-absorbed will trigger your body to produce less. You will need to make sure you express enough to avoid mastitis (and to be comfortable), but no more than that. Also watch closely for the signs of mastitis so if it starts you can treat it before it takes hold.

Best of luck hun, I hope you are doing ok.