if it is really worrying you then move him...
i am sure it wont happen - but no need to worry unnecessarily - too many other things to worry about.
though would you then worry he was far away???