AFAIK its safe to start taking it any time during the third trimester.
I started drinking tea at the beginning of the third trimester and then started the tablets just over a week ago.