I don't think so, someone would have to take her to visit, so logically that's you. IMO he is still technically a part of your life because of DD so it's appropriate. Now if your DH took her that would be inappropriate

Hope he's ok?