you have a good excuse not to let him taking your daughter

To let your daughter live with your hu*****and, away from Mom and brother is not good for your daughter, period.

Don't let him do that, just for the sake of your daughter.

Take care a good luck!