Hermies/Females

i have 3 hermies out of 10 plants in the flowering room. Do i have to take out the hermies for the females?

If you leave them in there they will pollinate the rest of the females