What steampick is trying to tell you is that any of your neighbors that might be growing may get their plant pollinated even if you don't intend that.
Pollen grains are practically microscopic and will travel with the wind for great distances.
Typically if you are not looking for a male for breeding purposes, the general consensus would be to take him down.