If it's your only plant and you are not worried about pollinating other females by all means grow it out! Seeds are an inconvenience and will not affect the quality of the smoke. As far as trying to remove the male flowers forget it! They can pop up at any node on any branch even the main trunk. They can and most ofter will show up on the colas intermixed with the female flowers. Grow On!