Man, you just need to know your rights and stop being so paranoid. I'm not trying to knock you or anything, but that was a pretty dumb move, cautious as it may have been.
If cops come to the door, just don't answer (like you did) and you should be fine. If a cop was there with a search warrant, he'd probably have a few of his buddies with him (SWAT to be precise) and you'd hear about one knock before you hear "POLICE! SEARCH WARRANT!" and they come busting through your door.
If you're suspected of growing, there is no way a cop is going to come by and knock to see if you'll let him in first. Hes gonna go straight for the warrant. In fact, I think just the fact that a cop showed up at your door at all is a sign that you're NOT under investigation. Cops don't like to spook their suspects, because if they throw out all their plants before they serve the warrant, they'll have no case!
Next time a cop knocks on your door, smoke a bowl or something and calm down. As long as you don't answer the door, you'll be fine. And they can't get a search warrant for a noise complaint or the smell of weed, so I wouldn't be too worried about that.
Make some hash oil, and next time, don't waste a month's work for 5 minutes of paranoia.