It's fine..... I buy weight from a guy all the time and he invites me into his apartment all the time.
There has to be a mutual trust between you two..... I can clearly see my dealer trusts me.
He leaves me alone in the presence of POUNDS of weed and thousands of dollars(he'll be like, brb gotta take a leak... or he'll get something to eat from the fridge).... and he knows I wouldn't touch or take anything...