The inorganic form of selenium (I think its sodium selenate) is the only one you would have to worry about it building up. People have taken 200 mcg a day of other forms (such as L-selinimethionine)for very extended periods of times with no problems whatsoever. Mercury binds tightly with selenium and leaves us with very little selenium in our body, which plays a giant role in many important enzymes and thyroid functioning...so if you have mercury you could definitely use selenium.

I take 100-200 mcg a day and can notice it does give me some anxious symptoms but nothing serious.