Please tell me you've tried more than one mouse... From your description it's not clear whether you're talking about one or more than one (and I'm not 100% sure you're not talking about the trackpad button, either).
A mouse problem is most commonly IN the mouse.