Leaving the machine plugged in does indeed leave it grounded, but of course there is the risk of accidentally powering it up.
What I usually do is to leave it plugged in, but I have it plugged into a turned-off power strip. That leaves no power flowing to the machine, yet it remains grounded. Then, before doing anything inside the machine, I touch some metal part of the chassis (e.g. the screws on the outside, or the metal PCI covers, etc) to drain the static in my body. Then I am safe. Repeat this if you move around the room, since walking will build new static.
Antistatic armbands do work, but they're a pain in the ass, since they tether your body to an outlet (they're designed to go around your wrist and connect to something grounded, so that you drain static before you touch anything). Grounded mats and desktops are much more convenient, but overkill for occasional work.
Also, generally speaking, you can reduce (but not eliminate!) static buildup by increasing humidity in the room.
tooki