By request, here are some more detailed instructions. Adobe Photoshop is used here to create the mask, but any graphics applications like GraphicConverter, ColorIt!, or even ClarisWorks/AppleWorks can do the same thing.
1. Copy a frame of the current video for reference
Open up the movie in the QuickTime Player, scrub to any frame of the video, then select Edit:Copy.
2. Create a new image in Adobe Photoshop
Launch Adobe Photoshop, and select File:New. The dimensions will already be filled in with the frame size of your video. Once the new document is created, paste in the video frame.
3. Create a 1-bit mask
In Photoshop, create a new layer. Then, using any tool you like, fill in the area that you want to keep with black. I suggest using the rectangular marquee with an aspect ratio matching your video (probably 4 to 3).
4. Export the mask
Turn off the layer containing your video image. You should now have a black and white (transparent) image of just the mask. Select File:Save As Copy... and the mask as a flattened PICT file.
5. Apply the mask
Return back to the QuickTime Player with your movie. Select Movie:Get Movie Properties, then select your video track in the left pop-up, and Mask in the right popup. Hit the Set button, and select the mask file you saved.
6. Resize the resulting movie
I'm sure you can figure this one out.