I feel your pain, albeit somewhat less with Panther. It seems that no matter how much memory you have with Jag, this function is the bane of my right click existence of a file.
Paring down of the apps in your Application folder will help, as well as your external drives not being spun down.