I have not seen this with iPhoto5. Using a G4 15" 1 Ghz Powerbook and downloading 20 7-8 MB RAW image files from a Canon G6, the bottleneck I find is the USB speed. As iPhoto was building its database and processing the files, I was able to work in 5 other apps without any sort of slowdown. Looking in Terminal at the top command output, the iPhoto load was consuming between 11% and 22% of the CPU. Once iPhoto was completed with the processing, rotating RAW images, zooming, etc. in the iPhoto window was quick. I then hooked up a 20" Cinema display and noticed no difference in speed. The processing capability of the powerbook is way below what you have available, so I wonder if there is a cache that is causing the slowdown.
My test was a fresh download from camera, not an import of an old lib.
Craig