I suspect it has to do with the drop shadows of the windows, but shouldn't OS X have optimizations to prevent this?
It is especially visible in browsers when you watch videos from different sites. If a windows overlaps the video with just a few pixels, the performance is totally killed, but then restored once you remove the window.
I'm on a Macbook Pro 1.83, btw.