Title explains the whole problem!
It doesn't matter if it's windowed or fullscreen.
There's even a discussion about this problem in the Apple forums but it's already quite old and there is no solution posted.
Anyone has this as well?
OS X 10.8.3 - iTunes 11.0.2
(I think I have this since a very long time but haven't used the visualizer in a long time)