That's what you need to do, I'm afraid. iTunes will let you keep your music somewhere other than the Music folder, so while its own DB will still be encrypted you won't have the music to deal with. The same goes for movies. Generally, it's best to keep these things on another partition or a separate hard drive, to keep your boot system cleaner. That's only an aesthetifc issue, but aesthetics can be important.
That said, even if there isn't strictly a need to encrypt your stuff, there isn't really a performance penalty involved in doing so. It's easiest to just go ahead and encrypt them along with anything else unless there's a really compelling reason not to, and I can't think of any such reasons off the top of my head.