In some ways yes, in some ways no. It also depends which PDA you get. First of all, some of the low-end ones don't do MP3 at all, so check around. Sony's Clie line has a lot of great multimedia features, but some of its PDAs, while they come with a proprietary MP3 player, are not compatible with the Palm audio standards so you can't use the more advanced third-party ones like this one
here
It should be noted that to play music or use any kind of non-Palm files (i.e. JPEGs, Word docs, basically any computer file) on your PDA, you'll need an external card, which raises the cost a bit. I believe the highest capcacity memory stick you can get for a Clie is 1GB, but the stick alone would cost you significantly more than an iPod. A 512MB is around $200-250. PalmOne PDAs use SD cards, which go up to 512MB and are usually a bit cheaper.
I personally am a huge fan of PDAs (and iPods), but if I only had to have one, I would go with the PDA.
For TONS of additional information, see:
For Clie:
http://www.cliesource.com
For Palm:
http://www.palmonecity.com