Just popped in to offer a little commiseration, confirmation, observation and a little old-fashioned whining. It follows along with much that has been posted here already.
First off, I have crappy hardware: eMac 1.0 GHz/ATI graphics 32 MB/640 MB RAM (128MB of which just crapped out on me yesterday), running Tiger 10.4.11, and as of this writing, reasonably recent Firefox (v 3.0.6) and Flash plugin (10.something-or-other). I suck my content through an above average, consistent, robust Comcast 6.0 Mbps pipe.
Vimeo has ALWAYS been crappy for me. Every. Single. Time. Even when I wait for the entire file to load before hitting the "Go" button. If I want to watch Vimeo-hosted stuff, I gank it to my desktop and open it from there.
YouTube is hit or miss for me. I've done some experimentation with my own uploads, and I'm convinced that in order to get the best playback quality (even on my crappy gear) there must be certain procedures and settings to use from the beginning of production through to export and upload. I just haven't nailed down exactly what they all are yet. Don't have the time or inclination to get it all sussed.
There are YouTube videos that I run across that play pretty well all the way through, even in HD, and even if I don't wait for the whole file to load first. Not many, but some. From my casual pattern-recognition, these almost always seem to be generated professionally, and/or by people/companies that have discovered that magical and (as far as I've been able to find) undocumented set of production, and export procedures and settings. But just because they might be professionally produced doesn't mean that they will work well. I've also seen plenty of high-profile producers whose YouyTube videos don't play worth sh¡t. When there are amateur videos that play well, they are usually comprised of imagery that is quite static. Or, in the case of "busy" dynamic imagery, perhaps these amateurs have just accidentally stumbled upon a workflow that works well, or they have experimented like I have and made note of what works best.
Now, compare all of that crap-n-hassle of Flash-wrapped video to what happens when I grab the video podcasts via the iTunes store of the TED conference talks. Even on my crappy hardware I'm able to stream them at high quality in QT and they are clean, smooth and nearly flawless. With NO stuttering.
I'm led to believe that the problems many of us are complaining about here are ALL related to Flash encoding and how they are streamed.