[chadcole26]

I do video editing for a living. Windows at work, Mac at home. I thought I'd do a little real world test of all the equipment I use. I'll be honest, I didn't think the Macs would perform as well as they did.

Check it out at:

http://homepage.mac.com/schmotz/pages/ap65test.htm

Let me know if you've tried something like this. I'd be interested to know what others have experienced.Chad's Real Life Processor Test

[Thinine]

I'm as addicted as the next man, but those results just don't make any sense. I mean, a 466 MHz G4 beating a dual 2.2 GHz Xeon!?! I would go back through and make sure all the setting are the same for both, and I mean all!

[CheesePuff]

Yeah, the Intel Xeon processor is pretty powerful, and when you put two of them together with XP Pro it'll fly.

[chadcole26]

I'll take suggestions. Like I said, I'm not a super techie, so when it comes to optimizing Windows performance, I'm kind of lost. I haven't done anything in terms of settings to try and optimize any of my Macintosh computers. They run pretty much with the same setup as when I took them out of the box (admittedly I've installed plenty of OS updates, but nothing that would be considered performance enhancing.)

I'm in disbelief that a 466 G4 would out perform Dual Xeons on any task period, but I've watched with my own eyes.

Like, I said in my original post. I turned off everything that I could find on the Windows machines to help spur their performance on, but nothing helped. What I have taken note of though is that the difference between my 466 and 733 is statistically similar to the difference between the Sony VAIO and the Compaq EVO's. The EVO's outperform the VAIO like I had assumed they would and very similarly to the way the 733 out performed the 466. I turned on the task manager on the Windows machines during the second test run. The processor usage performance for the VAIO was pegged at 100% during the entire render. On the EVOs only one processor was working hard and it was only hovering at 50% to 60%. Added in later: I know that Premiere and XP are mulitple processor aware and capable, so maybe there's something that I need to "turn on" for a performace enhancement.

The result is that I'm really glad I have Mac's at home. I use them for my own personal projects and I've started using them for projects from school. I took them out of the box and they started working hard for me just like I expected them to. To say that I'm disappointed with what I have to work with everyday at school is an understatement. Especially after running this comparison test. I really did expect all three of the Windows machines to kick my G4 733's butt on this thing.

Again, I realize this is really an "Apples" to oranges comparison, but it's all I had to work with.

Let me know if you have any specific suggestions for leveling the playing field. I think it's terrible that a person can buy a computer and then have to worry about optimizing its performace just to achieve respectable results. So it goes in the dark side. Long live Apple!

[drmcnutt]

Originally posted by chadcole26:
I'll take suggestions. Like I said, I'm not a super techie, so when it comes to optimizing Windows performance, I'm kind of lost.


Again, I realize this is really an "Apples" to oranges comparison, but it's all I had to work with.

Let me know if you have any specific suggestions for leveling the playing field. I think it's terrible that a person can buy a computer and then have to worry about optimizing its performace just to achieve respectable results. So it goes in the dark side. Long live Apple!

You really wear your bias on your sleeve You should leave testing to professionals who understand the "techie" stuff and get results that aren't predetermined (even subconciously). Objectivity is more important to your results than you know. Your site reminds me of the recent XP switcher imposter, just what do you get for posting this "test"?

Disregarding the Megahertz myth arguements, Rambus memory is faster than the memory used in "six month old macs" or the recent DDR implementation some may argue. Also you test on probably the two least impressive major manufacturers of computers, Compaq and Sony. Try again with a Dell for a better comparison to Apple and their standards of production. At least it wasn't an Emachine.

Your results are very hard to believe. My own user experience is from a 266 beige G3 OS 9 at work to a recently supplied 800 G4 on OSX 2 there and I have a 400 G4 OSX 2 at home too, with a 1.8 GB XP machine. Just on the basis of my day to day experience I get by with the 400 fine but the 800 is more responsive for obvious reasons. The XP machine is pretty fast too though and I handle Graphic Design on both with very little difference in performance.

DRM

[chadcole26]

Overall, I agree with you, the machines I use at work do seem much more responsive than my Macs. That is why I really did expect them to mop the floor with this video rendering test.

I'm going to do another test using video I've imported and see if they perform as poorly when doing transitions and special effects. I have a feeling that maybe the PC's just don't handle the redering of Quicktime movies as well as they would their "native" AVI.

I'm not gaining anything from this test other than a real appreciate for the fact that even though it may look like Apple (Motorola) has dropped the ball when it comes to megahertz, we're really not that far behind. Is still think that out of the box, without any tweaking at all, these PC's should have whipped every one of my Mac's. I'm going to try and figure out why Premiere is not utilizing the dual processors on the Compaq machines, as I'm sure that would speed them up tremendously.

[JNG]

Interesting. if nothing else, perhaps this goes to show that there are perfectly valid ways of working with certain common document types and applications for which the Macintosh is, performance-wise, very competitive.

[sleepyrenderer]

I use Windows hardware all day at work for video editing (our district has been strictly Windows based for over 5 years and it doesn't look like that's going to change any time soon) and I use Macintosh at home.

Wow, I guess you better go speak to your boss and let him know if he replaced all those expensive dual Xeon workstations with used iBooks you could render your projects way faster.

I wish they'd included your super-rendering-workstation iBook in this Digital Video Editing bakeoff:
http://www.digitalvideoediting.com/2...w_macvspc2.htm
I bet it would have been at least twice as fast as any of the systems they tested (well, at least it definitely would have been if they let you do the "testing").

[chadcole26]

I never said this was a scientific test. I used the equipment I had avialable to me to test one of the features that I have to use most often in the product described.

As of yet, no one has offered me a solid solution for getting the XP machines to render faster. I have been reading up on this in the Adobe forums and have seen several suggestions I'm going to try when I get to work on Monday. Until then, my ridiculous finding only show that when it comes to importing a Quicktime movie (24fps, Cinepak codec) into Adobe Premiere 6.5, the render time on a stock Intel machine seems to be terrible when compared to "slower" processors.

I've never claimed that my G4's will run circles around a P4 or Xeon in other tests. This test was specific to something that I do on a regular basis.

[yzeater]

Guys, back off. Chad strikes me as a nice guy who truely has nothing to gain. I trust real world comparisons a lot more than the crap we find in mags. Agreed?

[drmcnutt]

Originally posted by yzeater:
Guys, back off. Chad strikes me as a nice guy who truely has nothing to gain. I trust real world comparisons a lot more than the crap we find in mags. Agreed?

Uh, No.


DRM

[drmcnutt]

Originally posted by chadcole26:
I never said this was a scientific test.

That's why your results are suspect. Even if you tweak your PC's to perform closer to the mac you still aren't playing on a level field as far as "testing" is concerned. You should'nt refer to it as testing, use something else.

DRM

[chadcole26]

How are the results suspect? I put three identical pieces of video on each timeline and told the program to render. Are you implying that I don't know how to read a stopwatch?

[sleepyrenderer]

It has to be something with the cinemapak codec on those specific PCs. Convert the source video to some more standard codec (you can use freeware virtualdub or tmpeg to do it) and run your test again. How about raw AVI, DV or mpeg1.

[edit: the reason I'm saying something has to be wrong is my school just got a bunch of Dual Xeon 2.4GHz systems donated from Dell/Intel and they positively scream. Our lab already had Quicksilvers (Dual 800s and DP1GHzs) w/ FCP and Premiere licenses. The PC's with Premiere are much faster. Now I've never tried Premiere using an iBook but I assume it wouldn't be faster than a Dual 1GHz...]

[drmcnutt]

Originally posted by chadcole26:
How are the results suspect? I put three identical pieces of video on each timeline and told the program to render. Are you implying that I don't know how to read a stopwatch?

I'm implying that you admittingly don't know the "techie" stuff of testing so you aren't really testing. Without a real "standard" to adhere to you are simply blowing hot air about nothing. I've described before how I think the "test" wasn't fair to the PC side (substandard manufacturers for one) and how unsubstantiated results can be more harmful than real world test results.

DRM

[Mark Tungston]

leave the dude alone, he's messing around and put it up just to get some feedback and comments. it's cool, his next test will be more comprehensive.

anyway it shows that a lot of things go into the performance and not just the hardware. it also shows that computer knowledge can contribute more to certain platforms in terms of speed while others give you the speed and usability.

[Mac Zealot]

Who cares?

Look, the tests on there, everything POSSIBLE was done to make the mac and PC equal.

Look, here we are comparing the machines OUT OF THE BOX! A typical video editor shouldn't have to go through 3 hours of configuration just to get his PC to go faster.

I believe what he says, as the magazines are sponsored by dell anyway.

[Codename]

First of all, Quicktime is a dog on Windows so anything you did with that can be discounted.

You also had a problem with your PC's utilizing their processors fully?

Easy, redo the tests with the apps set to realtime priority or at least "high" priority in task manager.

Open task manager, right click on the app's primary rendering process(the one you said was only getting 50% utlization) and set thew priority to realtime.

[Rabid Duck]

Originally posted by drmcnutt:


I'm implying that you admittingly don't know the "techie" stuff of testing so you aren't really testing. Without a real "standard" to adhere to you are simply blowing hot air about nothing. I've described before how I think the "test" wasn't fair to the PC side (substandard manufacturers for one) and how unsubstantiated results can be more harmful than real world test results.

DRM

The fact that you attribute a performance difference of that magnitude to "substandard manufacturers" is ridiculous. Putting aside the fact that Dell, the computer manufacturer you suggested testing with, often puts cheaper hardware in their computers than Sony does in order to cut costs, two computers from two different manufacturers with the same cpu, bus speed, ram, etc should not vary in performance by more than 10-20% or so. To imply that a 1.8ghz P4 Dell machine is twice as fast as a 1.8ghz P4 Sony is laughable (this would still put the Dell machine's render speed at 580, 76 seconds slower than the 733mhz G4's.) So don't rag on chadcole26 for not knowing "techie" stuff, as you haven't presented any valid techie arguments yourself.

That being said, I agree that the results are highly suspect. The culprit, as sleepyrenderer pointed out, is most likely the codec. I'm interested in seeing how the machines stack up after the clip on the windows machines is first converted to .avi. Also I'd check to make sure the render settings on all the machines are the same.

[drmcnutt]

Originally posted by Rabid Duck:


The fact that you attribute a performance difference of that magnitude to "substandard manufacturers" is ridiculous. Putting aside the fact that Dell, the computer manufacturer you suggested testing with, often puts cheaper hardware in their computers than Sony does in order to cut costs, two computers from two different manufacturers with the same cpu, bus speed, ram, etc should not vary in performance by more than 10-20% or so. To imply that a 1.8ghz P4 Dell machine is twice as fast as a 1.8ghz P4 Sony is laughable (this would still put the Dell machine's render speed at 580, 76 seconds slower than the 733mhz G4's.) So don't rag on chadcole26 for not knowing "techie" stuff, as you haven't presented any valid techie arguments yourself.

That being said, I agree that the results are highly suspect. The culprit, as sleepyrenderer pointed out, is most likely the codec. I'm interested in seeing how the machines stack up after the clip on the windows machines is first converted to .avi. Also I'd check to make sure the render settings on all the machines are the same.

Rabid,

I appreciate your reply to my post. My post was not filled with technical arguements because my problem wasn't technical in nature it was with the way the tests are portrayed and how harmful the "objective data" is to valid testing between platforms. My reference to manufacterers was based on personal research of PC manufacturers and their reviews in magazines and on the web. When there are tests done comparing cross platform performance I've seen Dell used as a resonable PC counterpart to Apple. This is reasonable to me and perhaps not to you, but there are differences and I know for a fact that Sony is plagued with performance problems while Compaq (http://www.compaqsucks.com/) has its own problems.

I know that every Dell out of the box isn't automatically the best, but neither are every Apple. My point was that compare Apples and oranges Dell is a better fit to Apple than Sony and Compaq. In the end it was an arguement based on personal research and not public opinion. I don't have a problem personally with the original poster I just wish he would call this activity something other than testing (testing implies objectivity). He comes off as an Apple zealot, apologizing for not knowing how to work with the icky windows and getting sucky results.

My Mac at work is optimized by our IT department to work in our office, some optimization is acceptable and necessary in both camps.

DRM

[idyll]

Actually.. There have been tests done with a few different manufacturers (Alienware, Sony, Compaq, etc.) and all of these machines had pretty much the same hardware (Pentium 4 1.8 Ghz all of them, I believe) and for some reason the Sony came out the slowest. Coincidence? Well.. I'm not quite sure..


Pardon my English, haven't been here long enough.

[Rabid Duck]

Originally posted by idyll:
Actually.. There have been tests done with a few different manufacturers (Alienware, Sony, Compaq, etc.) and all of these machines had pretty much the same hardware (Pentium 4 1.8 Ghz all of them, I believe) and for some reason the Sony came out the slowest. Coincidence? Well.. I'm not quite sure..


Pardon my English, haven't been here long enough.

Yes, some manufacturers make slower systems than others. My point was that something else is obviously the main culprit here, because the discrepancies in performance between the mac/pc systems is so large.

[Graymalkin]

There's two major things I can think of that are going to skew your results with these tests.

The first is the codec, with your test you're not showing the speed of the hardware, just the speed of the particular codec's implementation on the system. You're seeing the Cinepak encoder in Quicktime on Mac is beating the pants off the Windows encoder. Try the MPEG4, MJPEG, and Sorenson encoders as well using the same input file.

You're also transcoding the video instead of doing a raw encode. The Quicktime file has already been encoded once and when you export your Premiere movie it is translating the encoding of the original source QT file into the new encoding scheme. The Cinepak transcoding could be simply faster in the Mac implementation than Windows. Instead of using someone else's video import a DV file or some other raw video. Encode that using several of the available codecs on all the systems and post THOSE results. It will be interesting if the Macs continue to trounce the Xeon based machines. I doubt they will when given a raw video source to encode from. If possible use a third party codec with no Intel or Motorola specific enhnacements.

You're run into the prime Mac/PC comparison problems. Benchmark code is often times untuned code used to show the relative speeds of various systems using various architectures. Real world production code however is tuned as much as programatically possible in order to squeeze the most performance out of a particular platform. Look at iTunes' audio encoder, the same encoder performs better on a G4 than it does a G3 because of AltiVec enhancements.

PS. The reason only one processor on the Xeons seemed to be doing the work is because your particular video encoder (Cinepak) is old and not designed to run on multiple processors simultaneously. SMP systems don't magically run programs on both processors, the program has to be designed to do it. Rework the tests and post the results, I'm curious to see how the Macs stand up.

[chadcole26]

OK. I timed some more rendering this morning and got some results that are helpful in understanding some differences between the platforms.

I converted the Treasure Planet trailer (Cinepak, 24fps, 480x288 video) to AVI (no compression, 29.97fps, 720x432 video) and Quicktime (no compression, 29.97fps, 720x432 video). On the timeline I put the Universal Counting leader, the video clip, and 10 seconds of black video.

The Sony VAIO rendered the AVI version in 8:32, much better than with the original trailer. It rendered the Quicktimg version in 10:09, still much faster than the original. The Compaq EVO rendered the AVI version in 6:39. I believe that's almost twice as fast as the original trailer. Both processors were utilized with the primary processor hovering between 50%-60% and the second processor between 40%-50%. Based on the Sony's performance I expected to see a similar, slightly slower time with the Quicktime version. WRONG! The QT version rendered in 4:59. Both processor were utilized and hovered between 55% and 65% usage. The page file usage stayed at around 650, too, compared to only 410 for the AVI version.

Obviously, when you're importing preprepared video into these windows system, codec makes a huge difference. It was faster for me to export the video in QT and then import the new version into Premiere for the render than it was just to render the original trailer in Premiere.

I'm going to run this same version difference test at home tonight, only on the 733 G4, and post the comparisons. I have a feeling that this will really show the speed difference that most of us would have been looking for in the first place.

[Graymalkin]

Thanks for taking the time and interest to redo the tests under different conditions. A lot of people on these boards would have just assumed their original results had been perfect.

The Cinepak codec is as you probably know old and not necessarily the best codec to use for video anymore though because of its age it is extremely common. I don't have Quicktime Pro so I can't save the trailer you're using but I'm guessing the QT5 requirement and size of the file for the quality it is the file is encoded in Sorenson 3. Going from Sorenson 3 to Cinepak might be a billion times faster on the Macs as a matter of course. In that case the Macs are way faster and the amount of memory bandwidth and raw processing power of the Xeons doesn't make a difference one way or the other. I'll bet with the raw video the Macs are going to perform much slower because the rendering of the uncompressed video is a very memory bound application.

[mac freak]

That comparison makes absolutely minimal (if any) sense. As much as I'd love to believe an iBook is faster than a dual 2200 MHz Xeon, I just can't. In fact, the only thing faster about an iBook than a massive Intel-based server is the speed at which it would fly away in a colission with a Hummer.

However, Chad's comparo certainly gets a high score on the Feel Good scale!

[chadcole26]

OK! This is it. I promise this is the last time I'll make a big deal out of this. I finished three more tests (four total) They are posted on the website listed in the original post. I think the the end results are much more realistic. I know they leave me wishing I had a brand new Mac that I could try this out with.

Thank you to the few of you who offered suggestions. This has been a valuable learning experience for me, just from the perspective of having to examine video compression and what the different codecs have to offer.

If you don't feel like scrolling back to the top to see the page just click here!

[sleepyrenderer]

chadcole26,
Thanks a bunch for putting in the effort to update your test. I'm sorry about being such a sarcastic dick originally but your results were so far removed from my personal experience with my school's multimedia lab that I assumed you had just made up the results. Since then I've gone and hunted down a Cinepak encoded video and I encountered the same performance anomaly you uncovered. Your new results match my previous experiences of just how plain fast high-end PCs render DV. So once again I apologize for the sarcasm and thanks for the effort.

Just a small correction for your page. I don't think the Cinepak codec is Sorenson 3. The Sorenson codec only showed up in QuickTime 3.0 and Cinepak actually originated from Quicktime 1.0. It really is that ancient.

[swiz]

Originally posted by drmcnutt:


You really wear your bias on your sleeve

DRM

Oh and you dont do the same with all of your PC touting around here?

[drmcnutt]

Originally posted by swiz:


Oh and you dont do the same with all of your PC touting around here?

Yes you pegged me right, a real zinger!:o Yes the guy that puts the Compaq sucks link up is a PC touter, to the moon Sherlock!

DRM

To those who have trouble understanding my posts like swiz the above was sarcasm.

[chadcole26]

[B]
Just a small correction for your page. I don't think the Cinepak codec is Sorenson 3. The Sorenson codec only showed up in QuickTime 3.0 and Cinepak actually originated from Quicktime 1.0. It really is that ancient.

Thank you for that. My education continues. I opened the movie up to take a peek at the properties and found that it is indeed using the Sorenson Video 3 codec. I probably got stuck on the whole Cinepak thing from the first time I tried to export the video. It probably showed up as the default compressor in QT pro. I've changed the web page to reflect my new knowledge and have added a link to this forum.