This was going to be a response in the "Definitely Slower" thread, but Scott 'suggested' we start a new thread ... so here we are :-).

As promised, I managed to run my first, very rough, very unofficial, very hand-timed OS X vs. OS 9 multi-tasking comparison test. Here's what I did...

Methodology
I decided to use only apps which were available native on both systems for this round. This meant no Photoshop or Office tests, and somewhat limited my scope of testing. However, I wanted to make sure I was comparing like and like on each OS. So I used IE 5.1, iTunes 1.1.1, Fetch 4.0, and the Finder in both OS X and OS 9. The tasks I decided to test included:
[list=1][*] Import a selection (playing at a low level while importing) of songs from Natalie Merchant's Ophelia CD in iTunes 1.1.1[*] Upload a folder containing 63 image files totalling 2.9mb to my server using Fetch 4.0[*] Copy 34 files digital video files totalling 1.7gb from one hard drive to another in the Finder[*] Watch the 'Tyrannosaur' (28.3mb) Jurassic Park III trailer from www.apple.com/quicktime in IE 5.1[/list=a]

The order of the list above represents the order in which I started the tasks. I decided to include the time spent in the Finder starting each task as part of the total time. However, to minimize this time in both OS 9 and OS X, I pre-set the Finder windows and pre-launched all applications. So the amount of time spent on starting each task was trivial - a quick drag and drop in the Finder for the upload and the copy, and a click of a "Play" button in IE.

To simulate "real world" usage, I also switched among all four tasks a total of five times during each test. I didn't linger long in each one; just enough to simulate a quick "visual check" and then I moved to the next task. At the end of the switching, I made sure I wound up back in IE.

Instead of timing from the start of the tests to the absolute completion, I decided that iTunes was my 'background' task, and I would stop the clock when the last of the other three tasks had completed. The Quicktime movie was the foreground app, and when the movie completed, I simply restarted it until the other two tasks finished. I decided not to time until the completion of the iTunes rip, as I was importing the whole CD, and the last part of the test would have been just iTunes running. I wanted to see how things worked when everything was going. I only ran this benchmark once, as I didn't want to spend all evening doing this, and yes, it's hand-timed so the results may be off by some portion of a second. However, I had all the windows arranged so that I could see all activity at the same time, so the error is not likely to be too great ... nor would it have an impact on the results, as you'll soon see.

I tested my network connectivity before and after each test on each OS, using webservices.cnet.com's speed test, and found that there was no statistical difference in my connectivity speeds, so the movie and file upload were comparable on both systems.

Test System
As I said, I only have access to one test system - it's a G4/733 with 512mb of RAM, the stock 40gb hard drive, a second 60gb IDE drive, and its connected to the net via an AT&T cable modem. The systems used were OS 9.1 and OS X 10.0.2.

The OS X box was running Apache, FTP, and SSH daemons at the time of this test because I forgot to turn them off, and I was actually connected from my PC via FTP but not actively doing anything (again, because I forgot to disconnect!). On both machines, file sharing was disabled.

If someone wants to replicate the test on other systems, please feel free - I'd love to see results for a similar task list on other machines to see how they compare.

Results - Objective (minutes:seconds.tenths)
1st place: 05:22.41 - Mac OS X
2nd place: 07:18.40 - Mac OS 9

Mac OS X was roughly 25% faster in this series of tests.

Interpretation of Results - Subjective
This was an interesting test. There are obvious strengths of both systems which were brought out fairly well by my little exercise. At the conclusion of the test on OS X, iTunes had ripped two songs from Ophelia; at the conclusion of the OS 9 test, iTunes had ripped four songs -- I'm not sure how many songs iTunes for X would have ripped had I let it run for the same amount of time. The task that finished last in OS X was watching the movie; the task that finished last on OS 9 was the file upload, and I had seen the movie 2.5 times by then. The file copy on OS X was done nearly one minute faster than the file copy on OS 9.

What doesn't show in the numerical results was the difference in using these two systems while all this was happening. In OS 9, it was sometimes a bit tough (when doing the task switching) to get OS 9's attention; I had to click a couple times on some occasions; OS X was always responsive to the click. On both systems, iTunes performed admirably - nary a skip nor stutter on either system. The QuickTime movie, however, was an eye opener. When placed behind iTunes in OS 9, the movie became unwatchable - great chunks of video were skipped, and the sound cut out regularly. It didn't get better until I moved back to IE in the foreground. On OS X, however, the QuickTime movie in the background was just fine - it skipped a frame here and there (every 10 seconds?) if it wasn't in the foreground, but it was very very watchable.

In short, OS X felt like a much more usable system during this test. OS 9's version of iTunes appeared to work better in the background, probably because it was getting more CPU time than the OS X version was. But as a result, the file upload and Finder copy really suffered, which caused the poor score for the OS 9 system. OS X felt like it wasn't really even working hard yet, and was willing and able to do more. This is hard to express in words, but OS X felt like a very usable system in the middle of this test, while OS 9 felt like it was very very busy doing what I'd asked it to do, and I should just leave it alone for a while.

Please don't interpret this as a me saying "OS 9 sucks; OS X rules!" They are clearly different systems designed around different cores, and as a result, they each do certain things better than the other. However, I don't think there's any arguing (I'm sure someone will!) with the straight numerical results -- OS X was over 25% quicker at this particular multi-tasking benchmark than was OS 9. To me, that's a great indicator of the strength of the core OS X system.

-rob.

------------------
Visit macosxhints.com ... a community-built OS X hints and tips site.

I appreciate you writing this long and well written out post and your results are probably fairly accurate. However, try that on a G3 450 mhz and 128 megs of RAM and you will defintely see an entirely different story since you are however, on the best OS X machine available.

edit: did you mention that IE skips with any quicktime movie like crazy while IE in Classic runs like a charm (at least on my machine)

[This message has been edited by mindwaves (edited 05-09-2001).]

>>did you mention that IE skips with any quicktime movie like crazy while IE in Classic runs like a charm (at least on my machine)<<

No, in fact just the opposite - I mentioned that IE 5 on OS 9 skips like crazy in the background; IE 5.1 on OS X is nearly perfect in the background.

>>However, try that on a G3 450 mhz and 128 megs of RAM<<

The point of the tests I chose is that anyone can do them! Change out the QuickTime tests if you don't have a fast connection (run a local movie on both machines in Player in loop mode); find a set of files to copy around; etc. That was part of the point - take some initiative and do the tests yourself, instead of just saying "it'd be slow." If we can't test more than one machine, we'll never know. And I don't have another machine to test, so I can't volunteer. Plus I think the 30+ minutes I spent on this tonight was enough of a contribution on my part.

If you want to prove me wrong on other hardware, please do so -- we need more data points, and less "but it's so slow!" comments without backup. I'm not claiming a position one way or the other >in general<. On my machine, there's no doubt about it - X does many things at once better than 9 ever did. If that's not so on other hardware, let's prove it and send the results to Apple!

-rob.

Thank you very much for a extremely though out post. I appreciated it very much. This is one of the things that people are just not considering what happens when they use more then one app, which I bet 99.99999% of the people in this forum do.

I will try to simulate this test on my Dual G4 450 later tonight and then post the results, that way we can see what kinds of results we get with a middling, albeit dual, system.

------------------
"Imagine the disincentive to software development if after months of work another company could come along and copy your work and market it under its own name...without legal restraints to such copying, companies like Apple could not afford to advance the state of the art."
- Bill Gates, 1983 (New York Times, 25 Sep 1983, p. F2)

I agree with mindwaves... try this test on a B&W G3 and OS X will loose, perhaps even by a LARGE amount of time. Perhaps someone can perform a similar test on one (but also include boot time and app launch time).

While I'm using X, the overall FEEL of the system is SLOW, excluding the dock which is lightning fast 100% of the time. Try opening iTunes and just browsing around in your playlist for a while... it's sluggish when compared to OS 9.1. And don't even get me started on IE5.1 and OmniWeb... they're practically unuseable (yes, even OmniWeb 4.0 final).

Another issue that I've only heard a few other people talk about is QT5. It's just plain slow. It only manages to play movies at HALF the frame rate they're supposed to be at. And don't even attempt to play more than one at a time...

I do have 256MB of RAM, too. I'm sure the overall speed will improve with each update (as it seems to be doing, very slowly), but I don't think OS X will ever be as fast as 9.1 on anything less than a G4.

Oh, and I'll try and perform this test myself as soon as I can find the time. I'll have to modify a few things, but I'll be sure to make it a fair comparison.

------------------
. : [ Apfhex ] : .

[This message has been edited by Apfhex (edited 05-09-2001).]

I really don't feel like rebooting in OS 9 right now (it is fairly late); however, I can give you a rough estimate of things on my machine.

1) IE 5 for OS (9 (classic) starts up at least 2.5 times faster than IE 5.1 for OS X
2) iCab for OS 9 starts up at least 50% faster compared to OS X
3) any one of my preferences for OS 9 (except the extension manager) is wayyy faster in OS 9 than in OS X
4) And overall Finder responsiveness is a lot better in OS 9
5) Just watched the Jurassic trailer also and it is a lot better in IE 5 for OS 9. The movie plays with the sound like it should be unlike in OS X


I am not saying that I hate OS X. I am saying that on my machine which has the minimum amount of RAM with 450 mhz which is way far and beyond the 233 mhz required should be able to be just as fast (or a tad slower ) when compared to OS 9 which it is not. OS X will take years to mature just like how all the other OS's before that.

edit: I would also like to add why does OS X start up so slow? I mean that it is embarassing to watch it start up because it is so slow. I remember when I had the beta version of it 1H14 I believe and my computer started up in 1 min 20 seconds while OS 9 started up in 1 min 30 seconds. I was rather impresses that day and count wait to see what the final would look like. Today however, with the final version, OS X starts up in 2 mins 20 seconds while OS 9 starts up in about 1 min 20 seconds (I parred some extensions down). Why the slowness?

edit 2: I boot into OS 9 a lot to do some other things and that is why I primarily use app launching times as my benchmarks.

edit 3: There is absoultely no way that the dinosaurs portrayed in that movie could have been smarter than the dolphins, dogs, and whatever else that they said in that trailer. 65 million years of evolution have made us (all life forms) smarter and according to the trailer, if I am correct, there is no way that the dinosaurs culd have figured out to lock people up in a giant bird cage. Not possible at all. I, for one, will not see that movie. Now back to OS X!

[This message has been edited by mindwaves (edited 05-09-2001).]

great post, griffman! I'll try to run similar test this weekend on my Pismo, if I can still find this thread.

Now, I think a few of you are in the wrong thread. If your argument requires you to use the terms 'launch time' and 'startup,' you're who I'm talking to. This thread discusses multitasking ability. We all know that apps in X run slower than 9. I attribute this to the fact that in 9 they can hog the CPU. In X you get better management of system resources, which necessarily comes at the expense of raw app performance. By this logic, we should see improvements in raw multitasking performance. This thread exists to see if that is true.

I should stop right here, but I can't resist. Of course X will never be as fast as 9 running on the same hardware. The same is true for any major upgrade of any operating system. You get better features, and you will need a better machine to run them. I believe the hardware is almost fast enough to support the advances of OS X. Whether you agree with me will dictate whether or not you should buy the new OS.

Here are my results from ripping "Tool, Enema", copying 1.65gigs of files form one of my partitions to the other, and watching Jurassic 3 in IE. I have a G4 Dual 450, 448 RAM, stock 40gig hard drive, rage 128...

I started my timer when all processes were running and I was going to stop when the rip was done but I didn't realize that the rip was going to my system partition so it got stopped at track 7, so when I got into X, I stop the timer there. First was iTunes, then the copy, finally IE. I gave each application one minute before I moved onto the next one.

iTunes: I encoded at 160kbps. It probably had the entire second CPU to it's self in 9, and when it was in the foreground it shot up to 8x but when other apps were on top, it plummeted to 2x. In X it stayed at a constent 4.1x the whole time. itunes took advantage of both altivec and my second processor.

IE: Even when it was the top app in 9 the movie was skipping out. But under X it never missed a beat, no matter what else was going on.

Copy: well I just copied a 1.65 gig folder from one of my partitions to the other.

In both 9 & X, I had to quickly change apps when the movie ended to restart it. Also, I timed with my stop watch, nothing special so the numbers aren't' super accurate. Also, half way through X, I noticed that I forgot to close stuffit, at which point I did. But it was idle so I doubt that it had any impact on the tests.

MM:SS:Miliseconds
OS 9: 8:15:21
the copy ended at exactly 6:00:00, and iTunes ran out of disk space at 8:15:21, or track seven.

OS X: 8:10:00
the copy ended at 6:29:14 (this one surprised me)
I noticed that iTunes was in the middle of ripping track seven at 8:10:00 so the real time is probably a bit shorter.

I was truly surprised at these results, I didn't figure that 9 would fare so well. Probably because when I did bring iTunes to the front it had both CPU's to use that it was able to make up for lost ground. Basically the same for all tasks, they were slow but when I brought them to the front for their minute they made up for it. The movie was skipping the whole time, I didn't even watch it until I did the test in X it was so bad. Even when it was in the front. 9, was very unresponsive, it took a second or two to switch between apps, and over all felt very busy.

X on the other hand, was taking this in strides. the system was very responsive and I wish that I had CPU monitor up so I could see what the system load looked like. The movie played flawlessly no matter what I was doing, and all of the apps seemed to keep the same pace no matter what app was in front.

------------------
"Imagine the disincentive to software development if after months of work another company could come along and copy your work and market it under its own name...without legal restraints to such copying, companies like Apple could not afford to advance the state of the art."
- Bill Gates, 1983 (New York Times, 25 Sep 1983, p. F2)

>>This thread discusses multitasking ability.<<

Thank you, Lucy! The purpose of the test was not to compare anything about application launching, GUI responsiveness, or 'feel'. It was to test how well X did things at the same time, compared to 9.

I'm viewing X as a complete OS, so using app launching time becuase one 'reboots into 9 a lot' isn't really a point of comparison. If you only used X, you would not be launching apps all that often.

Yes, I realize this is an incomplete benchmark. Yes, there are other elements of system responsiveness that need to be fixed. But it wasn't designed to look at those areas - the question that was asked dealt strictly with multi-tasking in X vs. 9, so that's what I set out to test.

-rob.

Great report Griffman!
Someone mentioned something about how X runs on a g3 450 mhz. And I have to say that I am so far impressed with how it flows on my imac 450 w/192 ram. Sure it needs some secs to start up a app or two, but in overal when working on it, it rocks. The stability and its multitasking makes it (in my eyes on my IMac) a much more modern OS than OS 9. Even on my machine. =) I can't really complain on the speed either. When a app is up, it runs very well!
But it still needs some develop time before it reach a "complete" state.

------------------
---------
sniffer

!taerg si X !yeH

Ok, so I sort of missed the point of that. However, multitasking wise, OS X is fairly nice and better than OS 9 in that aspects. However, sometimes it slows down my system to a crawl and still in OS 9, I can swicth between running apps a lot faster than in OS X. I am sure that OS X has far better multitasking than OS 9, but it just takes too much processing power to do so.

Is anyone with a G3 willing or able to run a similar set of tests themselves? It's not that tough, and it might take a total of 30 minutes including the three restarts required.

It'd be nice to add some empirical data to the mix, and these are NOT tough benchmarks to run...

-rob.

I have got to say with multitasking OS X blows 9 away.. its much more responsive than before. X might look slow but under the hood it is very fast.

Also the faster machines and the MP are the machines we should be looking at, as they were designed for os x where as some of the older ones can just run it.

If the new machines are faster in X than 9 this means X is not slow on current equipment. After all machines will get faster not slower I must admit this is no comfort for my 266 iMac but it just a fact of life.

Thanks to everyone to doing a real test....

------------------
450DP/448MB/80GB/Rage/Zip250/ZipCD/Epson740

I Took The Red Pill

[This message has been edited by edddeduck (edited 05-09-2001).]

No real surprises here - file copies are faster & multi-tasking in 9 is pathetic, we kinda knew that.
But it's the raw speed and wind in the face that makes OS 9 feel faster, despite some interesting results with file transfers, OS 9 would have killed OS X if multi-tasking wasn't part of the agreement.

Aahh, multi-tasking - this has become quite a hit with the whole Mac community lately, I know I'm not giving it up anytime soon and in time, OS X will be a fine, well-polished OS in the very, very near future.

Great job and nice reporting - nice to see people go Gonzo just to show facts and establish standardized criteria for our beloved OS(s)

Cheers

Originally posted by osiris:
No real surprises here - file copies are faster & multi-tasking in 9 is pathetic, we kinda knew that.
But it's the raw speed and wind in the face that makes OS 9 feel faster, despite some interesting results with file transfers, OS 9 would have killed OS X if multi-tasking wasn't part of the agreement.

But that is just it, why do we test only one app at a time? Almost nobody had only one app running at a time, even my mom! She would have Netscape and iTunes and word up. I have alot more then just that, and I be most of the people in these forums who push their computers the hardest do even more. The thing is we should be testing in the real world, not a lab. Do a bunch of common tasks and see how they turn out. Hell, even in my own tests 9 come out just a little bit slower and the file transfur was done FASTER!

Some of the problems I realized about my tests, first who has iTunes in the front when they rip CD's? I don't, but for this test I did, and I even kept it there for a whole minute, same with the file transfur. In my mind a much better test would be to, burn the CD, (making sure it is going to my big hard drive and I don't get an error half way through) move the files and then browse the boards and wait until ALL of the tasks had completed in the background. This is how I do work, I am not going to waist my time watching the tracks rip of files move, I feel like I am in Winblows watching the little sheets of paper move...

I hope someone with a G3 can also run a test that is similer to mine and Griffman's that would give us a good idea of across the board benchmarks.
Low>>>>>Middle>>>>>& gt;High
G3>>>>><DualG4>>>>&g t;>733G4

------------------
"Imagine the disincentive to software development if after months of work another company could come along and copy your work and market it under its own name...without legal restraints to such copying, companies like Apple could not afford to advance the state of the art."
- Bill Gates, 1983 (New York Times, 25 Sep 1983, p. F2)

Just for grins, tonight I ran the same test in "single-tasking" mode. I made sure each task was the only thing running (other than DragThing on both X and 9, and Apache/SSH/FTP on X), and then timed each task from start to finish. I had been told (in email) that OS 9 would "kick X's butt" if the tasks were single tasking.

The problem is that my test suite was really designed as a multi-tasking trial. For instance, watching a web-based movie in IE will take as long as the movie is, assuming the net connection is fast enough to download before the movie ends. Similarly, uploading files to the web server is now strictly dependent on connection speed, which will vary slightly over the trial.

With that disclaimer, what I found was that OS X was basically just as fast as OS 9 with this set of single-tasking skills ... as the results show.

Copy 1.7gb from drive to drive:
OS X = 1:56.7
OS 9 = 2:00.3

Upload 2.9mb of data files to the web server
OS X = 3:32.7
OS 9 = 3:27.3

Watch Jurassic Park 28mb trailer
OS X and OS 9 = 2:24 (that's how long the trailer is; both easily downloaded the data file in about a minute]

Encode five iTunes tracks
OS X = 3:41.7
OS 9 = 3:42.3

The total test suite took about 11.5 minutes on either machine.

I can find no evidence, when either single- or multi-tasking, that OS X is a slow operating system. It's definitely got a slow Finder in some respects, but the core OS appears quite solid. The big advantage in OS X comes in with multi-tasking. If you assume my test results are combinable, then the OS X 11.5 minute single-task time drops about 50% to 5.5 minutes when all tasks are done at once; OS 9's 11.5 minute time only drops 36% to 7.25 minutes.

Just more data points for everyone to discuss ;-).

-rob.

It is odd, though, that OS X has so many problems divvying up IO access between different programs. iTunes skips in OS X far more often than it does in 9; various applications seem to be able to hog all disk access until I manually stop them; and so on. I've stopped using iTunes because when I have an MP3 playing, the rate at which my typing shows up on the screen slows considerably.

Maybe with an updated kernel we'll see how OS X can really do multitasking. Right now, I'd say it's worse than OS 9 in many of the things I do (I don't download lots of movie trailers off the internet or encode MP3s with iTunes so your benchmarks don't mean much to me).

Originally posted by tie:

(I don't download lots of movie trailers off the internet or encode MP3s with iTunes so your benchmarks don't mean much to me).

Well what do you do? And we can test that.

------------------
"Imagine the disincentive to software development if after months of work another company could come along and copy your work and market it under its own name...without legal restraints to such copying, companies like Apple could not afford to advance the state of the art."
- Bill Gates, 1983 (New York Times, 25 Sep 1983, p. F2)

>> iTunes skips in OS X far more often than it does in 9; various applications seem to be able to hog all disk access until I manually stop them; and so on<<

What hardware do you have? One of the things we're trying to do here is establish the difference between G4's and G3's, but nobody who seems to be complaining about the speed seems willing or able to run a test of their own. It's not that time consuming to run your own benchmark, and it answers a lot of questions.

I can also tell you that, on the G4/733, iTunes hardly ever skips (certainly no more than it does in OS 9), my typing speed is completely unaffected by any background tasks -- I'm ripping, listening, and downloading (my typical multi-tasking list) right now, and I certainly don't notice any difference while typing this response, nor is iTunes skipping.

So, please, someone with a G3, can you invest a bit of time for the benefit of the group? We're talking about 30 minutes of your time to help answer some questions with empirical data instead of subjective claims. I can sit here and run tests forever, but they'll continue to show the same results - on my G4/733, there's no speed penalty in actual operations for using OS X. I keep waiting for someone to chip in with some G3 figures that show a dramatic slowdown -- I'd actually like to see it, because we then have some ammo to give Apple to focus on G3 performance. Until someone is willing to do a bit of work, though, it's all just ranting.

>>worse than OS 9 in many of the things I do<<

Identify those things, design a simple test, and test it in both OS's. That's what I did - ripping, listening, uploading, and downloading represent a fair assessment of how I use my machine, so that's what I tested. If you don't agree, test something else. But please test SOMETHING, instead of saying "I'd say it's worse than OS 9...". I'm not disagreeing with your position at all, but help support it - show us how bad it is. Publish the test, and I'll duplicate it on the G4/733 to see if it's still true on a high-end machine.

I like discussions as much as the next person, but I don't understand the reluctance of the group to back their claims with results. I claimed OS X was fast on my machine, and have now run two separate benchmarks that support that conclusion. Is nobody on the "it's slow" side of the fence willing to do the same? Why not?

-rob.

[This message has been edited by griffman (edited 05-10-2001).]

Griffman, Great thread! Very resourceful!

None of the readers who have been claiming OS X is slow have responded with any tests from their machines. Doing so might help validate their claims.

[edit: I have a dual G4 now but next week I'm getting a G3/500 so I'll try it out.]
-john

[This message has been edited by SillyMonk (edited 05-10-2001).]

What's slow in X?

Printing to a laser printer w/ 18MB from OmniWeb 4.0

Clicking and moving things around in the Finder.

Navigating long folders.

Recovering from a crash.

Booting back and forth.

Having to move from mouse to keyboard too often.

Downloads. Whether IE or a Classic browser.

PID 0 Idle_thread using 35%

Boot times on B&W G3/450 w/ Ultra2SCSI is fast though. Installs are fast. And of course I can boot into OS X for doing Finder copies.

In 9.1 I can launch programs as fast as "Snap!" or the time to hide and show an application in OS X.

OS X really likes having 512MB or more even if just web browsing, doing email and printing while using TextEdit. Nothing else. 640MB.

My boot volume in 9.1 is on mirror raid - meaning it "self-repairs" (which is slow) but insures data integrity. Therefore saving my work, my system, my files.

We use to upgrade to deal with bloated code on mainframes, well Aqua is the new COBOL. sortof.

I'm really enjoying this thread rob! I'm going to conduct a few tests of my own when I get home....

... Perhaps someone would be willing to try an iMovie benchmark? I am VERY intersted in seeing how mach's real time mode stacks up against OS9s (un)co-opertive multitasking...

Originally posted by Gregory:
What's slow in X?

Printing to a laser printer w/ 18MB from OmniWeb 4.0%

This is a good point. My printer still isn't compatible with X so I don't know but maybe someone could try some laser printer benchmarks?

Clicking and moving things around in the Finder.

Let's leave the Finder out of this thread please. These tests should be a reflection of the core os's abilities.

Recovering from a crash.

Booting back and forth.

Having to move from mouse to keyboard too often.

This is really a matter of opinion first of all. Secondly it's totally unsupported and isn't a reflection of OSXs core abilites. I mean is the time it takes to boot back and forth really OSX fault? I find OSX boots faster than 9 too. Actually this would be a really good test: Startup Times!

Downloads. Whether IE or a Classic browser.

Try downloading something in OmniWeb. Then try downloading something in IE in 9 while it's in the background. Welcome to pre-emptive multitasking 101.

PID 0 Idle_thread using 35%

[/B][/QUOTE]
I belive (correct me if I'm wrong here) Idle_thread is just an indication of free processer power. If you have 35% of your processor free all the time that's a pretty good thing.

Good thread!

>>Downloads. Whether IE or a Classic browser. <<

Can you please test this? On both my old G4/350 and my G4/733, downloads have been notably faster on the X box than the 9 box -- a 20mb data file downloads roughly 10% - 15% faster; I don't have my data sheets at the office, but I ran multiple tests multiple times, and this was always true -- with the download running as the primary task on either platform.

And foobars is right - this wasn't a Finder test. We all know the Finder is slower if you try to use it like an OS 9 finder. The way I use it, though, I don't notice any speed differences. I haven't resized a Finder window since Final came out, barring a couple of disk image installer windows. I can move files faster in the OS X Finder than I can in the OS 9 Finder. Maybe I was inefficient in OS 9, but working with two column-view windows in OS X makes moves and copies incredibly fast and easy.

The following example assumes you have already have an open window with a bunch of files you want to move...

OS 9 way: Highlight selection of files to be moved, drag them onto folder, hit spacebar (or wait) for pop-open folder, drag to next folder in hierarchy, hit space bar (or wait), drag to next folder, repeat until you reach the destination, release mouse button. Don't let mouse button slip early or you've moved everything somewhere bad.

OS X way: In window #2, use the same number of mouse-clicks as space-bars in the OS 9 example to select the destination folder, highlight files in window #1, drag to destination in window #2, release mouse button. In my testing (and yes, I timed this, too), I can move files about 20% faster in OS X than in OS 9.

I'm not claiming the OS X Finder is perfect - it's not. Window resizing sucks, the alphabetical selection bug stinks, lack of labels is lousy, etc. But for what I use it for, it actually works quite well.

-rob.

Originally posted by griffman:
>> iTunes skips in OS X far more often than it does in 9; various applications seem to be able to hog all disk access until I manually stop them; and so on<<

What hardware do you have?

G4 450 with 256 MB RAM. It's hard to benchmark IO responsiveness to typing, but I'll see what I can come up with.

tie:

You might find it useful to open a terminal window with "top -u" running, or just launch ProcessViewer in the GUI. Take a look at the CPU utlization when typing is "fine", and compare it to CPU utilization when typing is "not fine". What's going on in terms of the apps taking the biggest chunk of CPU, and (at least in top) is there activity in the "page in" and "page out" areas, suggest swapping of pages in and out of RAM? Since you aren't running movies or ripping iTunes, what else are you doing while typing? Those tasks should show in top or ProcessViewer...

Finally, do you have File Sharing enabled? I've found that it can occasionally run amuck, and suck up 40% of the CPU just sitting there in the background...

cheers;

-rob.

Ok, my test results. Note that I did not do the IE 5.1 OS X Jurassic Park thing simply because IE 5.1 will launch but will not oad anything. Never happened before.

Here is what I did first:
Copy a folder containing 5,843 items for a total of 170.5 megabytes from one partition to another while playing 2 quicktime movies in the background both in 9 and X. I played 2 roads (792 kb) and the 1984 commerical (13.1 megs) looping of course during the file transfers.

The results:
OS X 4:57 mins/secs
OS 9 2:34 mins/secs

Quicktime in OS 9 did skip a couple of times during the test but it just blew OS X away in copying thousands of files from one partition to another. Quicktime in OS X did not skip at all however.

Launch times:

iCab: 2 (OS 9) | 4 (OS X)
IE 5 : 5 (OS 9) | 11 (OS X)

System: iMac DV+ 450 mhz G3 with 128 megs of RAM

I would also like to add that I dont use iTunes as my example because I just don't burn any music CDs. I just burn data as in files.

My quick test.

Ran Jurassic Park III in IE browser while copying 556 files totalling 1.4 Gig from one disk to another.

On a dual G4/500 with 512 RAM

OS 9 : 4 min 05 secs
OS 10 : 1 min 55 secs (files finished before movie finished)


------------------


[This message has been edited by SillyMonk (edited 05-10-2001).]

In a blatant attempt to keep this thread near the top, I will make some remarks:

mindwaves did not run the same, or even an equivalent test. iTunes was not being used to burn CDs, it was being used to rip them, so his disclaimer is out of place. I also think that's the key element to OS X winning the test: something processor intensive needs to be happening in the background, or you're not proving anything.

I will run the test on my 466MHz iBook w/ 320MB RAM as soon as I get a chance. I'm thinking of making it a little more dramatic though... something like adding stuff to the OS X side and demonstrating how it's still faster. I think when I run it I'll start a compile of XFree86 in the background

I'd wager money OS X will still come out ahead, or at least equal. I'd also like to invite anyone who doesn't believe me to come to my house and have it demonstrated. I live near Richmond, VA, USA. Maybe we can just meet at the opening of the Apple Store

mindwaves - thanks for being the first to run ANY sort of test on a G3; it's appreciated!

i'm wondering if it's a RAM issue -- can you try with the terminal open to "top -u" and see if you're getting pagein/pageout activity during the test? or is there some other task eating up cycles? those results seem way out of kilter.

perhaps it has do to with the number of files copied; i copied a relatively small number of larger files; you copied a large number of smaller files. i'll try tonight with a folder of smaller files and see how it goes.

-rob.

Hey,
All I said is that I didnt run ANY iTunes test so I am well aware that my test is different but the copying files from one partition to another is similar.

Here the the results with me running no applications besides the terminal of course and the system.

Processes: 35 total, 2 running, 33 sleeping... 76 threads 11:34:10
Load Avg: 0.03, 0.21, 0.28 CPU usage: 4.5% user, 8.2% sys, 87.3% idle
SharedLibs: num = 94, resident = 16.0M code, 1.10M data, 4.27M LinkEdit
MemRegions: num = 1650, resident = 13.7M + 1.95M private, 12.6M shared
PhysMem: 19.5M wired, 36.9M active, 36.5M inactive, 92.9M used, 35.1M free
VM: 393M + 45.1M 21950(0) pageins, 3174(0) pageouts

Whoa... really running nothing there with 87% idle and practically no load. Is that during the file copy?! I think that was the idea. And please do not get the idea that I'm flaming you or anything when I pointed out the lack of iTunes in your test. I was just explaining the discrepancy to the lurking population (which is always many times larger than the posting population). Ripping MP3s is far more processor intensive than copying files, and a far better test of multitasking.

Speaking of file copying, though, it would be very interesting to compare the results of using "/Developer/Tools/CpMac -r <srcdir> <dstdir>" vs. using the Finder to copy a directory...

mindwaves - ideally, three snapshots of top would be ideal:

one before, one during, one after.

include the top three or four apps listed, which will show the % of CPU they're using (make sure you do top -u to get those to the top of the list).

that would show us what it looked like during the copy, which is where we'd like to see what's going on...

-rob.

pardon my english, i'm french.

so, i got an iMac G3 400, 386 mo ram, ibm 20 go 7200 t/mn HD.
macOS 9.1 vs macOS 10.0.3

4 stuff to do :

> copying a file of 1.5 GO on a different folder
> watching Depeche Mode Enjoy the silence video on quicktime player, in loop.
> riping the two first songs of the first tommy hools LP, in iTunes. without listening to them, 160 khz.
> watching to Artificial Intelligence trailer in IE 5, Apple Website. DSL connexion.

and the winner is.. OS 9.1

OS 9.1 finish in 5'01
OS 10.0.3 finish in 8'26

big difference isn't it?

on a G3 OS 9.1 flies, while os 10.0.3 isn't flying (!)

there's is one thing i can notice. in OS 10, i was able to listening to the DM movie without shutoff. in OS 9, not, the sound was quite ok, but the images were lagging. multitasks rules there. but i prefer working on OS 9.1, right now.

but OS 10.0.3 is slow on a G3, today. that's a fact.

------------------
Florence mon amour

bateman - nice test, and the English was fine (better than some people I've met who didn't have the non-English-speaker excuse!). 9 is definitely the quicker system on your machine. it appears that the gulf between the G4's and the G3's is wider than i would have guessed it to be. any more data points?

-rob.

EDIT: arrrgh, a really long post trashed by UBB.

[This message has been edited by ratfink (edited 05-10-2001).]

Before, during after. Keep in mind that it flucates 10% or so during the test so...

Processes: 35 total, 2 running, 33 sleeping... 76 threads 13:18:42
Load Avg: 0.12, 0.20, 0.20 CPU usage: 2.8% user, 8.3% sys, 89.0% idle
SharedLibs: num = 94, resident = 12.6M code, 908K data, 3.24M LinkEdit
MemRegions: num = 1676, resident = 13.7M + 1.86M private, 10.9M shared
PhysMem: 19.4M wired, 32.4M active, 29.2M inactive, 81.0M used, 47.0M free
VM: 396M + 45.1M 23781(0) pageins, 3445(0) pageouts
------------------------------------------------------------------------------------

Processes: 36 total, 3 running, 1 stuck, 32 sleeping... 83 threads 13:22:44
Load Avg: 4.69, 4.48, 4.16 CPU usage: 67.2% user, 32.8% sys, 0.0% idle
SharedLibs: num = 94, resident = 14.8M code, 1.04M data, 3.62M LinkEdit
MemRegions: num = 1878, resident = 26.5M + 2.33M private, 16.7M shared
PhysMem: 19.6M wired, 32.7M active, 37.8M inactive, 90.1M used, 37.9M free
VM: 456M + 45.1M 25273(0) pageins, 3522(0) pageouts

PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
57 Window Man 38.8% 4:52.60 2 130 175 4.06M 4.70M 8.44M 26.5M
505 QuickTime 29.3% 0:54.73 6 118 169 9.31M- 7.20M 10.3M- 56.3M-
310 Finder 18.4% 1:10.98 4 110 251 7.46M+ 6.61M 9.77M+ 58.1M
503 top 5.9% 0:19.69 1 19 14 176K 220K 396K 1.31M
496 Terminal 4.4% 0:05.74 4 79 68 1.97M+ 4.58M 4.61M+ 39.7M+
55 ATSServer 1.4% 0:07.38 1 35 59 448K 1.83M 1.34M 9.29M

------------------------------------------------------------------------------------

Processes: 35 total, 2 running, 33 sleeping... 76 threads 13:24:21
Load Avg: 0.04, 0.22, 0.27 CPU usage: 4.5% user, 10.0% sys, 85.5% idle
SharedLibs: num = 94, resident = 14.9M code, 1.04M data, 3.62M LinkEdit
MemRegions: num = 1697, resident = 15.9M + 1.89M private, 13.2M shared
PhysMem: 19.4M wired, 31.9M active, 28.2M inactive, 79.5M used, 48.5M free
VM: 396M + 45.1M 25435(0) pageins, 3522(0) pageouts

PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
503 top 8.2% 0:27.31 1 19 14 176K 220K 396K 1.31M
496 Terminal 2.7% 0:07.65 4 79 68 1.98M 4.72M 4.62M 39.7M
57 Window Man 1.8% 5:02.71 2 115 166 4.08M 3.81M 7.64M 25.2M
0 kernel_tas 1.8% 2:21.63 21 0 - - - 13.7M 118M
310 Finder 0.0% 1:13.83 3 110 247 5.62M 6.59M 7.91M 55.4M


[This message has been edited by mindwaves (edited 05-10-2001).]

OS X is great with multi-tasking. Every time I boot my machine, I launch 7 programs (Clock, TextEdit, OmniWeb, AIM, Terminal, Mail, and iTunes). Now that's what I call multi-tasking... and I'm only on a 500MHz G3 iMac! I love OS X!

------------------
The mInD has posted,
The mInD that rule oVeR all,
The mInD oVeR everything,
The oVeRmInD

oVeRmInD911,
I don't mean to be rude, but that sounds like a very light load. The rough Mac OS 8/9 equivalent would be running the menubar clock (or Date and Time Preferences if you'd like), Simpletext, Navigator or Explorer, AIM, Eudora or Outlook Express, iTunes, and some other lightweight program (in place of Terminal).

I very often have that much or far more running all at once quite comfortably on Mac OS 8.6 on a beige G3/266MHZ.

While it seems that Mac OS X really does have great multitasking abilities, your seven programs probably aren't even making it sweat.

I don't have a stopwatch, but I am going to see if I can get one tommorow to do my version of your test.

I will do:

1. Rip latest World Party album 'Here Comes the Future' in background, while playing.
2. Upload a folder of 1.7MB of images to iDisk on a 56k modem.
3. Copy 63MB of RM movies from one partition to the next
4. Watch the Duality 80MB Star Wars movie.

I suspect number 4 will bring it all to tears on OS X and number two will bring sorrow on OS 9. I will see.


My machine: PowerBook G3 Lombard, 256MB RAM, 12GB HD, 4 partitions. X and 9 on separate partitions.

2. Upload a folder of 1.7MB of images to iDisk on a 56k modem.

Prepare for some long waits then!!

iMac DV SE 400, 13 GB HD, 384 RAM

My test: ripping RATM
Duplicating 374 MB folder (some large files some small)
Send myself large movie file as attachment Mail in X 10/ OE in 9
Watching large Fellowship of the Ring Teaser in QT 5 Pro

OS X 10.0.3

System is totally fscked. Mail has a seizure and becomes vegetative. FOTR trailer totally looses all quality (maybe 5 frames per second in foreground, 1 or 2 in bg--sound skips a beat every now and then). iTunes rips at 1.1 times in fg or bg; I get through first three songs. Duplicating file is actually pretty quick--finishes first.

This is so awful that I shut everything down and stop the test after about 5 minutes.

OS 9.1

Things actually work pretty well. In foreground, trailer plays almost perfectly, and better than in X in bg; but sound cuts off completely in bg (and frames drop to 1 every second or 2). iTunes rips at 1.2 in bg, 1.7 in fg. Duplicating the folder is noticeably slower though, even in fg; X wins on that one. OE hangs in there pretty well, and I'm really disappointed that 9 is obviously kicking X's ass on this G3...

That is, until OE has an aneurism and crashes, taking my whole system down!

Winner by default: X!!!!!!!!!!!!!!!!!!

I had to laugh. Kind of perfect, doncha think?

------------------
Hoom hom hmmmmmmmmmmm...

[gorgonzola]

Originally posted by Group51:
4. Watch the Duality 80MB Star Wars movie.

OT, but Duality rocks!

<edit>

*smack* -- forgot to put in the relevant part of the post

One really interesting test that we could do a bit later on (because X doesn't support it yet ) is to see how priority levels affect these tests. The problem is that the nice/renice commands aren't interpreted by the Mach scheduler yet, but Apple is apparently working busily away on this and kernel build 128 (something like that) has some fixes in the scheduler area.

This way, OS X would give you tremendous control over which app gets CPU time. The OS would no longer have to assume that the frontmost app is the one that you really want priority given to. Also, pushing up the Finder's priority setting would likely lead to some speed improvements.

Just something to look forward to.

------------------
the oddball newsletter
------------------
it's only after you lose everything that you're free to do anything

[This message has been edited by gorgonzola (edited 05-11-2001).]

Ok, I did it. Here's the set up:

Powerbook (Firewire) 500 mhz G3, 384 megs RAM, 10.0.3 and 9.1 (VM off in 9.1):

The test:
1) Rip Built to Spill "Keep it Like a Secret"
2) Play Charlie's Angels (large) and Emperor's New Groove (large) from the public beta (it was a pain in the ass to extract, but thanks to griffman's great site, it was doable)
3) Copy my bloated system folder from one partition to the other (659 megs).

Running on OS X at the time:
BBEdit
terminal
Mail (checking every 5 minutes)
Internet Connect (connected)
Omniweb (3 browser windows open)
iTunes: playing moby/radiohead/tori amos mix, ripping aforementioned CD
Classic (running, but no apps, prolly asleep a this point)
Process viewer
QT: playing two movies
Apache, mysql, ssh, applefileserver (should've turned this off).

Running in OS 9.1
iTunes: just ripping, not playing because I'm having sound difficulties in 9.1 (I have no idea why)
remote access (connected)
QT: playing both movies

The results: OS X 9:29, OS 9.1 9:01. It looks like OS 9.1 was a bit faster, but the QT movies were unwatchable because I let the copy be the foreground app for much of the test. When it was in the background, it slows significantly. A truer test would have been the file copy in the back like it was in OS X.

Overall, the QT movies' playback in X was smoother (not great, but ok) and there were a ton more things going on. Switching between apps was smoother in X than 9.1. iTunes ripping was better in 9.1 (about 1.4 vs. 1.0). Overall, I'd say X won because I was doing more and I could continue reading omniweb and looking at the movie playing in the dock while all this was going on. Good test, griffman.

can someone did this kind of test on a powerbook G4?

single G4..

------------------
Florence mon amour

No numbers here, but some observations: Last night I wrote a Perl script to recursively seek out and copy image files that fitted a certain naming convention, and copy them (in one case from a CD) to a firewire drive, creating directories to put them in.

I was amazed at how quickly the thing ran, even when copying from a CD - it was effortless. But the main thing is that I was able to write a perl script in the first place - this saved me far more time than trying to do the same on Mac OS 9 either manually, with MacPerl or Applescript.

This is the true speed advantage of MacOS X...

well I did my own test kind off I am running RC5 in the terminal at 9 if you are not familiar this compleatly taxes both my processors. in the processveiwer the bars are pegged. and I try opening up other apps and typing. in OS 9 there would be a five second deally every once any a while for the cycle to get back to the key strokes. but in OSX its like RC5 isn't running at all. Ym speed has also imporved from about 7.6 Mkeys/sec to 8.9Kkesy/sec I have A dual 500 but RC5 takes advantage of both processors in 9 and X, so OS is the only differance. very cool I am now addicted to OSX http://forums.macnn.com/cgi-bin/Foru...ML/000203.html

------------------
I Have a Super Computer

Since 10.02 I've felt that OSX is as fast if not faster than 9.1 on my setup. 1st of all, I bought my G4 with OS-X in mind. I've used Apples since the Apple II plus, and before the Dual Processor and OSX, I was headed for AMD Athalon land. So OSX was(is) VERY important to me. My setup: Dual 500mhz G4, 768MB of RAM, 2 IBM Desktar Hardrives (40gb & 45gb), Firewire Sony Spressa CD Burner, Epson 740i, & Cannon N650U scanner.

The Test:
Quicktime Player: Final Fantasy Trailer 480 x 260 looping
Internet Explorer Quicktime plug-in: Crouching Tiger Trailer (Medium) looping
Copy my MP3 folder 4.02gb (1,340 files) from one drive to another
Burn 79min 24sec CD
Operations were started in the order above. The test was finished when the files were copied and the CD burned.

OSX 10.03 (with Classic in the background)
Total Time : 14min 0sec
-Copy File : Done in 9min 40sec
-CD Finished: 11min 20sec
-2min 40sec penalty (explained later)

OS9.1 (VM off)
Total Time: 22min 14sec
-Copy File: 8min 40sec
-CD Finished: 20min: 25sec
-1min 20sec penalty

OSX: Screenshot of OSX Test
The CPU intensive task of preparing the tracks (converting MP3s to AIFF files) definitely tasked the system. The Crouching Tiger trailer dropped very few frames if any and the Final Fantasy trailer dropped a frame every 4-5 seconds without fail. After the music track preparation or during the actual CD burn, both Quicktime movies played without dropping frames whether in the foreground or background. The sounds from both of the trailers never stopped or stuttered. The OS was slower but still usable. Menus took about 1 second to open, and dock menus took 2-3seconds. I was able to launch the terminal, text edit, and grab and use them with no issues. BUUUUT, right after the CD burn finished, I got the colorful spinning wheel of death in ALL open applications. I was able to switch between apps but I couldn't force quit any applications. This required the 3rd hard reboot of OSX since the 10.01 update, but the 1st one that didn't involve Classic and Macromedia Flash4. The 2min 40sec penalty was for the time it took me to restart, Fsck 4 times to rid myself of all disk errors and reboot into a fully functional OS-X desktop environment.

OS9: Screenshot of OS9.1 Test
A very different experience. The task of preparing the music tracks was stilling going at 12minutes (when OSX was already done with the entire task). Both Quicktime movies became Quicktime slide shows, both dropping a large number of frames. The sound stuttered and cut out. Menu opening, window opening, and scrolling performance dropped drastically . After the track preperation (about 13minutes) the Quicktime movies improved as long as either IE or Quicktime was in the foreground. Changing to iTunes or the finder caused frames in both movies to dropped. All this and iTunes for some reason decided to drop the last music track. So it prepared only 15tracks and burned only 15 tracks. I estimated that it would have taken iTunes about 50seconds to prepare the dropped track and 30 seconds to burn it., hence the 1min 20sec penalty.

It can only get better

------------------
-Toyin

ToyinFayemi.com

[This message has been edited by Toyin (edited 05-11-2001).]