Welcome to the MacNN Forums.

jerseyfreeze · 01:27 PM

I know that it's impossible to put objective ratings on the themes created and shared with these forums, but I find it hard to believe that every theme reviewed by MacThemes is significantly above average.

Here are the overall scores for the 25 themes reviewed so far:

Aluminum Alloy: 9.1
Aqua: 9.1
Aqua Pro: 8.0
BBX Mercury: 9.6
Brushed: 8.9
EYLO: 8.8
FatalE: 8.1
GuiPod: 7.5
Gershwix: 7.8
JetBlack: 9.0
Latium: 8.8
Mac OS XP: 8.8
Milk: 8.0
NEOS: 6.8
NeXT: 7.5
Power Metal: 9.0
Rhapsodized: 9.4
SS Studio Pro: 8.2
Shinobi: 7.4
SImple X: 8.5
SmoothStripes: 8.6
TiSkin: 9.6
Titanium: 8.6
[fear]Platinum: 8.4
iStyle: 9.0

Average overall score: 8.5
Percentage of scores 9 and above: 32
Percentage of scores 8 and above: 80
Lowest score: 6.8

All of this means I take MacThemes reviews with a gigantic grain of salt. It's like car magazines that are in love with BMWs; You can't gain much from their comparisons because you know they will always favor the Bavarian manufacturer.

I wish MacThemes would rethink its rating system.

-Jason

bwahahax · Apr 25, 2004, 01:25 PM

Yeah. I noticed that too.

idle · Apr 25, 2004, 02:12 PM

Maybe its just that some of the themes posted at resex are missing?

All of the themes reviewed at MT so far have been made very thoroughly and with a lot of effort. Personally i see no reason to bash some of them with really harsh ratings....

regards, [ idle. ]

rhythmicmoose · Apr 25, 2004, 02:17 PM

And to think that the 6.8 is NEOS, one of the most tangible, well-constructed themes out there... Some of these are a bit off.

BUT!

Keep in mind the type of themes MacThemes reviews. The scale seems to be built to accommodate ResEx-quality themes as well. I'm sure if those were ever to be reviewed, the range of ratings would be dramatically increased.

EDIT: Damnit, idle, you posted that while I was writing.

jerseyfreeze · Apr 25, 2004, 02:18 PM

Originally posted by idle:
Maybe its just that some of the themes posted at resex are missing?

All of the themes reviewed at MT so far have been made very thoroughly and with a lot of effort. Personally i see no reason to bash some of them with really harsh ratings....

regards, [ idle. ]

Except that it's already understood that MacThemes only reviews complete themes. There should be no shame in getting a 5 if your theme is being compared with other complete themes.

Why create some rating system just to make all theme creators feel good about themselves? It doesn't help me (the MacThemes user) distinguish between good themes and bad.

My personal opinion is the rating system should be discarded. A good, relatively fair review is fine. You don't need to append arbitrary numbers to the end of each review.

-Jason

Jaey · Apr 25, 2004, 02:39 PM

I personally don't pay attention to the scores, I make my own judgements. The ratings do seem to match my own fairly often, so I like to go by those.

You do have a valid point, but I don't agree with it. But as idle said, all of the themes reviewed at MacThemes are good themes. I wouldn't want to see a single one of them with a low rating.

The one thing I would like to see, maybe as a separate rating, is a score for originality.

phillryu · Apr 25, 2004, 03:37 PM

This issue was actually brought up with software scores before, and the fact is, our software and theme archives have turned into a collection of the most useful, popular, and best themes and software. The main reason with this is because we have limited time to review themes and software, and frankly, we'd much prefer to add to our archive higher quality themes rather than lower ones to "balance" the scores out.

That having been said, I don't think I ever expected the scores to be like this when I started the site. Believe it or not, just four months ago, the theming world was actually a lot less mature, and much slower. JetBlack, NEOS, TiSkin, ShapeShifter, GarageBand, and even Panther updates for SmoothStripes was brand new. We had one reviewer - me, and scarily enough, that seemed enough to soon run out of themes to review. And as I expected to eventually start reviewing incomplete and/or less mature themes, I thought the score scale made sense. Luckily, I say, I was wrong. (About running out of stuff to review.)

So I wouldn't go as far as to say we are a fan site raving about Britney Spears albums or BMW's. But I will admit that our archives are a tad biased and do not represent the entire theme universe. And as the webmaster of the site, I'm going to go ahead and say it'll remain that way. I'd much rather review the newest, and best themes rather than first outings or simpler modifications. I doubt Kevin would've wanted his first SimpleX theme reviewed. However, I will take this slant into consideration in the future, and perhaps readjust scores to bring some more diversity into our archives, if themes truly deserve the C or D, or even F. Hopefully, it won't come to that, but if it's necessary, I'll do it.

phillryu · Apr 25, 2004, 03:39 PM

Originally posted by Jaey:
The one thing I would like to see, maybe as a separate rating, is a score for originality.

That's a pretty good suggestion, and one that I've previously included in the Reviewer's Weight score, but may add as a separate category, especially after ditching "Extras".

quandarry · Apr 25, 2004, 06:25 PM

ratings on macthemes means squat.

anyone can rate a theme on their webpage.

means nothing...

what counts is that the author is happy with his product and that maybe just maybe one person has chosen to use it.

everything else is bulls!t thrown around by peeps for no useful purpose.

do you really think it matters whether neos gets 5.0 and that the windows ports get 10.0 on macthemes.

slap yourself.

<----insert pokey guys here.

flamingmoe · Apr 25, 2004, 07:56 PM

Originally posted by quandarry:
ratings on macthemes means squat.

anyone can rate a theme on their webpage.

means nothing...

what counts is that the author is happy with his product and that maybe just maybe one person has chosen to use it.

everything else is bulls!t thrown around by peeps for no useful purpose.

do you really think it matters whether neos gets 5.0 and that the windows ports get 10.0 on macthemes.

slap yourself.

youre just mad becuz your themes are crap!!!

all of your themes get maybe 4.0 from me. that would serve a good balance to macthemes, don't you think???

SwarmyCurve · Apr 25, 2004, 08:33 PM

For as much as I love MacThemes (the news updates, insider info, design from Adam Betts, etc) the reviews I always skip over. The scores are just wildly inaccurate, in my humble opinion.

Some of the themes scores are way, way too high. I won't name names, since that is rude to the authors of them and considering I have never made a theme myself, who am I to shoot down other peoples work.

Based on this grading system, I would hope the upcoming review of Swizzzzes Cappuccino to be about an 11.2. The orbital variant is one of my three fav. themes now, alongside Aluminum Alloy and Milk (which got me interested in themes in the first place).

quandarry · Apr 25, 2004, 08:39 PM

Originally posted by flamingmoe:
youre just mad becuz your themes are crap!!!

all of your themes get maybe 4.0 from me. that would serve a good balance to macthemes, don't you think???

wibs · Apr 25, 2004, 09:04 PM

Sorry phill, but I have to agree with everyone else. I treat reviews more like a Theme Spotlight than an actual review, and enjoy reading the comments about what people like and dislike. But those scores are completely useless, except for objective categories like extras... oh wait that's not a category anymore

. For other categories, for example Usability, themes don't fall into nice numbers for me. They're either unusable, somewhat usable but with problems, or usable. As for aesthetics, I can judge what I think is pretty or not without any help, thanks

.

I love the reviews for being there, but as I said I just tend to pretend that they're not reviews at all.

flamingmoe · Apr 25, 2004, 09:50 PM

Originally posted by quandarry:

i recieved a sleepy smiley from quandarry. a 11 years old boy with no sense of mannership. on an internet forum. from somewhere far, far away.

man, my life is so over!!!

swiz · Apr 25, 2004, 10:48 PM

I like to read the review comments, read the user contributed comments and leave it at that. I dont look at the scores or the star rating from users. I think most people look at the star rating and click either the highest or lowest star to help sway the result toward their own preference. Not showing the star rating until ater the user has voted would actaully be useful at least.

digitaljames · Apr 25, 2004, 11:24 PM

Originally posted by swiz:
I like to read the review comments, read the user contributed comments and leave it at that. I dont look at the scores or the star rating from users. I think most people look at the star rating and click either the highest or lowest star to help sway the result toward their own preference. Not showing the star rating until ater the user has voted would actaully be useful at least.

He lives! The early reports of his demise at the hands of a disgruntled wood chipper crew were grossly inaccurate.

Welcome back!

Gimmee skins....

quandarry · Apr 26, 2004, 01:14 AM

Originally posted by flamingmoe:
i recieved a sleepy smiley from quandarry. a 11 years old boy with no sense of mannership. on an internet forum. from somewhere far, far away.

man, my life is so over!!!

apparently you still have to find a life...

a little hint get another nick...against me you'll lose.

Ozzpot · Apr 26, 2004, 02:35 AM

I think the scores are on average about right on MacThemes. They do review only the best themes so I think they justify 8 or so out of 10 on average. A theme should be judged in comparison with all the themes out there, not just the others on MacThemes.

I do think that NEOS's score is a travesty though. I have used NEOS for ages now, and I find it beautiful, unique, elegant and a joy to use. Very rarely does it suffer from the problems usually associated with dark themes. I would be happier with a score of around 8.

Also, I think people can be a little anal about a score of 5 being the average, just because it is mathematically halfway. But if someone rated your girlfriend 5 out of 10 you'd be pretty annoyed, right?

jerseyfreeze · Apr 26, 2004, 07:09 AM

Originally posted by Ozzpot:
I think the scores are on average about right on MacThemes. They do review only the best themes so I think they justify 8 or so out of 10 on average. A theme should be judged in comparison with all the themes out there, not just the others on MacThemes.

But it doesn't make sense to judge them in comparison with themes that MacThemes refuses to review (and probably rightly so). How do we know what some other theme would've gotten had it been reviewed?

In a perfect world, the scores I listed above, when plotted, would form a perfect bell curve with the vast majority of scores being around 5.0.

A car magazine can't do a comparisong of semi-luxury sport sedans, but then score them in comparison with supercars and SUVs.

-Jason