[silverflyer]

Currently there are 7 external drives connected via firewire to the computer, is there a raid card that will allow some or all of these to be set up in a raid format?

thank you.

[silverflyer]

Also, does anyone here have experience with softraid 3?

[mduell]

What's your price range, which G5 PowerMac revision, how many drives do you need, and what RAID levels do you want?

You can use OS X's software RAID 0 or 1 (or a combination) with your existing drives; it's not great (like most (all?) software RAID), but it's functional. I haven't seen any hardware RAID cards for Firewire devices, so for hardware RAID you'll need to buy a new multibay enclosure. eSATA enclosures that support hardware RAID 0 and 1 are reasonably priced; about $250 for a 5 bay enclosure plus $100 for a performant Mac-compatible eSATA card. If you need more drives you could go with a 12 bay enclosure for about $750, but you'd need a $500-1000 eSATA RAID card (which would support RAID 0, 1, and 5) to go with it.

[OreoCookie]

What do you need? More speed? extended storage? How valuable is the data?

[silverflyer]

It is a G5 powermac dual 2.0, that is all I know.

There may very well be a new Mac Pro soon however, the requirements are data integrity and speed for video editing.

does something like this require a Raid card in the computer?

http://eshop.macsales.com/shop/firewire/usb/raid_1/Gmax

[OreoCookie]

We can't give you advice unless you tell us what you want to use the drives for. If you don't know, we can't give you any advice. Unless you get a self-contained hardware RAID (à la WiebeTech RT5 or so), there is no easy way to migrate RAIDs (without losing your data). In your particular instance, a RAID with FireWire drives won't give you much of an edge speedwise and also capacity-wise, I'm not sure whether it's useful to concatenate several FireWire drives.

SoftRAID won't give you an edge, it doesn't have RAID-levels OS X' built-in RAID has, neither has the enclosure you've linked to.

If you want to think about storage solutions, you have to think about what kind of data you have, how precious it is, the storage capacity and speed you need, and a backup solution.

[silverflyer]

Ok, one more question. Can I make a RAID array using OSX Disk utility and of external drives of various sizes, and do I need to move the data on them somewhere else first?

This particular array will be for archiving of video projects and files. The data is VERY important.

[mduell]

Apple menu->About this Mac->More Info; what's the Model Identifier?

For data integrity and speed, you want RAID10, which every RAID enclosure/controller supports.

What size are the drives you have and how full are they? If you want to reuse the disks you already have in a RAID array, what type are they (IDE or SATA)?

You can made a software RAID of different sized externals in OS X, but it offers limited advantages compared to a proper setup.

How much data do you have and how fast is to growing?

There's a lot of knowledge available in this forum from people who know a lot about media management and storage hardware, but you need to provide more details for a meaningful recommendation.

[chris v]

Originally Posted by silverflyer 

Ok, one more question. Can I make a RAID array using OSX Disk utility and of external drives of various sizes, and do I need to move the data on them somewhere else first?

This particular array will be for archiving of video projects and files. The data is VERY important.

1. Yes, you can make a software RAID array out of a bunch of external Firewire disks using Disk Utility, should you choose to do so. I have done this, just to see if I could, and it worked fine at the time, though I only left it for a few days. You will lose all the data on them when you format the RAID array.

2. If the data is REALLY important, I wouldn't do it the software way. I would buy one of the multiple-drive enclosures that offers RAID 1 & that has a Firewire 800 connection. That way, you'll have data integrity and reasonable throughput.

3. Then, I'd back that up off site, weekly, or monthly.

[silverflyer]

Originally Posted by mduell 

Apple menu->About this Mac->More Info; what's the Model Identifier?

For data integrity and speed, you want RAID10, which every RAID enclosure/controller supports.

What size are the drives you have and how full are they? If you want to reuse the disks you already have in a RAID array, what type are they (IDE or SATA)?

You can made a software RAID of different sized externals in OS X, but it offers limited advantages compared to a proper setup.

How much data do you have and how fast is to growing?

There's a lot of knowledge available in this forum from people who know a lot about media management and storage hardware, but you need to provide more details for a meaningful recommendation.

Thanks for the reply.


I will have to check when I get back to the Office it is in, all I know now is it is a dual 2.0ghz Power mac G5

Raid 10, for absolute data integrity? is striping a good idea for the working drive? The way Final cut pro works, it is better if you have two separate drives, for those two we would like speed as well as integrity of the data.

The two drives we have now that will go in an external raid are 2 500gb drives inside the mybook external drives, as of right now, I can not tell you if they are IDE, ATA or SATA. The other 6 drives are of various sizes from 500gb down to 120gbs. We are considering using them as one large array for archival use only. The smaller ones are full, with one of them only having 1gb left.

Data could potentially be growing very fast, it depends on how fast work comes in.

[silverflyer]

We also set up 2 terabyte external drives as strictly backup.

[silverflyer]

Originally Posted by chris v 

1. Yes, you can make a software RAID array out of a bunch of external Firewire disks using Disk Utility, should you choose to do so. I have done this, just to see if I could, and it worked fine at the time, though I only left it for a few days. You will lose all the data on them when you format the RAID array.

2. If the data is REALLY important, I wouldn't do it the software way. I would buy one of the multiple-drive enclosures that offers RAID 1 & that has a Firewire 800 connection. That way, you'll have data integrity and reasonable throughput.

3. Then, I'd back that up off site, weekly, or monthly.

1) I was going to try today but thought it wise to backup the data first, so It is moving over to a different external as we speak.

2) Are there enclosures like that in which we can use different size drives?

3) back up, I agree with.

[mduell]

Originally Posted by silverflyer 

Raid 10, for absolute data integrity? is striping a good idea for the working drive? The way Final cut pro works, it is better if you have two separate drives, for those two we would like speed as well as integrity of the data.

The two drives we have now that will go in an external raid are 2 500gb drives inside the mybook external drives, as of right now, I can not tell you if they are IDE, ATA or SATA. The other 6 drives are of various sizes from 500gb down to 120gbs. We are considering using them as one large array for archival use only. The smaller ones are full, with one of them only having 1gb left.

Data could potentially be growing very fast, it depends on how fast work comes in.

I don't know what you mean by "absolute"; RAID10 is striping and mirroring. Striping is a great idea for a working drive, since it gives you a substantial increase in throughput.

Reusing small drives from a MyBook probably isn't a great idea (they're not that big, hassle to get out, warranty is voided). It sounds like you have about 2TB on all your externals combined.

I'd suggest two of these 5 bay enclosures (eSATA with hardware RAID 0 and 1). Buy at least three (and preferably five) 1TB drives for each enclosure. You'll also need an eSATA card ($100ish), since I don't think the PowerMacs have any extra SATA ports that you could use with a eSATA bracket ($5), but which one you need depends on your PowerMac model. Set up each enclosure as a RAID0 across all disks and set up an rsync cron job to sync the two arrays at a reasonable time interval for your business needs (probably every hour). While this doesn't quite offer the availability of RAID1 (which you probably don't really need), I think you're better off since it has a lower probability of data loss and it's more portable (you can unplug one of the enclosures and plug it in anywhere and have your data available). All in, the cost is about $1650-1950 depending on how big the array is and if you need an eSATA card; I'd encourage you to go with the 5 drive arrays if you can afford it now for performance and less future hassle (upgrading isn't terribly hard, but it takes a bit of time).

Originally Posted by silverflyer 

2) Are there enclosures like that in which we can use different size drives?

In a RAID array? No. As JBOD? Yes.

[silverflyer]

By "absolute" I meant no chance of data loss if one drive fails.

RAID 10 sounds like the way to go, thanks for your assistance. I am new to RAID-ing and am trying to learn as much as I can, as fast as I can.

I also have to work in a budget, and build things over time. it would be nice if we havd a money tree outback and could buy whatever we needed today,. but the next big purchase will be the Mac Pro, then we can look into buying more and larger drives for the 5 bay raid enclosure. It is for this reason I am trying to use as much of what we already have, in the best way possible.

I think we will indeed use the 6 remaining drives as a JBOD, and use the 500gb drives as seperate drives for the moment until we get 2 more and then have 2 separate raids for the working disks, and then a bunch of them for archival purpose.

Which leads me to one more question, is it possible with Disk Utility to set those remaining disks up as a JBOD that will function well enough for archival purposes until we can afford to replace them with a better solution?

[OreoCookie]

Originally Posted by silverflyer 

Ok, one more question. Can I make a RAID array using OSX Disk utility and of external drives of various sizes, and do I need to move the data on them somewhere else first?

If you create an array, it's akin to `partitioning' and initializing them, you will lose all data on that disk.

Originally Posted by silverflyer 

This particular array will be for archiving of video projects and files. The data is VERY important.

Then I don't think you will make do with any kind of software RAID. There are a few considerations to be made:
(i) If you need large amounts of storage on one volume, you will need a RAID5. This is currently not possible via software (neither OS X' RAID nor SoftRAID can do this) in OS X.
(ii) Except for JBOD (= just a bunch of disks), your harddrives need to be of the same size. JBOD concatenates your drives which gives you zero speed advantage and the only thing it does is create one large volume. With OS X, if you lose one drive of a JBOD, you will lose all your data. Not recommended.
(iii) JBOD and RAID0 increase the probability of failure significantly. It is not recommended unless you have a backup of your data or the data is not worth a lot to you (e. g. as a scratch volume, a RAID0 is quite useful). If you use the drives via FireWire, RAID0 won't even give you an advantage in terms of speed.

If you have n drives, the probability that your RAID fails due to hardware is approximately n times the failure rate of a single drive. Add the probability of user error and this is quite significant.
(iv) The only cheap solutions (hardware or software) will be able to provide RAID levels 0 (= faster, but much higher risk of failure) and 1 (= mirroring, you will lose a lot of capacity). If you need anything beyond that, I can't recommend it.

So if I were you, I'd consider the following two things, both of which are easier and probably cheaper than a RAID: get a large 1 TB drive and use it. If you use a SATA enclosure, it will be as fast as an internal drive (you use the same interconnect after all). It'll be much faster and easier to configure than a RAID with FireWire drives.
Or if you really need a cheap, but slow solution, have a look at a Drobo: if all you want to do is archive your data, this might be a feasible solution. Furthermore, it acts as an external USB harddrive and you can plug it into other computers if necessary (something you can't do with RAIDs, unless it's a hardware RAID).

A `real' (hardware) RAID solution will be in the ball park of $2k+ (including drives). A WiebeTech RTX600 will cost $2.7k without drives, the older RT5e (which uses PATA drives) will cost $1.7k without drives. Those are cheap hardware RAID solutions.

[mduell]

Originally Posted by silverflyer 

By "absolute" I meant no chance of data loss if one drive fails.

RAID 10 sounds like the way to go, thanks for your assistance. I am new to RAID-ing and am trying to learn as much as I can, as fast as I can.

I also have to work in a budget, and build things over time. it would be nice if we havd a money tree outback and could buy whatever we needed today,. but the next big purchase will be the Mac Pro, then we can look into buying more and larger drives for the 5 bay raid enclosure. It is for this reason I am trying to use as much of what we already have, in the best way possible.

I think we will indeed use the 6 remaining drives as a JBOD, and use the 500gb drives as seperate drives for the moment until we get 2 more and then have 2 separate raids for the working disks, and then a bunch of them for archival purpose.

Which leads me to one more question, is it possible with Disk Utility to set those remaining disks up as a JBOD that will function well enough for archival purposes until we can afford to replace them with a better solution?

If one drive fails, the RAID0 array that it's in would go offline, and you would be able to continue working from the other RAID0 array.

Note I recommended RAID0+syncing rather than RAID10 for your setup; they're similar, but not exactly the same.

I understand you have a budget to work with, but what's the cost of losing one of your external drives?

JBOD does not solve any problem you have (redundancy or performance), and adds a new failure mode.

Originally Posted by OreoCookie 

(iv) The only cheap solutions (hardware or software) will be able to provide RAID levels 0 (= faster, but much higher risk of failure) and 1 (= mirroring, you will lose a lot of capacity). If you need anything beyond that, I can't recommend it.

Two enclosures running RAID 0, 1, or 10 is usually (IME - there are exceptions) the best solution when you consider capacity, price, recovery time, and performance (no data loss is assumed). See below for the comparison a more expensive product.

Originally Posted by OreoCookie 

Or if you really need a cheap, but slow solution, have a look at a Drobo: if all you want to do is archive your data, this might be a feasible solution. Furthermore, it acts as an external USB harddrive and you can plug it into other computers if necessary (something you can't do with RAIDs, unless it's a hardware RAID).

With the high enclosure cost and limited number of bays, Drobo offers little price advantage, and of course poor performance. The enclosures I suggested above can be plugged in to any USB port with a $25 SATA-USB adapter.

Originally Posted by OreoCookie 

A `real' (hardware) RAID solution will be in the ball park of $2k+ (including drives). A WiebeTech RTX600 will cost $2.7k without drives, the older RT5e (which uses PATA drives) will cost $1.7k without drives. Those are cheap hardware RAID solutions.

The Steel Vine based product I linked to from AMS is real hardware RAID 0/1/10.
$3700 buys you a slow (RAID5 write penalty) 5TB array with the WiebeTech, while $2000 buys you a fast (RAID0 performance) 5TB array with the pair of AMS enclosures.

[OreoCookie]

Originally Posted by silverflyer 

I think we will indeed use the 6 remaining drives as a JBOD, and use the 500gb drives as seperate drives for the moment until we get 2 more and then have 2 separate raids for the working disks, and then a bunch of them for archival purpose.

This is a very bad configuration, if the probability of drive failure is 5 % (just for the sake of numbers), the probability that a 6-drive JBOD fails is 30 % (= 6x5 %)! (BTW, the number of 5 % may seem bit high, but it is still realistic. A study with google's drives, they have found that the average probability of failures of harddrives in the first three months and in the third year is about 5 %.)

So please give us an idea of
(0) your budget! (and no, as cheap as possible is not a budget )
(i) the amount of storage you need now
(ii) the expected data growth you will have
(iii) the amount of data you want to access on one single volume
(iv) the kind of data
(v) your speed requirements
(vi) and sort the data by priority (e. g. old projects could be moved to backup drives)

Storage solutions are complex, very complex. I recommend you have a look at this thread. Two members discuss their solution and their reasoning behind it.

For example, if you don't need more than 1 TB of volume space, you could get one or two 1 TB drives and make a RAID1. Older projects are stored on separate drives which the clients are billed for (that's what chasg does, although AFAIK he is a professional photographer, not a video guy).

Perhaps you need a scratch volume? Then you can create a RAID0 which is faster, but nothing is lost if it fails (it's a scratch drive after all). These are all considerations that will make your solution cheaper and more adapted to your specific needs.

@mduell
If you need something professional, then you will want an enclosure with hardware RAID5 capability. Your solution cannot deliver that.

I don't think your proposed solutions fit in well here, because of the lack of technical expertise. E. g. the RAID0 + rsync solution is problematic, because (i) rsync isn't as reliable on OS X as on other platforms, (ii) it needs to be set up properly (not a problem if you know what a cron job is and how to use the command line, probably a problem if you don't) and (iii) this is a silent error solution (no warnings by rsync to the user unless (s)he goes through the system's log files regularly.

That's why I have proposed the Drobo: if performance is not critical, it does what it says it does, is easy to set up and upgrade. That's worth $$$.

I'm not even convinced the OP really needs a RAID/JBOD: without having more information (amount of contingent storage needed, data growth per year, etc.), I think it's not wise to advise for or against a particular solution.

If we had full ZFS support in OS X, we'd have more flexible tools at our disposals and external SATA enclosures would be a lot more appealing than they are now (you can create RAIDZ volumes which give you RAID5- or RAID6-like protection (protection against one or two failed drives using cheap `dumb' external enclosures) without the write-hole (which is less important than data safety and integrity), but right now you can't. (Unless you set up a FreeBSD/Linux/OpenSolaris storage server for just that.)

(Just this weekend, our sys admins upgrade our storage here, they're going from RAID5 to RAID6 and upgrade the firmware of our storage boxes, the IT honcho makes a big deal out of it )

[mduell]

Originally Posted by OreoCookie 

@mduell
If you need something professional, then you will want an enclosure with hardware RAID5 capability. Your solution cannot deliver that.

Oh please; claiming that RAID5 is more professional than RAID1 (with or without RAID0) is absurd. They both have their pros/cons and thus places, but professionalism isn't the argument.

Originally Posted by OreoCookie 

I don't think your proposed solutions fit in well here, because of the lack of technical expertise. E. g. the RAID0 + rsync solution is problematic, because (i) rsync isn't as reliable on OS X as on other platforms, (ii) it needs to be set up properly (not a problem if you know what a cron job is and how to use the command line, probably a problem if you don't) and (iii) this is a silent error solution (no warnings by rsync to the user unless (s)he goes through the system's log files regularly.

In that case run RAID10 (in hardware) with or without a hot spare; still comes out ahead of the expensive RAID5 boxes for cost and performance.
Two enclosures ($450) + ten 1TB drives ($2000) yields 4TB protected with hot spare at ~$600/TB; One RTX600 ($2700) + six 1TB drives ($1200) yields 4TB protected with hot spare at ~$1000/TB. The former has better write performance and no downtime when you lose a drive (data accessible during rebuild, something a professional may desire).

Originally Posted by OreoCookie 

I'm not even convinced the OP really needs a RAID/JBOD: without having more information (amount of contingent storage needed, data growth per year, etc.), I think it's not wise to advise for or against a particular solution.

I'm halfway with you there, but it's so hard to get information out of the OP

Originally Posted by OreoCookie 

(Just this weekend, our sys admins upgrade our storage here, they're going from RAID5 to RAID6 and upgrade the firmware of our storage boxes, the IT honcho makes a big deal out of it )

6 is nice since it lets you comfortably run larger arrays without going to 50.

[OreoCookie]

Originally Posted by mduell 

Oh please; claiming that RAID5 is more professional than RAID1 (with or without RAID0) is absurd. They both have their pros/cons and thus places, but professionalism isn't the argument.

No, that's not what I said.
All RAID levels have their professional applications, but professional RAID systems have more than two RAID levels

[chris v]

As to the integrity of software RAID arrays, I can offer one point of anecdotal evidence. I have an old G4 QS Dual 1.0 box that I was trying to breathe a little more life into a few years ago, so it occurred to me to make a RAID 0 out of the two internal 80 gb IDE drives in the machine using Disk Utility, to speed up read/write a little bit. (I made a poor-man's RAID 1+0 by adding a 160 gb SATA drive & PCI card, then set CCC to back up from the RAID array every night)

It worked great for about 2.5 years, then one day, after a power failure, the RAID array just wasn't there any more. Upon boot, I got the question mark, had to boot from the backup, and there were both of the 80 gb drives, no physical problem with either one, but blank, and unnamed, and certainly not part of any RAID array. I had to rebuild the array and copy everything back from the 160 gb backup, despite there being nothing physically wrong with the disks, motherboard, or ATA bus/ribbon. Something bad just happened when the power went out. This is why I personally don't recommend software RAIDs.

[silverflyer]

I have not abandoned you all in this thread, I appreciate the knowledge shared here very much, right now we are taking what has been said and learned in this thread, and using it to evaluate our needs a bit more intelligently.

Thank you all.

[drdidg]

As it was not specified, that I have seen if your system is the Dual G5 2.0Ghz that has PCIe or PCI-X, I will share what I did with my Dual 2.0Ghz with PCI-X.

I looked high and low for a long time for a cheap RAID 5 solution. I wanted redundancy and a recoverable setup, so RAID 5 was my choice. PCI-X obviously and eSATA with support for multiplexing was also a must as I was not in the mood to have cluttered wires going to my external enclosure. It took me forever to find the card that I wanted, and it was my chance that I happen to find one, as it seemed no one had any info on the subject unless your talking about $500+ cards.

30 mins rummaging through Microcenter cards, and I found one that met all requirements. The Norco-4618.

I paid $79 for the card at Microcenter. I picked up an Addonics external RAID case and I think I got the ST5X1PM model for $199.

I monitor cheapstingybargains.com via RSS feed and found a couple of Samsung 750GB drives for $114 each. I already had an extra 750GB lying around to use as well. Got all three up and running in RAID 5 at 1.4TB of total space available for just over $500 with drives. This was a couple of months back, so likely to be even cheaper by now. I tested the card in 10.4.9 and 10.5.4 in both regular and server edition of OS X. The one thing I found to be a slight pain is that the card only seems to work in my system in the top slot, so it was just tight to get it up in there, but works like a charm. It may not be the fastest scratch disk in the world for what your after, but for a cheap redundant data solution, I am yet to find anything better.

[OreoCookie]

The description says that RAID5 works only on Windows … are you sure there is RAID5 support for OS X?

[drdidg]

Originally Posted by OreoCookie 

The description says that RAID5 works only on Windows … are you sure there is RAID5 support for OS X?

Yeah, that one caught my eye as well on the box. Thankfully at Microcenter the box could be opened where inside it had a notice that there are drivers for it on the CD included. But yes, it does support RAID 5 in full. It comes with a funky Java based configuration tool to get it up and running. You can goto the SiS chip site to grab the drivers and application from them directly as well. Nothing too exciting, but if your using a multiplexing setup, the card has 4 eSATA ports so you can hook up to 20 drives to the thing. IMO the one major fallback of this, is no decent amount of dedicated RAID memory on the card. Just enough to scrape by, but at 1/5th to 1/10th the price of like the Promise cards that Apple uses, its hard to beat.

[mduell]

OS-specific RAID level availability implies its being done in the driver or app... eww.

The Promise box that Apple sells is in an entirely different class.

[drdidg]

Originally Posted by mduell 

OS-specific RAID level availability implies its being done in the driver or app... eww.

The Promise box that Apple sells is in an entirely different class.

Well can check for yourself it has the Silicon Image 3124 RAID controller chip inside it.

And yes the Promise is in a different class at a mere 12x the price.

As the OP never seemed to list his model number or if using PCI-X or PCIe that sure does not help. PCIe has so many RAID controllers available its sick, but PCI-X is nothing but a PITA to find anything especially as it is becoming an outdated bus type.

[mduell]

SiI 3124 is just an eSATA chip with Port Multiplier support... it doesn't do any RAID in hardware. You should be connecting one of it's channels to a box that does hardware RAID.

[OreoCookie]

It's a software RAID, hardware-capable RAID cards for PCs cost more than twice as much and they have a sizeable cache (128 MB or more) rather than 4. The two are not the same thing. If you look at the specs in more detail, you will see that it admits explicitly that RAID5 is done in software.

Hardware RAIDs are independent of the underlying OS, the RAID functionality is provided in hardware and you have to configure the RAID via the bios of the card. With software RAIDs, if you screw up your OS installation, chances are that your RAID is gone, too. I doubt the RAID survives a reinstall of the OS.

[drdidg]

Originally Posted by OreoCookie 

It's a software RAID, hardware-capable RAID cards for PCs cost more than twice as much and they have a sizeable cache (128 MB or more) rather than 4. The two are not the same thing. If you look at the specs in more detail, you will see that it admits explicitly that RAID5 is done in software.

Hardware RAIDs are independent of the underlying OS, the RAID functionality is provided in hardware and you have to configure the RAID via the bios of the card. With software RAIDs, if you screw up your OS installation, chances are that your RAID is gone, too. I doubt the RAID survives a reinstall of the OS.

Well I have reinstalled the OS several times and never lost anything. I have all my data on DVD-DL's just in case, but just interesting you should mention that.

When I start pushing mine via the network, now granted mine acts more of a media server and not a processing powerhouse, I will usually see the CPU start to increase usage up to a max of about 5%. For me and my needs that is perfect and quite acceptable. Yeah good RAID cards have RAM seems like 256MB is becoming the new standard and a dedicated I/O chip. The Norco-4629 seems to be a better step towards it with a I/O processor on board, but still no decent RAM cache. If I was doing this for work, yeah I would look at my solution and laugh. But on a home budget just building for fun, the price point can't be beat if your willing to sacrifice a bitty of your CPU power.

When I first saw the article I thought this forum thread was more of a discussion on options not just a single guy asking for thoughts. So I dumped my thoughts and experiences here if not for anything but to help anyone else who may have thought the same and wanted to come for info.

Do we know yet if the OP is on PCI-X or PCIe?