[Biest]

Hi,

First of all Thanks for all replies. I am still in the planning stage of the whole system because of the size (requirements will follow) and the specs. I wasn't to sure where to post this, as this involved close to nearly all Apple Hardware products and I am personally tending towards a Mac Pro solution.

So the whole idea is this... My father and I have been accumulating a quite substantial DVD collection (somewhere in 850+ range). We are currently using software for Video rental stores to keep track of everything. To add to this we have roughly 1000+ albums. Yes one could call us crazy. At the moment the DVDs are stored multiple locations across the house, as the storage in a single place would cost a lot of space. The main idea my dad has had since the announcement of the Apple TV is a media center. We don't have a HDTV (yet), but are pretty such else setup well considering the space restrictions (the main media rig is in my dad's home office, which is similar to a balcony within the house... really hard to explain). The media center should be able to do the following:

1. Use Frontrow, Quicktime and iTunes as the media players and distribution system.
2. Be able to store all the data (if I use DVD2one to shrink then down we are still at 5TB+)
3. Pretty secure data-wise and have a certain ease of use.

At the moment I have been considering multiple Idea:

Mac Pro plus XServe Raid (highly unlikely because of the price and the fact that nobody seems to be able to tell me if I can upgrade if I only buy the 1TB version)

MacPro plus LaCie - Biggest S2S with SATA II 3Gb/s PCI-Express Card 4E

MacPro plus Sonnet: Fusion D800RAID - 8-Drive SATA Storage System with PCI Express RAID Controller (6TB version)

MacPro or iMac or Mac Mini plus external HDDs (Beowulf-Method as like to call it, either USB Hub or FireWire Daisy-Chain)

MacPro plus home build storage cluster (i have two old Athlon Socket A computer lying around, so those would be converted and running linux)

Additional to this "basic" set up we would add a Apple TV and Airport station and the software required (Handbrake, Mactheripper)

I know this is crazy. I have been trying to tell my dad that it is insane to say the least and could cost an enormous amount of time and money, but he is so stubborn I can't help it. I have goggled and researched a lot and from what i saw the best options is the RAID systems, but those are also expensive.

The MacPro would have the following specs:

2x 2.0 Ghz Xeon (I doubt i'll ever need the .66 extra for encoding a movie, I have the 15 minutes for that)
2GB RAM
250 GB standard (i would add another 500 for the music)
FiberChannel card (only with XServe, the other two options provide the required cards)

The iMac:

24"
2.33GHz Intel Core 2 Duo
2GB 667 DDR2 SDRAM - 2x1GB
500GB Serial ATA Drive
NVIDIA GeForce 7600 GT 256MB SDRAM

The Mini:

Superdrive Version with 2 GB RAM

So what are the opinions, other then it is crazy... I am not stupid enough to realize that myself.

To the Mods: Please move the thread if in the wrong section. I am a bit confused as to where to put it.

Cheers,

Biest

[mduell]

Wow, and I thought my 1TB/200 movie collection was getting big.

First off, compared to the movies, the music is going to be rounding error. Assuming 60 minutes per CD, even at 256kbps you're only looking at 110GB.

Second would be to pick the file format for the movies, since it will determine the size: The whole FrontRow/AppleTV/iTunes/Quicktime stack doesn't support raw dvd rips (either VIDEO_TS folders or single MPEG2-PS files), so that's out. I think the best approach would be to Handbrake them at a good bitrate (2900kbps video + 160kbps audio puts you right around 3Mbps total). That puts you at 2.64GB per 2 hour film, or about 2.33TB for 900 films.

Next up, storage. You don't really need high performance storage on the way in (ripping/encoding DVDs) since you're limited by the encoding speed there, or the way out since we're only playing 3Mbps streams. You definitely need some level of protection; ripping all those DVDs is going to take a long time and it would suck massively to lose it. I think a Drobo is a good balance of price/performance for you, and it has protection against drive failures. Drop in four 750GB* drives, and you're looking at $1431 for 2.05TB. The really nice part about the Drobo is that you can upgrade the capacity on the fly, so when you get close to filling those 2TB, start swapping the 750s for 1TB (or larger) drives and you'll have the additional capacity online immediately; none of the other products you linked to would allow this. Or you could buy another Drobo. Either way that's a decision to be made down the line after you've ripped 800 DVDs.

Now it's time to think about the head-end computer. Again, high performance isn't really needed; a more important consideration is a computer that you don't mind having on all the time. I think a mini is the way to go; from there you can share with the rest of the computers in the house (via AFS or NFS or SMB) and the AppleTV (via iTunes).

You may want to pick up another box (in addition to all your personal machines and the head-end) just for ripping/encoding. A Mac Pro (2.66Ghz actually makes sense, given the overall cost of the machine) makes sense here, as does another mini depending on your budget and timeframe. Or if you're comfortable with PCs, after the Intel price drops in August you'll be able to build a quad 2.4Ghz Core 2 Duo box for well under a grand that would be great for ripping/encoding.

Depending on how much space you have under your TV, you could stick the Mac mini + Drobo under there and skip the AppleTV altogether. Another thing to consider is buying a UPS (uninterruptable power supply), so the system will stay up through brief power outages and give you time to shut down during longer ones. Losing power abruptly while writing data is a good way to corrupt the contents of your drive. Even a smallish (~700VA) unit will be fine for a Mac mini and Drobo.

Links to buy:
Newegg.com - data robotics drobo External USB 2.0 Interface Robotic Storage Array - Retail
Newegg.com - Seagate Barracuda 7200.10 (Perpendicular recording) ST3750640AS 750GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive - OEM
Newegg.com - HITACHI Deskstar 7K1000 HDS721010KLA330 (0A34915) 1TB 7200 RPM 32MB Cache SATA 3.0Gb/s Hard Drive - OEM

* At current prices, 4x500GB+Drobo is 64 cents per GB, 4x750GB+Drobo is 63 cents per GB, and 4x1TB+Drobo is 70 cents per GB, so I think 750s are the way to go (given the ease of future upgrades), but 1TB isn't a bad option. Or you could just buy two 750GB drives now, rip 260 movies, then buy another hard drive of whatever capacity makes sense at that time.

[Biest]

Thanks for the reply. I am just a tad bit confused about how you get your price per gig. As far as i have calculated a 500 is .24 or are you calculating the price of a second drobo into that?

Newegg.com - Seagate Barracuda 7200.10 (Perpendicular Recording) ST3500630AS 500GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive - OEM (for reference)

I have already told my father about and he refuses to consider it because of "bad" reviews, even though I can't find any. I am going to try it again, but at some point I am going to tell him that he can plan it himself.

To get back to the topic. The drobo option would give me this (as I want to go all Mac... I am sick and tired of WIndows and my Mom is supposed to use it as well, so Linux is not the best, esp. when I am only home for 3 weeks a year... research and college keep my busy):

MacPro for the ripping, encoding
MacMini as the distribution station
Apple TV as the receiver, or maybe skip the mini for that?

About the music. I dunno how mental my dad will go with the iTunes music store if he can access it, and you can never have enough storage for backup etc.

Thanks again.

edit//

The whole system will be in europe. so a UPS isn't really necessary.

[mduell]

Originally Posted by Biest 

Thanks for the reply. I am just a tad bit confused about how you get your price per gig. As far as i have calculated a 500 is .24 or are you calculating the price of a second drobo into that?

Newegg.com - Seagate Barracuda 7200.10 (Perpendicular Recording) ST3500630AS 500GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive - OEM (for reference)

4x500GB (4*$115) + Drobo ($500) = $960 for 1500GB, which is $0.64/GB.
4x750GB (4*$230) + Drobo ($500) = $1420 for 2250GB, which is $0.6311/GB.

This assumes you fill it out at the start... of course you don't have to, you can start with 2x500, add a 750 later, add a 1TB later, and then swap one of the 500s for a 1.5TB even later. You always 'lose' (for protection) the capacity of the largest drive in the box.

Originally Posted by Biest 

I have already told my father about and he refuses to consider it because of "bad" reviews, even though I can't find any. I am going to try it again, but at some point I am going to tell him that he can plan it himself.

Consider what?

Originally Posted by Biest 

To get back to the topic. The drobo option would give me this (as I want to go all Mac... I am sick and tired of WIndows and my Mom is supposed to use it as well, so Linux is not the best, esp. when I am only home for 3 weeks a year... research and college keep my busy):

MacPro for the ripping, encoding
MacMini as the distribution station
Apple TV as the receiver, or maybe skip the mini for that?

Yes.

Originally Posted by Biest 

About the music. I dunno how mental my dad will go with the iTunes music store if he can access it, and you can never have enough storage for backup etc.

Again, music is rounding error compared to movies.

Originally Posted by Biest 

The whole system will be in europe. so a UPS isn't really necessary.

Have the Europeans come up with a magical system for preventing mother-nature/human/capacity induced blackouts?

[Biest]

Originally Posted by mduell 

4x500GB (4*$115) + Drobo ($500) = $960 for 1500GB, which is $0.64/GB.
4x750GB (4*$230) + Drobo ($500) = $1420 for 2250GB, which is $0.6311/GB.

This assumes you fill it out at the start... of course you don't have to, you can start with 2x500, add a 750 later, add a 1TB later, and then swap one of the 500s for a 1.5TB even later. You always 'lose' (for protection) the capacity of the largest drive in the box.

Thanks

Originally Posted by mduell 

Consider what?

A drobo. I had the idea a while ago and he rejected it

Originally Posted by mduell 

Have the Europeans come up with a magical system for preventing mother-nature/human/capacity induced blackouts?

Well the only blackouts we have had are induced by human error. And mother-nature usually shows her face before striking to you turn things off

[chatam]

I am quite charmed by the Drobo thing, mainly because of the lack of restrictions in the choice or addition of hard disks sizes and types, but one aspect is unclear to me.

How is the security of the data achieved in the event of disk failures? It appears that no redundancy or parity data is provided, nor a spare space, and no explanation is available on the Drobo site.

Anybody knows about this ?

[ninahagen]

A note on sound quality.

iTunes as your main music source runs against any sense of good sound. MP3s will not even approach the sheer sound quality of even a basic audiophile set up, CD or vinyl.

Are you going to encode all your albums? (I hope not)
What amplification will you be using? Speakers?

nina

[mduell]

Originally Posted by chatam 

I am quite charmed by the Drobo thing, mainly because of the lack of restrictions in the choice or addition of hard disks sizes and types, but one aspect is unclear to me.

How is the security of the data achieved in the event of disk failures? It appears that no redundancy or parity data is provided, nor a spare space, and no explanation is available on the Drobo site.

Anybody knows about this ?

Yea, the explaination on their website kind of glosses over how the protection works ("Automatically protects your data with no setup or configuration... Knows where your data is stored on the disks so Drobo knows the most effective way to protect your data and repair itself in case of a disk failure or data corruption"). The data sheet does a little better in explaining what you're protected against.

Drobo protects all your data against the loss of 1 disk (and a second/third, if your data would fit on the remaining disks, and it has time to rearrange your data after the first/second failure) using parity data. It's a lot like RAID5, but with more automation and flexibility.

[chatam]

Thank you, mduell, for the hint.

In fact, while I am currently using Raid 5 (slight saving on disc space) and Raid 1 (severe loss on disc space), the possibility of using differently sized disc and to add them when needed is quite appealing, provided the 100% disc failure security is maintained.

Chances of a double disc failure at the same time appear to be quite thin.

[besson3c]

Biest,

You'll get get more value for your money looking at MythTV based options. Plus, you can do a lot more stuff with MythTV such as use it as a PVR, strip out commercials, and install many of the plug-ins that are available to extend its functionality even further.

AppleTV has the eye-candy and ease of setup, but MythTV has the features and capabilities.

[Biest]

Camino had a glitch here.... sry didn't see it

[Biest]

Originally Posted by mduell 

Yea, the explaination on their website kind of glosses over how the protection works ("Automatically protects your data with no setup or configuration... Knows where your data is stored on the disks so Drobo knows the most effective way to protect your data and repair itself in case of a disk failure or data corruption"). The data sheet does a little better in explaining what you're protected against.

Drobo protects all your data against the loss of 1 disk (and a second/third, if your data would fit on the remaining disks, and it has time to rearrange your data after the first/second failure) using parity data. It's a lot like RAID5, but with more automation and flexibility.

This might help with this question ( Data Robotics, Inc. | Drobolator ).

From what I understand the space of the largest disk is lost. So incase one of the other drives fails the data is somewhere on the other disks and is then copied to the the "empty" disk. This saves you from singular data loss.

About 2+ failing at the same time. I have never really seen that except in cooperate setting and as we alll know those are in the 1000+ disk range.

I'll look into mythTV, I am guessing the AppleTV has nearly died right now cause of the Minis capabilities

Originally Posted by ninahagen 

A note on sound quality.

iTunes as your main music source runs against any sense of good sound. MP3s will not even approach the sheer sound quality of even a basic audiophile set up, CD or vinyl.

Are you going to encode all your albums? (I hope not)
What amplification will you be using? Speakers?

nina

The encoding will be via lossless audio and to be honest by dad has vinyls, but I haven't heard him play 1 in 15 some odd years. I have thought about the possible problems and am considering using the MacPro via a sound card as the output for the music. I have to look into it. Currently the sound setup are two large speakers by Nexus (as far as I can remember) and an amp by Denon (don't quote me on that and those are also in the 10+ year range of usage. There are a couple other components left of my dad olds setup, but those are mostly older then I am. Maybe we also might upgrade to optical equipment at some point, since the mac is set up for that

[ninahagen]

Originally Posted by Biest 

The encoding will be via lossless audio and to be honest by dad has vinyls, but I haven't heard him play 1 in 15 some odd years. I have thought about the possible problems and am considering using the MacPro via a sound card as the output for the music. I have to look into it. Currently the sound setup are two large speakers by Nexus (as far as I can remember) and an amp by Denon (don't quote me on that and those are also in the 10+ year range of usage. There are a couple other components left of my dad olds setup, but those are mostly older then I am.

If he hasn't played the LPs in 15 years, what on earth do you want to encode them for?

Do you want a great sound experience, or are you mostly interested in the technical challenge of organizing all that media?

[OreoCookie]

I don't think sound quality will be an issue here and you will not hear a difference to an mp3, ogg or aac encoded at a sufficiently high bitrate. If you want to hear the subtle differences, you need first-rate equipment and first rate source material and trained/good ears.

I think you'll be fine with the MacPro's stock soundcard and 160+ kBit VBR aac. The same goes for movies, newer algorithms are a lot better than MPEG2 and it's quite probable that the artefacts you (at lower bitrates) see are due to MPEG2 rather than having chosen too low a bitrate. I usually encode my movies at 1000 kBit at full resolution and I cannot see the difference to my DVDs.

I've tried MythTV in a personal project, but it doesn't come nowhere near Front Row in usability (although the feature set is smaller).

Concerning storage, I haven't heard of Drobo, but it seems like a great product to me. Certainly the cheapest hardware-based RAID5-like storage solution I've seen. You will always lose storage capacity in favor of security, but if you have combine more than two harddrives and need reliable storage, there is no way to prevent that from happening.

There is a more serious limitation of Drobo, or rather USB: you can only have volumes which are no larger than 2 TB, i. e. if you use 4 1 TB harddrives, you'd have to split your capacity into two volumes. An XRaid seems unpractical for several reasons: (i) it's very expensive and (ii) it's very loud. You can expand it with your own harddrives, but you need to buy drive trays from Apple (this is common practice in the storage market). A WiebeTech RT5e seems like an alternative whose price is somewhere in the middle. However, this would exclude a Mac mini as you would need an eSATA connector.

So I would propose the following setup: get yourself a storage/encoding station. Put it somewhere far away as it will probably be noisier. It doesn't have to be a MacPro, it can be anything with a reasonable amount of processing power (probably you will be the limiting factor as you have to manually change DVDs and such). Then get a Mac mini/iMac for your living room. Connect to your storage server/encoding machine via your network of choice. For mp3s, you need very little bandwidth, even wireless networks more than suffice, for movies, you will probably need a wired network for satisfactory results.

I would reckon completing your project will take several years, so don't underestimate the logistics of all this: categorizing your movies. It'll be much easier with songs as iTunes will take care of that. And your Mac mini/iMac can help encoding.

[mduell]

Originally Posted by OreoCookie 

There is a more serious limitation of Drobo, or rather USB: you can only have volumes which are no larger than 2 TB, i. e. if you use 4 1 TB harddrives, you'd have to split your capacity into two volumes.

True, but the wonderful thing about the Drobo compared to a 'dumber' RAID box (at least, in this price space) is that the size of the volumes is dynamic. So with four 1TB drives you have 3TB of usable space, in however many volumes you want, so long as the total size is only 3TB. Put another way, if one of your volumes as 1.5TB in it, the other volume can also have 1.5TB in it; you're not limited to a fixed capacity of 2TB in the first volume and 1TB in the second.

[OreoCookie]

No, sure, the Drobo is a very nice product, dead simple to use and provides redundancy. To be honest, I haven't heard of it before … 

[Biest]

Originally Posted by OreoCookie 

No, sure, the Drobo is a very nice product, dead simple to use and provides redundancy. To be honest, I haven't heard of it before …

Thanks for the reply and other ideas.

The main problem I have with a RAID 5 option is the rebuilding the array. I am only home 3 weeks a year and i somehow doubt that my dad has the time to rebuild an array... And the simplicity is also good, since I think even somebody else could perform an upgrade.

I'll think about the RAID option, but that is I think for later arrays.

Originally Posted by ninahagen 

If he hasn't played the LPs in 15 years, what on earth do you want to encode them for?

Do you want a great sound experience, or are you mostly interested in the technical challenge of organizing all that media?

I am not going to rip the vinyls... that was never an option. I'll just rip the CDs

[OreoCookie]

Originally Posted by Biest 

Thanks for the reply and other ideas.

The main problem I have with a RAID 5 option is the rebuilding the array. I am only home 3 weeks a year and i somehow doubt that my dad has the time to rebuild an array... And the simplicity is also good, since I think even somebody else could perform an upgrade.

I'll think about the RAID option, but that is I think for later arrays.

But what exactly is the alternative here?
If a drive fails in a RAID5 (or a Drobo, for instance), your dad will be able to continue to use his media center. If in any non-redundant solution the drive fails, your dad's media center is gone. You'd have to start ripping everything from scratch, etc. RAID5 arrays also usually heal automatically: you just need to replace the drive that has failed and viola.

Of course if a drive in a RAID needs to be replaced, your dad has to do it. But if he needs to replace the oil in his car, he needs to do that, too. Otherwise his engine might be damaged and he can't use his car anymore: such a project is technically demanding, make no mistake about it.

[Biest]

Originally Posted by OreoCookie 

But what exactly is the alternative here?
If a drive fails in a RAID5 (or a Drobo, for instance), your dad will be able to continue to use his media center. If in any non-redundant solution the drive fails, your dad's media center is gone. You'd have to start ripping everything from scratch, etc. RAID5 arrays also usually heal automatically: you just need to replace the drive that has failed and viola.

Of course if a drive in a RAID needs to be replaced, your dad has to do it. But if he needs to replace the oil in his car, he needs to do that, too. Otherwise his engine might be damaged and he can't use his car anymore: such a project is technically demanding, make no mistake about it.

I understand that... It is just that it saves a lot of time for us with a drobo for the start. If we ever get to the point where we get the HDDs and what not together to rip everything, I am considering a RAID5 based system. Just out of the sake that I can build a server out of that.

A minor think is also that my dad spends most of the week on the road between Europe, US and Asia. I am being serious here. During the week he is usually at home for about 3 days, if that, so I am guessing that a drobo for now is the cheapest and adequate solution for now. Until we decide to finally remodel the house, so we can rewire it completely and maybe be able to create a proper server.

I am not letting the RAID idea get out of my mind. You said earlier, this is going to take years, so till then I hope we will find a better solution or switch to a RAID5. I would personally then take over the drobo cause I need to keep my personal research data somewhere as well, if i ever get into grad school... (and for those who are curious... observational and instrumental astrophysics is my field)

[OreoCookie]

If a drobo does what it promises it does, it gives you some redundancy -- probably enough for what you want to do. It really sounds like an adequate solution.

I guess that the bottleneck will be encoding the DVDs and CDs -- something you need to do by hand.

PS Glad to help a fellow physicist

[Biest]

Thanks for the help. Anyway... yeah it will be fun as soon as grad school hits.. i am somehow hoping i can get into UIUC from their dark matter survey in chile, so i can use that drobo properly

[chichow]

UIUC? as in Shampoo-Banana?

[Leonard]

Originally Posted by OreoCookie 

No, sure, the Drobo is a very nice product, dead simple to use and provides redundancy. To be honest, I haven't heard of it before … 

It's a newer machine I think. Cali Lewis mentioned one in her GeekBrief podcast... she bought one. This is only the second time I've heard of it, and I haven't had a chance to check it out. Sounds cool, but I've never cared for USB for external HD storage, Iv'e preferred Firewire. But USB 2.0 drives are common.

[Biest]

Well I had thought about directly linking the MacPro to the TV, and it turns out that you need a X1900XT to use the apple DVI to Video adapter. Which dumbfounds me a bit, since it works with the on-board on the mac mini, but not a 7300GT. Any thoughts on that?

[mduell]

Originally Posted by Biest 

Well I had thought about directly linking the MacPro to the TV, and it turns out that you need a X1900XT to use the apple DVI to Video adapter. Which dumbfounds me a bit, since it works with the on-board on the mac mini, but not a 7300GT. Any thoughts on that?

Probably saved 30 cents a board by skipping the circuitry they'd need for video out... it somewhat surprises me that they'd do that on the Mac Pro, since the 7300GT in the iMac supports the adapter. Perhaps it's to up-sell people to the X1900.

[Biest]

Originally Posted by mduell 

Probably saved 30 cents a board by skipping the circuitry they'd need for video out... it somewhat surprises me that they'd do that on the Mac Pro, since the 7300GT in the iMac supports the adapter. Perhaps it's to up-sell people to the X1900.

Thanks for the reply.... So I have finally decided on what to recommend...

MacPro 2.66 Ghz with a 23" (so it can replace a 5 year old dell laptop)
MacMini (1GB RAM)
Drobo connected to mini
4x 750GB Seagates

Maybe:

Elgate EyeTV 310 (as we have satellite) connected to mini

If anybody has any more thoughts on this.. please go ahead

[mduell]

Buy the LCD from someone else (like Dell) if you're not stuck on the looks of it's case.