[ajprice]

I thought I would post this here rather than Networking, to get a wider audience and hopefully more help. If it needs to be moved, no problem.

I work for a small publishing company that has experienced significant growth in the last three years. Our IT infrastructure is at best 'quaint' and at worst laughable.

Sales, events, accounts and circulation depts work on PCs. Editorial and Production work on Mac. We all have our own Mac, which is not linked to anything else — a brief fire in the editorial office could therefore wipe out 20 years of historical backups as well as destroy the company.

My enviable task is to get the Ed/Prod department up an running with a server (I am fed up of looking for USB flash drives to transfer docs about), an offsite backup and remote access (some of the editors travel quite a bit nowadays.)

I am not a techy person really, so any advice on the above would be most welcome, either direct or by referral! I am working with a theoretical budget of <£5k, which will no doubt influence my options dramatically. This is becoming 'business critical' and I'm looking at getting some movement within the month.

FYI my mag is a b2b focusing on additive manufacturing, rapid prototyping and 3D printing and has been around since 1992. Some of you might find it interesting — TCT Magazine - Rapid Product Development, Prototyping and Manufacturing Technologies — subs are free in the UK.

Thanks.

[Phileas]

We use dropbox and box.net for most of our files. Automatic, continuous backup plus remote accessibility.

It's up to the individual to decide whether or not cloud based services are secure enough. We, after talking to both providers, decided that they were and have never looked back.

It would take too long to explain the services here, so take a look for yourself: box.net and dropbox.com

The only files we still keep local are video files, where the amount of data produced is simply too large to be moved via the net.

[bstone]

Any backup onsite is not a backup, it's a copy. Keep that in mind and live by that rule.

Simplest way to backup is use Mozy. $5 a month for unlimited space. Not bad, eh?

You need a server, OS X Server (tho you can do it with Linux or Windoze) to set up a domain, password, authentication, etc. I would set your OS X Server to backup everyone home directory off site every night.

The end.

[Phileas]

Originally Posted by bstone 

You used to need a server, OS X Server (tho you can do it with Linux or Windoze) to set up a domain, password, authentication, etc. .

Hah


(Just ribbing you. There are perfectly good reasons to maintain your own server of course.)

[bstone]

Originally Posted by Phileas 

Hah

I didn't write that. I said the truth- you don't need an OS X Server in order to authenticate the OS X clients. You can had an Active Directory or Open Directory through linux. Take your pick.

[Laminar]

Whoosh.

[OreoCookie]

Can you tell us more about your requirements? How much storage do you need? Do you want to have a central user database? What is your networking infrastructure (GBit ethernet, wireless LAN, etc.)? What is your internet connection? What hardware do you have (server + clients)?

[Nivag]

If you added up everyones work in progress files, what's the total?

As for transferring files around, can't you switch on file sharing on each machine? One of my clients doesn't have a server and each designer keeps their files local to them but the work drive is setup to be shared so the other designers have access. Not ideal, but they do backup each machine each night to tape then offsite a set when they remember.

[Moderator]

If I were in charge I'd buy a Mini Server, with firewire raid if necessary (depends on you storage needs) I'd back that up to an external disk (mybook or whatever).

Then I'd setup another machine offsite, can be anywhere. I'd open up the afp ports on that offsite router and then I'd set the mini server in the office to backup to my offsite location nightly.

So youd have mini server backing up onsite to disk thru time machine and offsite to disk nightly using a different backup utility like chronosync. The one thing is for that first offsite backup you will need to physically bring the data over for a dump. after that you can run backups nightly and it will just send what has changed. That initial copy would probably be too much to send through the internet like that.

You could also sign up for mozy or something but that takes a long time to get up to date, is slow, and will be a pain to get back if you need you data fast.

You can PM me if you need more info.

[bstone]

Mozy is only slow the first time. I have a friend who recently did 85gb and it took 3 days. After that, however, it does nightly incrementals and it super fast and totally silent. As far as getting stuff back, huh? I've used it a bunch and it was never a pain in the ass.

[besson3c]

I think you need to analyze your storage needs, associated costs, and come up with a comprehensive disaster recovery plan that includes rehearsing for these scenarios.

As sound as Moderator's advice is, no backup solution is one-size-fits-all. Some companies might have trouble relying on a random offsite machine receiving data via AFP. For starters, where will it be hosted? What happens when that machine is offline or having problems (will you be aware of this)? How do you recover data from this machine? How much will this cost in storage/bandwidth/etc.? Security and privacy are certainly also a concern... Unless you are tunneling AFP through an SSH tunnel this data will be sent in the clear.

What you want to avoid is a workflow that can fail silently without you knowing about it before it is too late. You also want to avoid the stress of trying to figure out how to recover data from this backup solution when the ganja hits the fan.

The general premise of an onsite copy + offsite backup is generally a good idea. For onsite copy there are a plethora of solutions I'll leave you with, what was been recommended is fine.

The offsite backup is a little more complicated. Storage on general shared hosting/virtual private server sort of solutions can be expensive. If you aren't familiar with Linux or Windows I'm pessimistic about the prospects of finding a good cost effective Mac-based solution for you. I'd be much more inclined to look at cloud based solutions such as Amazon S3, Dropbox (which uses Amazon S3), and the others (Mozy, Carbonite, etc.) Amazon S3 solutions are good choices if the costs make sense for you, since Amazon's infrastructure is excellent and is relied upon by many companies.

The reason why it is hard to just point you at one of these is because Amazon S3's costs are metered - you pay for how much you use, both in bandwidth and actual storage. Prices in services like Dropbox are based on these costs with extra padding for whatever bandwidth you use. If you are trying to come up with the most cost effective solution, a pure Amazon S3 solution and their reduced redundancy storage might be best, but of course Dropbox offers what I understand to be an excellent interface and great ease-of-use.

So, there are a lot of variables to consider here.

[ajprice]

Thanks for the input so far guys, all very helpful. I'll try to answer questions as best I can.

Can you tell us more about your requirements? How much storage do you need? Do you want to have a central user database? What is your networking infrastructure (GBit ethernet, wireless LAN, etc.)? What is your internet connection? What hardware do you have (server + clients)?

A backed up collected magazine folder and PDF print files usually takes 2 CD-R's, call it 1-1.5GB per magazine, this year we'll be publishing about 50 issues all in of the 6 magazines we produce. There is a wired ethernet system through the building, and a wifi router. The internet connection is broadband, speed isn't that great at the moment (probably lucky to get 5mbps on a speed counter), but a faster connection is going to get looked into along with the server and other 'wish list' items. Computer hardware is Mac Pro's and high end iMacs for production (one designer also has an iMac at home for work), editorial is a mix of iMacs and MacBook Pro's, sales are PC laptops, with a few iMacs. There's no central server at the moment, file transfers are done through USB memory sticks, emails and OS X public folder drop box.

If you added up everyones work in progress files, what's the total?

As for transferring files around, can't you switch on file sharing on each machine? One of my clients doesn't have a server and each designer keeps their files local to them but the work drive is setup to be shared so the other designers have access. Not ideal, but they do backup each machine each night to tape then offsite a set when they remember.

As above with the magazine production files, at a guess roughly 50-75GB of files will get backed up this year over all the magazine issues, and other publications like show guides etc. Completed magazines are backed up to CD-R, and the Macs use Time Machine backups to portable drives, not sure of the PC backup system, but sales do have an online sales database called Capsule (?)

[OreoCookie]

(1) I agree with the others when stressing the importance of off-site backups. Online services are helpful if your internet connection (the upstream!) is fast enough to handle the data. I like Dropbox, but since the storage offered is too small, it's not helpful to me as a full backup solution (I use it for current work documents; even if I were to get 100 GB, that still wouldn't be near enough for all of my data). If online services such as mozy or Amazon S3 don't fit your needs, perhaps a rented server can solve the problem?

(2) Server hardware. The Mac mini server is great, but it doesn't offer anything in terms of fail safety. Speed-wise it's plenty. I'd probably re-use the oldest Mac Pro you have (get a high-end Mac mini for that person instead, he/she will love you!) and put in more storage. I'd probably move to a central user database, at least for the desktops. Things like print services and such can also be set up easily. Since you don't have a resident

(3) What is the total amount of data you want to store/your users use? (Not per year or something, how much data needs to be retained in total?)

If you can make do with 6-8 TB of storage (the storage you buy should be at least twice as big as the amount of data you have now!), you can just fill all bays of the Mac Pro with 2 TB drives. Then you can get a Drobo S to back up your data.

If your storage needs exceed said 6-8 TB, you need to invest in a professional storage solution such as a Promise RAID (the expensive ones with 16 drives) or a Drobo Pro (8 drives, expandable to 2x8 drives). If you get, say, a Drobo Pro, you can connect it to the Mac Pro's second GBit Ethernet port and use iSCSI (SCSI over ethernet, very fast). The advantage of the Drobo Pro is that you can add storage as you need it and it's reasonably fast.

There are other storage solutions, if you give us your specific requirements (total storage, how much of that data changes, importance/redundance of data, etc.). You can get a much cheaper solution if you look into these questions before buying.

You can also build an iSCSI storage box (which you can then connect to your server) with relative ease, but the problem I see is to get your hands on quality hardware. You can, for instance, buy quality pc hardware with lots of space for harddrives, sufficiently many sufficiently large harddrives and then set up an OpenSolaris or FreeBSD server using ZFS. It's not that hard to do, but certainly not as easy to set up as a Drobo, for instance. I'd be realistic about that and prefer more expensive solutions that are easier to maintain.

[besson3c]

Originally Posted by OreoCookie 

You can also build an iSCSI storage box (which you can then connect to your server) with relative ease, but the problem I see is to get your hands on quality hardware. You can, for instance, buy quality pc hardware with lots of space for harddrives, sufficiently many sufficiently large harddrives and then set up an OpenSolaris or FreeBSD server using ZFS. It's not that hard to do, but certainly not as easy to set up as a Drobo, for instance. I'd be realistic about that and prefer more expensive solutions that are easier to maintain.

Or Solaris. OpenSolaris is what Fedora is to Redhat - generally more experimental.