Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > Applications > How to merge 12 HD with many duplicates

How to merge 12 HD with many duplicates
Thread Tools
Mac Enthusiast
Join Date: Apr 2001
Location: Adelaide, Australia
Status: Offline
Reply With Quote
Dec 26, 2013, 05:15 PM
 
Happy XNY

I have a problem where I have many usb drives including a few NTFS that I need to consolidate to one drive.
Total data stored is in excess of 6GB but many files are duplicated across the drives and many exist multiple times on each drive.

I have tried copying them all to a large 8TB RAID drive, but while copying the second or third drive there is always some sort of failure (I believe the enclosure may be faulty)
The aim of this method was to get them all into one physical volume and then use some sort of duplicate finder to clean up.

Having failed many times I'm wondering about software that might copy unique files from one disk to another.
(then I might get this done with a single smaller drive in lieu of the potentially faulty RAID)

It would first copy disk 1 in its entirety to an empty destination drive.

Then it would copy disk 2 to the same drive, but only files and folders which didn't now exist on the destination would be copied.

Next it would copy disk 3 to the same drive, but again files and folders which didn't yet exist on the destination would be the only files copied.
and so on...

Does such software (with a decent GUI) exist?
     
Clinically Insane
Join Date: Jun 2001
Location: planning a comeback !
Status: Offline
Reply With Quote
Dec 26, 2013, 06:03 PM
 
You could try this:

DeltaWalker for Mac OS X: Compare/Merge Files, Synchronize Folders; Two and Three-Way

I would think you need to start with HD 1 and 2, merge and eliminate, then mirror the result against HD 3 etc...

I doubt there is a off-the-shelf software that can compare 12 HDs concurrently.

-t
     
Mac Enthusiast
Join Date: Apr 2001
Location: Adelaide, Australia
Status: Offline
Reply With Quote
Dec 26, 2013, 07:06 PM
 
Thanks for that. This might just work.
I would copy the first drive to my empty destination drive.
On completion I would then connect one of my other drives and compare to my destination.
Using the Deltawalker 'Copy selected from first to second' feature I would copy ONLY the files that were currently missing from the destination.
I would then repeat for drives 3 through 12 hoping I don't experience issues like I have in the past (corruption on the destination drive)

I'm assuming the previously used destination drive is faulty, but could it perhaps be a compatibility issue between NTFS and OSX or Corruption on the original drive?
How could I stress test my RAID enclosure?
     
Clinically Insane
Join Date: Jun 2001
Location: planning a comeback !
Status: Offline
Reply With Quote
Dec 26, 2013, 08:21 PM
 
By default, OS X can only READ NTFS drives, not write to it.
To write to NTFS, you would need an extension like Paragon NTFS. Paragon NTFS for Mac OS X software

Maybe that was the issue ?

-t
     
Mac Enthusiast
Join Date: Apr 2001
Location: Adelaide, Australia
Status: Offline
Reply With Quote
Dec 27, 2013, 04:54 AM
 
No, I was aware of the NTFS limitation and I wasn't trying to copy or sync anything to the NTFS drive.
Perhaps OSX does though, during large copy operations.
     
   
Thread Tools
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Top
Privacy Policy
All times are GMT -4. The time now is 12:44 AM.
All contents of these forums © 1995-2014 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2014, Jelsoft Enterprises Ltd., Content Relevant URLs by vBSEO 3.3.2