MacNN Forums (
-   Applications (
-   -   How to merge 12 HD with many duplicates (

SunSeeker Dec 26, 2013 05:15 PM
How to merge 12 HD with many duplicates
Happy XNY

I have a problem where I have many usb drives including a few NTFS that I need to consolidate to one drive.
Total data stored is in excess of 6GB but many files are duplicated across the drives and many exist multiple times on each drive.

I have tried copying them all to a large 8TB RAID drive, but while copying the second or third drive there is always some sort of failure (I believe the enclosure may be faulty):brick:
The aim of this method was to get them all into one physical volume and then use some sort of duplicate finder to clean up.

Having failed many times I'm wondering about software that might copy unique files from one disk to another.
(then I might get this done with a single smaller drive in lieu of the potentially faulty RAID)

It would first copy disk 1 in its entirety to an empty destination drive.

Then it would copy disk 2 to the same drive, but only files and folders which didn't now exist on the destination would be copied.

Next it would copy disk 3 to the same drive, but again files and folders which didn't yet exist on the destination would be the only files copied.
and so on...

Does such software (with a decent GUI) exist?
turtle777 Dec 26, 2013 06:03 PM
You could try this:

DeltaWalker for Mac OS X: Compare/Merge Files, Synchronize Folders; Two and Three-Way

I would think you need to start with HD 1 and 2, merge and eliminate, then mirror the result against HD 3 etc...

I doubt there is a off-the-shelf software that can compare 12 HDs concurrently.

SunSeeker Dec 26, 2013 07:06 PM
Thanks for that. This might just work.
I would copy the first drive to my empty destination drive.
On completion I would then connect one of my other drives and compare to my destination.
Using the Deltawalker 'Copy selected from first to second' feature I would copy ONLY the files that were currently missing from the destination.
I would then repeat for drives 3 through 12 hoping I don't experience issues like I have in the past (corruption on the destination drive)

I'm assuming the previously used destination drive is faulty, but could it perhaps be a compatibility issue between NTFS and OSX or Corruption on the original drive?
How could I stress test my RAID enclosure?
turtle777 Dec 26, 2013 08:21 PM
By default, OS X can only READ NTFS drives, not write to it.
To write to NTFS, you would need an extension like Paragon NTFS. Paragon NTFS for Mac OS X software

Maybe that was the issue ?

SunSeeker Dec 27, 2013 04:54 AM
No, I was aware of the NTFS limitation and I wasn't trying to copy or sync anything to the NTFS drive.
Perhaps OSX does though, during large copy operations.
All times are GMT -4. The time now is 02:22 PM.

Copyright © 2005-2007 MacNN. All rights reserved.
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

Content Relevant URLs by vBSEO 3.3.2