Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > macOS > Sparsebundle band size

Sparsebundle band size
Thread Tools
reader50
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Jan 12, 2023, 05:00 PM
 
By default, Time Machine or Disk Utility creates sparse bundle images with a band size of 8 MB. In the past, macOS had problems with images that went significantly beyond 50,000 bands - they're all stored in a "bands" folder inside the bundle. So you have a folder with 50K+ files, and a stable size limit somewhere over 400 GB.

This stability problem seems to have been fixed - under normal operation at least. When something goes wrong, and the image won't mount within the time limit (seems like 90 seconds), you're stuck up a tree. Disk Utility won't repair unless you can first mount an image. I haven't found a method to repair a sparsebundle without mounting it.

When Time Machine stores to a remote server, it always creates a sparse bundle image, so it's working with HFS+ internally. Not whatever native format is used by the server. And I've lost several Time Machine backups over the years because a sparsebundle became unstable, and there was no way to repair it. I'm talking about images in the 2-12 TB range.

Image just lost: 2.4 TB, ~340,000 bands allocated.
A different image, stable so far: 12 TB, ~650,000 bands allocated.

The obvious solution is to declare an image with a larger band size. So the OS isn't fighting a folder that could hold over a million bands, on top of fixing a filesystem problem. Go to 256 MB or 1 GB per band, so the file count remains below 50K files. You can create images with larger bands using hdiutil in Terminal, or use a utility. I've created such bundles, and converted existing TM bundles to a copy with larger band size. In each case, Time Machine rejects the new bundle, and starts creating a new one. With 8 MB bands of course.

• hdiutil convert (requires an outfile. It will not convert an image in place. So it always creates a modified copy.)
• hdiutil compact (does not allow the band size to be changed.)
• hdiutil resize (does not allow the band size to be changed.)
• hdiutil attach -noverify (forgot to test this before giving up.)

I've used tmutil commands inheritbackup and setdestination - neither has worked.

• tmutil setdestination won't accept a remote disk image - only a network folder.
• tmutil inheritbackup appears to reject my copied TM images because they're read-only: a normal state of affairs for a TM backup image.
• TM prefs pane does not even show mounted images, so you can't create an image, mount it, and select it.
• Using TM prefs pane to select everything at once (TBs to back up) does not make TM create image bundles with larger band size.

The Time Machine prefs file is at /Library/Preferences/com.apple.TimeMachine.plist but does not contain the default band size for images.

------------------------
Right now, I'm watching TM create a new network image. It will take several days. Fortunately, I've got multiple backups, so I'm still covered. But what a waste of time, especially as it's happened multiple times before.

For future reference, anyone know how to make TM create or accept a remote image with bigger band sizes? Or how to repair an image bundle without mounting it?

Things I didn't try before giving up:
• Force-mounting the image for repair with "hdiutil attach -noverify". But mounting a damaged image can make the OS unstable.
• tmutil backup image (to a custom image with bigger bands). I expect this to fail with a remote image. But maybe I could create it locally, let TM use it, then copy it to a remote volume. Then use "tmutil inheritbackup" to select an already-accepted image at the new location.
• Test DiskWarrior on a remote sparsebundle image. It will fix unmounted volumes, maybe it can handle unmounted images.
     
ghporter
Administrator
Join Date: Apr 2001
Location: San Antonio TX USA
Status: Offline
Reply With Quote
Jan 13, 2023, 08:08 PM
 
I’d worry about force-mounting a file unless you had more than one copy of it. Not because theOS might glitch, but because your sole back up file might be corrupted in the attempt the OS makes to mount it. File systems like to adjust boundaries and stuff when they do things without verification.

I think Disk Warrior is worth a try. At worst it won’t work, but IIRC, DW tends to give useful, understandable messages - these could point out what’s fudged with the image.

Glenn -----OTR/L, MOT, Tx
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Jan 18, 2023, 09:12 PM
 
I have two network backups.

• QNAP NAS - this is the image that crapped out. Probably caused by a crash-on-sleep of my Mac during a backup.
• A RAID array hosted from an older MBP. This is the older image, going back to 2019.

I tried copying the MBP image over to the NAS. It eventually worked, though the copy took 2 days. Then I had to Verify the image with Disk Utility, which took ~20 hours. Add in a couple reboots of my Mac Pro. tmutil inheritbackup finally accepted the copied image, giving me extensive network backups on both again. Technically an upgrade, as the new NAS image goes back farther than it did before.

I still haven't managed to get TM to accept an image with bigger band sizes, so all backups remain more vulnerable to corruption than they should be.
     
   
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 01:16 PM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,