Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > macOS > Warning: disk full corrupted catalog structure and thus the whole disk

Warning: disk full corrupted catalog structure and thus the whole disk
Thread Tools
Dr.Michael
Registered User
Join Date: Oct 2003
Status: Offline
Reply With Quote
Oct 8, 2004, 03:37 AM
 
Hi,

this is a warning.
Yesterday I found my Powerbook with 48kB disk space left after I had converted a DVD during my absence.

I cleaned up the clutter, freed 10GB and could work and logout/login without problems. But after a restart the powerbook did not find the drive any more. I started from my external fw backup drive and ran a disk diagnosis with the apple disk utility which terminated on error. So no repair was possible. The OS X partition wasn`t even mountable.

I started from a linux partition to see if I could access my last day work from there but it also failed to mount the OS X drive and reported severe corruption.

I thought these problems were windows only!
After a reinstall of my backup I checked the drive and found everything ok. No hardware problem.

So my conclusion is that the catalog structure can be damaged by software if the drive is full. Please make sure that this does not happen to you (at least not without a backup).
     
cpac
Professional Poster
Join Date: Jul 2001
Location: New York, NY
Status: Offline
Reply With Quote
Oct 8, 2004, 10:14 AM
 
Originally posted by Dr.Michael:
So my conclusion is that the catalog structure can be damaged by software if the drive is full. Please make sure that this does not happen to you (at least not without a backup).
Um - yep.
Less than about a Gig of free drive space can quickly result in corruption problems and the like. Good job having a backup!
cpac
     
Tsilou B.
Senior User
Join Date: May 2002
Location: Austria
Status: Offline
Reply With Quote
Oct 8, 2004, 11:38 AM
 
Originally posted by cpac:
Um - yep.
Less than about a Gig of free drive space can quickly result in corruption problems and the like. Good job having a backup!
This should NEVER happen in a good OS. It's one thing if the computer crashes and unsaved data is lost and/or files that had been saved in that moment are corrupted. But the entire hard drive corrupted? I've never seen that in Windows, I've never seen something like that in Linux, however, I haven't had such problems with Mac OS X, either, and I had an iBook that had less than 50 MB free space for half a year.
     
mishap
Mac Elite
Join Date: Jul 2002
Location: Boston
Status: Offline
Reply With Quote
Oct 8, 2004, 11:43 AM
 
i dont know if i've actually been that low on disk space, but i got to the point where all my user prefs get reset (like default dock settings)...never had a problem with hard drive corruption.
     
cpac
Professional Poster
Join Date: Jul 2001
Location: New York, NY
Status: Offline
Reply With Quote
Oct 8, 2004, 11:50 AM
 
Originally posted by Tsilou B.:
This should NEVER happen in a good OS. It's one thing if the computer crashes and unsaved data is lost and/or files that had been saved in that moment are corrupted. But the entire hard drive corrupted? I've never seen that in Windows, I've never seen something like that in Linux, however, I haven't had such problems with Mac OS X, either, and I had an iBook that had less than 50 MB free space for half a year.
Well here's the thing - if your disk gets truely full, there's nowhere to write down the file catalogue type thingy, which means that it appears your disk is unreadable.

That said, your data should still be there, and be recoverable using appropriate utilities....
cpac
     
absmiths
Mac Elite
Join Date: Sep 2000
Location: Edmond, OK USA
Status: Offline
Reply With Quote
Oct 8, 2004, 12:01 PM
 
Originally posted by cpac:
Well here's the thing - if your disk gets truely full, there's nowhere to write down the file catalogue type thingy
That doesn't make much sense if it is true. The file catalog normally resides on the disk, so room doesn't have to be "made" to write it at system shutdown (or failure). The disk getting 100% full (which BTW has happened to me about a dozen times in the last few months without any corruption, using 10.3.x on a TiBook) processes can get crazy, memory runs out, etc, and that can wreak havoc - but there is no reason for a drive to get corrupted simply because it is full.
     
Millennium
Clinically Insane
Join Date: Nov 1999
Status: Offline
Reply With Quote
Oct 8, 2004, 03:26 PM
 
Originally posted by absmiths:
That doesn't make much sense if it is true. The file catalog normally resides on the disk, so room doesn't have to be "made" to write it at system shutdown (or failure). The disk getting 100% full (which BTW has happened to me about a dozen times in the last few months without any corruption, using 10.3.x on a TiBook) processes can get crazy, memory runs out, etc, and that can wreak havoc - but there is no reason for a drive to get corrupted simply because it is full.
I don't know; cpac's explanation does make sense. That said, I'm surprised that OSX's preflight routines wouldn't have caught something like this. What program were you using to rip the DVD?

Also, can we be certain that the corruption was actually caused by the hard drive being full, and that this wasn't a coincidence of some kind? For example, might you have lost power to the PowerBook, perhaps due to running out of batteries?
You are in Soviet Russia. It is dark. Grue is likely to be eaten by YOU!
     
proton
Senior User
Join Date: Nov 2000
Status: Offline
Reply With Quote
Oct 9, 2004, 11:41 AM
 
Originally posted by Dr.Michael:
So my conclusion is that the catalog structure can be damaged by software if the drive is full. Please make sure that this does not happen to you (at least not without a backup).
I can't see any reason for you to make this conclusion other than them happening close to each other.

On my old Mac (iMac 266, 6 GB Hard Disk) I ran it out space numerous times and no harm ever came to it. I seriously doubt this is due to you running out of disk space, but is instead due to some other problem.

Smells like a coincidence.

- proton
     
tooki
Admin Emeritus
Join Date: Oct 1999
Location: Zurich, Switzerland
Status: Offline
Reply With Quote
Oct 9, 2004, 12:25 PM
 
At about 50MB free space, Mac OS X tells applications that the disk is full, and will refuse to allow them to write files (which is why sometimes the prefs get trashed, because they are prohibited from being written). This is to prevent the directory from having nowhere to be written.

It's likely that something else happened.

That said, it's still a bad idea to let the disk fill completely. It's always wise to leave 5-10% free.

tooki
     
tooki
Admin Emeritus
Join Date: Oct 1999
Location: Zurich, Switzerland
Status: Offline
Reply With Quote
Oct 9, 2004, 12:26 PM
 
Originally posted by absmiths:
That doesn't make much sense if it is true. The file catalog normally resides on the disk, so room doesn't have to be "made" to write it at system shutdown (or failure). ...
Not true. The directory is of variable size and of arbitrary location, so it is entirely possible to run out of space for it, though the file system never allows this.

tooki
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 9, 2004, 01:49 PM
 
Originally posted by tooki:
Not true. The directory is of variable size and of arbitrary location, so it is entirely possible to run out of space for it, though the file system never allows this.

tooki
I remember that MicroMat Tech from the MacFixIt Forums used to talk about corruption resulting from the free space getting too low on an HFS -formatted drive. Of course, this was the same guy who claimed that BootCD copies all of OS X onto a 10 MB RAM disk so I tend to take what he says with a grain of salt...

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
bmhome1
Dedicated MacNNer
Join Date: Jul 2003
Status: Offline
Reply With Quote
Oct 9, 2004, 08:43 PM
 
MicroMat Tech is one of THE most knowledgeable individuals in the Mac world about HFS format and partition issues (who bothers trying to help others on the web at least). Only an idiot would describe OSX from a 10MB RAM disk, the translation from or reporting about that is the problem.
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 9, 2004, 10:02 PM
 
Originally posted by bmhome1:
Only an idiot would describe OSX from a 10MB RAM disk, the translation from or reporting about that is the problem.
http://www.macfixitforums.com/php/sh...&Number=620226

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
bmhome1
Dedicated MacNNer
Join Date: Jul 2003
Status: Offline
Reply With Quote
Oct 9, 2004, 10:34 PM
 
That discussion sure seemed illogical as you stated. I wish it had continued onward to straighten it all out. I do know MicroMat Tech spends an insane amount of his own time online helping out others and his advice is usually competent and often referenced to solid documentation.
     
yukon
Mac Elite
Join Date: Oct 2000
Location: Amboy Navada, Canadia.
Status: Offline
Reply With Quote
Oct 10, 2004, 03:10 AM
 
AFAIK (take this with salt), there is space reserved for the catalog and other structures, but this can overflow. When it does, there needs to be extra space available, or previous parts of the catalog are overwritten, causing the FS to lose track of files and generally get messed up.

Of course, in this situation, you'd need to run out of space simultaneously in the normal way, and run out of free pages for new catalog data. This sounds kinda difficult, though possible.....If you run for a while on very little space (what is it, 20mb?), have greater than (say) 120,000 files, and have a fragmented drive (bloating the disk structures, as they have to keep track of each fragment), then in that difficult situation, you could easily have a problem.

Keeping free space available is a good idea, it speeds up the drive (barefeats, and others have benchmarked), decreases fragmentation, apparently keeps you safer (?)....easier said than done at times, especially if you're stuck with ATA100 and below.

edit: ok, not that much salt.
( Last edited by yukon; Oct 10, 2004 at 10:18 PM. )
[img]broken link[/img]
This insanity brought to you by:
The French CBC, driving antenna users mad since 1937.
     
Dr.Michael  (op)
Registered User
Join Date: Oct 2003
Status: Offline
Reply With Quote
Oct 10, 2004, 04:22 PM
 
Originally posted by Millennium:
I don't know; cpac's explanation does make sense. That said, I'm surprised that OSX's preflight routines wouldn't have caught something like this. What program were you using to rip the DVD?

Also, can we be certain that the corruption was actually caused by the hard drive being full, and that this wasn't a coincidence of some kind? For example, might you have lost power to the PowerBook, perhaps due to running out of batteries?
Hi,

I used ffmpeg 0.0.9p which is a wrapper for a bunch of commandline tools. I encoded a DVD to a SVCD and the process where everything stopped was while the last disk image (to burn with Toast) was created. Don't exactly remember which tool does this.

Maybe any of these tools is responsible for the mess, but I agree with Tsilou B. that the management regions of the drive should be protected by apple against third party software access.

This also happens to Windows machines. We had the problem with a Win2k server before but never with vms or solaris (although they usually have very full drives).

And no, my powerbook wasn't on battery power. Unfortunately my powerbook does not run for more than 7 hours with full cpu load on battery :o). But I always hope that sometime...
     
Dr.Michael  (op)
Registered User
Join Date: Oct 2003
Status: Offline
Reply With Quote
Oct 10, 2004, 04:23 PM
 
Originally posted by tooki:
At about 50MB free space, Mac OS X tells applications that the disk is full, and will refuse to allow them to write files (which is why sometimes the prefs get trashed, because they are prohibited from being written). This is to prevent the directory from having nowhere to be written.

It's likely that something else happened.

That said, it's still a bad idea to let the disk fill completely. It's always wise to leave 5-10% free.

tooki
Will the info show then that <50 MB are free?
My info window said 48kB free disk space.
     
tooki
Admin Emeritus
Join Date: Oct 1999
Location: Zurich, Switzerland
Status: Offline
Reply With Quote
Oct 10, 2004, 05:24 PM
 
Yes, my understanding is that the Finder will report the actual amount of free space, but that the operating system will return a disk full error to programs that attempt to create new files on disk.

tooki
     
SMacTech
Mac Elite
Join Date: Nov 2001
Location: Trafalmadore
Status: Offline
Reply With Quote
Oct 10, 2004, 06:42 PM
 
Originally posted by tooki:
Yes, my understanding is that the Finder will report the actual amount of free space, but that the operating system will return a disk full error to programs that attempt to create new files on disk.

tooki

And hopefully the program will actually not ignore the error.

I assume you have 10.3, as I didn't see where that was mentioned. 10.2 was known for allowing the disk to become full.
     
SMacTech
Mac Elite
Join Date: Nov 2001
Location: Trafalmadore
Status: Offline
Reply With Quote
Oct 10, 2004, 06:45 PM
 
Originally posted by Dr.Michael:

This also happens to Windows machines. We had the problem with a Win2k server before
It sure will happen with Win 2k server and it is the reason why I always monitor the disk space [servers alive] and I get a page when it is below 1GB and about 5GB on our SQL server.
     
theolein
Addicted to MacNN
Join Date: Feb 2001
Location: zurich, switzerland
Status: Offline
Reply With Quote
Oct 10, 2004, 11:51 PM
 
This is why I always do my work on a seperate partition. I have three partitions, one for Sytem and apps, one for data, and one that is for working files. I also try to keep my stuff backed up every few days.

But I suppose stuff like this can happen to anyone. It almost happened to me a year ago with XCDRoast, when it started filling my drive my temporarry data. Luckily I caught it in time.
weird wabbit
     
Dr.Michael  (op)
Registered User
Join Date: Oct 2003
Status: Offline
Reply With Quote
Oct 11, 2004, 01:53 AM
 
Originally posted by tooki:
Yes, my understanding is that the Finder will report the actual amount of free space, but that the operating system will return a disk full error to programs that attempt to create new files on disk.

tooki
Maybe this is the problem that caused the bug.
The files I created were very large (> 1gig). So a protection against creating NEW files on disk full would not be sufficient because the existing files grow until nothing goes.
     
yukon
Mac Elite
Join Date: Oct 2000
Location: Amboy Navada, Canadia.
Status: Offline
Reply With Quote
Oct 15, 2004, 01:19 AM
 
Does anyone have a more solid number of how much space you must have to avoid the problem altogether? Apple apparently believes 50mb, but surely that's padded as a buffer. ISTR the micromat tech guy saying 20mb was the size the disk structures would grow by when it's reserved space is filled. I need to fill up as much space on the drive as possible (only for a few minutes), so an actual number or range would be perfect . Plus, it would be nice to know anyway.
     
Millennium
Clinically Insane
Join Date: Nov 1999
Status: Offline
Reply With Quote
Oct 15, 2004, 07:57 AM
 
Originally posted by Tsilou B.:
This should NEVER happen in a good OS. It's one thing if the computer crashes and unsaved data is lost and/or files that had been saved in that moment are corrupted. But the entire hard drive corrupted? I've never seen that in Windows, I've never seen something like that in Linux, however, I haven't had such problems with Mac OS X, either, and I had an iBook that had less than 50 MB free space for half a year.
It used to happen fairly frequently in Windows and even Linux, on those occasions when you could get a Linux system to crash. What stopped it from occurring was the advent of journaled filesystems, which have ways of protecting against this sort of thing. Windows XP now uses NTFS by default, which is journaled. Linux 2.4 saw the advent of several journaled filesystems, the most popular being ext3 and ReiserFS.

Macs also have journaled filesystems, now, though they only came along with the 10.2.3 release, and at that time they were optional. They were made the default in 10.3 Panther, though. You can convert an existing drive to use Journaling without having to reformat.

Were you using Journaling when this happened? If not, then this is unfortunate but quite within the realm of possibility. If so, however, then there are larger problems, and we need to investigate further.
You are in Soviet Russia. It is dark. Grue is likely to be eaten by YOU!
     
tooki
Admin Emeritus
Join Date: Oct 1999
Location: Zurich, Switzerland
Status: Offline
Reply With Quote
Oct 16, 2004, 01:29 PM
 
My hunch is that journaling is not what made those filesystems more resilient to the problem in question -- it just happens that journaling was added to Windows and Linux at the same time as many other filesystem optimizations.

If a disk got so full that it couldn't expand its directory, then where exactly would there be room for it to create a journal, anyway? And besides, what kind of filesystem would make space for a journal (which is redundancy data]) without first making space for the directory (the primary data)?!? That would be like a car having tire pressure monitoring, but no speedometer or gas gauge.

tooki
     
   
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 08:23 PM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,