[Rainy Day]

Does anybody know if Mac OS X zeros out file slack? That’s the unused – but allocated – space at the end of a file. I’m not talking about unused free-space on the HD. This is the unused space at the end of a file which pads out to a multiple of the drive’s blocksize. If not zeroed, when the file is uploaded or eMailed, it’s possible the slack space might go along for the ride, potentially giving up confidential information.

If Mac OS X doesn’t zero it out, are there any tools to wipe it?

TIA.

[BLAZE_MkIV]

I've never heard of the term file slack. I doubt any OS zeros out extra space in the block since its a hard disk only phenomena. Also it won't get transferred along when you send the file. The length of a file is in bytes not blocks even though its rounded up to the nearest block on disk.

[reader50]

Hard drives are block devices - they cannot be addressed in smaller units. So when a file doesn't completely fill out the last block, a full last block will be sent to the HD anyway. It will either be padded out with extra zeros, or a memory dump of the block buffer. Which might contain unflushed data from the last block sent to disk. I don't know OS X's behavior, if it zeroes out block buffers between use.

It would be an odd security issue if OS X didn't zero buffers between uses. And considering how much faster the computer is vs the HD bandwidth, there's no performance penalty to zeroing. I'm going to guess that OS X does zero block buffers between uses. It would have to if they ever want security approval to sell to certain government agencies.

[BLAZE_MkIV]

If you have something in memory you need protected, passwords etc, you can tell the kernel that when you get memory and it doesn't cache, swap, etc.

But you wouldn't send the file to a non-block device, like over an network connection, in blocks. Sure TCP will break it up into packets but it would be really weird for any padding in a TCP packet to include padding from a block on the HD. Does TCP even pad its packets?

[ghporter]

BLAZE's answer is exactly what you're looking for. Any file you send will be ONLY that file, not the "slack" from the hard drive, which is only on the drive. It's not really part of the file, just the space allocated for the file, so there's no reason why it should go. When you send a file, the system reads to an "end of file" marker and stops there, so NOTHING from the hard drive that isn't actually part of the file goes anywhere. Padding in TCP packets is part of the network interface, not anything to do with the internal data structure of the hard drive.

[CharlesS]

I decided to test this to find out for myself. I used TextWrangler (a Carbon-based text editor that edits in place) to save a text file onto a disk image. Wrote out a text file, deleted some text from the end, saved, checked with a hex editor. There was no file slack - the extra data was deleted. Next, I did the same operation, using a hex editor instead of a text editor to delete the extra data from the end. This time, I did see file slack on the disk image. So my guess is that while simply truncating a file the way a hex editor would do may result in file slack, in practical usage most applications will just be writing out the whole file, and not resulting in file slack. One thing to note is that Cocoa apps usually write atomically - they write a new file to the disk and then delete the old one once the new file has been successfully written - which means the old file will actually still be on the disk in its entirety (and could possibly be undeleted with something like Data Rescue). You could take care of that by using Disk Utility's "Erase Free Space" option, though.

[Rainy Day]

My question is really whether or not Mac OS X zeros out file slack, not whether or not it’ll get transferred when sent over the InterNet. I simply used that as an example. Probably should have used burning a CD as an example, as i’m fairly certain file slack is transferred with disk to disk copies, or CD mastering/burning.

I understand the nuances of block devices vs serial, but am not convinced there aren’t situations where the file slack won’t be transferred on InterNet uploads. I know that in the pre-Mac OS X days there were instances where file slack did get transferred. For example, where MacBinary format was used to preserve the resource fork, file slack was often included too. (Begs the question of whether file compression, like gzip, might include file slack. I would hope not, because it would be inefficient to do so, but it could.)

Be all that as it may, the question of whether or not file slack gets transferred with eMail or other uploads becomes moot if Mac OS X zeros it out. And that’s really what i want to know.

@CharlesS: Which HexEditor to you prefer? I think i might like to do some experiments of my own.

[larkost]

Rainy Day, file slack is only going to be a problem if programs are reading the actual blocks on a device, and that is really uncommon nowadays. I remember this being a problem with older versions of Word, but that was some Microsoft programmers being over-creative and trying to eek extra performance by skipping the OS layers.

If you are making block-to-block copies of a disk (not of files, but of the whole disk as a block device), then you are going to carry around non-file data that happens to also be in those blocks. But most programs have no access to the raw blocks, but rather access files as a stream through an API. In theory you could do a buffer-over-read, but the API's governing file access nowadays pretty much guarantee that this is going to get you an error rather than extra data.

So burning a CD of files is not going to have this issue, nor is anything with files being uploaded through regular programs.

Note that it is possible for something resembling file slack within a single file if the program tries to get cute managing its own files either with memory overwrites, or writing within the same file. This was also an issue with MS Word where it would grab contiguous block from the memory it managed and so write some data from one file into another. But that is not the same thing.

[CharlesS]

Originally Posted by Rainy Day 

My question is really whether or not Mac OS X zeros out file slack, not whether or not it’ll get transferred when sent over the InterNet. I simply used that as an example. Probably should have used burning a CD as an example, as i’m fairly certain file slack is transferred with disk to disk copies, or CD mastering/burning.

Just tried it, and the file slack is not transferred with either disk-to-disk or burning a CD-RW.

I understand the nuances of block devices vs serial, but am not convinced there aren’t situations where the file slack won’t be transferred on InterNet uploads. I know that in the pre-Mac OS X days there were instances where file slack did get transferred. For example, where MacBinary format was used to preserve the resource fork, file slack was often included too. (Begs the question of whether file compression, like gzip, might include file slack. I would hope not, because it would be inefficient to do so, but it could.)

It doesn't.

Be all that as it may, the question of whether or not file slack gets transferred with eMail or other uploads becomes moot if Mac OS X zeros it out. And that’s really what i want to know.

OS X leaves the file slack there. But it's not really a big deal.

@CharlesS: Which HexEditor to you prefer? I think i might like to do some experiments of my own.

I use a hex editor that I wrote myself but never released. Unfortunately, this means you can't get it, but it's ideal for me to do a test like this with, since I know exactly how it works.

[Cold Warrior]

I like 0xED but don't know if it does what you guys need it to.
http://www.macupdate.com/info.php/id/22750/0xed

[BLAZE_MkIV]

How much info are you really gonna get in < 512 bytes. Sure its allot of characters but what are the odds of hitting a sweat spot in a file were the data has enough context that you know what it is.

[cypherpunks]

This site says that mac's disk utility does not wipe file slack: How to wipe file slack on OS X

Not sure how useful file slack is in forensic computer examination though.

So what's the best way to securely wipe free space? How about the scrub utility (available on macports)?

[BLAZE_MkIV]

Take a drill press to the HD when your done with it. Otherwise there are ways to recover data.

[ghporter]

Originally Posted by BLAZE_MkIV 

Take a drill press to the HD when your done with it. Otherwise there are ways to recover data.

Even if you drill a hole through the platters, something could be recovered. IF someone wanted to spend the time and money. IF someone wanted YOUR information that badly. IF your information was worth even investigating in the first place.

A "secure wipe" app, one that overwrites the whole drive in several passes then writes random characters on a final pass, will make your data "virtually" unrecoverable. This process meets DoD standards for wiping a drive that has had classified information (up to Secret) on it, meaning that NSA would have to work very hard at getting anything at all from the platters. DoD does not allow this process for Top Secret only because there is the slightest chance that someone, somehow could come up with some sort of process that, given unlimited time and resources, could reconstruct something from such a drive.

In other words, when you dispose of a drive, protect your personal information by using a secure wipe app and stop sweating.

[tooki]

Well, then clearly the only safe way to dispose of the drive is to melt it in a volcano. Do I smell a business opportunity?

1. Drop drive in volcano.
2. ???
3. Profit!

[Spheric Harlot]

Originally Posted by ghporter 

In other words, when you dispose of a drive, protect your personal information by using a secure wipe app and stop sweating.

Agreed.

If somebody is THAT interested in your data, they already have it - unless you live in a high-security bunker and your computer is completely unconnected to the outside, and is locked in a safe whenever visitors enter the building (or some such scenario).

If you need to ask how to dispose of your data before disposing of a drive, then secure wipe is certainly enough.

[P]

Originally Posted by tooki 

Well, then clearly the only safe way to dispose of the drive is to melt it in a volcano. Do I smell a business opportunity?

1. Drop drive in volcano.
2. ???
3. Profit!

There is another way: thermite.

Or just cut them in half with an axe and distribute the pieces across the planet.