[Dalgo]

I was wondering what would be a good thing to do if I was ever given a null pointer when I tried to allocate something. I'd assume that there'd be no point in putting an alert on the screen since if there wasn't enough memory available to allocate whatever I wanted, then there probably isn't enough memory avaliable to make an alert panel. I don't really want to make my program quit unless it was really necessary and I would like to alert the user to the fact that the reason whatever they tried to do didn't work because it was a memory error. I never allocate huge things with malloc so it wouldn't be an issue with me not being able to get enough contiguous memory, it'd be an issue that there just wasn't enough memory available. I know with OS X, it isn't very likely that there won't be enough memory, but it is an issue that I should address.

[eyadams]

Well, when I get a NullPointerException I log it to stderr via Exception.printStackTrace()...

Oh, wait a minute. You might not be programming in Java. Sorry, couldn't resist :-)

But seriously, I'd say the correct behavior depends on what you're doing. Displaying an alert won't necessarily require the same amount or more RAM than whatever you're trying to allocate, so displaying an alert is certainly a viable option. NewsWatcher under OS 9, for example, will display an alert indicating that it's out of RAM and recommends things the user can do to correct the problem.

If you can't allocate your resource, and can't display an alert, but can otherwise continue processing, that might be acceptable as well. Just keep track of what you'd like to do when more RAM is available.

Or you could just quit with an approriate status. For a pure Unix app, you would exit with a non-zero value to indicate an error; I'm not sure how OS X handles exit codes.

Erik

[Dalgo]

Don't be too proud of java. Doesn't stderr just throw it into the console? I could do the same thing in C. I think that the default behaviour of printf, if I'm not running a program from the shell, is to put it into the console. Of course I'd like a response in the GUI. I'll try returning codes and see what happens.

[gregomni]

The only time a memory allocation will fail on OS X is when:
(a) all the RAM is being used,
(b) all swap space on all drives is completely full, and
(c) there are no pages in memory that can be discarded (for example, code pages which can be discarded without swapping them out because they can be reread from the executable file on disk instead)

So basically, if a memory allocation fails on OS X you are completely screwed. Don't worry about it except for very large allocations: i.e. put error checking around allocating space for multi-megabyte images or things like that so you can present an alert panel or whatever.

But for the vast majority of "normal" allocations I wouldn't bother even checking for null pointers - segfaulting and your program exiting when you try to access the null pointer soon after is probably the best you can manage under those conditions anyway. (Or NullPointerException in Java - whatever the default action is is likely the best you can do.)

[Dalgo]

Originally posted by gregomni:
The only time a memory allocation will fail on OS X is when:
(a) all the RAM is being used,
(b) all swap space on all drives is completely full, and
(c) there are no pages in memory that can be discarded (for example, code pages which can be discarded without swapping them out because they can be reread from the executable file on disk instead)

I guess that I have a question about unix memory management then. Suppose that I have opened and closed a whole bunch of programs so the memory is pretty fragmented. Now, I know that unlike Mac OS9, the memory that is allocated for my program doesn't have to be, and proably isn't, continguous. However, I had thought that the amount of memory that I needed for each individual call to malloc had to be available as a contiguous block of memory. So if I use malloc to allocate 10 bytes and then another 10, the first ten would be a contiguous block of memory as would be the second ten, but the first ten and the second ten didn't need to be anywhere near eachother in memory. So that'd mean that if I could be requesting, say 100 bytes, and there was 2 megs free, total, but the largest fragment of that 2 megs was only 99 bytes then malloc would fail.

So the question is: will the kernel defragment what is in memory if I request an amount of memory that is larger than the largest fragment, but is still smaller than the total available amount?

[gregomni]

I guess that I have a question about unix memory management then. Suppose that I have opened and closed a whole bunch of programs so the memory is pretty fragmented.

There's nothing to fragment. :-) Your sentence presupposes that all applications share the same heap. They each have their own. The whole idea of protected memory is that applications have entirely disjoint memory spaces. Not just that two processes can't write in each others memory, they literally can not even address it or know that it is there. I can write to memory location 372, and you can write to memory location 372, and they are different 372's.

Let's take a step back: The way OSX deals with memory is by splitting it up into "pages". Each page is a 4kb chunk of memory that is either resident (really in memory) or swapped out (saved to disk temporarily in the swap file). Which pieces of memory are resident and which are swapped out is totally transparent to programs (and to most of the OS itself as well).

When a program tries to look at a page of memory (either to read, write, or execute code on it), if that page happens to be swapped out it will cause a "page fault" that the OS has to handle. The OS reads the needed page out of the swap file and somewhere into RAM (The important thing is: where it puts it in RAM and where my program thinks it is can be totally different. This is called address translation.) Then the OS lets the program that needed the page continue on, using the page which is now in RAM, totally unaware that anything happened. If there was no free RAM, the OS will have to make room for the new page by swapping out some other page that is currently in memory (probably by writing it to the swap file). That other page could be in your program, say. Neither of us will notice. Just the next time an instruction in your program tries to access anything on that page, the address translation gear kicks in, it sees your page isn't in memory at all, and the OS loads it back in.

However, I had thought that the amount of memory that I needed for each individual call to malloc had to be available as a contiguous block of memory.

Through the magic of address translation, only memory addresses that are on the same page are REALLY right next to each other. In my program memory address 4094 and 4095 are definitely next to each other in RAM but 4095 and 4096 could be mapped to vastly different parts of main memory, or one of those addresses could be swapped out to disk and the other still in memory or whatever. (Because with 4k pages the page boundaries are: 0-4095, 4096-8191.) So there is no such thing as a contiguous blocks more than 4k big.

So that'd mean that if I could be requesting, say 100 bytes, and there was 2 megs free, total, but the largest fragment of that 2 megs was only 99 bytes then malloc would fail.

Fragmentation still happens, but only within an individual process. My program could have 2 megs free, total, on various pages it has asked for from the OS but the biggest fragment could be only 99 bytes. In that case, when I request 100 bytes, my malloc() routine looks through its fragments, doesn't find anything big enough, and requests a brand new empty page from the OS and uses 100 bytes of that. The way that the OS gets an empty page is to chuck an existing page out of memory, just like described above.

When my program exits, the operating system throws away the address translation table it had for me and marks all the pages I was using as free, to be reused.

I'm not sure I'm explaining this very well, but there it is. :-)

[Dalgo]

Thanks. I guess that Unix is a lot smarter than I thought that it was.