[Zim]

Depending on the progs I run, I can end up with 5,6,7,8 VM swap files in /var/vm, causing a real slow-down as I switch from process to process.

I can gain speed back by killing the processes involved, but the swap files stay.

Is there a way to clear those out without rebooting?

Thanks,
Mike

PS. I have 512M, not adding more memory.

[osxisfun]

I'd love to know too.

My system feels so much better after a reboot after a few days up ...

[wadesworld]

clearing out VM files while they're in use is a bad idea.

Wade

[Zim]

Let me rephrase...

"Is there a good/safe way to clear those out without rebooting? If so how".

Thanks,
Mike

[Millennium]

Clearing out these files without rebooting is extremely dangerous, because you have no way of knowing which files are still in use by running applications or even the system itself. The only code which absolutely cannot be paged out to VM is the virtual memory code itself, because there would be no way to get it back out of VM. Everything else, from applications to drivers (except the VM driver) to vital OS code, is fair game.

My advice is to Just Not Do It. It is a pain, and it is inconvenient, but it is so not worth the risk.

[osxisfun]

New App Idea?

loop
VM caches
if cache belongs to running app then ignore
else
delete cache


end

[someone_else]

Sounds like osxisfun and zim both need more memory. Your system should never "feel" more responsive after a reboot, because after a clean start nothing is cached in memory. If it does "feel" faster, then your working set of applications is too large for the amount of memory you have and you need more memory. Or run less applications.

Memory is so cheap, why would you not buy more if it's needed?

[Moonray]

Originally posted by someone_else:
Sounds like osxisfun and zim both need more memory. Your system should never "feel" more responsive after a reboot, because after a clean start nothing is cached in memory. If it does "feel" faster, then your working set of applications is too large for the amount of memory you have and you need more memory. Or run less applications.

Memory is so cheap, why would you not buy more if it's needed?

If you need gigabytes it's not so cheap anymore and won't fit in most Macs.

-

[Moonray]

Originally posted by Zim:
I can gain speed back by killing the processes involved, but the swap files stay.

Why don't you quit them normally?

10.0 - 10.2 had a slighty different way and did create equal-sized vm files. They got automatically deleted when they were no longer used. I suppose they might still get deleted in 10.3 if they hold no valid contents any more but don't know fo sure. However if the files are bigger it's not as likely that one is unused because the allocated space can be fragmented.

You can try and quit all or as many running applications as possible and see if that has an effect. You must not delete them yourself.

-

[mbryda]

Mac memory IS PC MEMORY. It's PC100/133 or DDR or whatever in SODIMM or DIMM or DDR. Nothing special about it anymore.

Why are you worrying aobut the files in /var/vm? They are supposed to be there, and OSX takes good care of cleaning them up as needed. As long as you have space, all is fine.

What I have found is that after a few weeks, the system does get a little pokey (currently on 65+ days uptime). My solution is to log out and let the computer put itself to sleep (about an hour) over night. Then when I start back in the AM, things are working perfectly fast again. My thinking is that either some process starts hoggimg RAM/CPU after a couple weeks or the VM system just needs flushed.

[osxisfun]

I have 1.5 gigs.

I was thinking of going to 2 or 2.5.

I think its the fact that i have so many darn things open. although i do need them all as i tend to watch out and make sure i am using what i have up.

but when i reboot.... ahhhhh... of course then i start to open up alll my apps again...

[Zim]

Originally posted by mbryda:
Mac memory IS PC MEMORY. It's PC100/133 or DDR or whatever in SODIMM or DIMM or DDR. Nothing special about it anymore.

Why are you worrying aobut the files in /var/vm? They are supposed to be there, and OSX takes good care of cleaning them up as needed. As long as you have space, all is fine.

I'm "worrying" about them b/c in fact it does NOT do a good job of cleaning them out. Even after I quit all processes, the files remain until a reboot. Rebooting and re-launching the same things produces a marked speed-up. If it did a "good job" that would not be the case.

Nothing personal, but I was seeking the seeking the answer to the question I posed. I am well aware of adding or not adding memory. I simply wanted to know if there was any way besides a reboot to clear these out and un-clog the sluggish system. I'm glad you do not have these problems. I do. It appears I am not the only one.

Mike

[Earth Mk. II]

Since you said it depends on which programs you run, maybe one of your apps is more memory intensive than you think, or has a memory leak someplace?

If you have 512 MB of RAM, you typically shouldn't have so much VM swapping going on that your computer slows down. If you're running something that crunches a lot of data all at once (a large image in Photoshop, for example), then you have to expect ram from other apps to get swapped out, which would cause a slow down when switching to other apps. But under normal use, you really shouldn't be swapping as much as you seem to say you are. OS X's memory management tends to be very efficient.

As an aside, the swap files themselves have little effect on actual performance of your computer, unless you're running low on disk space and actively used swapfiles become fragmented. Really, the number or size of your swapfiles is totally insignificant. It's the amount of your currently running app's data that fits into your physical RAM that counts.

iirc, there's a cron job that runs nightly to help clear out some unused swapfiles, and there are utilities out there that will execute that job at an arbitrary time. If the swapfiles existence really bothers you, then I suggest leaving your computer on overnight or finding ones of those utils.

However, since you seem to be saying that these symptoms occur when certain apps are running, I want to say that the most likely cause of your poor performance is a memory leak in one of those apps, or that one of those apps is very memory intensive. If that's the case, rebooting, clearing your VM cache, or whatever is just treating a symptom - not actually fixing the problem.

[mbryda]

Originally posted by Zim:
I'm "worrying" about them b/c in fact it does NOT do a good job of cleaning them out. Even after I quit all processes, the files remain until a reboot. Rebooting and re-launching the same things produces a marked speed-up. If it did a "good job" that would not be the case.

Do you log out or simply end everything? I've noticed a similar slowdown at about 14 days upime. A logout and let it figure things out nearly always does the trick. I'm thinking it's maybe a slow leak in WindowManager or some other load on login process that gets cranky after many days up.

The iMac I'm typing this on has:
Mail
iTunes
iCal
Terminal with D.Net client
Safari
Word
Excel
Enterouge
StickyBrain
Preview
TextEdit
Acrobat
iSyns
iMovie
MYOB

running within 512 M ram, and has been up 65 days. Still speedy enough to capture video in iMovie and overall speediness is fine.

My /var/vm looks like this:

Code:
-rw------T  1 root  wheel   67108864  8 Jan 19:05 swapfile0
-rw------T  1 root  wheel   67108864  8 Jan 22:48 swapfile1
-rw------T  1 root  wheel  134217728  9 Jan 22:01 swapfile2
-rw------T  1 root  wheel  268435456  9 Jan 22:12 swapfile3
-rw------T  1 root  wheel  536870912  7 Mar 23:57 swapfile4
-rw------T  1 root  wheel  536870912 11 Mar 21:45 swapfile5

On 10.3.2, if I quit all apps, within, say 5-10 mins swapfile5 will go away. If I log out and let it sit, swapfile4 will go away.

My top looks like this:

Code:
Processes:  76 total, 3 running, 1 stuck, 72 sleeping... 203 threads   08:04:57
Load Avg:  3.96, 3.20, 2.73     CPU usage:  72.5% user, 27.5% sys, 0.0% idle
SharedLibs: num =  127, resident = 43.7M code, 2.32M data, 7.82M LinkEdit
MemRegions: num = 13398, resident =  169M + 12.6M private,  124M shared
PhysMem:  77.9M wired,  285M active,  139M inactive,  503M used, 8.71M free
VM: 5.92G + 85.6M   2453576(0) pageins, 1679967(0) pageouts

  PID COMMAND      %CPU   TIME   #TH #PRTS #MREGS RPRVT  RSHRD  RSIZE  VSIZE
26770 top          7.0%  0:03.25   1    16    26   436K   428K   812K  27.1M 
26758 FirstEdge    5.6%  0:27.23   3    73   130  3.02M  12.8M  8.30M   115M
26746 nmbd         0.0%  0:00.47   1    12    25   584K   892K  1.41M  27.8M
26724 slpd         0.0%  0:00.02   6    29    37   236K   872K   912K  30.4M
26718 lookupd      0.0%  0:00.66   2    36    70   412K   808K  1.18M  28.5M
26701 dnetc        0.8% 97:03.51   2    15    30   280K   704K   880K  36.9M
26659 iTunes      12.0% 28:31.35   4   239   356  9.89M  21.2M+ 19.5M   137M
26553 Microsoft    1.6% 15:07.32   8   199   268  8.39M  22.1M  18.0M   146M
26546 iMovie       0.0%  4:25:43   6   287   650  12.1M  33.8M  23.8M   192M
26542 Finder       0.0% 24:30.73   1   104   267  4.70M  24.0M  11.2M   153M
26107 Mail         0.0%  4:17.87   8   159   385  10.6M  12.5M  17.6M   164M 
25815 umount       0.0%  0:00.60   1    11    15     0K   332K   172K  17.6M
25777 tcsh         0.0%  0:00.05   1    13    22   348K   528K   620K  22.1M
25776 login        0.0%  0:00.03   1    13    37     0K   320K   208K  26.9M
25663 RetroRun     0.0%  0:37.55   1    10    18    52K   332K   460K  27.7M
25457 iSync        0.0%  0:35.43   2    87   219  1.45M  5.36M  2.38M   114M
25447 System Eve   0.0%  0:02.37   1    56   104   784K  2.87M  7.76M   100M
25318 Adobe Read   8.1% 91:48.48   4    81   291  6.73M  9.36M  10.8M   165M 
25070 iChatAgent   0.0%  0:00.60   2    64    89   504K  1.39M   824K  94.7M
25067 TextEdit     0.0%  0:06.15   1    76   132  1.08M  4.73M  1.53M   107M
24958 Microsoft    6.1% 70:39.96   3    84   286  1.71M  19.6M  2.04M   165M
24891 Microsoft    0.2%  2:04.72   2    67   125   976K  5.54M  1.69M  99.3M
24890 Microsoft    0.0% 21:11.03   4   101   294  6.89M  18.9M  14.1M   197M
24453 Preview      0.0%  1:23.60   2    94   227  2.41M  13.4M  3.71M   120M
24284 tcsh         0.0%  0:00.47   1    13    23   300K   528K   644K  22.1M
24282 login        0.0%  0:00.03   1    13    38     0K   320K   180K  26.9M
24281 Terminal    19.4% 16:34.01   4    74   214  2.27M  8.50M+ 6.20M+  114M
24273 iCal         1.1% 51:15.70   3   208   400  10.0M  15.7M  12.3M   140M
24269 WeatherPop   0.2%  4:40.25   1    88   122  1.76M  5.07M  2.22M   106M
24264 AppleSpell   0.0%  0:00.54   1    55    84     0K   976K   320K  62.0M
23959 Safari      20.8% 60:51.20   7   172   708  51.9M  27.2M+ 62.4M+  242M
23924 mount_webd   0.0%  0:19.63   7    46    37   292K   652K   488K  30.4M
23920 LCCDaemon    0.0%  0:02.27   1    64    97   624K  1.14M  1.12M  93.9M
23919 iCalAlarmS   0.0%  0:04.06   1    60    98   432K  1.36M  1.20M  93.9M
23909 StickyBrai   1.1% 34:43.67   2   155   192  2.81M  5.86M  3.11M   142M
23908 HotKey       0.0% 12:28.49   4   147   121   412K  3.02M  1.03M  94.6M
23906 Palm Deskt   0.8% 42:36.39   1    51   116   664K  4.98M  1.18M  98.0M
23892 MirrorAgen   0.0% 10:15.36   3   139   119  1.69M  4.50M  2.65M   105M
23891 Image Capt   0.2%  0:09.90   1    60    92   452K  2.49M  1020K  98.3M
23887 TWAINBridg   0.0%  0:05.58   2    78   114   480K  3.26M  1.08M   103M
23876 SystemUISe   0.0%  0:51.93   2   206   243  1.67M  6.91M  4.39M   117M
23875 Dock         0.0%  2:35.23   2   123   196  1.43M  10.4M  7.16M   112M
23873 pbs          0.0%  0:00.52   2    32    46   208K   760K   616K  44.1M

So, you can see, I am hitting my system hard and things are just fine. I'd say look at your apps when things get sluggish - one may be hitting memory hard or have a spike in CPU usage.

The existance of multiple swap files doesn't affect performance other thatn the obvious hit of swapping RAM to HD. OSX will dynamically allocate and deallocate them as needed. It does take some time, though, as it wants to make sure it won't be needed.

My money is on one of your processes going awry after a couple weeks - a simple logout overnight should fix it without a reboot.

[iamnotmad]

Originally posted by Zim:
Depending on the progs I run, I can end up with 5,6,7,8 VM swap files in /var/vm, causing a real slow-down as I switch from process to process.

I can gain speed back by killing the processes involved, but the swap files stay.

Is there a way to clear those out without rebooting?

Thanks,
Mike

PS. I have 512M, not adding more memory.

Hey Zim, this looks like it might do what you want. Take a look. Cache Out X

[Zim]

Originally posted by iamnotmad:
Hey Zim, this looks like it might do what you want. Take a look. Cache Out X

Great!!! that does look like something that could help. Anyone know how it does its work on the swap files? (I downloaded it, but there was little documentation).

I think it is a memory leak situation, just trying to figure out the way with the least impact on other open Apps to get this unix process (hence the kills) to clean itself up. Then again with all of the Apple Updates that require reboots, its usually is taken care of.

Thanks!
Mike

[mbryda]

Originally posted by Zim:
I think it is a memory leak situation, just trying to figure out the way with the least impact on other open Apps to get this unix process (hence the kills) to clean itself up. Then again with all of the Apple Updates that require reboots, its usually is taken care of.

Monitor Top and Activity Monitor and see what is using the most memory. Ive found that Safari loves memory when left open for about a week. It's taken 90MB or so opening 1 page after a week. Also, iPhoto loves memory, as does Photoshop, with PS not liking to release memory.

Also, when it gets sluggish, quit all apps to leave the OS in RAM and see what is using all the memory. Monitor your TOP and Activity Monitor and you will find the culprit.

[Basilisk]

Great!!! that does look like something that could help. Anyone know how it does its work on the swap files? (I downloaded it, but there was little documentation).

I've used it before and as near as I can tell it just rm -f's the swapfiles. After doing that your next step should be a reboot, which is what you're trying to avoid. I certainly wouldn't trust the stability of a system that's had that done, its asking for trouble.

I'm "worrying" about them b/c in fact it does NOT do a good job of cleaning them out. Even after I quit all processes, the files remain until a reboot. Rebooting and re-launching the same things produces a marked speed-up. If it did a "good job" that would not be the case.

Second guessing dynamic_pager is not a good idea. First off, it really will clean up the files if there are no active pages in the file. Your problem is likely to be the allocation pattern of your usage has left just a few system-owned pages in each file, hence no VM files can be removed even after all applications have quit.

But you're focusing on the fact the swapfiles are gone after a reboot as the explanation for the speedup. This is almost certainly not true. A reboot does lots of other things that could just as easily explain the speedup that have nothing to do with the VM system. Drivers are restarted, lookupd and friends are cycled, caches are cleared.

Unless you are actually seeing a high number of pageins/pageouts when the slowdowns occur (with accompanying disk activity) you are probably barking up the wrong tree.

So the most important question is one I haven't seen you answer yet... When things get sluggish for you is there a lot of accompanying disk activity? Does vm_stat in Terminal report a sharp increase in pagein/pageout activity? If so, then maybe the VM system (or low memory due to leaks) is the problem. Otherwise I would be looking elsewhere for an explanation.


Alex

[Zim]

Originally posted by Basilisk:
Your problem is likely to be the allocation pattern of your usage has left just a few system-owned pages in each file, hence no VM files can be removed even after all applications have quit.
....
So the most important question is one I haven't seen you answer yet... When things get sluggish for you is there a lot of accompanying disk activity? Does vm_stat in Terminal report a sharp increase in pagein/pageout activity? If so, then maybe the VM system (or low memory due to leaks) is the problem. Otherwise I would be looking elsewhere for an explanation.
Alex

Yes, "one thing in each swap file" is my suspicion too b/c I see a tremendous thrashing of hard drive activity. That is why I have suspected VM all along, and noted that things get worse as the # of swap files grows. Even with 512M of RAM, simpl switching from Terminal to Mozilla can take 45 seconds of virtual lockup. Quitting all Apps does not seem to reduce the # of swap files. I'm glad VM works so well for most people, but just not seeing it for me under certain circumstances.

Looks like I'm back to the occassional reboot. Ah well, was hoping there was someting like a vm_restart -graceful command

Cheers,
Mike

[Earth Mk. II]

Originally posted by Zim:
...switching from Terminal to Mozilla ...

Out of curosity -- which build of Mozilla? Milestone release, official alpha/beta release, or a nightly?

[JNI]

Originally posted by Zim:
Yes, "one thing in each swap file" is my suspicion too b/c I see a tremendous thrashing of hard drive activity. That is why I have suspected VM all along, and noted that things get worse as the # of swap files grows. Even with 512M of RAM, simpl switching from Terminal to Mozilla can take 45 seconds of virtual lockup. Quitting all Apps does not seem to reduce the # of swap files. I'm glad VM works so well for most people, but just not seeing it for me under certain circumstances.

Looks like I'm back to the occassional reboot. Ah well, was hoping there was someting like a vm_restart -graceful command

Cheers,
Mike

I still don't see how your condition points to the VM system working poorly. If you have run out of physical memory, you have to swap no matter whether it is to one file or to many. I would guess that that actual paging of the needed data on disk is at least an order of magnitude more than the fact that it has to get the data from multiple files.

The bottom line of your problem still seems like you need more memory. 512M isn't a lot when you have apps like Mozilla that are memory pigs. It seems like even if it were to cache every page you've gone to in the last month, the ones you are constantly hitting should not be getting paged out if Mozilla was partitioning their paged data well. They shouldn't be swapping in and out the whole month's work of cache to get at a single page. That seems like a problem in Mozilla, not the OS. If it is because their bloated application code is being paged, well that's because they need to pull in all kinds of stuff just to kick start again. If you need the code, and it is paged, well, it just has to swap to get it if you don't have enough physical memory. When you start fresh they just haven't needed certain parts of the code yet, but will eventually. When you kick start again, they just need to get you back to the bloated state that you left their app. I think that is a side effect of the fact that they break up their app into like 100 or more libraries, and so if they need just one function from each of those, it probably has to page in all those libs just to kick start. And then there is all that Java stuff that they use (in the name of being cross platform). Jars are notoriously slow archive formats (on all platforms) for containing chunks of code that need to be accessed randomly.

I'm not saying you aren't having issues. I just think you are pointing at the wrong causes.

[Rainy Day]

Originally posted by Zim:
Yes, "one thing in each swap file" is my suspicion too b/c I see a tremendous thrashing of hard drive activity. That is why I have suspected VM all along, and noted that things get worse as the # of swap files grows. Even with 512M of RAM, simpl switching from Terminal to Mozilla can take 45 seconds of virtual lockup. Quitting all Apps does not seem to reduce the # of swap files. I'm glad VM works so well for most people, but just not seeing it for me under certain circumstances.

Looks like I'm back to the occassional reboot. Ah well, was hoping there was someting like a vm_restart -graceful command

For what it's worth, i have the same (virtually identical) problem. I have even less RAM (320 MB) and have made the decision not to add more RAM to this old computer because it max's out at 512 MB, and is just too much money to put into a computer i expect to replace in the not too distant future.

I've noticed that when the number of swapfiles grows to about 8, which usually takes about two weeks (given my usage), the machine's performance degrades like you mention (and to the point where a reboot is in order).

I've suspected the problem is related to "virtual memory fragmentation," where one process may require bits and pieces of virtual memory from multiple swapfiles. I don't know exactly how the VM goes about its job, but from what i surmise, it's not as efficient as it could be. It appears to me that when virtual memory spans multiple swapfiles, each file must be read in its entirety. I don't know that this is indeed the case, but it's doing an awful lot of thrashing if that's not what's going on. I can be doing something as simple as switching between a page in Safari (or iCab) and a page in Tex-Edit (both of which should easily fit within the 320 MB of my system) and it might thrash for 60-120 seconds.

When the system gets like that, i can close down all app's, but the swapfiles do not seem to disappear, and relaunching the app's does not yield any increase in performance. Only a reboot seems to do the trick.

As for logging out and letting it sleep overnight: Okay, but a reboot is no more inconvenient at that point, and a whole lot faster.

I have read about a way to restart the VM, but i don't think it's graceful and have never trusted the safety of that approach enough to try it.

[JNI]

Originally posted by Rainy Day:

When the system gets like that, i can close down all app's, but the swapfiles do not seem to disappear, and relaunching the app's does not yield any increase in performance. Only a reboot seems to do the trick.

As for logging out and letting it sleep overnight: Okay, but a reboot is no more inconvenient at that point, and a whole lot faster.

I have read about a way to restart the VM, but i don't think it's graceful and have never trusted the safety of that approach enough to try it.

From what I understand, when you quit apps the system still keeps around what it can in memory (and therefore possibly swap) for those apps for the case that you might start the same app again. That can possibly make relaunches faster. If waiting overnight seems to clear it, it sounds like maybe the system times out at some point waiting, and eventually cleans out all the old stuff and starts again. Or maybe some cron job or something has come along and needed a bunch of memory which causes the swap info to clear.

If that is the case, then, yes it does sound like the VM is maybe less than ideal. But it may be a side effect of some attempts at optimizations that just happen to result in worse case scenarios for you guys that have such little memory. A reliable way to quit all apps and reset the VM might be a good tool to have, though it may be a problem to truly quit all applications that may be swapped out (like system services and such). So your periodic reboot seems like your best bet unless either someone can come up with a tool to refresh your systems, or you decide it is in your best interest to get more memory (or a new machine.)

You might try something like quitting all the piggie apps and then run some simple hand-rolled tool that allocates a huge amount of memory and just spins through it and then quits just to see if that might force a cleansing. Just a thought.

[Rainy Day]

Originally posted by JNI:
From what I understand, when you quit apps the system still keeps around what it can in memory (and therefore possibly swap) for those apps for the case that you might start the same app again.

I don't know much about the internals of the VM system, so i really can't dispute that. But i am at least a little skeptical of it because it seems to me the overhead to do that would be greater than the overhead of simply freeing the space and reallocating as needed. Also, i suspect that the VM works by process ID, and a subsequent relaunch would have a different process ID.

If waiting overnight seems to clear it, it sounds like maybe the system times out at some point waiting, and eventually cleans out all the old stuff and starts again.

I think the person advocating this said the system goes to sleep, so if something magical happens, it must happen before it goes to sleep (or when the system awakens).

If that is the case, then, yes it does sound like the VM is maybe less than ideal. But it may be a side effect of some attempts at optimizations that just happen to result in worse case scenarios for you guys that have such little memory.

Entirely possible.

A reliable way to quit all apps and reset the VM might be a good tool to have, though it may be a problem to truly quit all applications that may be swapped out (like system services and such).

Speaking strictly for myself here, the biggest hassle of rebooting is the quitting of all apps (because of all the web pages and documents i have open). So once my app's are closed down, there's really no reason for me not to reboot. So even if such a program existed, why would i use it? It really wouldn't save me much time, and as it is now, i only need to reboot once a fortnight, so i can live with it.

So your periodic reboot seems like your best bet unless either someone can come up with a tool to refresh your systems, or you decide it is in your best interest to get more memory (or a new machine.)

Yes, and what this thread has done is confirm my suspicion that having 512 MB isn't going to do a great deal more for me than the 320 MB i already have, because i can expect to have the same problems still. I think a new machine is really the only cost effective solution for my situation.

You might try something like quitting all the piggie apps and then run some simple hand-rolled tool that allocates a huge amount of memory and just spins through it and then quits just to see if that might force a cleansing. Just a thought.

And it's a good thought too. But i've already tried that: Nothing short of a reboot seems to be effective.

[JNI]

Originally posted by Rainy Day:
I don't know much about the internals of the VM system, so i really can't dispute that. But i am at least a little skeptical of it because it seems to me the overhead to do that would be greater than the overhead of simply freeing the space and reallocating as needed. Also, i suspect that the VM works by process ID, and a subsequent relaunch would have a different process ID.

I know I've seen on the Java dev forums lots of discussion of the insane overhead of the Java classloaders in finding classes within jar files, pulling them out, possibly decompressing them, byte checking, security checking, and binding them up and how Apple has tried to deal with this problem. Apparently the Java dev guys are just livid that Sun chose zip files for their container format. So if you too are using something like one of the Mozilla based browsers that uses a lot of Java stuff mixed into the app, I can imagine the advantage of pulling a fully JIT'd and linked app out of the VM storage versus having to build it back up from scratch would be significant.

I think that wouldn't be as much a problem with Cocoa or Carbon apps. I believe the pre-binding thing eliminates some linking overhead, but, a big app can still utilize lots of dynamic libs that might be loading and (hopefully) unloading. There is a form of garbage collection in Cocoa that is happening, and huge amounts of temp objects are created all the time. And of course most apps have some amounts of memory leakage no matter what.

I think the person advocating this said the system goes to sleep, so if something magical happens, it must happen before it goes to sleep (or when the system awakens).

Well if Apple can do it, then presumably someone else could figure out a way to do it safely. It's 'simply' a matter of figuring out what that is. Of course, ideally, Apple would officially support a way to do it, either an app that could be run manually or run in the background to clean up a bit more frequently. But those are big 'ifs'.

Speaking strictly for myself here, the biggest hassle of rebooting is the quitting of all apps (because of all the web pages and documents i have open). So once my app's are closed down, there's really no reason for me not to reboot. So even if such a program existed, why would i use it? It really wouldn't save me much time, and as it is now, i only need to reboot once a fortnight, so i can live with it.

Yes, it would be nice to have a way to tell an app to save it's entire state and then restore that on launch. Again, presuming the data was still in memory, it would still be faster than doing it all manually. Another 'if only' ...

what this thread has done is confirm my suspicion that having 512 MB isn't going to do a great deal more for me than the 320 MB i already have, because i can expect to have the same problems still. I think a new machine is really the only cost effective solution for my situation.

Everything I've seen people talk about a minimum of 512M. But minimum is not usually any where near ideal. And if you happen to use exceptionally piggie apps, then it will really be off. I have 896M and don't see any real swapping issues. I currently have 2 swap files, and my machine has been up about a week or so. Of course, I don't use any of the Mozilla browsers because they just feel laggy to me. The rendering is good, but the app itself seems logey. I use Safari, unless I have to fire up something else for a particular site, and then I just quit the app immediately afterwards.

Anyways, I'm sure Apple is constantly looking for way to improve the system, and who knows, the next rev might just make some improvements there. We'll just have to see.

Until, then, let's hope the economy starts rolling along again soon so we can all move up to Dual (or Quad!) G5s with Gigs and Gigs of memory!

[Rainy Day]

Originally posted by JNI:
I know I've seen on the Java dev forums lots of discussion of the insane overhead of the Java classloaders

AFAIK, i'm not using any Java based app's. Not using any of the Mozilla browsers either. iCab and Safari suit my needs just fine. iCab has a real small RAM footprint, although i often have a lot of web pages open, which can chew up a lot of RAM (and is likely the greatest source of VM fragmentation that i have).

I think you're on the money with your assessment of Cocoa and Carbon apps.

Yes, it would be nice to have a way to tell an app to save it's entire state and then restore that on launch. Again, presuming the data was still in memory, it would still be faster than doing it all manually.

I had developed AppleScripts to save away the URL's of the pages i had open in iCab and Safari, but after tabbed browsing came along, i wasn't able to grab all the URL's anymore, so those scripts more or less don't work now. I might revisit that issue again, just to see if anything has changed in the AppleScript dictionaries. I don't really care if the page contents are preserved in memory; preserving URL's and reloading the pages are sufficient for my purposes. I can always go brew a cup of tea while the pages load!

Everything I've seen people talk about a minimum of 512M. But minimum is not usually any where near ideal. And if you happen to use exceptionally piggie apps, then it will really be off.

When MacOS X first came out, everyone was saying 300MB is the minimum. Now it's drifted upward a bit. But you know, it's really all a bit subjective and depends so much on one's usage (as you note). If i'm going to run iDVD or iMovie, i'm going to need a lot more RAM than if i'm just running a browser, eMail and a word processor. YMMV.

Until, then, let's hope the economy starts rolling along again soon so we can all move up to Dual (or Quad!) G5s with Gigs and Gigs of memory!

Here in Oregon, the economy has been in the dumpster for the last three and a half years. I've pretty much given up on it and my plan now is to start buying lottery tickets!

[Zim]

Originally posted by JNI:
From what I understand, when you quit apps the system still keeps around what it can in memory (and therefore possibly swap) for those apps for the case that you might start the same app again.

Yes, I've long suspected that myself. My belief is based upon using Preview to go thru digital pics after a trip. I'll launch 10-20 at a time, figure out which ones I want to keep, and delete the rest. If I mistakenly include a "kept" photo in the next batch of photos, it comes to screen MUCH faster than others (or something like that), making me think that some amt of "caching" is going on.

As said, usually this would be a "good thing", but if there is not a good expiry time for this, then it becomes a bloating "bad thing".

And yeah, when 10.0 came out, my 512M was supposedly WAY more than enough. I also will not be investing more in this machine, waiting til Apple refreshes the line-up, and til job situation is more stable.

Essentially it would seem a "VM defrag" tool is needed (or the existing one needs to be improved, and/or tweakable)

Thanks all,
Mike

[CatOne]

Originally posted by Zim:
Yes, I've long suspected that myself. My belief is based upon using Preview to go thru digital pics after a trip. I'll launch 10-20 at a time, figure out which ones I want to keep, and delete the rest. If I mistakenly include a "kept" photo in the next batch of photos, it comes to screen MUCH faster than others (or something like that), making me think that some amt of "caching" is going on.

As said, usually this would be a "good thing", but if there is not a good expiry time for this, then it becomes a bloating "bad thing".

And yeah, when 10.0 came out, my 512M was supposedly WAY more than enough. I also will not be investing more in this machine, waiting til Apple refreshes the line-up, and til job situation is more stable.

Essentially it would seem a "VM defrag" tool is needed (or the existing one needs to be improved, and/or tweakable)

Thanks all,
Mike

Why is it a bloated, bad thing? Because launches could be faster? If you launch a new app and an old one has been cached, it will be flushed if there's not enough free RAM.

JUST SAY NO to tinkering with the VM files. The OS is smarter than you are in these respects. UNIX has been paying attention to these things for going on 30 years now. Worrying about something from a naieve reading of top or listing of /var/vm is ridiculous.

[Rainy Day]

Originally posted by Zim:
Yes, I've long suspected that myself. My belief is based upon using Preview to go thru digital pics after a trip. I'll launch 10-20 at a time, figure out which ones I want to keep, and delete the rest. If I mistakenly include a "kept" photo in the next batch of photos, it comes to screen MUCH faster than others (or something like that), making me think that some amt of "caching" is going on.

Yes, i've seen app's cache files too, but that's entirely different than the proposition that the VM keeps memory allocated for a recently terminated process (i.e. application) just in case the app is subsequently relaunched.

I am skeptical of that proposition, but wouldn't be at all surprised if the VM takes its own sweet time to deallocate resources for terminated processes, perhaps at its next garbage collection cycle? As far as i know, the VM of MacOS X is the standard UNIX VM. UNIX in general works off of process ID's. Since unique process ID's are assigned when an application is launched, each launch of an application will produce a different process ID. So there is no easy way for the VM to know it is the same app. Furthermore, under UNIX, it is possible (even common) to have multiple concurrent instances of any given app, each with a different process ID. Besides, it seems to me the overhead of figuring out if a new application launch is an app the VM has seen before would exceed that of just allocating new resources for this new process ID.

Based on my own observations of quitting memory hog app's, and not seeing any immediate difference in the number of swapfiles, it is entirely possible the VM marks a terminated app's resources for deletion, but waits until its next garbage collection cycle before disposing of those resources. If that is so, then it would explain why logging out and waiting hours would clear things up (i.e. all user app's have been quit, and there is time for the garbage collection cycle to do its thing). This would imply that similar results might be acheived by quitting memory hog app's, and letting the computer sit (still logged in). But one should not expect quite as dramatic of results because fewer app's have been quit, etc.

What i do know is that all preexisting swapfiles are deleted on system startup by this code in the /etc/rc startup script (Jaguar), so a restart is a sure way to clear everything out:

Code:
# Make sure the swapfile exists
if [ ! -d ${swapdir} ]; then
    ConsoleMessage "Creating default swap directory"
    mount -uw /
    mkdir -p -m 755 ${swapdir}
    chown root:wheel ${swapdir}
else
    rm -rf ${swapdir}/swap*
fi

[mbryda]

I think the culprit may be the WindowManager or orher GUI related processes. When you have multiple windows open, it uses more RAM to track them (I apologize- I forget which process), and if you are running low, it will swap these out to disk. There is probably a bug in the code that this RAM is not deallocated in a timely fashion, resulting in the slowness. Logging out/in will kill that process and leaving it at the logon screen will allow the VM subsystem to sense a need for less swapfiles and reduce them accordingly. Logging out will only leave the bare processes in memory, killing any non-essential PID's and "defragmenting" the RAM allocation. That's why it works to aleviate sluggishness without a reboot.

The other thing is that the VM system is working as designed. Say you have 3 apps open that need 100MB of RAM and you only have 50MB free. You open another app that needs 150 MB. All that 100MB is going to be swapped to disk to make room for the new program. You work in that program for a bit and then quit it and go back to one of the others (#1). The VM system has to swap that back in to RAM. If you use program #2, the VM system has to swap that back to RAM. And so on for app #3. Once you get #1,2,3 back in RAM, the performance should be a lot better, unless you have another process hogging RAM. No VM system swaps things back to RAM until you use them. It will just swap as needed back to RAM and/or disk.

Hope that helps clear things up.

[Zim]

Originally posted by CatOne:
Why is it a bloated, bad thing? Because launches could be faster? If you launch a new app and an old one has been cached, it will be flushed if there's not enough free RAM.

JUST SAY NO to tinkering with the VM files. The OS is smarter than you are in these respects. UNIX has been paying attention to these things for going on 30 years now. Worrying about something from a naieve reading of top or listing of /var/vm is ridiculous.

It is a "bloated bad thing" if a "defrag" (or equivalent) needs to happen when page-outs for a single app are spread among different swap files (if that happens)... given the thrashing I've seen and the very few apps open at that time, something less than optimal is certainly going on.

If the OS were so smart, I'd never see this problem.

As for naive, gosh, I've only been using unix daily since 1988(?) (unix at work, Mac by night). Glad to know we have superior people like yourself. Maybe you could use small words (so that we can follow along) and explain why the smart OS takes so long to recover the memory and reduce the swap_file thrashing even after the apps themselves are no longer running?

Lest anyone mistake things, I'm generally thrilled with the OS, and that Mac is finally unix-based, but certainly not thrilled with this one particular effect of the OS.

Mike

[ASIMO]

Excellent points, Zim and others. This less than ideal behavior irks me about OS X, and I feel it cathartic to reboot every couple of weeks or so to regain some speed and space.

Churchill, on OS X, just before his other, more famous quote: "Mac OS X is like democracy."

[Rainy Day]

Originally posted by mbryda:
I think the culprit may be the WindowManager or orher GUI related processes. When you have multiple windows open, it uses more RAM to track them (I apologize- I forget which process), and if you are running low, it will swap these out to disk. There is probably a bug in the code that this RAM is not deallocated in a timely fashion, resulting in the slowness.

Logging out/in will kill that process and leaving it at the logon screen will allow the VM subsystem to sense a need for less swapfiles and reduce them accordingly. Logging out will only leave the bare processes in memory, killing any non-essential PID's and "defragmenting" the RAM allocation. That's why it works to aleviate sluggishness without a reboot.

Well previously i tried just logging out and back in again. Didn't help. Rebooted and everything was fine. Apparently you have to logout and wait several hours (such as overnight). I've never tried that, and probably won't; much faster (and no more inconvenient) to just reboot (which will fix the problem).

I suspect what it boils down to is this: If you don't have enough RAM, expect to have to reboot regularly at certain intervals, the length of which depends on the amount of RAM you do have, and your usage patterns. In other words: YMMV.

Originally posted by Zim:
It is a "bloated bad thing" if a "defrag" (or equivalent) needs to happen when page-outs for a single app are spread among different swap files (if that happens)... given the thrashing I've seen and the very few apps open at that time, something less than optimal is certainly going on.

If the OS were so smart, I'd never see this problem.

For what it's worth, i agree completely with your assessment. Based on my experiences with the dynamic pager (VM), i'd have to say it's nothing to brag about (at least not under the load imposed by MacOS X's GUI). I suppose if you have enough RAM, the dynamic pager is doing very little work on your system, so you never see its ugly thrashing under load. But when you have to wait 60 seconds or longer just to switch between already open windows, you kind of come away with the impression that it just isn't as efficient as it should be. And once it's pushed to the point that the thrashing begins, you can't just back off, close up a couple of app's, and have the system revert back to normal efficiency.

Now it may be that the dynamic pager just isn't tuned properly for our hardware configurations. I mean, i would like to think that a quarter century is sufficient time to develop a decent dynamic pager. So i have wondered if larger swapfiles might not reduce the thrashing? Because i strongly suspect it's not just switching out one page and switching in another, but rather going back and forth between the same few pages (i.e. swapping out memory to page A, loading in page B, then swapping out B and loading back in A, etc., back and forth, until it gets all the bits and pieces it needs to complete whatever it thinks it needs to do.) I figure if it has larger swapfiles, chances are better that all the pieces will lie in a single swapfile. I also have to note that the official minimum RAM supported by MacOS X is 128 MB, and swapfiles are set at about 78 MB. Maybe an optimal size for a 320 MB system is closer to 200 MB? Hmmm. Might be worth a try!

But i will say MacOS X's VM system is better than MacOS 9's, and i suspect far better than anything M$ is capable of producing. But those comparisons don't change the fact it appears to behave like a dog on my Mac. Just my 2¢ worth and clearly a personal opinion.

[mbryda]

Originally posted by Rainy Day:
Well previously i tried just logging out and back in again. Didn't help. Rebooted and everything was fine. Apparently you have to logout and wait several hours (such as overnight). I've never tried that, and probably won't; much faster (and no more inconvenient) to just reboot (which will fix the problem).

That I don't know - I always do it at the end of a working session. I come back to a computer ready to work once again and at normal speeds. I do think that the VM system may wait a while before figuring things out, possibly to make use of a decent amount of cached information. Which has me wondering if there's a setting somewhere to control the idle time before a VM flush.

I suspect what it boils down to is this: If you don't have enough RAM, expect to have to reboot regularly at certain intervals, the length of which depends on the amount of RAM you do have, and your usage patterns. In other words: YMMV.

I have a feeling it's more to do with the usage patterns than anything else. The wife's iBook with 256MB and 2 users logged in seems pretty snappy, even after lots of uptime. I'm wondeting if the usage and apps make a bigger deal than we are thinking.

So i have wondered if larger swapfiles might not reduce the thrashing?

Could be - that's how MS does it - just one huge pagefile.sys that grows and is supposed to shrink base don load. I would think you may be right - the larger the swapfiles, the less thrashing - as long as they are kept contiguous.

I also have to note that the official minimum RAM supported by MacOS X is 128 MB, and swapfiles are set at about 78 MB. Maybe an optimal size for a 320 MB system is closer to 200 MB? Hmmm. Might be worth a try!

The minimums are pointless. MS said you would run Win2k on a Pentium/133 with 64MB. I've run on a K5/133 (faster thana Pentium) with 96MB and it was unbearable. You calso can try running WinXP in 128MB, but the speed is abysmal on anything less than 512MB. I ran it on a P3/1.2Ghz with 256 for 2 years, and opening up Notes, Office, and Mozilla would bring that system to its knees with disk thrashing.

Same with a modern Linux distro on 256MB. It can be done, but it ain't pretty.

But i will say MacOS X's VM system is better than MacOS 9's, and i suspect far better than anything M$ is capable of producing. But those comparisons don't change the fact it appears to behave like a dog on my Mac. Just my 2¢ worth and clearly a personal opinion.

Don't know about OS9 - I didn't get a Mac until 2002, and never liked the "classic" interface at all - OSX is what got me switching. I would say the VM systems on the 2 systems are close with none having the clear advantage. I'd say OSX is better, as with XP if you run to VM, you can watch the screen redraw line by line. But, the realistic minimum for both OS's is 512MB, regardless of what they say the minimums are.

[hudson1]

My general impression was the the number of swapfiles was irrelevant as long as it wasn't some very big number (indicating a possible leak). Isn't it the frequency of pageins and pageouts that's the real indicator of memory status?

[m2bored]

I have 768MB of RAM and have this exact same problem!!!

I dont use any large apps either.
I have the following open at all times.

Safari
iChat
Appleworks
Mail
iCal
a menubar app called Deskshade Plus
and another menubar dealie called...something i cant remem

and then occasionally i will use address book, itunes and a VNC app

That is iT!!!

I feel like I need a fresh install...I have been upgrading since 10.1...