Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > Mac OS X > Spotlight permenantly estimating indexing time

Spotlight permenantly estimating indexing time
Thread Tools
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 8, 2008, 11:52 AM
 
So last week, I did a spotlight search on my MacBook which for some reason resulted in a kernel panic.

I had to rebuild the directory structure using Disk Warrior - took me a while to get it working due to stupidity


Since then, Spotlight has been estimating indexing time... for about a week. I'm bored now and I want to be able to launch apps through it and search my email.

Console harbors these messages:



I've tried deleting my .spotlight-v100 with no effect. I've also tried sudo mdutil -E / to re-index the drive with no luck.

Can anyone lend me some wisdom?
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 06:14 AM
 
Everything I search for related to this issue points to Time Machine being a problem. I'm not running Time Machine and it's been turned off for months.

Any ideas?
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 12:29 PM
 
OK, so using Spotless, I can see that Spotlight chokes after 1.82mb, even after deleting the index. Is there a way I can see what files it's stuck on?

It may well be that I need to run Disk Warrior again and do another check on the drive.
     
Clinically Insane
Join Date: Oct 2000
Location: Los Angeles
Status: Offline
Reply With Quote
Nov 10, 2008, 12:46 PM
 
Have you tried excluding your drive in the privacy pane for a while and then trying to index it again? That usually cures difficult Spotlight problems.

"The natural progress of things is for liberty to yield and government to gain ground." TJ
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 12:49 PM
 
Originally Posted by Big Mac View Post
Have you tried excluding your drive in the privacy pane for a while and then trying to index it again? That usually cures difficult Spotlight problems.
No I haven't. I thought the mdutil commands would kind of override that. I'll give it a shot.

Thanks.
     
Clinically Insane
Join Date: Oct 2000
Location: Los Angeles
Status: Offline
Reply With Quote
Nov 10, 2008, 01:15 PM
 
It probably won't help but you never know.

"The natural progress of things is for liberty to yield and government to gain ground." TJ
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 01:17 PM
 
Doesn't seem to have. My OS X 10.5 index size is 2.0MB. There must be a log somewhere telling me what it's actually doing...

Console is still filling up with mds errors.
     
Grizzled Veteran
Join Date: Mar 2004
Status: Offline
Reply With Quote
Nov 10, 2008, 03:39 PM
 
Originally Posted by seanc View Post
OK, so using Spotless, I can see that Spotlight chokes after 1.82mb, even after deleting the index.
Is there a way I can see what files it's stuck on?

sudo fs_usage -f filesys mds mdworker mdimport

...should produce a ton of text for you to look at. [normally it's not too noisy]
Use ctrl-C to cancel execution.


Originally Posted by seanc View Post
Console is still filling up with mds errors.
Are they still mostly about "Importer start failed"?

I recall a case or two where older (pre-Leopard) mdimporter items caused Spotlight to act weird, when users migrated in older apps (some apps store importers inside their package). I have no idea if this is the case here but, here are two commands that can dig up a list of existing ones:

# if your locate database is up to date, this one is very fast:
locate mdimporter |grep mdimporter$ |grep -v /Volumes/ |
while IFS= read -r thisItem; do ls -ld "$thisItem"; done

# this one is slower but guarantees accuracy, plus sorts by date (oldest last):
sudo find -x / -name \*.mdimporter -exec ls -ldtT {} +

Anything dated 2006 or earlier might be unaware of how Leopard's Spotlight likes to do things.

idunno.
-HI-
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 04:35 PM
 
This command sudo fs_usage -f filesys mds mdworker mdimport gave me the below output (I trimmed it - just imagine it duplicated lots):

Code:
21:27:50 fcntl 0.000010 mds 21:27:50 fcntl 0.000006 mds 21:27:50 fcntl 0.000005 mds 21:27:50 fcntl 0.000005 mds 21:27:50 fcntl 0.000004 mds 21:27:50 fstat 0.000007 mds 21:27:50 open . 0.000020 mds 21:27:50 close 0.000008 mds 21:27:50 fstat 0.000005 mds 21:27:50 open . 0.000011 mds 21:27:50 close 0.000006 mds 21:27:50 fstat 0.000003 mds 21:27:50 open . 0.000008 mds 21:27:50 close 0.000005 mds 21:27:50 read 0.166497 W mds 21:27:52 read 2.439737 W mds 21:27:54 read 1.560676 W mds 21:27:55 open /Volumes/Spotless 2 0.000029 mds 21:27:55 fcntl 0.000006 mds 21:27:55 close 0.000010 mds 21:27:55 open 0.0000
It doesn't seem to be doing much, but it seems to be doing that a lot.

Yep, the errors in the console are still about 'Importer start failed'.

This command:

locate mdimporter |grep mdimporter$ |grep -v /Volumes/ |
while IFS= read -r thisItem; do ls -ld "$thisItem"; done


Showed me that nothing was older than September 2007.


It's just broken

I don't know whether to blame my recent disk issues or something in Leopard.

The only common thing between my previous disk issues and my setup now, is the hard drive and Leopard installation. I'm running it in a different MacBook which rules out the RAM, logic board and everything else of the other MacBook for causing issues.
     
Grizzled Veteran
Join Date: Mar 2004
Status: Offline
Reply With Quote
Nov 10, 2008, 04:55 PM
 
Hmm, your mds seems to be doing a lot of nothing (file-wise).

Here's an example of mine when I double-click on a text file:
Code:
16:45:25.041 read F=4 B=0x108 8.119014 W mds 16:45:28.399 fcntl F=6 <PATHPKG_CHECK> 0.000010 mds 16:45:28.399 open F=14 (R_____) /Users/halito/Desktop/todo.rtf 0.000012 mds 16:45:28.399 fcntl F=14 <GETPATH> 0.000003 mds 16:45:28.399 close F=14 0.000009 mds 16:45:28.399 write F=42 B=0x8c 0.000014 mds 16:45:28.399 fstat F=12 0.000004 mds 16:45:28.400 open F=14 (R_____) . 0.000010 mds 16:45:28.400 pwrite F=24 B=0x4000 O=0x00911000 0.000038 mds 16:45:28.401 close F=14 0.000009 mds 16:45:28.401 fsctl 0.000010 mds 16:45:28.401 fcntl F=6 <PATHPKG_CHECK> 0.000009 mds 16:45:28.401 lstat /Users/halito/Desktop/todo.rtf 0.000040 mdworker 16:45:28.401 getattrlist /Users/halito/Desktop/todo.rtf 0.000009 mdworker
I don't know what all the " F=14" stuff is about, and the rest... but you can see it's doing something.

Are there no entries for mdworker in yours?
-HI-
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 05:17 PM
 
Interestingly, I ran it again in a bigger terminal window and got more information than I got last time.

Code:
22:11:05.386 read F=4 B=0x248 2.615739 W mds 22:11:05.386 fcntl F=6 <PATHPKG_CHECK> 0.000038 mds 22:11:05.386 fcntl F=6 <PATHPKG_CHECK> 0.000024 mds 22:11:05.387 fcntl F=6 <PATHPKG_CHECK> 0.000023 mds 22:11:06.533 open F=8 (R_____) /Volumes/Spotless 2 0.000028 mds 22:11:06.533 fcntl F=8 <GETPATH> 0.000006 mds 22:11:06.533 close F=8 0.000011 mds 22:11:06.534 open F=8 (R_____) 0.000012 mds 22:11:06.534 fcntl F=8 <GETPATH> 0.000004 mds 22:11:06.534 close F=8 0.000007 mds 22:11:07.117 read F=4 B=0x52 1.730327 W mds 22:11:08.313 read F=4 B=0x52 1.196571 W mds 22:11:11.109 read F=4 B=0x52 2.796104 W mds 22:11:14.775 read F=4 B=0x52 3.665684 W mds 22:11:17.121 read F=4 B=0x52 2.345962 W mds 22:11:18.319 read F=4 B=0x52 1.197924 W mds 22:11:21.536 open F=8 (R_____) /Volumes/Spotless 2 0.000037 mds 22:11:21.536 fcntl F=8 <GETPATH> 0.000007 mds 22:11:21.536 close F=8 0.000014 mds 22:11:21.537 open F=8 (R_____) 0.000013 mds 22:11:21.537 fcntl F=8 <GETPATH> 0.000004 mds 22:11:21.537 close F=8 0.000008 mds 22:11:23.778 read F=4 B=0x52 5.459219 W mds 22:11:26.642 read F=4 B=0x1ff 2.863773 W mds 22:11:26.642 fcntl F=6 <PATHPKG_CHECK> 0.000013 mds 22:11:26.642 fcntl F=6 <PATHPKG_CHECK> 0.000005 mds 22:11:26.642 fcntl F=6 <PATHPKG_CHECK> 0.000004 mds 22:11:26.642 fcntl F=6 <PATHPKG_CHECK> 0.000004 mds 22:11:26.642 fcntl F=6 <PATHPKG_CHECK> 0.000015 mds 22:11:27.112 read F=4 B=0x52 0.470034 W mds 22:11:28.323 read F=4 B=0x52 1.211156 W mds 22:11:31.113 read F=4 B=0x52 2.789521 W mds 22:11:32.783 read F=4 B=0x52 1.669703 W mds 22:11:35.114 read F=4 B=0x52 2.331000 W mds 22:11:36.533 open F=8 (R_____) /Volumes/Spotless 2 0.000030 mds 22:11:36.533 fcntl F=8 <GETPATH> 0.000006 mds 22:11:36.533 close F=8 0.000011 mds 22:11:36.534 open F=8 (R_____) 0.000013 mds 22:11:36.534 fcntl F=8 <GETPATH> 0.000004 mds 22:11:36.534 close F=8 0.000008 mds 22:11:37.129 read F=4 B=0x52 2.015732 W mds 22:11:38.206 read F=4 B=0x53 1.076770 W mds 22:11:38.328 read F=4 B=0x52 0.122015 W mds 22:11:44.786 read F=4 B=0x52 6.457802 W mds 22:11:47.116 read F=4 B=0x52 2.329680 W mds 22:11:48.335 read F=4 B=0x52 1.218854 W mds 22:11:51.117 read F=4 B=0x52 2.781833 W mds 22:11:51.269 read F=4 B=0x1c9 0.152433 W mds 22:11:51.269 fcntl F=6 <PATHPKG_CHECK> 0.000021 mds 22:11:51.269 fcntl F=6 <PATHPKG_CHECK> 0.000017 mds 22:11:51.289 read F=4 B=0x11f 0.019645 W mds 22:11:51.289 fcntl F=6 <PATHPKG_CHECK> 0.000017 mds 22:11:51.289 fstat F=14 0.000008 mds 22:11:51.289 open F=8 (R_____) . 0.000018 mds 22:11:51.289 close F=8 0.000012 mds 22:11:51.346 open F=8 (R_____) 0.000009 mds 22:11:51.346 fcntl F=8 <GETPATH> 0.000004 mds 22:11:51.346 close F=8 0.000006 mds 22:11:51.356 read F=4 B=0x244 0.067454 W mds 22:11:51.356 fcntl F=6 <PATHPKG_CHECK> 0.000011 mds 22:11:51.356 fcntl F=6 <PATHPKG_CHECK> 0.000008 mds
There's nothing in there from mdworker, mds seems to be hogging the show.

mdworker isn't running under Activity Monitor.

If I
Code:
locate mdworker
I get the following results:
Code:
/Developer/Documentation/DocSets/com.apple.ADC_Reference_Library.DeveloperTools.docset/Contents/Resources/Documents/documentation/Darwin/Reference/ManPages/man8/mdworker.8.html /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mdworker /usr/share/man/man8/mdworker.8 /usr/share/sandbox/mdworker.sb
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 10, 2008, 05:57 PM
 
The "F=14" is just the file descriptor. When you open a file in code, you get a file descriptor to it, which is just a number referring to the open file. You can pass this file descriptor to functions like read() and write() to work with the file. We don't need to worry about the file descriptor.

One thing that might help is to check to see what files mdworker and mdimport have open - if one of them opened a file before you ran the fs_usage tool, and then got stuck, the file could still be open. I dunno whether this will help at all, but it's worth a try:

Code:
sudo lsof | egrep mdworker\|mdimport\|mds
Of course, you could run the fs_usage program before you start a clean re-indexing of the drive... but you'd end up with a really huge log file. There might be a way to read the info from the FSEvents log... but then again, Spotlight uses FSEvents heavily and I suppose it's possible that the logs in .fseventsd could have become damaged by your file system troubles. Did you notice any errors related to fseventsd in your console/system logs?

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 06:03 PM
 
I'm starting to think mdworker and mdimport are corrupt or missing since I'm seeing no record of them running anywhere.
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 10, 2008, 06:05 PM
 
The .fseventsd log, which mdworker and mdimport make use of, could be corrupt.

Have you seen any errors related to fseventsd?

edit: also, if mdworker and mdimport are not running, they may have crashed. Are there any crash logs for them in /Library/Logs/CrashReporter (note: this would be the /Library at the root of your drive, not the one in your home folder)?
( Last edited by CharlesS; Nov 10, 2008 at 06:28 PM. )

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Grizzled Veteran
Join Date: Mar 2004
Status: Offline
Reply With Quote
Nov 10, 2008, 06:07 PM
 
mdworker isn't running under Activity Monitor.
That certainly sounds like a problem.

What do you get for this?
ps axcru |sed '1p;/ md/!d'
(procs starting with md)

And just for the record:
mdutil -a -s
?
-HI-
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 06:30 PM
 
FSevents errors in console:

Code:
10/11/2008 23:19:24 fseventsd[47] bumping event counter to: 0x50fa7 (current 0x0) from log file '0000000000050d3a' 10/11/2008 23:19:36 fseventsd[47] log dir: /Volumes/Windows XP/.fseventsd getting new uuid: 3245C730-2A27-4AEA-8848-143B9AE5A813 10/11/2008 23:19:36 /System/Library/CoreServices/coreservicesd[66] /dev/fsevents filter failed! (Bad file descriptor)
Processes starting with md:
Code:
root 40 0.0 0.4 105948 7712 ?? Ss 11:19pm 0:01.38 mds
No mdimport or mdworker

mdutil -a -s
Code:
/: Indexing enabled. /Volumes/Windows XP: No index.
Disk Warrior says all is OK.

I did find these related to mdworker in console:

Code:
10/11/2008 23:19:35 com.apple.launchd[1] (0x10f400.mdworker[299]) getpwuid("4294967294") failed 10/11/2008 23:19:35 com.apple.launchd[1] (0x10f400.mdworker[299]) PID 143 "AppleVNCServer" has no account to back it! Real/effective/saved UIDs: 0/4294967294/0 10/11/2008 23:19:35 com.apple.launchd[1] (0x10f400.mdworker[299]) PID 139 "ARDAgent" has no account to back it! Real/effective/saved UIDs: 0/4294967294/0 10/11/2008 23:19:35 com.apple.launchd[1] (0x10f400.mdworker[299]) Exited with exit code: 1 10/11/2008 23:19:35 com.apple.launchd[1] (0x10f400.mdworker) Failed to check-in!
I disabled remote management for now.
     
Grizzled Veteran
Join Date: Mar 2004
Status: Offline
Reply With Quote
Nov 10, 2008, 06:35 PM
 
Originally Posted by CharlesS View Post
One thing that might help is to check to see what files mdworker and mdimport have open - if one of them opened a file before you ran the fs_usage tool, and then got stuck, the file could still be open. I dunno whether this will help at all, but it's worth a try: sudo lsof | egrep mdworker\|mdimport\|mds
Files already opened... excellent point.

Looks like lsof's '-c' switch eliminates the need for piping to egrep (in this case):
sudo lsof -c md
...works nicely.
-HI-
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 10, 2008, 06:37 PM
 
Okay, since we've got some FSEvents errors, it's possible that the .fseventsd logs could be corrupt. You could try deleting them - the problem is that I have no idea whether this is safe to do or not so you might want to back up any important data first. If you want to try this, I'd boot into Single-User Mode first (to cut down on the number of daemons that might be relying on FSEvents for something), then run the following:

Code:
/sbin/fsck -fy mount -uw / cd / mv .fseventsd fseventsd.bak reboot
That might solve your problem. Hopefully it doesn't muck things up worse, but I'd hope that if the .fseventsd directory isn't there, Leopard would be smart enough to rebuild it (after all, a drive migrated from some older OS X version wouldn't have the .fseventsd directory there). Do back up first though.

I've got to go somewhere for a while, so good luck with your issue - I'm keeping my fingers crossed.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 06:38 PM
 
I'd forgotten to post the output of that command.

Code:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME mds 40 root cwd DIR 14,2 1462 2 / mds 40 root twd DIR 14,2 1462 2 / mds 40 root twd DIR 14,2 1462 2 / mds 40 root twd DIR 14,2 1462 2 / mds 40 root twd DIR 14,2 1462 2 / mds 40 root txt REG 14,2 1788848 2005783 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mds mds 40 root txt REG 14,2 59568 33934 /System/Library/PrivateFrameworks/MDSChannel.framework/Versions/A/MDSChannel mds 40 root txt REG 14,2 43691 3472829 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/0.indexGroups mds 40 root txt REG 14,2 2048 3472835 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/0.indexCompactDirectory mds 40 root txt REG 14,2 2731 3472840 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexGroups mds 40 root txt REG 14,2 2591600 2006835 /System/Library/PrivateFrameworks/SpotlightIndex.framework/Versions/A/SpotlightIndex mds 40 root txt REG 14,2 8224 3472834 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/0.indexDirectory mds 40 root txt REG 14,2 524288 3472828 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/0.indexIds mds 40 root txt REG 14,2 32768 3472839 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexIds mds 40 root txt REG 14,2 8224 3472845 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexDirectory mds 40 root txt REG 14,2 1024 3472846 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexCompactDirectory mds 40 root txt REG 14,2 84400 15647 /usr/lib/liblangid.dylib mds 40 root txt REG 14,2 126648 6337 /System/Library/Frameworks/CoreFoundation.framework/Versions/A/Resources/tokruleLE.data mds 40 root txt REG 14,2 10235872 2008602 /usr/share/icu/icudt36l.dat mds 40 root txt REG 14,2 5151552 82065 /usr/share/langid/langid.inv mds 40 root txt REG 14,2 5468448 15740 /usr/lib/libmecab.1.0.0.dylib mds 40 root txt REG 14,2 1059776 773402 /usr/lib/dyld mds 40 root txt REG 14,2 133242880 2222522 /private/var/db/dyld/dyld_shared_cache_i386 mds 40 root 0r CHR 3,2 0t0 66006404 /dev/null mds 40 root 1 PIPE 0x3fb6db0 16384 mds 40 root 2 PIPE 0x3fb6db0 16384 mds 40 root 3r REG 14,2 2512 3777941 /private/var/run/utmpx mds 40 root 4u KQUEUE count=0, state=0x2 mds 40 root 5r DIR 14,2 1462 2 / mds 40 root 6 FSEVENT mds 40 root 7u unix 0x454d110 0t0 ->0x3fb4dd0 mds 40 root 8r DIR 14,2 1462 2 / mds 40 root 9r REG 14,2 126648 6337 /System/Library/Frameworks/CoreFoundation.framework/Versions/A/Resources/tokruleLE.data mds 40 root 10u REG 14,2 28 3472825 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/indexState mds 40 root 12r DIR 14,2 1564 3472824 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2 mds 40 root 13u REG 14,2 840 3778253 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/journalAttr.11 mds 40 root 14u REG 14,2 2731 3472840 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexGroups mds 40 root 15u REG 14,2 32768 3472839 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexIds mds 40 root 16u REG 14,2 4096 3472837 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexHead mds 40 root 17u REG 14,2 0 3472838 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexUpdates mds 40 root 18u REG 14,2 65536 3472841 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexPostings mds 40 root 19u REG 14,2 65536 3472842 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexTermIds mds 40 root 20u REG 14,2 131072 3472843 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexPositions mds 40 root 21u REG 14,2 65536 3472844 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexPositionTable mds 40 root 22u REG 14,2 8224 3472845 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexDirectory mds 40 root 23u REG 14,2 1024 3472846 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexCompactDirectory mds 40 root 24u REG 14,2 131072 3472847 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/live.0.indexArrays mds 40 root 34u REG 14,2 1675264 3472848 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/store.db mds 40 root 35u REG 14,2 1675264 3472849 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/.store.db mds 40 root 36r DIR 14,2 1462 2 / mds 40 root 37u REG 14,2 4214 3472851 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/journalLive mds 40 root 39u REG 14,2 0 3472852 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/journalExclusion mds 40 root 40u REG 14,2 1504239 3472853 /.Spotlight-V100/Store-V1/Stores/89AC1BF0-4EF6-4635-9577-1A6C1F3DC2B2/journalSync mds 40 root 42r REG 14,2 5151552 82065 /usr/share/langid/langid.inv mds 40 root 63u unix 0x4532c38 0t0 ->0x4532cc0
Still no signs of mdworker and it's not looking at anything but system files and spotlight files at the moment it seems.
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 10, 2008, 06:51 PM
 
Originally Posted by CharlesS View Post
Okay, since we've got some FSEvents errors, it's possible that the .fseventsd logs could be corrupt. You could try deleting them - the problem is that I have no idea whether this is safe to do or not so you might want to back up any important data first. If you want to try this, I'd boot into Single-User Mode first (to cut down on the number of daemons that might be relying on FSEvents for something), then run the following:

Code:
/sbin/fsck -fy mount -uw / cd / mv .fseventsd fseventsd.bak reboot
That might solve your problem. Hopefully it doesn't muck things up worse, but I'd hope that if the .fseventsd directory isn't there, Leopard would be smart enough to rebuild it (after all, a drive migrated from some older OS X version wouldn't have the .fseventsd directory there). Do back up first though.

I've got to go somewhere for a while, so good luck with your issue - I'm keeping my fingers crossed.
Well I don't mind being a guinea pig, it's broken anyway.

You'll be pleased to know that it didn't break anything and it made a new .fseventsd

I got excited when mdworker appeared in Activity Monitor, but it didn't stick around for the party

Thanks for the help so far Charles & Hal, eventually we'll get it working again.
     
Grizzled Veteran
Join Date: Mar 2004
Status: Offline
Reply With Quote
Nov 10, 2008, 10:09 PM
 
Speaking of FSEvents, I've found >fseventer< to be a magnificent tool for
both troubleshooting and learning (via occasional peeks under the hood).

Maybe it will reveal something to us here, now?

(after entering your password, start it running, and collapse the toolbar).
-HI-
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 01:34 AM
 
Originally Posted by seanc View Post
Well I don't mind being a guinea pig, it's broken anyway.

You'll be pleased to know that it didn't break anything and it made a new .fseventsd

I got excited when mdworker appeared in Activity Monitor, but it didn't stick around for the party

Thanks for the help so far Charles & Hal, eventually we'll get it working again.
Did mdworker leave a crash log behind?

Also, did you try nuking .Spotlight-V100 again after rebuilding .fseventsd?
( Last edited by CharlesS; Nov 11, 2008 at 01:53 AM. )

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 01:55 AM
 
Here's something odd I found via Google:

http://discussions.apple.com/thread....sageID=8242041

You wouldn't have the Safari 4 beta installed, would you?

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 07:15 AM
 
FSeventer is interesting, but unless I'm missing something, it doesn't keep a complete log. When something is finished with, like .Spotlight-V100, it disappears.

Unfortunately, I don't have Safari V4 either.

I re-installed the 10.5.5 combo update but that didn't make a difference unfortunately.

The same problems appear under a brand new user account.

If I kill mds, mdworker will appear, running as root and start doing some intensive CPU work, but it stops after a while and closes. I sometimes have an mdworker running under my user.

I can't think of what to do other than an archive and install, which isn't a very good fix because we won't have solved the problem for our own benefits or for future searches.

Right now, I want to (somehow) reset every bit of Spotlight back to the beginning and see what happens.
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 12:53 PM
 
So I take it you did try deleting .Spotlight_V100 after you had a fresh .fseventsd, and it didn't work?

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 01:18 PM
 
I couldn't remember, so I've just done it again.

Booted into single user mode, deleted .Spotlight-V100, moved .fseventsd (i'll delete this later) and moved com.apple.timemachine.supported even though I'd tried deleting this before.

I watched mdworker scan some files in the terminal, but it's stopped now, just mds. So still no luck.

From what I've found out googling, I suspect the importer start failed messages to be a relatively unimportant bug in 10.5.5.

I think the errors that are most important are these:

Code:
11/11/2008 18:09:04 com.apple.launchd[1] (0x100890.mdworker[382]) getpwuid("4294967294") failed 11/11/2008 18:09:04 com.apple.launchd[1] (0x100890.mdworker[382]) Exited with exit code: 1 11/11/2008 18:09:04 com.apple.launchd[1] (0x100890.mdworker) Failed to check-in!
After these messages mdworker stops accessing the file system (shown in the terminal) and it's just mds.
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 01:58 PM
 
The getpwuid() function is related to the passwd system - in olden days, that used to use NetInfo, but that changed at some point to an LDAP-based system. With NetInfo, that used to be rebuildable by nuking /var/db/netinfo and /var/db/.AppleSetupDone. My guess would be that now we would nuke dslocal and DirectoryService inside /var/db like so:

Code:
fsck -fy mount -uw / cd /var/db mv dslocal dslocal.bak mv DirectoryService DirectoryService.bak mv .AppleSetupDone AppleSetupDone.bak reboot
If all goes well, that should get you into the Setup Assistant where you can make yourself a new users database.

DISCLAIMER: I haven't tried this myself, so do this at your own risk - if your machine gets messed up worse than it was, it's not my fault. Making sure your home folder is backed up would probably be a good idea.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 02:22 PM
 
I'll have a poke around Directory Utility, I did have it logging onto my offices domain - well I tried anyway. That was a long time ago though.

I'd rather break it trying to fix it. If needs be, I'll slave the drive up, migrate my user to another Mac and start again. At least THEN it'd be fixed.
( Last edited by seanc; Nov 11, 2008 at 02:29 PM. )
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 02:29 PM
 
Actually, looking at that error message... why the heck is it passing ULONG_MAX-1 (or -2 if it's using a signed int for some reason) to getpwuid()? That's not a valid UID...

Something's definitely corrupted here, it's just a matter of finding out what.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Addicted to MacNN
Join Date: Aug 2008
Location: The New Posts Button
Status: Offline
Reply With Quote
Nov 11, 2008, 02:32 PM
 
This is like an episode of House for geeks. More sarcasm people!
sorry for interrupting your thread
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 02:34 PM
 
What?

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Addicted to MacNN
Join Date: Aug 2008
Location: The New Posts Button
Status: Offline
Reply With Quote
Nov 11, 2008, 02:37 PM
 
Originally Posted by CharlesS View Post
What?
You guys are working hard together diagnosing seanc's problem. You guys are like his dignostic team... only computer geeks.

Passable?
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 02:39 PM
 
I thought that was kind of the point of this forum.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Addicted to MacNN
Join Date: Aug 2008
Location: The New Posts Button
Status: Offline
Reply With Quote
Nov 11, 2008, 02:40 PM
 
...you do realize I'm not trying to make fun of you guys, right? I just found this much more intense and complicated than "why does itunes duplicate all my music" threads.
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 02:41 PM
 
The laugh track is disabled in here Dakar, sorry.
     
Addicted to MacNN
Join Date: Aug 2008
Location: The New Posts Button
Status: Offline
Reply With Quote
Nov 11, 2008, 02:44 PM
 
Originally Posted by seanc View Post
The laugh track is disabled in here Dakar, sorry.
No, it's my fault. I thought everyone loved House and would get the compliment. I'll quit contaminating your thread now.
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 02:45 PM
 
Originally Posted by Dakar V View Post
...you do realize I'm not trying to make fun of you guys, right? I just found this much more intense and complicated than "why does itunes duplicate all my music" threads.
Do you use Spotlight? Does yours work?

Mine doesn't.

I can take the easy way out and nuke and pave, or I can try and work out what the problem is in the hopes that I can learn something.

iTunes duplicates songs because it has/had an annoying setting/click through to let iTunes do the organizing that most people ignore.
     
Addicted to MacNN
Join Date: Aug 2008
Location: The New Posts Button
Status: Offline
Reply With Quote
Nov 11, 2008, 02:46 PM
 
Originally Posted by seanc View Post
I can try and work out what the problem is in the hopes that I can learn something.
I know, that's what makes this interesting.
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 02:47 PM
 
Originally Posted by Dakar V View Post
I know, that's what makes this interesting.
Pfft, K
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 02:58 PM
 
Fine, here's a cat picture if that'll make you feel better.



Now, back to the thread.

Did the DirectoryService stuff rebuild properly, and did it fix anything?

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 03:05 PM
 
Originally Posted by CharlesS View Post
Did the DirectoryService stuff rebuild properly, and did it fix anything?
So far, spinning pinwheel of death (on boot screen)

Should have booted it in verbose mode and will do if it doesn't boot soon.

OK, here's what a verbose output gives me.

com.apple.launchd[1] (come.apple.mDNSResponder[22]): getpwnam("_mdnsresponder") failed
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 03:37 PM
 
Well crap.

If you go in via Single-User mode again and have a look in /var/db, did anything fail to get rebuilt?

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 03:41 PM
 
Everything got rebuilt apart from .AppleSetupDone

I've moved everything back, but we've disturbed something else now.
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 03:50 PM
 
Crap, it's still doing the same thing?

Did it rebuild all of the individual folders inside dslocal, or only some of them?
( Last edited by CharlesS; Nov 11, 2008 at 03:59 PM. )

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 04:02 PM
 
I haven't looked inside the folders yet.

I'm seeing if Disk Warrior can do anything with it, I don't know what I expect it to do, but I paid for it so I might as well try.
I've got it hooked up to another machine now so I'll have a poke around.
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 04:28 PM
 
OK, it's alive again.

In my stupidity, I'd moved my back files into the new dslocal & DirectoryService directories, rather than overwriting them.

Replaced them and now it's booting again.

I've just watched another working MacBook start indexing the one having issues via TDM. It seems mdworker launches under root, nobody and my user. Could it be that I'm missing "nobody"?
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 04:29 PM
 
Oh and by the way, Spotlight is indexing.





Thanks CharlesS and Hal.
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 04:40 PM
 
If you wanted to know whether you were missing nobody, you could check /var/db/dslocal/nodes/Default/users/ to see if nobody.plist was in there. But since your problem is now fixed, I guess we'll never know. It looks like we also can't know whether the problem was solved by our monkeying around with /var/db/ or whether it was the indexing by the other MacBook that did it. Oh well, if it's working, that's the important thing.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
seanc  (op)
Moderator
Join Date: Apr 2005
Location: Cambridge, UK
Status: Offline
Reply With Quote
Nov 11, 2008, 04:50 PM
 
Could it have been the permissions on the folders in /var/db that we moved? OS X re-created them, then I copied the contents back into them.

Nobody.plist is in there and it definitely exists because I've just seen mdworker running under it.

It's a shame that we don't know what the problem was. Perhaps we'll find out one day.
     
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 11, 2008, 04:54 PM
 
I doubt it was permissions, since the relevant parts are read/writeable only by root, and if you're root, you can read/write anything no matter what the permissions are. If nobody was gone, that could definitely do it - it would explain the nonsensical value that was getting passed to getpwuid(). But we can't know for sure.

Oh well. Sometime I'm going to install Leopard on a spare Mac and experiment a bit to find out exactly what the steps are to nuke the user data so it rebuilds properly - will be good to know the next time something like this turns up. It's a shame that it apparently isn't as simple as just nuking /var/db/netinfo back in the day. I do notice that some of the stuff in there is installed by BaseSystem.pkg on the install discs - perhaps the proper thing to do is to delete everything that's not in the install package, and leave the default stuff alone.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
 
Thread Tools
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Top
Privacy Policy
All times are GMT -4. The time now is 07:23 PM.
All contents of these forums © 1995-2015 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2015, Jelsoft Enterprises Ltd., Content Relevant URLs by vBSEO 3.3.2