[Max]

The system is barely usable. Both mdworker and Finder are hogging CPU usage, and the following repeating message (or a slight variant) is all that appears in the system log, 24 X 7:

mdworker32[2457]: *** process 2457 exceeded 500 log message per second limit - remaining messages this second discarded ***
mdworker32[2457]: ** Unrecognized DjVu Message: \ n ** Message name: ^CIFFByteStream.corrupt_id
--- last message repeated 499 times ---

Ideas?

[Big Mac]

What happens if you start in Safe Mode? Hold down the Shift key at startup.

mdworker is the process behind Spotlight. If it's choking that badly I'd suspect possible hard drive problems.

[Max]

Originally Posted by Big Mac 

What happens if you start in Safe Mode? Hold down the Shift key at startup.

mdworker is the process behind Spotlight. If it's choking that badly I'd suspect possible hard drive problems.

Well, for some reason that I can't determine, Spotlight apparently decided to do a complete reindexing of my external clone drive several days ago. So, yes, that could be a factor. However, system performance has been progressively deteriorating since getting this machine in the fall (the logs are filled with entries I've never seen on previous machines — I'll post a few if you think it would be helpful) this is just a new "wrinkle." (The reindexing apparently finished this morning, and performance has improved a bit...it's now just as bad (i.e., it's miserable) as it was before it became barely usable!)

What's puzzling me, however, is not the mdworker activity per se (previous logs are filled with such entries, which do seem to correlate with some of the system's poor performance), but the "** Unrecognized DjVu Message: \ n ** Message name: ^CIFFByteStream.corrupt_id" entry, which began last week after I downloaded a textbook that was in DjVu format. It seems coincidental, because I have a number of such files on my machine, and there's never been a problem related to them, so I have no idea why suddenly such a file format seems to have triggered this kind of response. And, FWIW, the 3-line entry

mdworker32[xxx]: *** process xxx exceeded 500 log message per second limit - remaining messages this second discarded ***
mdworker32[xxx]: ** Unrecognized DjVu Message: \ n ** Message name: ^CIFFByteStream.corrupt_id
--- last message repeated 499 times ---

is repeated every few seconds (the process number "xxx" seems to change only after dozens of entries), and at the end of the day, the system log is filled with them, and no other entries!

[Big Mac]

It's from mdworker. Spotlight is choking trying to index those files, and it's choking really hard. That's what the mdworker error is indicating. Can you exclude those files from the Spotlight index (at least temporarily) and see if that helps?

Have you thought about reinstalling the OS?

[Max]

Originally Posted by Big Mac 

It's from mdworker. Spotlight is choking trying to index those files, and it's choking really hard. That's what the mdworker error is indicating. Can you exclude those files from the Spotlight index (at least temporarily) and see if that helps?

Again, things have improved since the reindexing stopped. But the log messages haven't stopped. And I'm unclear which "files" are actually the problem. Like I said, there's only one file that I downloaded in the past week that's in DjVu format, and its name (it's a textbook) bears no apparent relationship to the apparently offending file, "^CIFFByteStream.corrupt_id."

Have you thought about reinstalling the OS?

I'd really like to narrow things down before I embark on a "nuke 'n' pave" project. I'm not entirely sure it's entirely the OS, as the problems I've experienced began on Day #1: display resolution is not right, sudden complete backing up to my other (Time Machine) external drive (i.e., as if the TM drive had no files on it...or as if all of my internal drive's files had changed), a kernel panic, freezes, Finder errors, constant SBBODs, etc., plus a regular multitude of odd messages in the system log. What I'd really like to do is try to see if the log messages can somehow help diagnose the cause(s).

[Big Mac]

Based on the list of symptoms you've just presented, if I were in your position I would have reinstalled the OS long ago. Your problems are much more serious than mdworker and CIFFByteStream.corrupt_id whatever that might be referring to. (Taking a wild guess based on the error, I would think that Spotlight is indicating it can't read the CIFFByteStream in the DjVu file that it's trying to read and index. Google turns up no hits for CIFFByteStream, but it sounds like the file format of whatever those files are.)

Reinstalling the OS doesn't mean nuke and pave - it doesn't mean erasing the hard drive. You're just reinstalling the OS (doing an Archive and Install as it used to be called before Apple simplified things by making Archive and Install the default installation type).

[Max]

Originally Posted by Big Mac 

Based on the list of symptoms you've just presented, if I were in your position I would have reinstalled the OS long ago. Your problems are much more serious than mdworker and CIFFByteStream.corrupt_id whatever that might be referring to. (Taking a wild guess based on the error, I would think that Spotlight is indicating it can't read the CIFFByteStream in the DjVu file that it's trying to read and index.

The name of the file is "Coddington---An_introduction_to_ordinary_differential_equations _%28Dover_1989%29_%28275p%29.djvu." What could that possible have to do with "CIFFByteStream?"

Google turns up no hits for CIFFByteStream, but it sounds like the file format of whatever those files are.)

I found a couple of references, but they'r not in English, and Google Translate isn't perfect, so I couldn't quite figure out what they were saying:

"Unrecognized DjVu Message" ^CIFFByteStream.corrupt_id - Google Search

Reinstalling the OS doesn't mean nuke and pave - it doesn't mean erasing the hard drive. You're just reinstalling the OS (doing an Archive and Install as it used to be called before Apple simplified things by making Archive and Install the default installation type).

I'd still like to get an understanding of whatever the underlying cause(s) may be. It seems that some understanding of what the log is trying to say might help to avoid going back to "square one," only to have the same situation begin all over again.

Here's a few of the recurring "odd" log messages that make no sense to me:

"Periodic CFURLCache" messages; for example:
Mail[23069]: Periodic CFURLCache Insert stats (iters: 360) - Tx time:0.178208, # of Inserts: 4, # of bytes written: 42835, Did shrink: NO, Size of cache-file: 20615168, Num of Failures: 0
SoftwareUpdateCheck[30843]: Periodic CFURLCache Insert stats (iters: 440) - Tx time:1.387898, # of Inserts: 5, # of bytes written: 27850, Did shrink: NO, Size of cache-file: 11061248, Num of Failures: 1
Safari[23033]: Periodic CFURLCache Insert stats (iters: 3625) - Tx time:2.436306, # of Inserts: 2, # of bytes written: 69156, Did shrink: NO, Size of cache-file: 33603584, Num of Failures: 0
Dictionary[23639]: Periodic CFURLCache Insert stats (iters: 10229) - Tx time:1.792374, # of Inserts: 1, # of bytes written: 23482, Did shrink: NO, Size of cache-file: 50331648, Num of Failures: 1

"INSERT-HANG-DETECTED" messages; for example:
mdworker[29521]: INSERT-HANG-DETECTED: Tx time:6.712662, # of Inserts: 0, # of bytes written: 0, Did shrink: NO
Mail[23069]: INSERT-HANG-DETECTED: Tx time:9.541911, # of Inserts: 134, # of bytes written: 655704, Did shrink: NO
Dictionary[23639]: INSERT-HANG-DETECTED: Tx time:3.728335, # of Inserts: 4, # of bytes written: 44681, Did shrink: NO
SoftwareUpdateCheck[30843]: INSERT-HANG-DETECTED: Tx time:4.561553, # of Inserts: 2, # of bytes written: 610876, Did shrink: NO
Safari[23033]: INSERT-HANG-DETECTED: Tx time:4.252848, # of Inserts: 13, # of bytes written: 807307, Did shrink: NO

mdworker[215]: Component boundaries mismatch (VTIMEZONE VTIMEZONE\n\n)

mdworker[36275]: *** Failed to decode 7-bit data, treating as binary
mdworker[36275]: *** Failed to decode us-ascii data, treating as binary

mdworker[36380]: socket(PF_ROUTE) failed: Operation not permitted
sandboxd[36383]: mdworker(36380) deny system-socket
sandboxd[36383]: mdworker(36380) deny network-outbound /private/ var/run/mDNSResponder