Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > macOS > macOS High Sierra

macOS High Sierra
Thread Tools
reader50
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Sep 19, 2017, 03:36 PM
 
A High Sierra GM support document has been posted by Apple, concerning APFS. Highlights:

• Fusion drives and HDDs are not converted by the GM. Fusion was supported by some of the betas.
• SSD conversion is automatic - no opt-out option. Presumably High Sierra will work from HFS+ if it were cloned to an HFS+ SSD.
• APFS drives are not readable by Sierra or earlier. At least, USB drives formatted as APFS can't be. No word on Sierra updates for compatibility.
• APFS drives cannot be shared over AFP. SMB or NFS sharing only.
• Boot Camp will work, but can't read or write APFS.
• Time Machine will back up APFS as normal. As TM almost always uses HDDs, it is implied that High Sierra backs up APFS volumes to HFS+ volumes.

It is not clear if Fusion / HDD support is coming in a later update. The 27" iMacs come with Fusion drives standard today, so Apple has motivation to finish HDD support. And I expect HDD support for APFS would significantly improve Time Machine performance.

Still no word on file checksumming.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 19, 2017, 07:07 PM
 
Can't say that I'm surprised about the lack of fusion drive APFS support. I saw a lot of reports on MacRumors about problems with fusion drive and time machine backups during the beta phases, and I guess that Apple just could not get the bugs worked out. Better delay support than half-arse it to the world. APFS is a big deal, so I had decided not to participate in the beta program.

However, rest assured that when I see that Apple has not fixed some of their "inconvenience" bugs that came along with the ride for (or were introduced by) earlier versions of macOS, Apple will be hearing from me for sure.
This one time, at Boot Camp, I stuck a flute up my PC.
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Sep 25, 2017, 06:39 PM
 
Ars has their High Sierra review posted. In particular, there is updated info from the APFS section.

• HDDs are supported by APFS. They aren't converted by default, but you can do it manually. Even for a boot drive. It's just Fusion that is MIA today.
• Boot Camp not only can't read/write APFS, it can't even see a Mac volume is present. You can't boot back to macOS from Boot Camp because you can't select an APFS volume. Keep an HFS+ boot option around. Or option-boot, then update Startup Disk.
• There is currently no de-duplication process to complement APFS cloning. Probably a good thing until more bugs are detected and worked out.
• eGPU support is here, but apparently intended more for developers than the gaming public.
• HS performs a weekly checksum of the EFI for tampering. Mac Pros flashed from 4,1 to 5,1 may or may not get flagged. It's done via a commandline tool, which also has the ability to dump the current firmware. While Apple intends it against hacking, it may be quite enabling for enthusiast hacking.

I spotted a couple other items of interest, but failed to take notes. And the Ars article is getting mobbed, causing server errors - I'll recheck later.
     
Chongo
Addicted to MacNN
Join Date: Aug 2007
Location: Phoenix, Arizona
Status: Online
Reply With Quote
Sep 25, 2017, 07:55 PM
 
I guess those of us with 3TB Fusion drives will wait until the new file format is supported. (2013 27” iMac.)
"The blood of the martyrs is the seed of the church" Saint Tertullian, 197 AD
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Sep 25, 2017, 10:30 PM
 
The other item I'd noticed is: APFS is undocumented outside of Apple. They plan to release documentation at some point. So High Sierra is the only OS that can address it. I wouldn't put much faith in Sierra's beta support, unless Apple updates it.

So no 3rd party support for say, disk repair/recovery tools. No drivers for Windows or Linux. Just the one OS version that can access or repair it. Which is a .0 release.

Make backups, and keep Time Machine up to date. APFS sounds nifty, but beware the early adopter pitfalls.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 26, 2017, 04:07 AM
 
Haven't read the review yet, but some comments:

Originally Posted by reader50 View Post
• There is currently no de-duplication process to complement APFS cloning. Probably a good thing until more bugs are detected and worked out.
Deduping eats RAM. As in, all of it. It is of quite limited utility on a consumer setup as well, so I don't think Apple will integrate it into the OS. Remember how they handled filesystem compression in 10.6? No live compression facility, but files were compressed on delivery to save space in the default install. Worked pretty well.

I can see a third-party utility doing off line de-duping, but as I said, use case is limited.

Originally Posted by reader50 View Post
• HS performs a weekly checksum of the EFI for tampering. Mac Pros flashed from 4,1 to 5,1 may or may not get flagged. It's done via a commandline tool, which also has the ability to dump the current firmware. While Apple intends it against hacking, it may be quite enabling for enthusiast hacking.
That tool has been there for Sierra as well, it just didn't run regularly.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Sep 26, 2017, 05:13 AM
 
Deduping is so intensive because it requires heavy files comparison. But if file checksums get enabled, deduping would be easy. You'd still do a byte comparison for safety, but you'd only do it on files with identical checksums. As the chances of a collision are negligible, in practice you'd only compare files you end up deduping. No wasted passes.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 26, 2017, 07:31 AM
 
So, how much longer until Photos Agent, photosanalysisd, and related processes quit draining my battery and exhausting my RAM? I had to shut down last night, because it was getting late. Today the laptop is on battery, but instead of getting 10 hours of battery life, I'm getting a little over 5.

Edit: Might be finished.

However, now my Mac uses 5 GB of RAM after starting up, rather than 1.5 GB in 10.12, according to iStat Menus (now version 6). What gives?
This one time, at Boot Camp, I stuck a flute up my PC.
     
Thorzdad
Moderator
Join Date: Aug 2001
Location: Nobletucky
Status: Offline
Reply With Quote
Sep 26, 2017, 08:55 AM
 
Seems there's a bit of a security issue with HS and earlier versions.
The kicker is this:
While the app in the video is unsigned—and as a result can't be installed on a default Mac installation—the vulnerability can be exploited by signed apps as well. All that's required to digitally sign an app is a membership in the Apple Developer Program, which costs $99 per year.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 26, 2017, 09:32 AM
 
Originally Posted by reader50 View Post
Deduping is so intensive because it requires heavy files comparison. But if file checksums get enabled, deduping would be easy. You'd still do a byte comparison for safety, but you'd only do it on files with identical checksums. As the chances of a collision are negligible, in practice you'd only compare files you end up deduping. No wasted passes.
I know how deduping works, but if you check the system reqs for ZFS - which has de-duping and relies heavily on file checksumming in every part of the file system, RAM requirements start to run away quickly. According to tests of ZFS on FreeBSD, you need 5GB of RAM per TB of storage that you want to keep live de-duping active on:

https://wiki.freebsd.org/ZFSTuningGuide

Apple only recently stopped shipping Macs with 4GB RAM, and they do ship models with 3TB storage. This is too much for a feature that will have very limited utility in a consumer model. Better to have some form of offline de-duping.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 26, 2017, 09:39 AM
 
Originally Posted by Thorzdad View Post
It seems like there is a bug in the keychain. Not specifically related to High Sierra - just a pissed off security researcher who think they should have delayed the release and given him a bounty.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 26, 2017, 09:42 AM
 
Originally Posted by And.reg View Post
However, now my Mac uses 5 GB of RAM after starting up, rather than 1.5 GB in 10.12, according to iStat Menus (now version 6). What gives?
Siri saw reader's post above yours and silently enabled dedupe.

What are you looking at when you're comparing - Wired + Active + Inactive, or Wired + Active? Because if you're including Inactive, it is a pointless measurement. That can happen if Apple has, for instance, improved the disk caching as part of the APFS introduction.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 26, 2017, 09:47 AM
 
An interesting piece in the Ars review about the server features ported over: An Apple Update server (similar to WSUS for Windows), a Time Machine Server, and an updated file server over SMB. The last I can ignore, but the other two are interesting. Considering taking an old MBA together with an external drive and placing in the cabinet next to the router to act as a server for the last two. It is only so I won't have to listen to the noisy HDD, but still.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 26, 2017, 09:56 AM
 
Originally Posted by P View Post
Siri saw reader's post above yours and silently enabled dedupe.
What is dedupe? I've never heard of that term before.



Originally Posted by P View Post
What are you looking at when you're comparing - Wired + Active + Inactive, or Wired + Active? Because if you're including Inactive, it is a pointless measurement. That can happen if Apple has, for instance, improved the disk caching as part of the APFS introduction.
I'm looking at:

This one time, at Boot Camp, I stuck a flute up my PC.
     
Chongo
Addicted to MacNN
Join Date: Aug 2007
Location: Phoenix, Arizona
Status: Online
Reply With Quote
Sep 26, 2017, 10:39 AM
 
Does anyone who has update have a Fusion drive?
"The blood of the martyrs is the seed of the church" Saint Tertullian, 197 AD
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 26, 2017, 11:21 AM
 
Originally Posted by And.reg View Post
What is dedupe? I've never heard of that term before.
De-duplication. Reader and I were discussing it above your post, so I made a little joke. De-duplication is when the computer can detect that the file at /Users/user1/photo1.jpg is actually the same as the file at /Users/user2/my_great_photo.jpg, and so save space by only storing it once until one of the files are changed. Actual de-duplication works on parts of files, even, but that is the principle. It is a nice feature, but it uses lots of RAM, hence my little joke.

Originally Posted by And.reg View Post
I'm looking at:

OK, so it isn't even showing the Inactive part. Interesting.

I don't know why the memory usage is higher. Go to Activity Monitor and check which app is using more data, I guess.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 26, 2017, 12:41 PM
 
This? The sum hardly adds up to the claimed 5-6 GB in my recent usage, and that's with the same Applications that I used to have open on 10.12.

This one time, at Boot Camp, I stuck a flute up my PC.
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Sep 26, 2017, 03:10 PM
 
Originally Posted by P View Post
I know how deduping works, but if you check the system reqs for ZFS - which has de-duping and relies heavily on file checksumming in every part of the file system, RAM requirements start to run away quickly. According to tests of ZFS on FreeBSD, you need 5GB of RAM per TB of storage that you want to keep live de-duping active on: ...
They must be creating per-block checksums on the fly, for every block of every file. For maximium deduping of common blocks in unrelated files. That would produce the scaling problem.

I suggest only an identical-file process, especially for consumer installs. And like you say, leave more complete deduping to an offline utility.

With just identical files, pull existing checksums from the file system. Pick the largest files with matching sums that haven't been compared recently, and pull blocks in succession for comparison. Should require less than a MB of memory space, easily set to a low priority, safe to run in the background.
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Sep 26, 2017, 04:11 PM
 
Originally Posted by Chongo View Post
I guess those of us with 3TB Fusion drives will wait until the new file format is supported. (2013 27” iMac.)
Apple has confirmed APFS support is coming to Fusion and HDDs in a future update.

Ars confirm

Press release for High Sierra:
Originally Posted by Apple
• APFS currently supports every Mac with all‑flash internal storage — support for Fusion and HDD Mac systems will be available in a future update.
Since HDDs are supported already, Apple presumably means they'll eventually convert boot HDDs by default.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 26, 2017, 06:23 PM
 
Originally Posted by reader50 View Post
They must be creating per-block checksums on the fly, for every block of every file. For maximium deduping of common blocks in unrelated files. That would produce the scaling problem.

I suggest only an identical-file process, especially for consumer installs. And like you say, leave more complete deduping to an offline utility.

With just identical files, pull existing checksums from the file system. Pick the largest files with matching sums that haven't been compared recently, and pull blocks in succession for comparison. Should require less than a MB of memory space, easily set to a low priority, safe to run in the background.
Deduping on a server is indeed per block, and it makes sense in a multiuser system, but I just don't see that the savings are that great for a single user. Why should I put the same file in more than one spot? Mistakes happen, but they can't be that common. Unless the savings are great, why spend CPU cycles and RAM on it?
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 26, 2017, 06:24 PM
 
Originally Posted by And.reg View Post
This? The sum hardly adds up to the claimed 5-6 GB in my recent usage, and that's with the same Applications that I used to have open on 10.12.

Those are just your processes, though. You're not showing system processes.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 26, 2017, 07:28 PM
 
What about this?

This one time, at Boot Camp, I stuck a flute up my PC.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 26, 2017, 07:53 PM
 
Originally Posted by And.reg View Post
However, now my Mac uses 5 GB of RAM after starting up, rather than 1.5 GB in 10.12, according to iStat Menus (now version 6). What gives?

That is not necessarily a bad thing. The test is how well your Mac performs when memory resources become scarce. Simply adding up the numbers in that column doesn't tell you a ton. I don't see anything alarming in your screenshots.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 26, 2017, 07:57 PM
 
Well my memory usage used to be 2.1 GB under same conditions before updating to macOS High Sierra.

Meanwhile, photoanalysisd is once again running in the background, for the second day, and heating up my laptop when plugged in to an outlet. I did wait until upgrading to High Sierra before reimporting my photo library and creating all new albums. But the problem is like what is reported at:

https://www.reddit.com/r/MacOS/comme...ntentious_new/

I have 2,444 photos and 169 videos, taking up just under 10 GB of library. So, this photoanalysisd is intermittently on and off. Weird...
This one time, at Boot Camp, I stuck a flute up my PC.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 26, 2017, 07:59 PM
 
Is Photos showing processing photos? https://discussions.apple.com/thread...art=0&tstart=0
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 27, 2017, 12:23 PM
 
No. It shows one photo in People and no processing information.

Still really bumped that about 6 GB of RAM are being used instead of about 2 GB, I would have thought that High Sierra would have made memory management more efficient, but instead, I have about 30% less RAM available for other apps/processes.

Originally Posted by besson3c View Post
That is not necessarily a bad thing. The test is how well your Mac performs when memory resources become scarce. Simply adding up the numbers in that column doesn't tell you a ton. I don't see anything alarming in your screenshots.
Huh? What test, what is High Sierra testing my memory for? As soon as I boot up my Mac, it bloats up to 4-5 GB of RAM used, instead of less than 2 GB under 10.12. That, combined with not understanding your post about the "test," has me confused.
This one time, at Boot Camp, I stuck a flute up my PC.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 27, 2017, 12:27 PM
 
Originally Posted by And.reg View Post
No. It shows one photo in People and no processing information.

Still really bumped that about 6 GB of RAM are being used instead of about 2 GB, I would have thought that High Sierra would have made memory management more efficient, but instead, I have about 30% less RAM available for other apps/processes.
That's not how it works though.

A higher number does not necessarily = bad. Memory is used for caching, because it is fast. Using available memory when it is available is fine, and can be a sign of smart coding. Absolutely needing/requiring that memory is another thing, which is why how things behave when memory is actually low is really what to be concerned over. The stuff that isn't actually needed should be reallocated in these conditions.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 27, 2017, 12:28 PM
 
Originally Posted by And.reg View Post
Huh? What test, what is High Sierra testing my memory for? As soon as I boot up my Mac, it bloats up to 4-5 GB of RAM used, instead of less than 2 GB under 10.12. That, combined with not understanding your post about the "test," has me confused.

There is no literal test, I meant rhetorical test, as in "what is most telling". That being said, that photo service probably shouldn't be spinning out of control that way.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 27, 2017, 07:07 PM
 
Awww yeah!
And.reg's List of Annoying Bugs in 10.13 (and a few fixed/remedied bugs from previous versions of macOS) - and request for help

As of October 16, 2017
21 unresolved bugs

Link to this post: http://forums.macnn.com/showthread.p...76#post4389776

Bugs in 10.10: http://forums.macnn.com/showthread.p...83#post4295983
Bugs in 10.11: http://forums.macnn.com/showthread.p...18#post4334618
Bugs in 10.12: http://forums.macnn.com/showthread.p...01#post4367001
Bugs in 10.13: this post



An ongoing bug that has been prevalent in all of the previous versions of macOS that I have documented, but I cannot seem to find:

Sometimes, when navigating through Finder multi-column view using the arrow keys, multiple tabs, and copying/pasting files, and switching apps will inexplicably cause the Finder window column sizes to expand threefold, and if you navigate back, they stay like that. This bug has NOT been fixed in 10.13.



From bugs reported in 10.10: http://forums.macnn.com/showthread.p...83#post4295983

Bugs 2, 4, 12 (see below), 15, and 20 have NOT been fixed in High Sierra. Bug 17 is not significant.

Bug 2, by the way, is a very OBVIOUS and ANNOYING bug that Apple should be ASHAMED with themselves for not fixing. Go read the post!

#12. When un-muting your volume for the first time after booting the computer, the volume control will show a fraction of a dial:
In 10.10:
In 10.11:
Same in 10.12 and 10.13, and was never remedied. Either make a continuous slider option for the volume buttons, or change the volume meter display to make it continuous.



From bugs reported in 10.11: http://forums.macnn.com/showthread.p...18#post4334618

Bugs 1 appear to have been fixed in High Sierra.

Bug 14 is in review.

Bug 15 is out of review, and has STILL NOT been fixed in High Sierra.

Bug 16 was likely caused by my modem, not by Apple, as the modem was hogging my connection to download software and update itself, with the updates getting backlogged from previous forced restarts.



From bugs reported in 10.12: http://forums.macnn.com/showthread.p...01#post4367001

Bugs 1, 3, 4 (tested on AMD 560), 5, and 8 appear to have been fixed in High Sierra.

Bugs 2, 6, and 11 have NOT been fixed in High Sierra.

Bugs 7, 9, and 10 are still in review. Concerning bug 7, in my case, I had to re-install macOS Sierra 10.12 to eliminate this bug. I have not seen it reappear in 10.13.



Also, on the unnumbered bug that I have reported:
http://forums.macnn.com/showthread.p...18#post4334618

Since upgrading to 10.12, Bug 12 APPEARS to have finally been remedied. During the 10.12 beta phase, I reported that the bug can still be reproduced if, instead of clicking on an open application window, you click on the corresponding Dock icon for that application.

^ This ANNOYING problem STILL happens *intermittently* in 10.13, and is not fixed.


On my note from past bug lists: Concerning Preview being extremely slow when editing the color properties of a photo, this continues to be the case, but to a slightly lesser extent than in 10.12.



NEW BUGS IN 10.13:

1. Adding a large number of albums and photos for the first time in Photos throws an erroneous message when trying to import one more picture, it "couldn't be opened because you don't have permission to view it." It is almost as if there is a certain upper limit of file sizes that may be added before this message is thrown. The solution, contrary to the message says, is to restart Photos and then re-import the files. This was NOT a problem in 10.11 and likely not a problem in 10.12.

2. When using other apps while iTunes is syncing, the Play/Pause button for your iTunes music is now stuck on Play only and will not pause music if you tap it on the TouchBar. This was NOT a problem in 10.12, because in 10.12, the button was always updated to Play/Pause and just worked.

3. The font for the letter "a" in Reminders looks too similar to "o" in that it is now often difficulty to tell either letter apart. Technically not a programming error, but still an unnecessary annoyance that Apple introduced with 10.13.

4. There are several issues with external monitors. (Testing on a 2017 Pro MacBook 15" with AMD Radeon Pro 560 connected to 4K LG monitor)

4a. Sometimes, connecting the external monitor will not register.
4b. Waking the laptop from sleep with the external monitor connected and the laptop monitor display brightness to zero will still force the laptop brightness back up... it's supposed to stay off while the external monitor comes back on.
4c. Disconnecting the cable for the external monitor from my laptop will not return the GPU mode to the integrated GPU. Instead, the dGPU will stay activated even though no open apps require it. Sometimes, I have to force it to iGPU via gfxCardStatus. This was NOT a problem in 10.12.
4d. At this stage, reconnecting the external monitor can completely freeze the laptop.

5. When toggling the volume up/down on the TouchBar, and when connected to an external display, the corresponding volume icon does NOT display below the middle of the screen. (But it does show when pressing Mute.) This was NOT a problem in 10.12.

6. Yet ANOTHER issue with external monitors: When disconnecting from an external monitor (2017 Pro Macbook with 560) the night before with the laptop monitor on zero brightness before disconnecting, then middle brightness after disconnecting, then shutting down a laptop, then booting up the following morning, the laptop starts up with the brightness setting on MAXIMUM, rather than the previous display brightness setting. This was NOT a problem in 10.12.

7. Yet ANOTHER issue with using multi-column view in Finder. This is an example of why I create these threads and send reports to Apple, because of the frustration that it causes. When using the Back key command to navigate previously-accessed subfolders, sometimes, the navigation just doesn't work. Here are steps to reproduce the issue:
1. Select a sidebar tab in Finder. I use one of my tags.
2. Navigate using multi-column view to a file X folders deep.
3. Optional step: Find and copy a file in that subfolder.
4. Press Command-Shift-C.
5. Navigate to a flash drive, or any other folder, whatever.
6. Now try pressing Command-[ to go back to the original folder, then Command-] to go forward again. Guess what. It doesn't go forward. Not only that, but selecting an item in the Back button menu doesn't do anything.
This actually WAS a problem in 10.12 that I did not think to report.

8. AND AGAIN with multi-column view in Finder. If I have the sidebar option selected to Downloads, and I want to drag a link out of Safari, to the display's top edge to Expose all of the windows, the drag the link over the Finder window, wait two seconds, then drop the link into the Downloads folder, it doesn't do it, the link just fades back toward the Safari toolbar. This was NOT a problem in 10.12, because in 10.12, link dragging through Expose just worked!

9. When a webpage is loaded in Safari that links to images in the same directory as the HTML file, and those images are reuploaded with a different file size, sometimes, Safari will display the revised webpage with the old image sizes. For instance, if the webpage is a subframe, Safari will display the webpage with the old cached image sizes. However, if the webpage is its own entire frame, and reloaded, the correct image sizes will be displayed. This problem leads a web developer to be mislead that the files were not reuploaded or that the web page code was incorrect. I do not recall if this was a problem in 10.12.

10. I have TextEdit set to open HTML files as RAW text HTML code, not as a reformatted page. At random just a moment ago, this checkbox was "unset" and the same HTML pages that I have opened and saved multiple times in the past hour just opened as an HTML webpage. I do not recall if this was a problem in 10.12.
( Last edited by And.reg; Oct 16, 2017 at 10:36 AM. )
This one time, at Boot Camp, I stuck a flute up my PC.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 28, 2017, 04:18 AM
 
Originally Posted by And.reg View Post
#12. When un-muting your volume for the first time after booting the computer, the volume control will show a fraction of a dial:
In 10.10:
In 10.11:
Same in 10.12 and 10.13, and was never remedied. Either make a continuous slider option for the volume buttons, or change the volume meter display to make it continuous.
It is possible to adjust the volume fractions by holding shift and/or option when pushing the volume adjust buttons. Are you sure that you haven't done that?

Your other links are broken for me.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Online
Reply With Quote
Sep 28, 2017, 07:39 AM
 
No one ever told me to use shift/option while holding the volume buttons. Going by my own poor memory, I had never heard of that before to do continuous adjustment.

*shrug* those were the links that I saved from the previous year. Whoever changed the forum settings didn't relink those symbolically.
This one time, at Boot Camp, I stuck a flute up my PC.
     
ghporter
Administrator
Join Date: Apr 2001
Location: San Antonio TX USA
Status: Offline
Reply With Quote
Sep 30, 2017, 10:13 AM
 
De-duping on a consumer level? OH BOY do I need that!

I have multiple copies of a huge number of photos, multiple copies of various PDFs for reference material, etc.... Yeah, this consumer really needs duplicate identification.

Do I need it on-the-fly and in the background? Probably not. I need it occasionally, sort of as a housekeeping thing. "User Glenn has 458GB in duplicate files in 59 folders. Would you like to review these files?" That's what I'd want.

At the moment I don't know if I would prefer it to be native or third-party, but there are already third party apps that identify duplicate files. The problem is that all of them I've tried are a pain to use. I'd want something that told me about not only duplicates, but also which copy I'd opened or read most recently, and maybe even tell me about folder usage data (most recently used, most frequently used, etc.).

I know this isn't specific to the HS beta, but as an arche consumer I know my online storage (whatever kind of "disk" I'm using) would be much more efficient if I was more aware of, or "protected from" excessive disk use from multiple copies of files.

Glenn -----OTR/L, MOT, Tx
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 30, 2017, 03:08 PM
 
Deduping on a file level seems like it should be easy if you have the duplicates identified. Just delete all but one and hard link the rest - and probably write protect that file to make it clear that it shouldn't be touched.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 30, 2017, 03:11 PM
 
Isn't dedup more about individual file segments rather than complete files?

For example, docker containers operate this way with image segments.
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Sep 30, 2017, 05:16 PM
 
That's the typical server-version of deduping. We discussed that somewhat earlier. But it's too memory-intensive to run in the background on consumer systems.

File-level deduping though is safe to run in the background. For example, I usually import photos into Photos. But I leave the originals, because I don't trust irreplaceable family pix to Apple's semi-closed ecosystem. And I want to crossreference them via folder.

With automatic hardlinking (what APFS would do) I could have my separate access, without burning the gigs for full separate copies. Wouldn't burn those gigs in my Time Machine backups either.

I've upgraded systems so many times, that I believe I have many duplicate files scattered about. From changing organizational design over the years. They won't always have the same names either.
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Sep 30, 2017, 09:16 PM
 
I got interested by the topic of dedup, and decided to make a little tool to scan for files that could be deduped on the per-file level, at least:

http://charlessoft.com/CloneClub.zip

Heavy disclaimers apply:

- This doesn't actually dedup anything; it currently just does a pre-scan to see how many duplicated files there are, and how much space is taken up by these duplicated files.

- If any files are already clones, they'll show up as false positives here. Unfortunately, there's no way for me to tell whether a file contains cloned blocks or not (well, there's ATTR_CMNEXT_PRIVATESIZE, but it's been returning 0 for me on any files that aren't brand new. According to the docs, ATTR_CMNEXT_PRIVATESIZE returns the size of any blocks that aren't part of either a clone or a snapshot; I suspect that Mobile Time Machine is probably using snapshots, causing every file on the disk's ATTR_CMNEXT_PRIVATESIZE to go to 0 when it updates itself).

- For simplicity's sake, I'm currently not looking at any files that we don't have write access to, or any files that have extended attributes or more than one hard link.

- Also, this tool takes forever to run, and chews through a lot of RAM (it topped out around 3 GB on my system). There are a few ways I could probably make it faster, but it's probably always going to be slow by nature of what it has to do.

Anyway, I'd be curious to see what results you guys get. On my machine, I'm getting over 50 GB of duped files (!), so unless I've got a bug in the size reporting, there might be some benefit to making a third-party dedup tool.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Sep 30, 2017, 11:48 PM
 
@Charles, it won't run on Sierra. I'm holding off on updating until more bugs are fixed. Does the code actually require HS? edit - I modified the minimum OS in info.plist, but it's hardcoded somewhere else too. Still shows as needing HS.

The files deduping would be easy if file checksums were turned on in APFS. You'd only look at files highly likely to be identical. Along with keeping a log of comparisons that failed (checksum collisions) so they don't get rechecked until one of them is modified.

btw, does your utility allow for more than two copies of a file?
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Sep 30, 2017, 11:54 PM
 
Originally Posted by reader50 View Post
@Charles, it won't run on Sierra. I'm holding off on updating until more bugs are fixed. Does the code actually require HS?
Yeah, HS only. I figured there wouldn't be much of a point on OS versions that can't boot from APFS.

The files deduping would be easy if file checksums were turned on in APFS. You'd only look at files highly likely to be identical. Along with keeping a log of comparisons that filed (checksum collisions) so they don't get rechecked until one of them is modified.
AFAIK APFS doesn't have checksums for files, only for metadata. It's a shame (and it's why my tool takes so long to run; it's checksumming every file on the disk).

btw, does your utility allow for more than two copies of a file?
Yeah, it groups files that match each other, and adds the size of each of them (except for the first one) to the running total. You can click a file and see all the files that are identical to it.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Oct 1, 2017, 12:01 AM
 
When APFS was first announced, plenty of people asked about file checksums. It's only for metadata today, but I got the distinct impression it may become an option.

On the Mac, it could be an out-of-the-way option for people who want it. Probably only used for system files on iPhone. And the Watch wouldn't use it at all. I remain hopeful it will become available in a future update. Or that someone will create an extension to add it.

Apple would be the best party to add it, as it should be tied into Time Machine. ie - if a checksum fails, and TM has a copy that matches the sum, there should be a silent replacement. And a user dialog only if there isn't a clean copy, and it isn't a temp/cache file.
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 1, 2017, 12:08 AM
 
Originally Posted by reader50 View Post
When APFS was first announced, plenty of people asked about file checksums. It's only for metadata today, but I got the distinct impression it may become an option.

On the Mac, it could be an out-of-the-way option for people who want it. Probably only used for system files on iPhone. And the Watch wouldn't use it at all. I remain hopeful it will become available in a future update. Or that someone will create an extension to add it.

Apple would be the best party to add it, as it should be tied into Time Machine. ie - if a checksum fails, and TM has a copy that matches the sum, there should be a silent replacement. And a user dialog only if there isn't a clean copy, and it isn't a temp/cache file.
I do hope that that happens eventually. They do seem to be rolling things out piece by piece (there still doesn't seem to be any API that I can see for snapshots, and fast directory sizing doesn't seem to be used anywhere thus far). Unfortunately as far as I know there hasn't been any official word on it, and apparently they've made the claim in the past that they don't need it, so who knows.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 1, 2017, 12:11 AM
 
One thing that is pretty cool dedup-wise is that I can confirm that the file clones are block-based rather than file based. As in, if you clone a file, then add some stuff to the end of the clone, it doesn't make a new copy of the whole file, but rather retains the reference to the original blocks and just stores the new stuff you added. You can see this by checking ATTR_CMNEXT_PRIVATESIZE of a newly created clone file (as I noted earlier, it always seems to be zero on older files) before and after making a change to it. Quite nice.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Oct 1, 2017, 12:24 AM
 
Originally Posted by CharlesS View Post
Unfortunately as far as I know there hasn't been any official word on it, and apparently they've made the claim in the past that they don't need it, so who knows.
Your link is most of what I read at the time. The Apple engineers were interested in talking about it, and carefully ignored the use of 3rd party storage devices. What gave me hope is that no one ruled out file checksums. They just didn't commit to anything, and kept asking questions.

Any chance you could recompile CloneClub to run on earlier OSes? So those of us waiting to upgrade can check our dupe situation too. When I asked if it required HS, I meant if it had a HS-specific code dependency.
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 1, 2017, 12:33 AM
 
Done—it's now recompiled for Sierra. I think I didn't use any APIs that will break it there, but since I haven't tested it, no guarantees.

The other disclaimer is that if I ever do make this into a full deduper, it'll have to go back to HS-only since you can't do that without APFS. That's still an "if" at this stage, though. Tell me what you think.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Oct 1, 2017, 01:44 AM
 
Run completed, took less than an hour. 500 GB partition, 315 GB used, 8.3 GiB duplicated. edit - RAM usage peaked at 655 MB.

It's finding mostly game files (especially Overgrowth), along with GarageBand files. Oh, and 6.2 MB of identical .DS_Store files. There's a scattering of other game and utility dupes too.

My results look plausible.
( Last edited by reader50; Oct 1, 2017 at 01:02 PM. )
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 1, 2017, 02:00 AM
 
Thanks!

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Oct 1, 2017, 12:58 PM
 
Originally Posted by reader50 View Post
Your link is most of what I read at the time. The Apple engineers were interested in talking about it, and carefully ignored the use of 3rd party storage devices. What gave me hope is that no one ruled out file checksums. They just didn't commit to anything, and kept asking questions.
What 3rd party storage devices? Internal 3rd-party storage devices are all but gone: The MBPs have storage soldered in now. The iMac requires you to remove a glued-on display to get at the drive. The Mac mini's last (or maybe latest) update made it significantly harder to access as well, and the trashcan Mac Pro has removable but extremely proprietary storage. Apple probably has stats on how many users have third-party external storage that isn't strictly backup, and probably knows that the number is small. If Apple implements an online backup service for Macs - they are clearly moving in that direction - the need for third-party storage will be gone from Apple's perspective.

CloneClub is running as I write this. Status bar looks full, but it is still running. Charles, what are you using to checksum? Because my MBP is at max all-core turbo all throughout the scan, and that implies that there is no AVX2 code at least (AVX will mostly disable turbo, and AVX2 will even reduce clocks below base clock as the chip power throttles. This is documented by Intel, but mostly not acknowledged unless you really dig down into the depths of the documentation).

EDIT: done. Didn't time it, but Activity Monitor says that it used 41 minutes of processor time. 500GB drive, 120 GB free, 690 MB duplicates.
( Last edited by P; Oct 1, 2017 at 01:11 PM. )
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
reader50  (op)
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Oct 1, 2017, 01:14 PM
 
Originally Posted by P View Post
What 3rd party storage devices? Internal 3rd-party storage devices are all but gone: {summary of current models} Apple probably has stats on how many users have third-party external storage that isn't strictly backup, and probably knows that the number is small. If Apple implements an online backup service for Macs - they are clearly moving in that direction - the need for third-party storage will be gone from Apple's perspective.
Recall that Macs are expected to last a long time. I'm not the only cheesegrater MacPro owner skipping the trashcan because of lack of internal expansion. And while Apple may offer online backup, today they recommend everyone add an external for TM backup. Even if they do offer online backup later, everyone should still have a local backup against internet outage, and for fast recovery. Internet speeds/prices remain a black eye in the US.

So I'm talking external storage, potentially more external drives than the number of Macs in use. And internal drives in legacy Macs, perhaps half of which remain supported. But even if we stick to Apple's internal OEM components, Apple cannot guarantee no bitrot will happen. Claiming that relies on statistics, but stats cannot take into account unexpected conditions. Like unplanned component failures elsewhere in the Mac (power supply voltage spikes), or if X-ray machines became routine in airports - or traffic stops at state borders in the southern US. Or if cosmic ray incidents at the surface picked up.

Apple should let us guarantee data integrity, regardless of how comfortable they feel with their hardware.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Oct 1, 2017, 02:46 PM
 
While I generally agree that Apple should support third-party storage, I don't think they will. If they feel that they can control their own storage units through checksumming in the SSD controller, I imagine that they won't bother doing anything in the filesystem.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Oct 1, 2017, 07:56 PM
 
Originally Posted by P View Post
While I generally agree that Apple should support third-party storage, I don't think they will. If they feel that they can control their own storage units through checksumming in the SSD controller, I imagine that they won't bother doing anything in the filesystem.
I mostly agree with your take. Two things, though: first of all, even if Apple does the checksumming on the SSD controller (presumably in hardware), there need to be some hooks in the filesystem to tell the OS when stored files have become corrupted and — if at all possible — need to be replaced by a backup or an older copy. So there needs to be some filesystem/OS integration one way or another. Secondly, given that Apple builds its own SoCs at least for all devices other than Macs, and the SoC houses the storage controller, it is not necessary to build checksumming into the controller, they could put the hardware acceleration units onto the the CPU bit and have the OS take care of that. That would expose more of the inner workings of checksumming to the OS which, I think, might allow for more end-user functionality. With current Intel-based Macs, though, Apple would have no choice but to put it into the controller. On the other hand, there is the perennial rumor that Apple is pondering to move its Macs to ARM.*

However, at the end of the day, I don‘t care how data checksumming is implemented, as long as it is implemented. Ditto for ECC RAM, that should be standard.


* Personally, I am quite sure that Apple has had a Project “reverse”-Marklar in the works since aeons ago (there was a rumored Apple A5-based MacBook), so that just in case it needs to and decides to switch, it can do so quickly. With the exception of the high-end MacBook Pro, Apple could conceivably switch most of its mobile Macs to ARM today if it decided to. Slowly, but surely, Apple’s chip team is creeping up to Intel according to any performance measure.
I don't suffer from insanity, I enjoy every minute of it.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Oct 2, 2017, 05:16 AM
 
I think Apple wants to keep the checksumming in the SSD controller for power reasons. Sending all that data to the SOC will cost lots of power. They do need hooks to the OS though, we agree on that.

Apple recently made a new dump of Darwin (open source Mac OS X kernel) to github, and this time included all the iOS parts for the first time. It truly is one development tree.

As for Mac on ARM... being as good as Intel isn't enough. Moving CPU archs again is hard and costs money, and I don't think they will unless their own ARM hardware is noticeably better. Thet do have that option in the back pocket, though.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
 
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 09:55 AM.
All contents of these forums © 1995-2015 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2015, Jelsoft Enterprises Ltd.,