Ever since reports of issues with large libraries on Apple's new iPhone 6 Plus surfaced, rumors have been flying around -- all claiming to know the true source of the problems, and Apple's future intentions
for the iPhone and iOS. MacNN
began some tests about a month ago to try and nail down the root cause of the issue -- and as we alluded to last week, we believe that the "problem" lies in the iOS, and not a hardware flaw related to NAND cells. Read on for our methodology, as well as our specific findings.
We obtained a wide variety of iOS hardware, including 64GB versions of the iPad 1, 3, 4, Air, and Air 2. Additionally, 64GB iPhones 4, 4s, and 5 were obtained. We sourced three iPhone 6 Plus 128GB configurations, as well as a single iPhone 6 128GB. On hand for the testing were also a 128GB iPad 4, and a 128GB iPad Air. All devices, other than the original iPad, were running iOS 8. Synchronizations were performed over USB 2.0 and eventually, 802.11n Wi-Fi, from both a 2010 Mac Pro, as well as a 2012 MacBook Pro with Retina Display.
To hit the 1,000 app "soft limit," as claimed by Internet reports, we sourced many small apps from the Apple iOS App Store -- we managed to get these apps to total 49.8GB, just under a pre-selected 50GB limit we imposed on ourselves. We also developed a second library of 1,300 apps, which could be contained in 100.7GB. All apps are iOS 5 compatible, or had an iOS 5 compatible version available, to suit the iPad 1's OS limit for some of the tests.
We also developed a 20,000-title MP3 library weighing in at 55.4GB, and a 20,000-strong AAC catalog hitting 51.3 GB. As applications aren't a single discrete file, but a bundle of files, the music libraries would far exceed the file count in the similarly-sized application libraries.
Method and observations stage one: application loading
All of the iOS devices were initially loaded with the 50GB, 1,000-app library we developed, and power-saving measures were disabled. Following the USB synchronization, 20 apps were run and used for an hour before recording observations.
The iPad 1 and 3 immediately developed extreme slowdown -- taps were not instant, and dragging items on the iPad "desktop" was incredibly slow. The iPad Air models had little, if any, tap lag, likely due to the processing power of those two models. All of the devices we tested manifested some crashing -- more than expected, but none so severe a device reboot was required. The tap lag on the older devices varied, and was not consistent measurement to measurement, and wasn't related to any device condition we could observe.
All devices were erased to factory specifications following the first test. We then repeated the test with the 100GB library on devices that could support it, and had the exact same manifestations. Crashing was no more prevalent with the 1,300 app library than with the 1,000 app library.
To determine if the synchronization method was potentially contributing to the reported issues, we then re-erased, and re-synchronized the devices with the 50GB and 100GB libraries over Wi-Fi. As expected, sync times were longer, but not mathematically more than expected. As the older iOS devices couldn't support this test, those weren't attempted.
The iPad 4 developed a slight tap lag, but not severe. As before, the iPad Air models didn't have much of a problem dealing with the library size. All devices had some instability, but not subjectively more than with the smaller library.
From the first battery of tests over eight days, initial observations pointed to an iOS issue, unrelated to connectivity method, as issues manifested with all devices. The different versions of iPads used different kinds, and speeds, of NAND and DRAM. Regardless of age or model of device, with libraries over 1,000 applications, instabilities developed. The commonality between the devices is little, as the hardware used different DRAM, processors, and even the screen (and these parts came from different manufacturers. This leaves the iOS as the source of the issues.
Method and observations stage two: media loading
Having proven that a really large amount of applications can cause instabilities, we shifted our attention to stored data other than applications. We then reset all the devices again. A suite of 50 apps was loaded, all taken from the previous libraries, and a mix of games, productivity tools, and media extenders. Having previously proven that sync method didn't matter for stability, we then loaded the MP3 library on all devices over the USB 2.0 connection.
The various media players we tried struggled a bit on scrolling on the older iPads, specifically the iPad 1, but we didn't develop any tap lag, nor did we see any instability. We repeated the test with the AAC library, and saw no issues either, other than that expected on the older iPad 1.
We then loaded both the AAC library and MP3 library to be stored simultaneously on the 128GB capacity devices. Again, no issues manifested with over 40,000 individual files loaded on the iPads and iPhones.
This second test, and its lack of resultant data, points away from NAND or controller issues related to number of files. The NAND media doesn't care what's stored on it -- data is data. If there was a problem with the devices being filled to near-capacity, it would have manifested itself when being loaded down with 40,000 music files. This again suggests an issue with the iOS, and addressing large amounts of applications, rather than dealing raw data.
Method and observations stage three: decreasing library size
For stage three, we stepped down library sizes by 100 apps per test. We first analyzed the 128GB devices, and the 1,300-count library using wired synchronization. Again, power saving measures were disabled. Following the USB synchronization, 20 apps were run and used for an hour before recording observations. Each 100-app step pruned the smallest of the applications first.
In all devices, instability dramatically stopped on all devices when we hit the 900 app point, with no subjective decrease in instability before that point. At 900 apps, the library still weighed in at 84.1GB.
We then repeated the test with the smaller 1,000 app library on all of the devices. Again, instability dramatically stopped on all devices when we hit the 900 app point, with no subjective decrease in instability before that point. The 900 app point was 43.1GB, not much smaller than the 50GB target size established.
All of the devices then had energy saving settings turned on, and were put into regular use, consisting of educational apps, web surfing, and game playing. Instability never manifested itself.
Method and observations stage four: stable library, full data
So, keeping library sizes under 900 apps as a factor in stability, we turned to the possibility of a combined instability. Using the 900 app, 84.1GB library initially, we filled the rest of all of the 128GB devices with MP3 and AAC files, drawn from the 20,000 song libraries.
Clearly, the iOS doesn't like to be full to near capacity -- we couldn't change settings on any device when the device was within 20MB of maximum storage. This aberration cleared up when we freed 100MB of space on all devices.
However, regardless of how full the iPads or iPhones were, instability didn't pop up at any point in the testing. We did the "regular use" testing, as we did in stage three. Besides keeping 100MB free being a bit of a challenge in everyday use, there were no other problems.
Conclusions and issues
This is by no means a comprehensive test, since we only tested three iPhone 6 Plus units. We can't comprehensively rule out a problem with some runs of NAND, but we didn't see any. Instead, we did see a pattern with iOS versions for years being unable to handle application libraries over 900 applications. Different manufacturers and techniques of NAND assembly were used across the generations of iOS devices, which points away from the TLC cells used in the iPhone 6 Plus being the cause of the problem.
However, consider the magnitude of 900 applications, or more, installed on a single device. We question the true utility of this, we wonder where this is practical, for even the most ardent user of the iOS. In our usage, more than 300 apps was unwieldy, and a manual search through double-digit quantities of home screens and folders is ridiculously hard. With an average of 20 apps per home screen, assuming some blank space, 900 apps spans 45 pages!
We queried the owners of the devices that we borrowed for the testing, and all but one claimed to have less than 16 everyday-use apps, with the average being 10. One outlier claimed to use 30 or more per day, but we had doubts of the veracity of this claim. We asked a prominent business in the Washington DC metro area with wide iOS utility what use patterns they were seeing, and they said that most users were using 10 or fewer apps per day, lending credence to our informal polling.
Rumors spread yesterday from another South Korean news agency that Apple was considering a retool of the device. If Tim Cook is unwilling to retool the iPod Classic because of the monumental engineering effort it would take to do so, why would a retool of the iPhone 6 Plus be likely, which would require more of an engineering effort to maintain the size profile? Shifting to a lower density storage medium would require a case size increase, above and beyond a motherboard redesign.
So, is the crashing a real problem for Apple? We don't believe it is -- loading 1000 apps on a device has to be such an unusual usage use of the technology, and while possible, we can't even imagine that even one percent of the user base is doing it.
We're not seeing any reports from Apple itself about a wide-spread issue, and the official guidance to users being doled out at the Apple store is to prune libraries to less than 700 apps installed on a device. Yes, this advice smacks of "you're holding it wrong" from iPhone 4 antenna issues, but a recall was widely rumored for the iPhone 4 for those problems that never materialized. This app limit is a non-issue for what we'd guess is 99 percent of users, is likely being stirred up beyond the actual magnitude of the problem by an Apple competitor, and doesn't conclusively point to a hardware problem at all.
-Mike Wuerthele (@MacNN_Mike