Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > News > Mac News > Exclusive Analysis: iOS app storage limits across range of devices

Exclusive Analysis: iOS app storage limits across range of devices
Thread Tools
NewsPoster
MacNN Staff
Join Date: Jul 2012
Status: Offline
Reply With Quote
Nov 8, 2014, 10:23 AM
 
Ever since reports of issues with large libraries on Apple's new iPhone 6 Plus surfaced, rumors have been flying around -- all claiming to know the true source of the problems, and Apple's future intentions for the iPhone and iOS. MacNN began some tests about a month ago to try and nail down the root cause of the issue -- and as we alluded to last week, we believe that the "problem" lies in the iOS, and not a hardware flaw related to NAND cells. Read on for our methodology, as well as our specific findings.

Materials:

We obtained a wide variety of iOS hardware, including 64GB versions of the iPad 1, 3, 4, Air, and Air 2. Additionally, 64GB iPhones 4, 4s, and 5 were obtained. We sourced three iPhone 6 Plus 128GB configurations, as well as a single iPhone 6 128GB. On hand for the testing were also a 128GB iPad 4, and a 128GB iPad Air. All devices, other than the original iPad, were running iOS 8. Synchronizations were performed over USB 2.0 and eventually, 802.11n Wi-Fi, from both a 2010 Mac Pro, as well as a 2012 MacBook Pro with Retina Display.

To hit the 1,000 app "soft limit," as claimed by Internet reports, we sourced many small apps from the Apple iOS App Store -- we managed to get these apps to total 49.8GB, just under a pre-selected 50GB limit we imposed on ourselves. We also developed a second library of 1,300 apps, which could be contained in 100.7GB. All apps are iOS 5 compatible, or had an iOS 5 compatible version available, to suit the iPad 1's OS limit for some of the tests.

We also developed a 20,000-title MP3 library weighing in at 55.4GB, and a 20,000-strong AAC catalog hitting 51.3 GB. As applications aren't a single discrete file, but a bundle of files, the music libraries would far exceed the file count in the similarly-sized application libraries.


Method and observations stage one: application loading

All of the iOS devices were initially loaded with the 50GB, 1,000-app library we developed, and power-saving measures were disabled. Following the USB synchronization, 20 apps were run and used for an hour before recording observations.

The iPad 1 and 3 immediately developed extreme slowdown -- taps were not instant, and dragging items on the iPad "desktop" was incredibly slow. The iPad Air models had little, if any, tap lag, likely due to the processing power of those two models. All of the devices we tested manifested some crashing -- more than expected, but none so severe a device reboot was required. The tap lag on the older devices varied, and was not consistent measurement to measurement, and wasn't related to any device condition we could observe.

All devices were erased to factory specifications following the first test. We then repeated the test with the 100GB library on devices that could support it, and had the exact same manifestations. Crashing was no more prevalent with the 1,300 app library than with the 1,000 app library.

To determine if the synchronization method was potentially contributing to the reported issues, we then re-erased, and re-synchronized the devices with the 50GB and 100GB libraries over Wi-Fi. As expected, sync times were longer, but not mathematically more than expected. As the older iOS devices couldn't support this test, those weren't attempted.

The iPad 4 developed a slight tap lag, but not severe. As before, the iPad Air models didn't have much of a problem dealing with the library size. All devices had some instability, but not subjectively more than with the smaller library.

From the first battery of tests over eight days, initial observations pointed to an iOS issue, unrelated to connectivity method, as issues manifested with all devices. The different versions of iPads used different kinds, and speeds, of NAND and DRAM. Regardless of age or model of device, with libraries over 1,000 applications, instabilities developed. The commonality between the devices is little, as the hardware used different DRAM, processors, and even the screen (and these parts came from different manufacturers. This leaves the iOS as the source of the issues.


Method and observations stage two: media loading

Having proven that a really large amount of applications can cause instabilities, we shifted our attention to stored data other than applications. We then reset all the devices again. A suite of 50 apps was loaded, all taken from the previous libraries, and a mix of games, productivity tools, and media extenders. Having previously proven that sync method didn't matter for stability, we then loaded the MP3 library on all devices over the USB 2.0 connection.

The various media players we tried struggled a bit on scrolling on the older iPads, specifically the iPad 1, but we didn't develop any tap lag, nor did we see any instability. We repeated the test with the AAC library, and saw no issues either, other than that expected on the older iPad 1.

We then loaded both the AAC library and MP3 library to be stored simultaneously on the 128GB capacity devices. Again, no issues manifested with over 40,000 individual files loaded on the iPads and iPhones.

This second test, and its lack of resultant data, points away from NAND or controller issues related to number of files. The NAND media doesn't care what's stored on it -- data is data. If there was a problem with the devices being filled to near-capacity, it would have manifested itself when being loaded down with 40,000 music files. This again suggests an issue with the iOS, and addressing large amounts of applications, rather than dealing raw data.


Method and observations stage three: decreasing library size

For stage three, we stepped down library sizes by 100 apps per test. We first analyzed the 128GB devices, and the 1,300-count library using wired synchronization. Again, power saving measures were disabled. Following the USB synchronization, 20 apps were run and used for an hour before recording observations. Each 100-app step pruned the smallest of the applications first.

In all devices, instability dramatically stopped on all devices when we hit the 900 app point, with no subjective decrease in instability before that point. At 900 apps, the library still weighed in at 84.1GB.

We then repeated the test with the smaller 1,000 app library on all of the devices. Again, instability dramatically stopped on all devices when we hit the 900 app point, with no subjective decrease in instability before that point. The 900 app point was 43.1GB, not much smaller than the 50GB target size established.

All of the devices then had energy saving settings turned on, and were put into regular use, consisting of educational apps, web surfing, and game playing. Instability never manifested itself.


Method and observations stage four: stable library, full data

So, keeping library sizes under 900 apps as a factor in stability, we turned to the possibility of a combined instability. Using the 900 app, 84.1GB library initially, we filled the rest of all of the 128GB devices with MP3 and AAC files, drawn from the 20,000 song libraries.

Clearly, the iOS doesn't like to be full to near capacity -- we couldn't change settings on any device when the device was within 20MB of maximum storage. This aberration cleared up when we freed 100MB of space on all devices.

However, regardless of how full the iPads or iPhones were, instability didn't pop up at any point in the testing. We did the "regular use" testing, as we did in stage three. Besides keeping 100MB free being a bit of a challenge in everyday use, there were no other problems.


Conclusions and issues

This is by no means a comprehensive test, since we only tested three iPhone 6 Plus units. We can't comprehensively rule out a problem with some runs of NAND, but we didn't see any. Instead, we did see a pattern with iOS versions for years being unable to handle application libraries over 900 applications. Different manufacturers and techniques of NAND assembly were used across the generations of iOS devices, which points away from the TLC cells used in the iPhone 6 Plus being the cause of the problem.

However, consider the magnitude of 900 applications, or more, installed on a single device. We question the true utility of this, we wonder where this is practical, for even the most ardent user of the iOS. In our usage, more than 300 apps was unwieldy, and a manual search through double-digit quantities of home screens and folders is ridiculously hard. With an average of 20 apps per home screen, assuming some blank space, 900 apps spans 45 pages!

We queried the owners of the devices that we borrowed for the testing, and all but one claimed to have less than 16 everyday-use apps, with the average being 10. One outlier claimed to use 30 or more per day, but we had doubts of the veracity of this claim. We asked a prominent business in the Washington DC metro area with wide iOS utility what use patterns they were seeing, and they said that most users were using 10 or fewer apps per day, lending credence to our informal polling.

Rumors spread yesterday from another South Korean news agency that Apple was considering a retool of the device. If Tim Cook is unwilling to retool the iPod Classic because of the monumental engineering effort it would take to do so, why would a retool of the iPhone 6 Plus be likely, which would require more of an engineering effort to maintain the size profile? Shifting to a lower density storage medium would require a case size increase, above and beyond a motherboard redesign.

So, is the crashing a real problem for Apple? We don't believe it is -- loading 1000 apps on a device has to be such an unusual usage use of the technology, and while possible, we can't even imagine that even one percent of the user base is doing it.

We're not seeing any reports from Apple itself about a wide-spread issue, and the official guidance to users being doled out at the Apple store is to prune libraries to less than 700 apps installed on a device. Yes, this advice smacks of "you're holding it wrong" from iPhone 4 antenna issues, but a recall was widely rumored for the iPhone 4 for those problems that never materialized. This app limit is a non-issue for what we'd guess is 99 percent of users, is likely being stirred up beyond the actual magnitude of the problem by an Apple competitor, and doesn't conclusively point to a hardware problem at all.

-Mike Wuerthele (@MacNN_Mike)
( Last edited by NewsPoster; Nov 10, 2014 at 10:31 AM. )
     
DCJ001
Dedicated MacNNer
Join Date: Oct 2007
Status: Offline
Reply With Quote
Nov 8, 2014, 03:32 PM
 
"we managed to get this in at 49.8GB, just under 50GB."

Okay. Just to be sure that I understand what you wrote, 49.8GB, is less than 50GB. Is that right?

Wow! I learn something new every day at MacNN.
     
Mike Wuerthele
Managing Editor
Join Date: Jul 2012
Status: Offline
Reply With Quote
Nov 8, 2014, 04:18 PM
 
Originally Posted by DCJ001 View Post
"we managed to get this in at 49.8GB, just under 50GB."

Okay. Just to be sure that I understand what you wrote, 49.8GB, is less than 50GB. Is that right?

Wow! I learn something new every day at MacNN.
There was a reason for the 50GB, which is apparent later in the analysis.
     
cgc
Professional Poster
Join Date: Mar 2003
Location: Down by the river
Status: Offline
Reply With Quote
Nov 9, 2014, 05:30 AM
 
Originally Posted by Mike Wuerthele View Post
There was a reason for the 50GB, which is apparent later in the analysis.
I think his point was that we all realize 49.8 is less than 50...or was this analysis sponsored by the #4 and the letter "q" (say it in Cookie Monster's voice for effect)?
"Like a midget at a urinal, I was going to have to stay on my toes." Frank Drebin, Naked Gun 33 1/3: The Final Insult
     
Mike Wuerthele
Managing Editor
Join Date: Jul 2012
Status: Offline
Reply With Quote
Nov 9, 2014, 01:42 PM
 
Very well, then. I clarified the language for the reasoning behind the 50GB limit.
     
FLBeachComber
Fresh-Faced Recruit
Join Date: Nov 2014
Status: Offline
Reply With Quote
Nov 9, 2014, 05:08 PM
 
Mind explaining to me why 1450 apps on an iPhone 5 64GB works fine while 1450 on an IPad Air 128GB with the same exact apps / screens (except where iPad Apps are substituted for iPhone Apps) causes the iPad Air 128GB to reboot every 30 seconds under iOS 8?

Also, you state that 900 apps takes 45 pages, yet the device will not display 45 pages. One needs to use folders and arrange folder/apps accordingly. I have a page / folders for Travel / Weather / Newsfeeds etc. Perhaps that you did not use folders for organization is a critical flaw in your testing - not to mention 45 Homescreens which does not exists.

And finally, when Apple states that the amount of apps is limited only by physical memory, who are you to use between 25% - 30% of the "review and testing" to question the need of that - something that is subjective and would never be allowed in any true "review and testing".

It's almost as if you went in with a biased opinion and went out to prove same.
     
nagromme
Senior User
Join Date: Dec 2000
Status: Offline
Reply With Quote
Nov 9, 2014, 05:45 PM
 
Sounds like it's not hardware. But the problem is still real and serious (it affects me for one):

- What if you collect games the way some people collect movies, books, or music? Hitting 1000 is easy if you've had an iPhone for years, and if you don't want to lose your progress/unlocks, you keep the games. Which worked fine on my old iPhone, not on my new Plus.

- A problem that affects only a small segment of users is still serious, even if lower-priority than one that hits everyone. What if a bug made iPhones crash for people with two Z's in their name? Probably well under 1%... still matters.

- If the iOS cannot handle 1000 apps, it shouldn't allow you to install them without warning. (But I predict it WILL be able to handle it again, once a fix is out.)

- This is a data loss bug: if you can't restore your old backup, you can and will lose vital files stored in your apps. ANY bug that can make you lose access to files and information is serious. (And I do mean "lose:" if I install my 1000-app iPhone backup on the Plus, it can't do much of anything--I cannot launch most apps at all.)

- People with the most apps (who actually care to NOT lose their old backup) are going to be the longest-term, most-loyal, and highest-end users. They're not the chunk to ignore. (And I don't think Apple will. Make sure of it: if you have problems, let Apple know! Not just forums.)

So, the NAND thing was an interesting theory, but what we really need is a software fix. (And not just for this--I know iOS 8 has other bugs.) Apple bit off more than they could chew in one year, it seems, and I hope they make it right soon.
nagromme
     
Mike Wuerthele
Managing Editor
Join Date: Jul 2012
Status: Offline
Reply With Quote
Nov 9, 2014, 05:55 PM
 
To FlBeachComber:

Its more the other way around -- with more devices on hand here than I think you have, we tested as best as we could.

We said clearly that we couldn't conclusively rule out a NAND issue in the iPhone 6 Plus, but since issues manifested across generations with a large quantity of apps, there is clearly a larger problem than one just with the 6 Plus and its NAND.

Who am I to use 25-30 percent of the review and testing to point out conclusions drawn from a multi-generational test? I'm the writer, that's who.

We were told by Apple to keep the App count down to maintain stability. We wanted to figure out why, so we did. This isn't a pro-Apple article, as Apple doesn't even really acknowledge that there's much of a problem.

My language on the 900 apps and 45 pages should have been more clear, that'll teach me to write and edit early Saturday morning - I'll rework that section. It takes a full component of home pages, as well as packed folders to hold, and access, all the apps.

Since you don't like our discovery, It's almost like you have confirmation bias with your single point of failure, and need to point it out to us. I've got no issue with you letting us know about your device and your problems, but our larger sample here than you have there points to an iOS issue, and not a hardware issue like you seem to think it is.

While you're here, I've been meaning to ask somebody who legitimately has all these apps installed -- what utility do you have with 1450 apps installed? Do you use all of them frequently?

Also, with your iPhone 5, is it on 7 and the problem manifests on 8 only? More fuel for the OS issue, and not a hardware problem.
     
FLBeachComber
Fresh-Faced Recruit
Join Date: Nov 2014
Status: Offline
Reply With Quote
Nov 9, 2014, 07:55 PM
 
No, my iPhone is now on 8.1 and has not had issues with any of the iOS 8 versions.

However, the iPad Air 128GB on iOS 7, which I suspect you did not test, has the same issues as the newer devices on iOS 8. However, the crashes with iOS 8 occur virtually every 30 seconds, compared to 5 minutes or longer with iOS 7.

You did not spend 25%-30% of your review summarizing of the results of your test. You spent 25%-30% of your review questioning why anyone needed more than 30-900 Apps.

That shows a clear bias.

With multiple support cases for up to a year on this issue with 128GB devices (not to mention over 5 replacement devices), Apple Senior Techs maintain there is no limit to number of Apps the device can handle - it is only limited to Physical Memory.

If I want to pay $100 or $200 more for extra memory, who are you to tell me how I can use it, when Apple tells me it's device can handle it. Perhaps the App Store would like to refund me ~$700 for the expenditures that their device cannot handle, which apparently is a fraudulent claim.
     
Mike Wuerthele
Managing Editor
Join Date: Jul 2012
Status: Offline
Reply With Quote
Nov 10, 2014, 08:55 AM
 
Actually, according to the word count, I spent less than 10% wondering how people used that many apps on a regular basis. The vast majority of the count was discussing results. This isn't bias, it's called examination of the issue from all sides. Do you think that a large percentage of users has that many applications in continuous use?

I'm not sure what your agenda is, here. I'm not saying that there's no issue -- in fact, I'm saying that there is! Clearly the iOS doesn't care for that many apps loaded at one time. There is no hardware problem, as we demonstrated, as the device is solid as a rock loaded down with media. How can you say it's a hardware problem with the iPhone 6 Plus when you see it on your iPad Air?

I'm not telling anybody how to use the storage. If you want to pack it full of apps, go nuts. Just realize that there's a demonstrable issue with the OS (which you've proven yourself with your iPad Air example) that causes problems with so many applications.

If you take the issue up with Apple, feel free to point to this article to back up your claims of a problem with many applications.
     
   
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 08:27 AM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,