Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Community > MacNN Lounge > Apple, Content Delivery, Bittorrent

Apple, Content Delivery, Bittorrent
Thread Tools
abbaZaba
Mac Elite
Join Date: Jun 2006
Location: Pittsburgh
Status: Offline
Reply With Quote
Sep 18, 2013, 03:46 PM
 
We are all frothing at the release of iOS 7, arguably the most anticipated iOS release and as has happened nearly every time in the past, a mad rush for the update leaves people with errors and slow downloads.

What if Apple incorporated BT or a BT-like system in iTunes where at least users updating through iTunes acted as peers and helped distribute the data they have already downloaded from the distribution node? Is lack of anything like this from BT stigma? Would this ultimately affect the ISPs negatively? Are there reasons I do not know about that make this unfeasible?

I would see something like this being an opt-in sort of thing, and perhaps even have the end user's upload limited to at most 50% of their upload speed. But I can only see a system like this being put into place as helpful for everyone involved: updates get out quicker, less load on distribution servers. Millions of people download these updates and even if 1 home connection is only a little bit of upload, times millions it could make a dent. I choose iTunes or the Mac App store as the programs to have this as doing it on iOS devices adds all sorts of hassle like turning off, not wanting to do it over cell networks, etc.

I will admit that I am not super familiar with how CDN works at the highest levels. I hear Apple uses Akamai CDN and that Akamai has about 40 nodes connected to the MBONE in regions around the world and that they use a BT-like system to distribute a file to those nodes, but from there it is single point-to-point handling all the downloads. Why is single point-to-point still in use at this level when I see multi-GB files transferred with, to me, great efficiency?

Perhaps someone with better knowledge of how higher-up CDN functions could shoot some holes in my thinking or how at the global/regional level distributing files in a BT-like way is a bad idea. I have no idea if I am totally off base here, but I would love to be better informed.
( Last edited by abbaZaba; Sep 18, 2013 at 05:46 PM. )
     
shifuimam
Addicted to MacNN
Join Date: Aug 2006
Location: The deep backwoods of the PNW
Status: Offline
Reply With Quote
Sep 18, 2013, 05:19 PM
 
I think there's a huge stigma against BitTorrent as a protocol at this point.

The bigwigs who make the decisions in large businesses are of the impression that BT is used exclusively for porn and piracy, whereas in reality major organizations like Canonical use it as the preferred content distribution protocol.

I asked in a forum why carrier firmware downloads (which can exceed 2GB each) are only available through shitty file hosting adware services like Uploaded.to, and the answer I got was that "BitTorrent is unsafe".

I honestly think that the biggest reason why something like a major iOS release isn't done through a p2p protocol (even a proprietary one) is the sigma behind such protocols. As far as how it might affect an ISP, it wouldn't be difficult to meter the p2p connection so that users on limited connections weren't sucking all their bandwidth just sharing iOS.
Sell or send me your vintage Mac things if you don't want them.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Sep 19, 2013, 04:18 AM
 
Torrent systems have been tried for distribution like this - I know Blizzard did it for WoW at one point. Customer reaction has been overwhelmingly negative. It works if your audience is tech-savvy and understands what's going on, but all too many people don't. I suspect it falls below Apple's threshold of "ideas we should bother with". Which is too bad, because systems like this can be amazingly powerful.

The security thing sounds like fig leaf. Bittorrent appears to use SHA-1 for computing the hashes (ie validating that a piece of the file that they get is the right one). SHA-1 is no longer recommended for use, as it may not be secure, but it still costs millions of dollars of computer time for a single collision. That's too expensive for anyone not a nosy secretive agency to bother with.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
shifuimam
Addicted to MacNN
Join Date: Aug 2006
Location: The deep backwoods of the PNW
Status: Offline
Reply With Quote
Sep 19, 2013, 11:28 AM
 
Well, that goes back to people assuming that BT is only used for malicious stuff.

The forum thread I got involved in regarding phone firmware downloads was ridiculous - people assumed that a download on Hotfile was inherently secure, because they "know who uploaded it". The reality is, BT is just as secure is much much faster - a 2GB download on Hotfile and similar sites can take hours and hours, whereas with even a few dozen seeders with good bandwidth, that same file might only take fifteen minutes on BitTorrent.

I think that it's feasible to do a distributed download option for people who want to use it - no reason to force it on everyone, but for people who don't mind sharing the bandwidth love, why not make it an option for them?
Sell or send me your vintage Mac things if you don't want them.
     
The Final Dakar
Games Meister
Join Date: Aug 2009
Location: Eternity
Status: Offline
Reply With Quote
Sep 19, 2013, 11:43 AM
 
Wouldn't the main objection be it uses more bandwidth? I mean, sure you're supposed to download this over wifi, but I'm sure it wouldn't be long that someone complained they couldn't figure out why they hit their cap.
     
shifuimam
Addicted to MacNN
Join Date: Aug 2006
Location: The deep backwoods of the PNW
Status: Offline
Reply With Quote
Sep 19, 2013, 12:29 PM
 
BT lets you limit the available bandwidth for upstream and downstream traffic. Seems like you could do the same with a P2P option to download your iOS updates.
Sell or send me your vintage Mac things if you don't want them.
     
The Final Dakar
Games Meister
Join Date: Aug 2009
Location: Eternity
Status: Offline
Reply With Quote
Sep 19, 2013, 12:30 PM
 
Originally Posted by shifuimam View Post
BT lets you limit the available bandwidth for upstream and downstream traffic. Seems like you could do the same with a P2P option to download your iOS updates.
This is not something the kind of person who does not even realize how bittorrent works would do.
     
shifuimam
Addicted to MacNN
Join Date: Aug 2006
Location: The deep backwoods of the PNW
Status: Offline
Reply With Quote
Sep 19, 2013, 12:43 PM
 
Originally Posted by The Final Dakar View Post
This is not something the kind of person who does not even realize how bittorrent works would do.
I get that - that's why I'm talking about an option. For people who know what they're doing, it'd be nice to have the option to use a distributed download system. For people who don't know what they're doing, they can keep using the shitty one-source download that's slow and error-prone.

And, Apple being Apple, I'm sure they could make a really nice, pretty configuration wizard so that even people who DON'T know what they're doing can get things set up pretty easily and safely.
Sell or send me your vintage Mac things if you don't want them.
     
The Final Dakar
Games Meister
Join Date: Aug 2009
Location: Eternity
Status: Offline
Reply With Quote
Sep 19, 2013, 12:55 PM
 
You'd have to set it to not share by default, meaning its usefulness as an implementation would be severely curtailed from the get go. I think that would be enough to make Apple think it's not worth the effort of implementation.
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 01:06 PM
 
It seems like a lot of effort and resources for something very few would use. It also opens Apple up to liability issues and class action suits.

Most people can handle waiting a day.
     
shifuimam
Addicted to MacNN
Join Date: Aug 2006
Location: The deep backwoods of the PNW
Status: Offline
Reply With Quote
Sep 19, 2013, 01:11 PM
 
If Apple could figure out a way to implement distributed downloads in a way that (a) doesn't suck bandwidth and (b) doesn't require knowledge of how BT works to use it (and I do think Apple could accomplish this; they specialize in dumbing things down), it could save them a shitload of money on bandwidth whenever a new iOS update goes public.
Sell or send me your vintage Mac things if you don't want them.
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 01:13 PM
 
Also, Apple isn't just for the less geeky. Sometimes the more geeky don't feel like hand ****ing each capacitor to get something done.

I know how to use BitTorrent, it's still a pain in my ass.

And to cap off the rant, I don't pay you ****ers enough already? You want to swipe my bandwidth now too?
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 01:15 PM
 
Originally Posted by shifuimam View Post
If Apple could figure out a way to implement distributed downloads in a way that (a) doesn't suck bandwidth and (b) doesn't require knowledge of how BT works to use it (and I do think Apple could accomplish this; they specialize in dumbing things down), it could save them a shitload of money on bandwidth whenever a new iOS update goes public.
I'd rather they just paid that money to Akamai.
     
shifuimam
Addicted to MacNN
Join Date: Aug 2006
Location: The deep backwoods of the PNW
Status: Offline
Reply With Quote
Sep 19, 2013, 01:27 PM
 
Wow.

I love BT. No ass pain over here.

Hell, Apple has the money and resources to just set up their OWN peers for distributed downloading of iOS, instead of a single-source download.
Sell or send me your vintage Mac things if you don't want them.
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 01:43 PM
 
Originally Posted by shifuimam View Post
I love BT. No ass pain over here.
Compared to just paying for your content?
     
shifuimam
Addicted to MacNN
Join Date: Aug 2006
Location: The deep backwoods of the PNW
Status: Offline
Reply With Quote
Sep 19, 2013, 01:50 PM
 
Originally Posted by subego View Post
Compared to just paying for your content?
I like the assumption that the only thing you can use BT for is piracy.

I also use it for things like downloading Linux distributions, because it's so much faster. And, as I pointed out in the newest thread about Netflix, content owners are so bitchy with who gets to see what that sometimes the only place to find something is on BT.

Don't go getting all high-and-mighty and self-righteous on me.
Sell or send me your vintage Mac things if you don't want them.
     
abbaZaba  (op)
Mac Elite
Join Date: Jun 2006
Location: Pittsburgh
Status: Offline
Reply With Quote
Sep 19, 2013, 03:11 PM
 
Originally Posted by The Final Dakar View Post
You'd have to set it to not share by default, meaning its usefulness as an implementation would be severely curtailed from the get go. I think that would be enough to make Apple think it's not worth the effort of implementation.
Did you read the entire first post?

1) I mentioned it being only for iTunes on a Mac/Windows- absolutely not on cell devices specifically for the caps on cell data

2) I mentioned it being an opt-in feature for those of use that want to help and know what we are doing

3) I mentioned out of the box limiting it to 50% of the user's upload bandwidth. my upload bandwidth on Comcast is about 1MB/sec (that is megabyte, not megabit) Even if we go with a conservative estimation of the average user's home upload bandwidth at 100KB/sec, and say we have 100,000 people opt in: that is 5GB/sec of bandwidth taken off the server. That is 5 iOS 7 iPad installs per second. Do you see how this could be beneficial?

4) This is Apple here, where they like to make technology invisible. I imagine in a well thought out scenario being a simple bandwidth test to test the user's upload limit and set it to 50% of that, and a checkbox to opt in. Probably some plist files super nerds can edit.

What company is better at making complicated technology invisible to the end user?
( Last edited by abbaZaba; Sep 19, 2013 at 03:23 PM. )
     
abbaZaba  (op)
Mac Elite
Join Date: Jun 2006
Location: Pittsburgh
Status: Offline
Reply With Quote
Sep 19, 2013, 03:15 PM
 
Originally Posted by subego View Post
Compared to just paying for your content?
Perfect example of stigma getting in the way of improving things. Not only do I always grab my linux distros via BT, I roll my own cloud solution with a small pool of computers using BTSync. This kind of mindset is what hampers possibly changing things for the better.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 19, 2013, 03:20 PM
 
I think Apple should continue doing what they are doing and master content delivery, because this is an important part of their business in iCloud, Siri, and I think this will only increase as time progresses.

If Google can handle content delivery in YouTube without the distributed system that BItTorrent uses, Apple can too, in theory (based on the technological demand). YouTube's daily bandwidth consumption would make iOS downloads look like a walk in the park. Apple needs to move in this direction too.
     
abbaZaba  (op)
Mac Elite
Join Date: Jun 2006
Location: Pittsburgh
Status: Offline
Reply With Quote
Sep 19, 2013, 03:20 PM
 
Originally Posted by subego View Post
And to cap off the rant, I don't pay you ****ers enough already? You want to swipe my bandwidth now too?
What percentage of the day are you even using the upload portion of your home pipe? Hell, they could enable it for the first 12 hours of a rollout and then disable it after which every new user attempting to grab the update does so via single point-to-point. Besides, if you aren't an early adopter of the new update, you wouldn't even be factored into this equation. Let me reiterate: it is for taking the initial load off, not a permanent distribution method.
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 03:21 PM
 
Originally Posted by shifuimam View Post
Don't go getting all high-and-mighty and self-righteous on me.


I wasn't.

The lady doth protest too much.
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 03:24 PM
 
Originally Posted by abbaZaba View Post
What percentage of the day are you even using the upload portion of your home pipe?
The percentage of the day I'm ****ing paying for it is 100%.

Apple can deal with it's own initial load.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 19, 2013, 03:29 PM
 
Originally Posted by subego View Post
The percentage of the day I'm ****ing paying for it is 100%.

Apple can deal with it's own initial load.

I agree, and often times downloads are slower when there is a simultaneous upload. Maybe our attitudes about this would be different if we didn't have to pay for internet twice in our cell and home internet usage, and if the total price for the two was similar to what we pay for electricity. Most store owners, for instance, don't seem to worry too much about people plugging in stuff to their electrical outlets.
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 03:30 PM
 
Originally Posted by abbaZaba View Post
Perfect example of stigma getting in the way of improving things. Not only do I always grab my linux distros via BT, I roll my own cloud solution with a small pool of computers using BTSync. This kind of mindset is what hampers possibly changing things for the better.
We're talking low friction.

Linux is not ****ing low friction.

You pay for low friction.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 19, 2013, 03:31 PM
 
Originally Posted by subego View Post
We're talking low friction.

Linux is not ****ing low friction.

You pay for low friction.

This is true, but I think the point he was making was that a low friction provider could provide the same tech he uses in Linux, and he's right about that.
     
abbaZaba  (op)
Mac Elite
Join Date: Jun 2006
Location: Pittsburgh
Status: Offline
Reply With Quote
Sep 19, 2013, 03:33 PM
 
Originally Posted by subego View Post
We're talking low friction.

Linux is not ****ing low friction.

You pay for low friction.
What exactly has riled you up so much that you need to swear in every post? I have never seen you like this in any thread before.
     
abbaZaba  (op)
Mac Elite
Join Date: Jun 2006
Location: Pittsburgh
Status: Offline
Reply With Quote
Sep 19, 2013, 03:47 PM
 
Originally Posted by besson3c View Post
This is true, but I think the point he was making was that a low friction provider could provide the same tech he uses in Linux, and he's right about that.
I grab the distros I need via BT on the OS I use mainly: OS X. Linux distros are just an example of a fairly large file I grab super fast over BT. In fact, they are around the size of a iOS update. It always maxes out the download on my not-too-shabby home connection. SXSW also releases a collection of their music after every festival, and it is totally legit and free. The torrent for that runs in the GBs. My point is that BT is used efficiently and often for legitimate means. The mindset it is only used for piracy is absurd.
     
mduell
Posting Junkie
Join Date: Oct 2005
Location: Houston, TX
Status: Offline
Reply With Quote
Sep 19, 2013, 03:58 PM
 
BT is a poor man's CDN. If the distributor is willing to pay for a CDN, using the CDN is more efficient on the network than BT.
     
abbaZaba  (op)
Mac Elite
Join Date: Jun 2006
Location: Pittsburgh
Status: Offline
Reply With Quote
Sep 19, 2013, 04:02 PM
 
Originally Posted by mduell View Post
BT is a poor man's CDN. If the distributor is willing to pay for a CDN, using the CDN is more efficient on the network than BT.
You are more the type of person I was hoping to pull into this thread: someone more knowledgable than I on how this stuff works at a high level and from whom I could learn some things. Not someone who cannot hold their in check.

So with their billions in cash, is Apple just not paying enough to their CDN? Why does this continue to happen every single update launch?
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 04:12 PM
 
Originally Posted by abbaZaba View Post
What exactly has riled you up so much that you need to swear in every post? I have never seen you like this in any thread before.
The **** you haven't!


It was a rant. Challenges to my rant will be met with rant.


Admittedly, it's somewhat fueled by the multiple accusations I'm somehow "part of the problem" for daring to suggest there are easier ways to get content, and that Linux distros are even remotely related to how the richest company on the planet pushes updates to its users.


Note to all: pirate (or not pirate) all you want. I have no shits to give either way, and I create content for a living. I like BitTorrent BTW. That doesn't make it lower friction than Apple, Google, Amazon, and Netflix.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 19, 2013, 04:13 PM
 
Originally Posted by abbaZaba View Post
You are more the type of person I was hoping to pull into this thread: someone more knowledgable than I on how this stuff works at a high level and from whom I could learn some things. Not someone who cannot hold their in check.

So with their billions in cash, is Apple just not paying enough to their CDN? Why does this continue to happen every single update launch?

If I could wager a guess, the CDN providers simply provide the infrastructure, it is up to Apple to decide how much of it to use, when to request more of it, and how to control how it is used.

I've personally had more problems with Apple server verifications than I've had with raw download speed.
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Sep 19, 2013, 04:23 PM
 
Originally Posted by besson3c View Post
If I could wager a guess, the CDN providers simply provide the infrastructure, it is up to Apple to decide how much of it to use, when to request more of it, and how to control how it is used.

I've personally had more problems with Apple server verifications than I've had with raw download speed.
I think it's Jobsian control freak DNA making them want to stay in-house.
     
Shaddim
Clinically Insane
Join Date: Apr 2003
Location: 46 & 2
Status: Offline
Reply With Quote
Sep 19, 2013, 04:47 PM
 
Originally Posted by subego View Post


I wasn't.

The lady doth protest too much.
Saw that coming.
"Those who expect to reap the blessings of freedom must, like men, undergo the fatigue of supporting it."
- Thomas Paine
     
mduell
Posting Junkie
Join Date: Oct 2005
Location: Houston, TX
Status: Offline
Reply With Quote
Sep 19, 2013, 07:58 PM
 
Originally Posted by abbaZaba View Post
So with their billions in cash, is Apple just not paying enough to their CDN? Why does this continue to happen every single update launch?
Apple is paying for the best CDN in the business. Why does what happen? Your initial post was about why they don't use BT, not asking about a problem BT could solve.

Originally Posted by abbaZaba View Post
I will admit that I am not super familiar with how CDN works at the highest levels. I hear Apple uses Akamai CDN and that Akamai has about 40 nodes connected to the MBONE in regions around the world and that they use a BT-like system to distribute a file to those nodes, but from there it is single point-to-point handling all the downloads. Why is single point-to-point still in use at this level when I see multi-GB files transferred with, to me, great efficiency?
Apple does use Akamai and Akamai does not use BT or MBONE as far as I know (is MBONE still a thing? I thought they gave up on it).

Akamai has about 2200 points of presence (averaging 60 servers each) on 1200 networks around the globe. So for most everyone on the globe they have servers either at your ISP, peered with your ISP at an IX, or one hop away. With a popular file like the iOS 7 download, it will rapidly be cached at most of those POPs (or possibly pre-staged, not sure if they use that) and served to everyone on that ISP. This is far more efficient than having a random guy in your city, another in germany, and another in thailand send you different bits of the file over various backhaul and through your ISP.

Put another way, large BT swarms usually have a few thousand peers all over the place. Akamai has a node at your ISP or a hop away.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Sep 19, 2013, 08:07 PM
 
Originally Posted by mduell View Post
Apple is paying for the best CDN in the business. Why does what happen? Your initial post was about why they don't use BT, not asking about a problem BT could solve.
Is Akamai regarded higher than Amazon CloudFront?
     
mduell
Posting Junkie
Join Date: Oct 2005
Location: Houston, TX
Status: Offline
Reply With Quote
Sep 20, 2013, 01:20 AM
 
lol, yea.
     
phantomdragonz
Mac Elite
Join Date: Aug 2001
Location: Near Boulder, CO
Status: Offline
Reply With Quote
Sep 20, 2013, 03:44 AM
 
Akamai is HUGE, the fact that MILLIONS of people were allowed to get the update today without a huge issue is testament to the ability of the existing setup.

the wiki page for akamai is interesting
http://en.wikipedia.org/wiki/Akamai_Technologies

it hosts Hulu, Netflix, among many others, I think a small 800mb (or so) update is NOTHING compared to their average bandwidth usage.
( Last edited by phantomdragonz; Sep 20, 2013 at 03:54 AM. )
     
   
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 03:50 PM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,