Welcome to the MacNN Forums.

besson3c · Apr 5, 2011, 03:09 PM

OpenSolaris Forums : [zfs-discuss] ZFS Going forward after ...

It looks like ZFS is being forked such that maintenance of the file system itself and not just userland stuff might take place away from Oracle. If Oracle opens up ZFS this way, perhaps the license stuff will change too? OSes involved with ZFS now include OpenSolaris, Nexenta, Illumos/OpenIndiana, and FreeBSD.

As you can see in this thread, MacOS representatives are involved in the discussion within Illumos, which might mean something.

nonhuman · Apr 5, 2011, 03:34 PM

The problem I see is that a fork can't change the license, only the license holder (currently Oracle). As long as the code is still derived from code under the CDDL, Apple (and Linux) won't be able to use it. From what I know of Oracle, it seems unlikely that they'll decide to re-license the existing code base out of the goodness of their hearts, especially as they're also the license holders for the GPLed Btrfs, which is their own project rather than acquired code (and is already in the Linux kernel). The only way that I can see OS X getting native (kernel-level) ZFS support at this point is some sort of commercial license from Oracle, which I also very much doubt will happen.

I'm not sure what version of the GPL Btrfs is under, but as long as it's not 3, I could see Apple maybe moving that direction instead.

besson3c · Apr 5, 2011, 03:50 PM

Originally Posted by nonhuman

The problem I see is that a fork can't change the license, only the license holder (currently Oracle). As long as the code is still derived from code under the CDDL, Apple (and Linux) won't be able to use it. From what I know of Oracle, it seems unlikely that they'll decide to re-license the existing code base out of the goodness of their hearts, especially as they're also the license holders for the GPLed Btrfs, which is their own project rather than acquired code (and is already in the Linux kernel). The only way that I can see OS X getting native (kernel-level) ZFS support at this point is some sort of commercial license from Oracle, which I also very much doubt will happen.

I'm not sure what version of the GPL Btrfs is under, but as long as it's not 3, I could see Apple maybe moving that direction instead.

Actually, according to the guys in the Illumos IRC channel, the exact rationale behind Apple dropping ZFS was never disclosed, but the license was NOT the problem since there were only a few proprietary parts of it including encryption that were optional components. The fact that these other projects are making changes to the source code is plenty evidence that the license was not the problem. The only concern is whether Oracle stops releasing source code after Solaris 11 sort of like Apple did with Darwin. If this were the case, the responsibility of maintaining the parts of ZFS that are open would have to be absorbed by one of these orgs.

nonhuman · Apr 5, 2011, 04:25 PM

Originally Posted by besson3c

Actually, according to the guys in the Illumos IRC channel, the exact rationale behind Apple dropping ZFS was never disclosed, but the license was NOT the problem since there were only a few proprietary parts of it including encryption that were optional components. The fact that these other projects are making changes to the source code is plenty evidence that the license was not the problem. The only concern is whether Oracle stops releasing source code after Solaris 11 sort of like Apple did with Darwin. If this were the case, the responsibility of maintaining the parts of ZFS that are open would have to be absorbed by one of these orgs.

The fact that people are making changes to the source code does not mean that the license isn't the problem. The CDDL is an open source license, but that doesn't mean that it has no terms that might conflict with other licenses. The same is true of the GPL, which requires that any project that 'makes use' of GPLed code must also be released under the GPL (which is why it's often described as a 'viral' license). The most recent version of the GPL (v3) made some changes to these terms that have made it unacceptable to many people, and as a result GPLv3 hasn't gained that much traction, and much important software (including, I believe, the Linux kernel) is still using GPLv2. In particular, Apple does not include any GPLv3 software with OS X, and this is at least part of the reason that they're using LLVM now instead of GCC: to avoid the additional encumbrances imposed by the GPLv3 (or CDDL) license.

Cold Warrior · Apr 5, 2011, 04:28 PM

Isn't Ellison best buddies with Steve? Seems like Larry could just sell a perpetual, non-revocable license to Apple under favorable license terms.

P · 05:02 PM

ZFS on the Mac is very much alive, although as a commercial product and probably not as a boot drive.

Btrfs is far from done. It's only "experimental", even though it was supposed to be done by 2008. While I think it's likely that Apple is working on something LIKE Btrfs, I would be very surprised if it were Btrfs itself.

Licensing: Btrfs must be GPLv2 if it's included in the Linux kernel (the kernel is GPLv2 and is going to stay so. There is no mechanism to upgrade it, and Linus Torvalds has made it very clear that he probably wouldn't have upgraded anyway), but including GPLv2 packages with the OS is something quite different from including them in the kernel. Including them in the kernel would force a GPL license of the entire kernel, which I don't think Apple wants.

The story as I remember it was that Apple wanted to use ZFS, but needed it licensed under a more permissive license than GPLv2. Sun was working on it, but the negotiations fell through. I actually think they fell through as the Oracle acquisition was being negotiated, so I don't think Ellison is going to give it to Apple for free.

Apple seems to be avoiding GPLv3, if the recent rumors of moving away from Samba are true.

besson3c · 05:24 PM

Originally Posted by nonhuman

The fact that people are making changes to the source code does not mean that the license isn't the problem. The CDDL is an open source license, but that doesn't mean that it has no terms that might conflict with other licenses. The same is true of the GPL, which requires that any project that 'makes use' of GPLed code must also be released under the GPL (which is why it's often described as a 'viral' license). The most recent version of the GPL (v3) made some changes to these terms that have made it unacceptable to many people, and as a result GPLv3 hasn't gained that much traction, and much important software (including, I believe, the Linux kernel) is still using GPLv2. In particular, Apple does not include any GPLv3 software with OS X, and this is at least part of the reason that they're using LLVM now instead of GCC: to avoid the additional encumbrances imposed by the GPLv3 (or CDDL) license.

What exactly are the problems then that cause conflicts? The FreeBSD team has been cherry picking pieces of ZFS for its OS, getting it to work with their kernel, but they in theory could write their own proprietary stuff on top of ZFS just as Apple could without having to give back like you would with a GPL license. AFAIK ZFS file system version updates are modular and can be applied without affecting the underlying file system, which is why all of these different vendors run different versions of ZFS and why you can apply these updates to your current running file system once they are available:

Code:
# zpool upgrade -v
This system is currently running ZFS pool version 15.

The following versions are supported:

VER  DESCRIPTION
---  --------------------------------------------------------
 1   Initial ZFS version
 2   Ditto blocks (replicated metadata)
 3   Hot spares and double parity RAID-Z
 4   zpool history
 5   Compression using the gzip algorithm
 6   bootfs pool property
 7   Separate intent log devices
 8   Delegated administration
 9   refquota and refreservation properties
 10  Cache devices
 11  Improved scrub performance
 12  Snapshot properties
 13  snapused property
 14  passthrough-x aclinherit
 15  user/group space accounting
For more information on a particular version, including supported releases, see:

http://www.opensolaris.org/os/community/zfs/version/N

Where 'N' is the version number.

I see no reason why Apple couldn't make their encryption/crypto pieces or any other stuff they want to add to ZFS proprietary if they wanted to just as Oracle did and just as other vendors such as Nexenta might as well if not so already. Besides, even just using the open parts of ZFS as-is without altering them, it is a vast improvement over HFS+, and I don't foresee Apple wanting to make frequent file system changes even if they would be prevented from doing so (and I don't see how this would be the case).

This really remains a mystery, but I'm starting to see that simplifying this as a case of licensing is not really accurate. The Illumos guys which are comprised of OpenSolaris, Nexenta, OpenIndiana, and FreeBSD guys can't explain Apple's decision, this was certainly never made public.

besson3c · Apr 5, 2011, 05:22 PM

Originally Posted by P

ZFS on the Mac is very much alive, although as a commercial product and probably not as a boot drive.

Btrfs is far from done. It's only "experimental", even though it was supposed to be done by 2008. While I think it's likely that Apple is working on something LIKE Btrfs, I would be very surprised if it were Btrfs itself.

Licensing: Btrfs must be GPLv2 if it's included in the Linux kernel (the kernel is GPLv2 and is going to stay so. There is no mechanism to upgrade it, and Linus Torvalds has made it very clear that he probably wouldn't have upgraded anyway), but including GPLv2 packages with the OS is something quite different from including them in the kernel. Including them in the kernel would force a GPL license of the entire kernel, which I don't think Apple wants.

The story as I remember it was that Apple wanted to use ZFS, but needed it licensed under a more permissive license than GPLv2. Sun was working on it, but the negotiations fell through. I actually think they fell through as the Oracle acquisition was being negotiated, so I don't think Ellison is going to give it to Apple for free.

Apple seems to be avoiding GPLv3, if the recent rumors of moving away from Samba are true.

Very interesting, I wasn't aware of this product.

I wonder if Apple pulled the plug while they were concerned over the future of Oracle/Sun and how all of this would shake down. I can understand a GPL license being incompatible, but what little I understand about the CDDL I don't see the problem with it. Apparently neither does Nexenta, which is a commercial company that makes ZFS storage appliances and also contributes to ZFS.

Not only is btrfs a ways off, but ZFS has grown for so long that anything else will have some *massive* catch up to do not only in stability, but features as well.

ZFS is really something quite special.

nonhuman · Apr 5, 2011, 05:33 PM

Originally Posted by besson3c

Very interesting, I wasn't aware of this product.

It does look cool, I can't wait to see what they release.

I wonder if Apple pulled the plug while they were concerned over the future of Oracle/Sun and how all of this would shake down. I can understand a GPL license being incompatible, but what little I understand about the CDDL I don't see the problem with it. Apparently neither does Nexenta, which is a commercial company that makes ZFS storage appliances and also contributes to ZFS.

Nexenta is using the OpenSolaris kernel which is also released under the CDDL. FreeBSD is using the BSD license, which is completely unrestrictive and so they can don't need to worry about incorporating other licenses into their code base as much (though they can't use GPL code as then they'd need to GPL everything)*.

In neither of those cases is anyone trying to incorporate code released under a somewhat restrictive open source license into a proprietary kernel.

Not only is btrfs a ways off, but ZFS has grown for so long that anything else will have some *massive* catch up to do not only in stability, but features as well.

Btrfs is not a ways off. Install the latest Ubuntu or Fedora today and you can be running Btrfs. I was running Btrfs several years ago without a problem. The only big restriction right now is that GRUB doesn't support it so you generally will still need an ext* partition to boot from, but this wouldn't be a problem for Apple, as they'd need to modify their own boot loader regardless of what new FS they went to. While Btrfs doesn't have feature parity with ZFS yet, the features that are missing are not ones that Apple customers are really going to care about or even notice. And they could simply be rolled out in future versions of OS X.

ZFS is really something quite special.

Agreed, but I don't know that it's necessarily so special that it's irreplaceable.

* I don't actually know the details of what FreeBSD is doing to make use of the ZFS code, I may be completely off. But the point is FreeBSD has very different licensing requirements from Nexenta or Apple when it comes to their kernel.

besson3c · Apr 5, 2011, 05:54 PM

Originally Posted by nonhuman

Nexenta is using the OpenSolaris kernel which is also released under the CDDL. FreeBSD is using the BSD license, which is completely unrestrictive and so they can don't need to worry about incorporating other licenses into their code base as much (though they can't use GPL code as then they'd need to GPL everything)*.

In neither of those cases is anyone trying to incorporate code released under a somewhat restrictive open source license into a proprietary kernel.

I don't understand your point. What does the license of the kernel have to do with this? AFAIK as long as you bring your own kernel to the party you can do whatever you want with ZFS, no?

Btrfs is not a ways off. Install the latest Ubuntu or Fedora today and you can be running Btrfs. I was running Btrfs several years ago without a problem. The only big restriction right now is that GRUB doesn't support it so you generally will still need an ext* partition to boot from, but this wouldn't be a problem for Apple, as they'd need to modify their own boot loader regardless of what new FS they went to. While Btrfs doesn't have feature parity with ZFS yet, the features that are missing are not ones that Apple customers are really going to care about or even notice. And they could simply be rolled out in future versions of OS X.

BTRFS is a ways off from being considered safe and ready for mass consumption... This part of development can take a very long time when you're talking about a file system. AFAIK there is more to making a file system like BTRFS bootable than simply adding bootloader support.

Agreed, but I don't know that it's necessarily so special that it's irreplaceable.

I don't know about irreplaceable, but it is miles ahead of HFS+, ext4, and NTFS...

mduell · Apr 5, 2011, 08:28 PM

I don't see the license as a huge deal. Apple isn't some penniless open source project run by two dudes, a cow, a chicken, and the internet. They can negotiate a commercial license with Oracle.

besson3c · Apr 5, 2011, 08:33 PM

Originally Posted by mduell

I don't see the license as a huge deal. Apple isn't some penniless open source project run by two dudes, a cow, a chicken, and the internet. They can negotiate a commercial license with Oracle.

Yeah, and I'm sure if they show up with their money truck that license would look like whatever they wanted it to

P · Apr 6, 2011, 04:27 AM

Originally Posted by mduell

I don't see the license as a huge deal. Apple isn't some penniless open source project run by two dudes, a cow, a chicken, and the internet. They can negotiate a commercial license with Oracle.

As I understood it, that was what they were working on with Sun, and the negotiations fell through. Sun was CTD at the time, they may have asked for 5 billion and a pony or something.

Originally Posted by nonhuman

Btrfs is not a ways off. Install the latest Ubuntu or Fedora today and you can be running Btrfs. I was running Btrfs several years ago without a problem. The only big restriction right now is that GRUB doesn't support it so you generally will still need an ext* partition to boot from...

You can now root and boot from it as well - it was announced here. Notice the bit in capitals? Not done yet - and there's still the GPL thing. I still thing that an HFS3 is more likely.

nonhuman · Apr 6, 2011, 09:14 AM

Originally Posted by besson3c

I don't understand your point. What does the license of the kernel have to do with this? AFAIK as long as you bring your own kernel to the party you can do whatever you want with ZFS, no?

The license for the kernel is relevant because native support, in a non-microkernel system (which OS X is, despite the Mach heritage), requires the filesystem to be integrated directly into the kernel. If it's not, you can't boot from that filesystem because the kernel doesn't know how to mount it or read data from it. However if you include GPLed code in your kernel, then your kernel needs to be GPLed or you're in violation of the license.

nonhuman · Apr 6, 2011, 09:17 AM

Originally Posted by P

You can now root and boot from it as well - it was announced here. Notice the bit in capitals? Not done yet - and there's still the GPL thing. I still thing that an HFS3 is more likely.

Actually, no:

Originally Posted by Colin Watson

You cannot yet use btrfs for /boot (although we're working on this), so you'll need to create an ext3 or similar /boot filesystem. I assume anyone able to use an experimental filesystem can cope with doing that in the installer's manual partitioner.

mattyb · Apr 6, 2011, 09:24 AM

Originally Posted by nonhuman

From what I know of Oracle, it seems unlikely that they'll decide to re-license the existing code base out of the goodness of their hearts

There is no goodness in any Oracle heart.

besson3c · Apr 6, 2011, 11:22 AM

Originally Posted by nonhuman

The license for the kernel is relevant because native support, in a non-microkernel system (which OS X is, despite the Mach heritage), requires the filesystem to be integrated directly into the kernel. If it's not, you can't boot from that filesystem because the kernel doesn't know how to mount it or read data from it. However if you include GPLed code in your kernel, then your kernel needs to be GPLed or you're in violation of the license.

Yeah, but ZFS is not GPL, I'm confused why the GPL keeps getting mentioned.

P · Apr 6, 2011, 02:52 PM

Originally Posted by besson3c

Yeah, but ZFS is not GPL, I'm confused why the GPL keeps getting mentioned.

You're right - and CDDL is even incompatible with GPL. For some reason I had it in my head that ZFS was licensed under GPL.

besson3c · Apr 6, 2011, 03:42 PM

Originally Posted by P

You're right - and CDDL is even incompatible with GPL. For some reason I had it in my head that ZFS was licensed under GPL.

It is btrfs which is GPL, which might also mean that it has even less of a chance of being adopted in OS X than ZFS.

My understanding of CDDL and ZFS is that while ZFS support definitely requires kernel integration (including in the FreeBSD kernels, there are kernel build options for ZFS), you can use ZFS with whatever kernel you want, and you can use the open parts of ZFS and the OpenSolaris kernel as you wish too. It is my understanding that the only parts of ZFS/Solaris that are not open relate to crypto/encryption and are pieces that are optional.

I see no reason why Apple can't take the open parts of ZFS and use them as is or at least build off of them. I suspect that what bothered Apple was the onerous task of taking responsibility for this code if ZFS was going to fold due to corporate business decisions. As it stands, there is cloudiness over the future of ZFS, but this cloudiness now is which organizations outside of Oracle will be involved and how. If the Illumos group (of which Apple is involved with) gains traction and involvement it won't matter what Oracle does to OpenSolaris, development will live on.

I'd like to think that Apple is very interested in ZFS... It helps them when it comes to snapshots and Time Machine, but also it allows them to build machines with a relatively small fast SSD read/write cache and cheap storage underneath for storage, which means they can sell low end computers and iOS devices with a lot of storage but without the I/O problems of HFS+ on a single SATA disk (which sucks).

nonhuman · Apr 6, 2011, 04:27 PM

Originally Posted by besson3c

Yeah, but ZFS is not GPL, I'm confused why the GPL keeps getting mentioned.

To illustrate the concept of why the license might be an issue...

I've already stated that I'm not very familiar with the CDDL, so I used the GPL to illustrate my point that the licensing of the file system can be important in terms of being able to use it in your operating system. And while the CDDL license may allow ZFS to be used in various kernels, it's entirely possible, that there are still provisions in it which Apple would find more onerous than an open source OS.

besson3c · Apr 6, 2011, 04:30 PM

Originally Posted by nonhuman

To illustrate the concept of why the license might be an issue...

I've already stated that I'm not very familiar with the CDDL, so I used the GPL to illustrate my point that the licensing of the file system can be important in terms of being able to use it in your operating system. And while the CDDL license may allow ZFS to be used in various kernels, it's entirely possible, that there are still provisions in it which Apple would find more onerous than an open source OS.

Ahhh, I see. Well, it certainly can't be disputed that licenses *can* be relevant.

I'll point out though that the commercial version of Nexenta which uses ZFS is not open source.

P · Apr 7, 2011, 03:53 AM

Originally Posted by besson3c

I'll point out though that the commercial version of Nexenta which uses ZFS is not open source.

If I understand the CDDL, you are free to combine CDDL code with proprietary code, as long as the CDDL code (along with any modifications you make to it) are licensed under the CDDL - that is, open source. If Nexenta modifies the ZFS code, they have to open it, but they can use the existing code together with other code that they write without sharing it.

besson3c · Apr 7, 2011, 04:21 AM

Originally Posted by P

If I understand the CDDL, you are free to combine CDDL code with proprietary code, as long as the CDDL code (along with any modifications you make to it) are licensed under the CDDL - that is, open source. If Nexenta modifies the ZFS code, they have to open it, but they can use the existing code together with other code that they write without sharing it.

But that doesn't make a whole lot of sense in terms of providing an explanation as to Apple's decision in the context of licensing, I don't think... I mean, one could write code that overrides existing open functions to skirt around not making fixes/modifications to the open source code available, right?

In the case of ZFS, file system revisions are installable patches/modules on top of the existing codebase that can be applied to a live file system and without even having to restart the machine (I believe). So, point being, there are a number of ways that Apple could be protective of any additions or modifications they make to the ZFS codebase if they wanted to.

Of course, I cannot fathom a possible reason why Apple would want to be protective of code fixes. New features, sure, but it would be in Apple's best interest to share fixes so that they are committed to the main code repository, audited, etc. I've yet to hear of Apple making secret fixes to the other open source parts of OS X... Like I said, I see nothing in the license that would prevent Apple from adding new features, coming up with their own file system revisions/version upgrades/whatever...

I remain convinced that Apple's decisions had nothing to do with licensing.