Missing documentation on how to build kernel with GPL exports re-enabled before building ZFS

[jittygitty]

I realize this is not a traditional "bug" but considering that the zfs "workarounds" (especially in regards SIMD fpu etc) that were necessary to overcome the recent removal of certain GPL exports sometimes had side effects that could be characterized as 'bugs' or regression of sorts, I think "many" or most building zfs from source would like to simply re-enable all the previously available GPL exports and have zfs use that method instead of workarounds.

So this request is to have clear instructions in official documentation on building ZFS on how a user can modify the kernel (perhaps even script provided by ZFS to do so) to re-enable all those GPL exports that "used to" exist.

If this already exists somewhere, I'd appreciate if someone would be so kind to simply point me to the link. thx!

[PrivatePuffin]

It might also be handy to document how to re-enable those and any potential side effects both from reenabling and the "workarrounds"... Just to enable people to make an informed decision, besides just politics/preference.

[stale]

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

[PrivatePuffin]

NotStale

[jittygitty]

Can someone well versed in Linux kernel building and modules please help with the following questions:

If one were to decide build ZFS by undoing any "EXPORT_SYMBOL_GPL" limitations, which of the following would be needed:

A. In KERNEL HEADERS ONLY: Simply do perl/sed etc search/replace to change EXPORT_SYMBOL_GPL(/EXPORT_SYMBOL(
B. Change ZFS MODULES and have them report that they are GPL modules. (How/where can I do that?)

Basically I would like to first accomplish this without a complete custom rebuild of the Linux Kernel, so modifying either the source files of ZFS itself and/or the Linux kernel HEADERS, but not the linux kernel or modules themselves.

To prevent needless posts, I should probably inform those that may be a bit confused about licensing issues, that this is about a local "END USER" and not anything to do with any sort of "distribution", so the above are fully within anyone's rights to do as they please on their own system. So please do not post anything except technical info advice, no license chatter here, thanks.

Basically I think I may want to do this myself in the near future, but I didn't know which of the above (A) or (B) would suffice.

Also, once wherever necessary EXPORT_SYMBOL_GPL has been turned to just EXPORT_SYMBOL, or if ZFS module has been made to declare itself as "GPL", will ZFS then build differently without any further configuration changes so that it stops using workarounds that have been implemented for fpu SIMD or other etc?

Can we get a list of all the features that will "forego the workarounds", and use linux native kernel exposed API methods?

For example I believe I read that at least for Crypto and checksums that nothing needs to be done and that ZFS will auto detect during building to see if the native Kernel APIs are available to it for use so it can skip the workarounds to export_symbol_gpl.

But would be good to have this verified and have a list of things that can auto-detect in this way and build the way we want.

Another question I have is how about the Linux page cache instead of relying on ARC? Is there a path to use Linux page cache?

In the end I think we should have a "script" that allows end users who so choose, to build zfs by either making zfs module declare itself as GPL or modifying kernel headers if that suffices to change export_symbol_gpl to export_symbol and then make sure the rest of ZFS is "aware" of these changes and bypasses any workarounds to enable full use of "all" kernel/modules APIs.

Of course I understand it may be prudent to keep this ability on the low so to speak so as to not "unnecessarily upset some". But it should be something I think that should be provided via external script if not included in zfs tree for sake of politeness.

[jittygitty]

If Not Distributing anything. What is the best way to set "ZFS_META_LICENSE" to "GPL" and make sure it propagates everywhere? Of course the point is that after building/compiling, all the modules will be claim "GPL" and will have full access to any of the Linux Kernel's "Export_Symbol_GPL" system calls. So what files would I need to edit to just declare all as "GPL"?

Are any of the License Declarations used from inside [module/os/linux/zfs/zfs_ioctl_os.c] ?

Yes, I could just do a recursive: perl -p -i -e "s/EXPORT_SYMBOL_GPL\(/EXPORT_SYMBOL(/;g" on entire Linux Kernel instead.
But I would prefer to just have all the zfs related built binaries to identify as GPL, so can anyone list the zfs files I'd need to edit?

[behlendorf]

The license for the project is specified in the top level META file as CDDL. Updating this file and rerunning ./autogen.sh && ./configure will result in configure using the specified license when checking if a symbol is available for OpenZFS to use.

[jittygitty]

Thanks! To @behlendorf for the info and to @sempervictus for the search/replace script he posted in his comment at #13397

grep -r CDDL include/zfs/|grep -v '\*'|grep -v bsd|cut -d':' -f1|while read FL ; do 
  sed -i 's|ZFS_META_LICENSE = CDDL|ZFS_META_LICENSE = GPL|; s|#define ZFS_META_LICENSE "CDDL"|#define ZFS_META_LICENSE "GPL"|' $FL
done

[jittygitty]

@behlendorf Is the code to autodetect if fpu SIMD symbol exports are still available, and to use those instead of the workarounds, is that still in the official stable releases as well as current master? I guess I wanted to make sure that the code won't just find fpu SIMD with "export_symbol_gpl" and build via the "workarounds" instead, not knowing zfs modules were changed to qualify. thx!

Update: @behlendorf Ok sorry, it seems you were saying autogen and configure check license as well so it should work fine. thx

[behlendorf]

@jittygitty that's right. If the symbols are provided by the kernel under a compatible license then they'll be used. Otherwise they won't. As a project we've always done our best to live within whatever limitations the kernel may prescribe. Occasionally, this does mean we need to implement our own version of some bit of functionality. This can be a fair bit of work, but in practice I've found this tends to make the OpenZFS code better and more portable. Not coupling the code too tightly to any specific platform is in fact quite a nice feature.

[jittygitty]

@behlendorf Sounds good. Fyi some want to make basically "all" exports gpl-only, and it may happen. If you're able to discuss bit further by email or similar please send me contact info. I will post a couple questions for you in the FIEMAP thread as well. Thanks!

[zfsbot]

the motives of posts like this are always suspicious, and your enthusiasm reminds me of being scammed.

[jittygitty]

@zfsbot Bots also remind me of being scammed ; ) But jokes aside why must you think I have some nefarious evil plans/motives?

[sempervictus]

@zfsbot - this sort of hackery is fine for build products which do not get redistributed. There are various "things" out there which can be built against Linux but not redistributed after for various reasons (often because the act of redist triggers GPL insanity). Proprietary binary drivers are a good example using DKMS to avoid the redist issue. Other patchsets not intended for redist have done similar things as well. The whole notion of GPL-only exports is grossly exclusionary, esp given that its done under the guise of protecting software freedom. Now if @jittygitty was trying to use the hidden VFS functions of ZFS for rootkit-like mechanisms... that would be suspicious. Openly discussing how to get around GPL's thick cloud of misguided crack-smoke is just part of the Open Source process.

[jittygitty]

@sempervictus Thanks, yea and I guess if I wanted to do a rootkit I'd be a lot smarter with C which I'm not : D and I can't remember an evil virus maker trying to convince someone to change code that he could easily change himself without asking :)

Now if the kernel Devs (Linus is much more level-headed but he's stuck with some fellows that went too extreme) would try to be more "just" and less hypocritical, they would "do away" with Export_Symbol_GPL and use Export_Symbol_COPYLEFT so that they recognize their original intent which is that they want to avoid closed-source binary blobs and have modified code be released under open source. This would have been a much more civil and just way to proceed versus the current license fiasco.

Very strong arguments can be made that OpenZFS can definitely qualify as a "non-derivative" work based on Linus' own past well archived remarks. So it is very hypocritical to treat OpenZFS or Ubuntu which complies with basically the very "intent" and reasons that Linus went with the GPL in 1992, to treat them just as bad, or actually "worse" than CLOSED-Source Binary BLOBS.
(And by worse I mean sometimes things got into the kernel which actually "closed source" drivers needed or benefited from, imagine if OpenZFS asked the kernel devs to put some new code (ie compatible with GPL) into the kernel to 'help' zfs module?)

If you read some of the email threads among the Kernel Devs, some are even proposing to somehow code into the kernel some DRM digital rights management (which kernel devs and the FSF (creators of GPLv2) have decried for years as horrible), so that they could somehow try to ENFORCE the GPL-Only "Exports" if some module tried to use a shim or other method to use indirectly.
They have been also contemplating trying to make "ALL" Exports to be GPL-only, to "change" all remaining ones to GPL-only.

I thought they were "champions" of copyleft "Open Source" and the spirit of "freedom", if that were the case why would they not "differentiate" between "CLOSED SOURCE" and try to punish other "copyleft" licenses of coders like them who wanted to protect the SAME freedoms for their own users but simply chose a "different" but still COPYLEFT license? Are they champions of some "magic" letters, ie G and P and L? Is there something special about those letters and no other copyleft license is worthy?
Even the FSF couldn't make their own GPLv3 to be "compatible" with the GPLv2 they created, should GPLv3 then be treated with just as much disdain as CLOSED-SOURCE because of that? (sorry I should of put this in #13415 instead of here I guess.)

[jittygitty]

Question: Would someone be willing to do a PULL-REQUEST to Linux Kernel repo with a patch of: EXPORT_SYMBOL_GPL to EXPORT_SYMBOL_COPYLEFT? Would be interesting to see the response from "everyone" including Kernel devs and Linus himself :)

[zfsbot]

@zfsbot - this sort of hackery is fine for build products which do not get redistributed. There are various "things" out there which can be built against Linux but not redistributed after for various reasons (often because the act of redist triggers GPL insanity). Proprietary binary drivers are a good example using DKMS to avoid the redist issue. Other patchsets not intended for redist have done similar things as well. The whole notion of GPL-only exports is grossly exclusionary, esp given that its done under the guise of protecting software freedom. Now if @jittygitty was trying to use the hidden VFS functions of ZFS for rootkit-like mechanisms... that would be suspicious. Openly discussing how to get around GPL's thick cloud of misguided crack-smoke is just part of the Open Source process.

maybe just don't use the GPL kernel then? I don't get it. you're benefitting from others' hard work while simultaneously insulting their intentions.

just use FreeBSD if you think the Linux developers are "smoking crack".

end users can do whatever they want, the project doesn't need to tell them that any more than we do in the license text.

[zfsbot]

Question: Would someone be willing to do a PULL-REQUEST to Linux Kernel repo with a patch of: EXPORT_SYMBOL_GPL to EXPORT_SYMBOL_COPYLEFT? Would be interesting to see the response from "everyone" including Kernel devs and Linus himself :)

please... this has been discussed to death. the OpenZFS project (@pcd1193182 ) has requested people not go to the LKML or do anything even close to what you've suggested.

raising the ire of the upstream kernel developers is not a good idea. it has not worked out well for anyone so far.

[behlendorf]

I'm closing this issue as the original question has been answered above. On behalf of the project I'd also request that people not take this to LKML.