Disable normalization implicitly when setting "utf8only=off".

[chungy]

Motivation and Context

When creating a new zfs file system and -o utf8only=off is passed as a parameter, the option could silently be overridden and switched to the on state due to the parent dataset having a normalization value != none. This is counter-intuitive and potentially harmful as it may go unnoticed until a non-UTF8 conforming file name is attempted to be created, at which moment the normal error about an invalid UTF8 sequence will be returned.

The zfsprops(8) manpage for utf8only has language already that might imply this: "If this property is explicitly set to off, the normalization property must either not be explicitly set or be set to none."

I describe the issue in #11892 with an example sequence of commands that triggers the bad behavior.

Description

lib/libzfs/libzfs_dataset.c already has code that detects when the normalization property is set and switches utf8only to on. Added another conditional to set normalization to none when utf8only is set to off.

How Has This Been Tested?

Compiled a version of ZFS with this change, ran my sample commands as I presented in the open issue, and checked the utf8only and normalization properties to make sure they are set as appropriate.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[chungy]

Has there been any progress on looking after this PR?

[behlendorf]

Thanks for reminding us about this and the clear explanation of the problem. I agree, this behavior is pretty unexpected and we should be honoring the explicit override. If you can 1) force rebase this PR against latest master branch, and 2) add your test case to the test suite, then we should be able to get this merged.

You'll want to add it to the tests/zfs-tests/tests/functional/casenorm/ directory, and make sure to include the new test script to the Makefile.am. You should be able to model the test after some of the existing test scripts, but let me know if you need a hand.

[chungy]

I've written a test case where I thought appropriate and ran it with both my code change and without, seems to work without issue.

[behlendorf]

Thank you, this looks good to me.