libzfs_core: add ZFS_IOC_TRACE envvar to enable ioctl tracing

[robn]

Motivation and Context

Understanding how the command line tools and the kernel module coordinate is tricky to understand sometimes, and while system tracing tools can show you the raw ioctl() calls, they usually have no idea about nvlists.

So, this adds a simple tracing facility that will dump ioctl() calls and returns.

Description

First two commits are some light reorganisation to make lzc_ioctl_fd() a single chokepoint where all calls come through, and which then call any platform-specific code.

Top commit is the meat: if theZFS_IOC_TRACE environment variable is set, then lzc_ioctl_fd() will put info about the call and return onto STDERR, with the source and dest nvlists if applicable.

Note that I'm only dumping nvlist contents, not trying to deal with all of zfs_cmd_t. Not that that wouldn't be useful, it's just not what I needed right now. Easily added later.

Some examples. zfs get receives results in nvlists:

$ env ZFS_IOC_TRACE=1 ./zfs get recordsize crayon/var
=== lzc_ioctl: call: ioc=0x5a12 name=crayon/var
=== lzc_ioctl: result: ioc=0x5a12 name=crayon/var errno=0
nvlist version: 0
	mountpoint = (embedded nvlist)
	nvlist version: 0
		value = /var
		source = crayon/var
	(end mountpoint)

	canmount = (embedded nvlist)
	nvlist version: 0
		value = 0x0
		source = crayon/var
	(end canmount)

...
	useraccounting = (embedded nvlist)
	nvlist version: 0
		value = 0x1
	(end useraccounting)

=== lzc_ioctl: call: ioc=0x5a05 name=crayon
=== lzc_ioctl: result: ioc=0x5a05 name=crayon errno=0
nvlist version: 0
	version = 0x1388
	name = crayon
	state = 0x0
	txg = 0x581489
	pool_guid = 0xeb8f8068290104ad
...
		com.klarasystems:fast_dedup = 0x0
		org.zfsonlinux:longname = 0x0
		com.klarasystems:large_microzap = 0x0
	(end feature_stats)

NAME        PROPERTY    VALUE    SOURCE
crayon/var  recordsize  128K     default
=== lzc_ioctl: call: ioc=0x5a3f name=[none]
nvlist version: 0
	message = zfs get recordsize crayon/var
=== lzc_ioctl: result: ioc=0x5a3f name=[none] errno=1

zpool wait has a request nvlist:

$ env ZFS_IOC_TRACE=1 ./zpool wait crayon
=== lzc_ioctl: call: ioc=0x5a05 name=crayon
=== lzc_ioctl: result: ioc=0x5a05 name=crayon errno=84
=== lzc_ioctl: call: ioc=0x5a53 name=crayon
nvlist version: 0
	wait_activity = 0
=== lzc_ioctl: result: ioc=0x5a53 name=crayon errno=84

Some have no nvlists, but you still get an idea of what's going on:

$ env ZFS_IOC_TRACE=1 ./zpool list crayon
=== lzc_ioctl: call: ioc=0x5a05 name=crayon
=== lzc_ioctl: result: ioc=0x5a05 name=crayon errno=84
=== lzc_ioctl: call: ioc=0x5a27 name=crayon
=== lzc_ioctl: result: ioc=0x5a27 name=crayon errno=84
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
crayon   444G   393G  51.2G        -         -    79%    88%  1.00x    ONLINE  -
=== lzc_ioctl: call: ioc=0x5a3f name=[none]
nvlist version: 0
	message = zpool list crayon
=== lzc_ioctl: result: ioc=0x5a3f name=[none] errno=1

How Has This Been Tested?

By hand, as above, on both Linux and FreeBSD. I'll let the buildbots make sure nothing is broken, but I don't expect anything weird because if the envvar isn't set, it should just be the fast path.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.