pyzfs / libzfs_core.c has no way to list filesystems or snapshots.

[georgyo]

Was about to write some python scripts to negotiate which snapshots needed to be sent to the remote side to replace some hacky and ugly bash / awk scripts.

pyzfs is listed as a prominent new supported feature in 0.8.0, and this seemed like a great way to go about it rather than parsing the commands. So the very first thing I did was read the docs and find libzfs_core.lzc_list_snaps(name)) which is a the first thing one would want to do when deciding to figure out which snapshots the local and remote have.

However, that just leads to a very non-intuitive error

>>> libzfs_core.lzc_list_snaps(b"zroot")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/site-packages/libzfs_core/_libzfs_core.py", line 81, in _f
    raise NotImplementedError(func.__name__)
NotImplementedError: lzc_list_snaps

I spend the next hour looking for code examples of anyone using pyzfs, to no avail....

Then I start reading the pyzfs source code and discover the _uncommitted decorator. So even though the python code is committed and clearly shows in the docs, it might not work.

Going further down the rabbit hole, I start reading the libzfs_core.c source code and indeed lzc_list_* is not defined.

Looking at all the functions in libzfs_core, I don't see a single way to list snapshots, filesystems, or volumes. Though I can get bookmarks and holds. Though getting bookmarks doesn't list regularly snapshots.

To create bookmarks, send, receive or many other things I need to know snapshot name. As a result I need to resort to parsing the output of the commands again, making it unclear what I would do with libzfs_core or pyzfs.

Issue #7230 is about making pyzfs useful for the 0.8.0 release but all the list functions are missing from there too.

And Issue #3907 has the original adding of the list code however it is far out of date.

This ticket is mostly to help other people know they are not crazy. The following functions DO NOT WORK in pyzfs in any version of zfs at this time of writing.

lzc_list
lzc_list_children
lzc_list_snaps
lzc_inherit
lzc_set_props
lzc_get_props

[ahrens]

I'm not sure why pyzfs is documenting/exporting functions that don't exist.

[richardelling]

AIUI, pyzfs was WIP when ClusterHQ closed.

See also #8956

[georgyo]

@behlendorf I would say this ticket is more then just incorrect documentation. Both pyzfs and libzfs_core is missing features that are required for it to be useful. As a result, nothing can be built on top of them that doesn't resort to parsing the command line tools.

#8956 does seem to be a much more useful library. It also seems to be missing the ability to list file systems and snapshots, however the libzfs does have have this ability.

[behlendorf]

I agree, and I didn't mean to imply this was solely an issue of documentation. We probably shouldn't have mentioned pyzfs in the release notes until it was a little further along. Some more work does need to be done to make pyzfs generally useful. What I can do is shed some a little light on why that functionality is missing.

The pyzfs library was designed to provide one-on-one wrappers for the stable libzfs_core.so API functions. Which it does. However, some important functionality still only exists in libzfs.so, and hasn't been refactored to depend on the new proposed libzfs_core.so interfaces. Which is unfortunately why those pyzfs wrappers aren't yet functional.

Let's go ahead and use this issue to track the that pending work. Any help wrapping this up would be appreciated. @loli10K handled the pyzfs porting and I'm sure has some thoughts on this as well.

[loli10K]

When pyzfs was integrated the main objectives were:

1. Provide users with an easy way to interact with ZFS core functionality (libzfs_core): i find Python easy to use (easier than compiling a C program anyway) and pyzfs was already available thanks to the good work done at ClusterHQ.

2. Verify the ABI to prevent breaking changes.

I think we did good integrating pyzfs into the main codebase: i personally find it quite useful to build my own software. Of course contributions to add missing functionality and improve the code are most welcome.

[andoriyu]

PyZFS aside, it would be nice for libzfs_core to have those functions...since libzfs is meant to be private.

lzc_exists not working with bookmarks is also a little weird.

[zegelin]

+1 for adding support for snapshots and other missing features to libzfs_core and by extension enabling the functionality in pyzfs.

Similar to @georgyo, I was hoping to write some Python scripts to do some experiments with snapshots, and went down my own rabbit-hole of discovering why lzc_list_snaps throws a NotImplementedError... And I eventually stumbled upon this ticket.

[andi8086]

+1 for making libzfs_core a useful C API. I also found several issues with it. For example the header is not self-contained and it took me 15 minutes to find out where it takes boolean_t from and how to include the two sys/time.h variants.. The header should include the headers it uses!

It happens I found a funny way to list snapshots using libzfs_core, namely bu utilizing lzc_channel_program. Didn't explicitely try it, but since the LUA script itself workd, it should also work with lzc_channel_program. This is cumbersome however :)

[je-marshall]

+1 for this - @andi8086, what do you have to include for boolean_t, I'm having the exact same problem! Can probably track it down myself but if you've already done the digging it would be good to know.

[georgyo]

Bumping this ticket, since libzfs_core is still missing features that are still actively documented in pyzfs documentation.

But with zfs-2.3.0 bring json output to many commands, maybe we can finally get to a point of no longer need to manually parse the output of zfs commands in scripts.