Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for ZFS pool io stats monitoring #374

Merged
merged 5 commits into from
Jun 29, 2022

Conversation

simplepad
Copy link
Contributor

This pull request adds support for ZFS pool io monitoring, using file /proc/spl/kstat/zfs/*pool_name*/io.

Reads and writes are recorded as bytes, not sectors, so there is no need to multiply it by 512.
Since zfs stat collection is different from block device stat collection, disk has to store filesystem type to determine which collection method should be used.
Also, io_ticks for zfs are recorded separetely for reads and writes, so to get a total I added io_ticks_read and io_ticks_write variables.

@anarcat
Copy link

anarcat commented Jun 27, 2022

this would be great to have. it's rather surprising to not see ZFS-related disk I/O stats in btop. i would understand we don't have it per dataset, but surely it should be registered in the merged view!

also, it would be nice to have pools instead of dataset in the disk usage listing, does this PR change that?

@simplepad
Copy link
Contributor Author

simplepad commented Jun 27, 2022

also, it would be nice to have pools instead of dataset in the disk usage listing, does this PR change that?

No, this PR is focused only on ZFS pool io monitoring, I didn't alter the listings logic in any way.

Copy link
Owner

@aristocratos aristocratos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, other than the requested change, awesome work :)

@aristocratos
Copy link
Owner

@simplepad
Oh right, just noticed, you should use the global Shared::procPath instead of hardcoded /proc/ in the paths, that way systems with custom proc locations could be supported. (Have not implemented an actual search for proc yet, but at least if/when it's added, all paths should already be using the variable).

@simplepad simplepad requested a review from aristocratos June 29, 2022 11:43
Copy link
Owner

@aristocratos aristocratos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome :)
Thanks for contributing!

@aristocratos aristocratos merged commit b0f5eb6 into aristocratos:main Jun 29, 2022
@simplepad simplepad deleted the zfs-pool-io-support branch June 29, 2022 11:58
@reefland
Copy link

I haven't tried the new btop yet, but in reviewing the comments above I don't see the /proc/spl/kstat/zfs/*pool_name*/io directory listed. I checked on my Desktop and Laptop.

$ zfs version
zfs-2.1.5-1
zfs-kmod-2.1.5-1

$ ls -l /proc/spl/kstat/zfs/vault
total 0
-rw-r--r-- 1 root root 0 Jun 30 17:40 dmu_tx_assign
-rw-r--r-- 1 root root 0 Jun 30 17:40 iostats
-rw-r--r-- 1 root root 0 Jun 30 17:40 multihost
-rw-r--r-- 1 root root 0 Jun 30 17:40 objset-0x183
-rw-r--r-- 1 root root 0 Jun 30 17:40 objset-0x195b1
-rw-r--r-- 1 root root 0 Jun 30 17:40 objset-0x36
-rw-r--r-- 1 root root 0 Jun 30 17:40 objset-0x3699
-rw-r--r-- 1 root root 0 Jun 30 17:40 objset-0x86
-rw------- 1 root root 0 Jun 30 17:40 reads
-rw-r--r-- 1 root root 0 Jun 30 17:40 state
-rw-r--r-- 1 root root 0 Jun 30 17:40 txgs

I only see it on my much older Ubuntu ZFS versions:

$ zfs version
zfs-0.8.3-1ubuntu12.14
zfs-kmod-0.8.3-1ubuntu12.14

$ ls -l /proc/spl/kstat/zfs/tank
total 0
-rw-r--r-- 1 root root 0 Jun 30 17:47 dmu_tx_assign
-rw-r--r-- 1 root root 0 Jun 22 22:14 io
-rw-r--r-- 1 root root 0 Jun 30 17:47 iostats
-rw-r--r-- 1 root root 0 Jun 30 17:47 multihost
-rw-r--r-- 1 root root 0 Jun 22 22:14 objset-0x10d
-rw-r--r-- 1 root root 0 Jun 22 22:14 objset-0x36
-rw------- 1 root root 0 Jun 30 17:47 reads
-rw-r--r-- 1 root root 0 Jun 22 22:14 state
-rw-r--r-- 1 root root 0 Jun 30 17:47 txgs

openzfs/zfs@371f88d

@simplepad
Copy link
Contributor Author

simplepad commented Jul 1, 2022

I was testing Debian Bullseye's last version of zfs-utils, and it had this file. Looking at the commit of openzfs you provided, it does look like my implementation is not compatible with the latest versions of zfs. That said, there shouldn't be any errors using the latest btop on your system, as it checks if this file exists before using it.

Also, could you please open an issue to track this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants