FreeBSD13/FreeBSD14 sysctl get zfs zroot read and write info, parseFreeBSDPoolObjsetStats function

[mingjunyang]

Host operating system: FreeBSD myhost 14.0-RELEASE FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716eee8ab4: Fri Nov 10 05:57:23 UTC 2023 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64

node_exporter version: master branch on commit c2dcc79

I have 2 pool in my system

 # zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
datas  43.7T   952K  43.7T        -         -     0%     0%  1.00x    ONLINE  -
zroot   236G  5.07G   231G        -         -     0%     2%  1.00x    ONLINE  -

I run the sysctl, unknown oid.

# sysctl -a 'kstat.zfs.pool.dataset'
sysctl: unknown oid 'kstat.zfs.pool.dataset'

I write the demo, maybe it's should work. I don't know if anyone fix this issues.

package main

import (
	"fmt"
	"golang.org/x/sys/unix"
)

func main() {
	zfsDatasetNames := []string{"datas", "zroot"}

	for zfsRootNameID := range zfsDatasetNames {
		sysCtlMetrics := []string{
			"nunlinked", "nunlinks", "nread", "reads", "nwritten", "writes",
		}
		for sysCtlMetricID := range sysCtlMetrics {
			zfsPoolMibPrefix := fmt.Sprintf("kstat.zfs.%s.dataset.objset-0x36.%s", zfsDatasetNames[zfsRootNameID], sysCtlMetrics[sysCtlMetricID])

			metricData, err := unix.SysctlUint64(zfsPoolMibPrefix)
			if err != nil {
				fmt.Println("couldn't get sysctl:", err)
			}
			fmt.Println(fmt.Sprintf("%s\t%d", zfsPoolMibPrefix, metricData))

		}
	}
}

# go run main.go 
kstat.zfs.datas.dataset.objset-0x36.nunlinked	0
kstat.zfs.datas.dataset.objset-0x36.nunlinks	0
kstat.zfs.datas.dataset.objset-0x36.nread	0
kstat.zfs.datas.dataset.objset-0x36.reads	0
kstat.zfs.datas.dataset.objset-0x36.nwritten	54
kstat.zfs.datas.dataset.objset-0x36.writes	4
kstat.zfs.zroot.dataset.objset-0x36.nunlinked	0
kstat.zfs.zroot.dataset.objset-0x36.nunlinks	0
kstat.zfs.zroot.dataset.objset-0x36.nread	0
kstat.zfs.zroot.dataset.objset-0x36.reads	0
kstat.zfs.zroot.dataset.objset-0x36.nwritten	0
kstat.zfs.zroot.dataset.objset-0x36.writes	0

And , I try build the project. Some error for me.

# go version
go version go1.21.5 freebsd/amd64

 go build
# github.com/prometheus/node_exporter/collector
collector/exec_bsd.go:65:4: undefined: labels
collector/exec_bsd.go:65:12: cannot use nil as bsdSysctl value in array or slice literal
collector/zfs_freebsd.go:338:6: undefined: strings
collector/zfs_freebsd.go:339:46: undefined: strings
collector/zfs_freebsd.go:343:28: undefined: zfsDatasetsNames
collector/zfs_freebsd.go:346:15: undefined: strings
collector/zfs_freebsd.go:350:22: undefined: fmt.SprintF
collector/zfs_freebsd.go:351:22: undefined: fmt.SprintF
collector/zfs_freebsd.go:352:60: undefined: poolObj

Maybe I can fix this issues, and I need a new FreeBSD version build.
So anyone try fix this before me?

[dswarbrick]

The fmt.SprintF is an obvious typo, introduced by #2753, as are the references to the missing import strings package.

TBH, it looks like that PR was blindly submitted without actually verifying that it successfully built on FreeBSD.

[mingjunyang]

I can fix this, I will send the PR.

This command list zfs dataset and objsetid.

# zfs list -oname,objsetid
NAME                                          OBJSETID
zroot                                               54
zroot/ROOT                                         387
zroot/ROOT/13.2-RELEASE-p4_2023-11-21_153555      3503
zroot/ROOT/13.2-RELEASE-p5_2023-11-21_180520      5057
zroot/ROOT/14.0-RELEASE_2023-11-21_181008          275
zroot/ROOT/14.0-RELEASE_2023-11-21_210444          666
zroot/ROOT/default                                 395
zroot/tmp                                           68
zroot/usr                                          643
zroot/usr/home                                     403
zroot/usr/ports                                    260
zroot/usr/src                                      650
zroot/var                                          267
zroot/var/audit                                     75
zroot/var/crash                                    657
zroot/var/log                                      773
zroot/var/mail                                     412
zroot/var/tmp                                      419

My code will import "os/exec" run the zfs command.

cmdOut, err := exec.Command("zfs", "list", "-tfilesystem", "-oname,objsetid").Output()
if err != nil {
	fmt.Println(fmt.Errorf("run cmd err, %w", err))
}
t := string(cmdOut[:])
// parse string and get objsetid and dataset name

And execute the parseFreeBSDPoolObjsetStats here.

func NewZfsCollector(logger log.Logger) (Collector, error) {
	c := zfsCollector{
		sysctls: []bsdSysctl{
		......
		},
		logger: logger,
	}
	c.parseFreeBSDPoolObjsetStats()
	return &c, nil
}
``

[dswarbrick]

My code will import "os/exec" run the zfs command.

That would go against node_exporter policy. See CONTRIBUTING.md:

A Collector may only read /proc or /sys files, use system calls or local sockets to retrieve metrics. It may not require root privileges. Running external commands is not allowed for performance and reliability reasons.

[SuperQ]

I wish we could do BSD builds in CircleCI or GitHub actions. We used to have buildkite, but nobody wanted to maintain the runners.

CC @conallob

[mingjunyang]

I try to list all zfs datasets name and objsetid without zfs list command, but I faild.

Maybe should run a textfile collector or a dedicated Exporter.

I build a local version for myself, use with zfs list command. And my PR comments the parseFreeBSDPoolObjsetStats function. If someone needed, maybe can build local too.

PR #2882

[seeplusplus]

If I understand correctly, this issue blocks further releases on FreeBSD. I've started looking at how we can fix this issue w/o using exec, but I'm new to Go and FreeBSD. I'm going to ope a PR to revert #2753. I don't see any reason not to, since it only adds functionality to FreeBSD and builds on that platform aren't even possible with its inclusion.