RFE/RFC: hd-idle extensibility: add option for a state file

[rct]

[This is really meant to be a discussion but that's not enabled on this repo.]

In thinking about what hd-idle currently does by processing /proc/diskstats and the limitation that it might not be able to detect disks that get spun up but don't actually do any reads or writes, I'm wondering about the possibilities of extending hd-idle:

A separate script could periodically check the state of a drive smarctl -n standby or other appropriate command, but it would need to know if a drive is supposed to be idle.

Instead of re-implementing the logic of hd-idle, it would be useful if it could get that state from hd-idle. The current logfile doesn't have enough info to be useful in this regard:

date: 2024-07-16, time: 00:13:02, disk: sda, running: 900, stopped: 43144
date: 2024-07-16, time: 01:06:05, disk: sda, running: 1080, stopped: 2102
date: 2024-07-16, time: 02:31:09, disk: sda, running: 960, stopped: 4143

However, hd-idle provides the state information when in debug mode:

disk=sda command=ata spunDown=true reads=1241608102 writes=645717976 idleTime=900 idleDuration=32466 spindown=2024-07-16T03:25:09 spinup=2024-07-16T02:31:09 lastIO=2024-07-16T03:10:09
disk=sdb command=ata spunDown=true reads=26937452 writes=607443752 idleTime=900 idleDuration=194140 spindown=2024-07-14T06:31:35 spinup=2024-07-14T06:05:34 lastIO=2024-07-14T06:16:35

So my current thought is adding a state file option to hd-idle, where the above data could be periodically written for use by other scripts and utilities. Just like logging, the user would have to take care not to spin up a disk every time the state file gets written.

I think this might be more general then other options like adding additional timers/command options to hd-idle to run a script to verify if drives are spun down

[firasdib]

I was in need of something exactly like this, and found this issue.

Is there any plan of adding something like this @adelolmo ? It would probably be even better to add it to the executable itself, hd-idle --state sda to get the state.

[adelolmo]

To be honest, I've been very reluctant to add any functionality that deviates from the original intention: put disks on standby.
Maintaining this piece of code has a very low priority for me and I don't see that improving in the future.
That said, getting the state of a disk shouldn't be a big task to implement.
@firasdib How do you imaging it working? Should it also return the state from all disks if no extra argument is given? Should it return plain text/json/yaml?

[adelolmo]

@rct Out of curiosity, what would be your usecase? I'm trying to understand how this change would be used.

[firasdib]

@adelolmo That makes perfect sense, it's a simple utility, but I believe this change remains true to the core purpose of the software. I primarily envision this being used for machine parsing, so any predefined parseable format is fine, for example JSON.

How it should work would depend on how expensive this operation is. If it's a cheap operation, then it would probably make sense to have the executable simply dump the data to stdout, and then the user can do what they want with it (write to disk, read as is, etc). If it's expensive, then having it write periodically to disk would probably be a better idea, and then consumers can read that file as they see fit.

Obviously this assumes the user has not configured the state file to be written to a disk hd-idle is supposed to put to standby.

For me, I want to continuously query the SMART data of my drives, without disturbing standby or deep EPC states. This utility already knows when drives are idle, and I can use that "for free".

[firasdib]

I took the liberty of simply adding a statefile, using the debug output format, see fork: master...firasdib:hd-idle:master

No clue if this is the best way to do it, but I'm not a wiz with Go, but I gave it a shot.

[Morikko]

Instead of re-implementing the logic of hd-idle, it would be useful if it could get that state from hd-idle. The current logfile doesn't have enough info to be useful in this regard

That's correct that the logfile is more statistics because printed after a full cycle of running-stopped-wakeup.

However, could not you use stdout logs:

Aug  8 00:14:55 enterprise hd-idle[9958]: sda spindown
Aug  8 00:14:55 enterprise hd-idle[9958]: sdb spindown
Aug  8 00:14:56 enterprise hd-idle[9958]: sdc spindown
Aug  8 00:17:55 enterprise hd-idle[9958]: sdb spinup
Aug  8 00:28:55 enterprise hd-idle[9958]: sdb spindown

Probably, the option -l is counter intuitive. I would rather expect all logs to be on stdout by default and when -l would be used that all logs are sent to the file.

[rct]

Sorry I missed the activity on this until now (got lost in too many GitHub notification emails)

@adelolmo asked:

Out of curiosity, what would be your usecase? I'm trying to understand how this change would be used.

It is related to #113 (comment) and #70 where hd-idle thinks the disk is spun down because no I/O is occurring but something (smartctl?) has caused the disk to spin back up.

Regarding the desire to keep hd-idle clear and single purpose (tracking state via /proc/diskstats, I was thinking along the same lines as @firasdib:

For me, I want to continuously query the SMART data of my drives, without disturbing standby or deep EPC states. This utility already knows when drives are idle, and I can use that "for free".

So add a co-utility that uses SMART status when hd-idle believes the drive is idle to ensure that it is in fact spun down.

Additionally there are other maintenance type jobs where it would be useful to check whether the disk is idle.