Change zfs_arc_meta_limit_percent default to 100 #10191
Conversation
As discussed on the mailing list: https://zfsonlinux.topicbox.com/groups/zfs-discuss/T2aca1048f5a31bb8/zfsarcmetalimitpercent-a-report-and-some-ideas-for-improvement-was-re-zfs-discuss-pmis-and-pm-in-arcstat-tuning TL;DR: the idea is to avoid inintuitivelly wasting 25% of ARC when people set primarycache=metadata. Also, fixed a small English verb-conjugation mistake in the comment.
|
And what abount absolute majority of people not using primarycache=metadata? I suppose the tunable was added for a reason. This looks to me you are optimizing default for single very specific use case. |
|
Hello @amotin.
It's not only people who have set primary=metadata (in which case the wastage would be total), but also anyone whose workload has less than 25% of "interesting" data (meaning data that would enhance performance to be kept in the ARC instead of getting evicted by more recently used / more frequently used metadata). I will update the PR description to reflect that. That said, I think it's actually a matter of balancing: (1) On one hand, wasting up to 25% of the ARC for everyone in the situation above. (2) On the other hand, evicting possibly "more interesting" data from the ARC in favor of metadata; again, IIUC this would happen only when the metadata is more frequently/recently used than the data it's replacing. I've never seen (and can't very well imagine) a case where (2) is true, but I have lived through a case where (1) was very true, and the default would have avoided me wasting countless hours on long-running backups while gigabytes of RAM were allocated to the ARC but unused nonetheless. |
Codecov Report
@@ Coverage Diff @@
## master #10191 +/- ##
===========================================
- Coverage 79.40% 65.95% -13.45%
===========================================
Files 385 304 -81
Lines 122603 105322 -17281
===========================================
- Hits 97350 69467 -27883
- Misses 25253 35855 +10602
Continue to review full report at Codecov.
|
|
I think this should be relegated to userspace, perhaps the ZED? I like the idea of Red Hat's It would also enable faster innovation in the heuristic algorithms, as developing pure userspace is way easier (though userspace C is I'd say more hostile to the developer, but that's another issue 😄 - but hey, there already is Lua interpreter in the project...) |
|
When #14359 was merged, this PR can be closed. |
7 participants
As discussed on the mailing list: https://zfsonlinux.topicbox.com/groups/zfs-discuss/T2aca1048f5a31bb8/zfsarcmetalimitpercent-a-report-and-some-ideas-for-improvement-was-re-zfs-discuss-pmis-and-pm-in-arcstat-tuning
TL;DR: the idea is to avoid non-intuitivelly wasting up to 25% of ARC in workloads with less than 25% of "interesting" data (meaning data that would enhance performance to be kept in the ARC instead of getting evicted by more-recently-used / more-frequently-used metadata). This wastage is total when primarycache=metadata, but would also happen to any workload with less than 25% (relative to the ARC size) of "interesting" data.
By changing the zfs_arc_meta_limit_percent default to 100%, these situations are avoided, and the decision about what to keep in the ARC is left to the MRU/MFU algorithms; people with special workloads that would benefit from such a "data reservation" (ie, metadata limit) are still free to tweak this parameter accordingly (or if they already do, their tweak will just keep working)
Also, fixed a small English verb-conjugation mistake in the comment.
Signed-off-by: Durval Menezes zfssign@durval.com
Motivation and Context
Description
How Has This Been Tested?
Types of changes
Checklist:
Signed-off-by.