Skip to content

Commit 9a5b183

Browse files
Michal Hockoakpm00
authored andcommitted
mm, percpu: do not consider sleepable allocations atomic
28307d9 ("percpu: make pcpu_alloc() aware of current gfp context") has fixed a reclaim recursion for scoped GFP_NOFS context. It has done that by avoiding taking pcpu_alloc_mutex. This is a correct solution as the worker context with full GFP_KERNEL allocation/reclaim power and which is using the same lock cannot block the NOFS pcpu_alloc caller. On the other hand this is a very conservative approach that could lead to failures because pcpu_alloc lockless implementation is quite limited. We have a bug report about premature failures when scsi array of 193 devices is scanned. Sometimes (not consistently) the scanning aborts because the iscsid daemon fails to create the queue for a random scsi device during the scan. iscsid itslef is running with PR_SET_IO_FLUSHER set so all allocations from this process context are GFP_NOIO. This in turn makes any pcpu_alloc lockless (without pcpu_alloc_mutex) which leads to pre-mature failures. It has turned out that iscsid has worked around this by dropping PR_SET_IO_FLUSHER (open-iscsi/open-iscsi#382) when scanning host. But we can do better in this case on the kernel side and use pcpu_alloc_mutex for NOIO resp. NOFS constrained allocation scopes too. We just need the WQ worker to never trigger IO/FS reclaim. Achieve that by enforcing scoped GFP_NOIO for the whole execution of pcpu_balance_workfn (this will imply NOFS constrain as well). This will remove the dependency chain and preserve the full allocation power of the pcpu_alloc call. While at it make is_atomic really test for blockable allocations. Link: https://lkml.kernel.org/r/20250206122633.167896-1-mhocko@kernel.org Fixes: 28307d9 ("percpu: make pcpu_alloc() aware of current gfp context") Signed-off-by: Michal Hocko <mhocko@suse.com> Acked-by: Vlastimil Babka <vbabka@suse.cz> Cc: Dennis Zhou <dennis@kernel.org> Cc: Filipe David Manana <fdmanana@suse.com> Cc: Tejun Heo <tj@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
1 parent 1d21229 commit 9a5b183

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

mm/percpu.c

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1745,7 +1745,7 @@ void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved,
17451745
gfp = current_gfp_context(gfp);
17461746
/* whitelisted flags that can be passed to the backing allocators */
17471747
pcpu_gfp = gfp & (GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN);
1748-
is_atomic = (gfp & GFP_KERNEL) != GFP_KERNEL;
1748+
is_atomic = !gfpflags_allow_blocking(gfp);
17491749
do_warn = !(gfp & __GFP_NOWARN);
17501750

17511751
/*
@@ -2191,7 +2191,12 @@ static void pcpu_balance_workfn(struct work_struct *work)
21912191
* to grow other chunks. This then gives pcpu_reclaim_populated() time
21922192
* to move fully free chunks to the active list to be freed if
21932193
* appropriate.
2194+
*
2195+
* Enforce GFP_NOIO allocations because we have pcpu_alloc users
2196+
* constrained to GFP_NOIO/NOFS contexts and they could form lock
2197+
* dependency through pcpu_alloc_mutex
21942198
*/
2199+
unsigned int flags = memalloc_noio_save();
21952200
mutex_lock(&pcpu_alloc_mutex);
21962201
spin_lock_irq(&pcpu_lock);
21972202

@@ -2202,6 +2207,7 @@ static void pcpu_balance_workfn(struct work_struct *work)
22022207

22032208
spin_unlock_irq(&pcpu_lock);
22042209
mutex_unlock(&pcpu_alloc_mutex);
2210+
memalloc_noio_restore(flags);
22052211
}
22062212

22072213
/**

0 commit comments

Comments
 (0)