KCSAN shows several btrfs data races, e.g. "BUG: KCSAN: data-race in btrfs_inode_safe_disk_i_size_write / fill_stack_inode_item" (kernel v6.6, amd64)

[ernsteiswuerfel]

KCSAN throws up several data races when running a btrfs filesystem. For the test run I used a v6.6 kernel built with KCSAN_EARLY_ENABLE=y, KCSAN_REPORT_ONCE_IN_MS=12000, KCSAN_STRICT=y, KCSAN_WEAK_MEMORY=y. I booted the machine and did a kernel build. During this period I got 218 data races.

I know that a data race detected by KCSAN is not necessarily a bug, but as 64 the 218 data races detected prominently mentioned btrfs, either in the trace or in the workqueue I thought it would probably a good idea to report it here.

4 times I got:

BUG: KCSAN: data-race in btrfs_inode_safe_disk_i_size_write / fill_stack_inode_item

write to 0xffff9b7a10739418 of 8 bytes by task 4490 on cpu 14:
 btrfs_inode_safe_disk_i_size_write+0xc1/0x150
 btrfs_finish_one_ordered+0x1ee/0xfb0
 btrfs_finish_ordered_io+0x1a/0x30
 finish_ordered_fn+0x1e/0x30
 btrfs_work_helper+0xfc/0x3f0
 process_one_work+0x35f/0x5d0
 worker_thread+0x415/0x6d0
 kthread+0x181/0x1b0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

read to 0xffff9b7a10739418 of 8 bytes by task 1497 on cpu 12:
 fill_stack_inode_item+0xb2/0x380
 btrfs_delayed_update_inode+0x1bb/0x290
 btrfs_update_inode+0xe3/0x180
 btrfs_finish_one_ordered+0x236/0xfb0
 btrfs_finish_ordered_io+0x1a/0x30
 finish_ordered_fn+0x1e/0x30
 btrfs_work_helper+0xfc/0x3f0
 process_one_work+0x35f/0x5d0
 worker_thread+0x415/0x6d0
 kthread+0x181/0x1b0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

Reported by Kernel Concurrency Sanitizer on:
CPU: 12 PID: 1497 Comm: kworker/u66:0 Tainted: G                T  6.6.0-Zen3 #3
Hardware name: To Be Filled By O.E.M. B450M Steel Legend/B450M Steel Legend, BIOS P8.01 03/14/2023
Workqueue: btrfs-endio-write btrfs_work_helper

19 times I got:

BUG: KCSAN: data-race in xas_clear_mark / xas_find_marked

read to 0xffff9b7bd7d5ce30 of 8 bytes by task 58136 on cpu 1:
 xas_find_marked+0x11a/0x4d0
 filemap_get_folios_tag+0x119/0x3d0
 extent_write_cache_pages+0x2a4/0xc50
 extent_writepages+0x7d/0xe0
 btrfs_writepages+0x21/0x40
 do_writepages+0xab/0x2c0
 filemap_fdatawrite_wbc+0xbb/0xf0
 __filemap_fdatawrite_range+0x94/0xc0
 filemap_flush+0x28/0x40
 btrfs_release_file+0xd4/0xf0
 __fput+0x1f8/0x4a0
 __fput_sync+0x5a/0x70
 __x64_sys_close+0x5b/0xa0
 do_syscall_64+0x63/0xa0
 entry_SYSCALL_64_after_hwframe+0x4b/0xb5

read-write to 0xffff9b7bd7d5ce30 of 8 bytes by task 685 on cpu 19:
 xas_clear_mark+0xa1/0x120
 __folio_start_writeback+0x3a3/0x4a0
 set_page_writeback+0x2f/0x90
 btrfs_page_clamp_set_writeback+0x10c/0x130
 __process_pages_contig+0x22b/0x2d0
 extent_clear_unlock_delalloc+0x66/0x90
 submit_compressed_extents+0x24e/0x7f0
 btrfs_work_helper+0x2dd/0x3f0
 process_one_work+0x35f/0x5d0
 worker_thread+0x415/0x6d0
 kthread+0x181/0x1b0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

Reported by Kernel Concurrency Sanitizer on:
CPU: 19 PID: 685 Comm: kworker/u65:11 Tainted: G                T  6.6.0-Zen3 #3
Hardware name: To Be Filled By O.E.M. B450M Steel Legend/B450M Steel Legend, BIOS P8.01 03/14/2023
Workqueue: btrfs-delalloc btrfs_work_helper

11 times I got:

BUG: KCSAN: data-race in wq_worker_tick / wq_worker_tick

read to 0xffff9b7a057699a8 of 8 bytes by interrupt on cpu 31:
 wq_worker_tick+0x5f/0x210
 scheduler_tick+0x111/0x120
 update_process_times+0xbf/0xf0
 tick_sched_timer+0xbe/0x130
 __hrtimer_run_queues+0x24f/0x330
 hrtimer_interrupt+0x1cb/0x3e0
 __sysvec_apic_timer_interrupt+0x5f/0xd0
 sysvec_apic_timer_interrupt+0x80/0xc0
 asm_sysvec_apic_timer_interrupt+0x1a/0x20
 kcsan_setup_watchpoint+0x27c/0x4f0
 btrfs_compress_heuristic+0x298/0x8e0
 compress_file_range+0x3ec/0xa40
 btrfs_work_helper+0xfc/0x3f0
 process_one_work+0x35f/0x5d0
 worker_thread+0x415/0x6d0
 kthread+0x181/0x1b0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

write to 0xffff9b7a057699a8 of 8 bytes by interrupt on cpu 28:
 wq_worker_tick+0x75/0x210
 scheduler_tick+0x111/0x120
 update_process_times+0xbf/0xf0
 tick_sched_timer+0xbe/0x130
 __hrtimer_run_queues+0x24f/0x330
 hrtimer_interrupt+0x1cb/0x3e0
 __sysvec_apic_timer_interrupt+0x5f/0xd0
 sysvec_apic_timer_interrupt+0x80/0xc0
 asm_sysvec_apic_timer_interrupt+0x1a/0x20
 kcsan_setup_watchpoint+0x27c/0x4f0
 HUF_compress1X_usingCTable_internal_bmi2+0xb91/0x2cb0
 HUF_compress4X_usingCTable_internal+0x25e/0x270
 HUF_compressCTable_internal+0xaf/0xc0
 HUF_compress_internal+0x415/0x6c0
 HUF_compress4X_repeat+0x62/0x90
 ZSTD_compressLiterals+0x2a3/0x400
 ZSTD_entropyCompressSeqStore_internal.constprop.0+0x16c/0x3e0
 ZSTD_compressBlock_internal+0xf3/0x340
 ZSTD_compressContinue_internal+0x387/0x1420
 ZSTD_compressEnd+0x43/0x210
 ZSTD_compressStream2+0xb3d/0xc90
 ZSTD_endStream+0x47/0xd0
 zstd_end_stream+0x21/0x40
 zstd_compress_pages+0x689/0x8f0
 btrfs_compress_pages+0x14c/0x170
 compress_file_range+0x4b6/0xa40
 btrfs_work_helper+0xfc/0x3f0
 process_one_work+0x35f/0x5d0
 worker_thread+0x415/0x6d0
 kthread+0x181/0x1b0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

Reported by Kernel Concurrency Sanitizer on:
CPU: 28 PID: 1512 Comm: kworker/u66:2 Tainted: G                T  6.6.0-Zen3 #3
Hardware name: To Be Filled By O.E.M. B450M Steel Legend/B450M Steel Legend, BIOS P8.01 03/14/2023
Workqueue: btrfs-delalloc btrfs_work_helper

10 times I got:

BUG: KCSAN: data-race in process_one_work / process_one_work

write to 0xffff9b7a082c1da0 of 8 bytes by task 48379 on cpu 21:
 process_one_work+0x37b/0x5d0
 worker_thread+0x415/0x6d0
 kthread+0x181/0x1b0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

read to 0xffff9b7a082c1da0 of 8 bytes by task 197 on cpu 19:
 process_one_work+0x367/0x5d0
 worker_thread+0x415/0x6d0
 kthread+0x181/0x1b0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

value changed: 0x0000000000000a41 -> 0x0000000000000a42

Reported by Kernel Concurrency Sanitizer on:
CPU: 19 PID: 197 Comm: kworker/u65:1 Tainted: G                T  6.6.0-Zen3 #3
Hardware name: To Be Filled By O.E.M. B450M Steel Legend/B450M Steel Legend, BIOS P8.01 03/14/2023
Workqueue: btrfs-endio-write btrfs_work_helper

Rather often I got BUG: KCSAN: data-race in __hrtimer_run_queues / hrtimer_active but only some times connected to btrfs, e.g.

BUG: KCSAN: data-race in __hrtimer_run_queues / hrtimer_active

read to 0xffff9b811e819e58 of 8 bytes by interrupt on cpu 17:
 hrtimer_active+0x7d/0xf0
 task_tick_fair+0x2f/0x270
 scheduler_tick+0x70/0x120
 update_process_times+0xbf/0xf0
 tick_sched_timer+0xbe/0x130
 __hrtimer_run_queues+0x24f/0x330
 hrtimer_interrupt+0x1cb/0x3e0
 __sysvec_apic_timer_interrupt+0x5f/0xd0
 sysvec_apic_timer_interrupt+0x80/0xc0
 asm_sysvec_apic_timer_interrupt+0x1a/0x20
 __page_table_check_zero+0xa3/0x190
 get_page_from_freelist+0x4a9/0xf20
 __alloc_pages+0x189/0x280
 __folio_alloc+0x16/0x60
 do_anonymous_page+0xe0/0x820
 __handle_mm_fault+0xaa6/0xab0
 handle_mm_fault+0x259/0x420
 exc_page_fault+0x18b/0x720
 asm_exc_page_fault+0x26/0x30

write to 0xffff9b811e819e58 of 8 bytes by interrupt on cpu 0:
 __hrtimer_run_queues+0x15d/0x330
 hrtimer_interrupt+0x1cb/0x3e0
 __sysvec_apic_timer_interrupt+0x5f/0xd0
 sysvec_apic_timer_interrupt+0x80/0xc0
 asm_sysvec_apic_timer_interrupt+0x1a/0x20
 kcsan_setup_watchpoint+0x27c/0x4f0
 btrfs_get_32+0x90/0x230
 __btrfs_check_leaf+0x188/0x23d0
 btrfs_check_leaf+0x1a/0x30
 btree_csum_one_bio+0x324/0x410
 btrfs_submit_chunk+0x759/0x790
 btrfs_submit_bio+0x1f/0x40
 write_one_eb+0x412/0x680
 btree_write_cache_pages+0x391/0x850
 btree_writepages+0xc6/0xe0
 do_writepages+0xab/0x2c0
 filemap_fdatawrite_wbc+0xbb/0xf0
 __filemap_fdatawrite_range+0x94/0xc0
 filemap_fdatawrite_range+0x2e/0x50
 btrfs_write_marked_extents+0xb4/0x200
 btrfs_write_and_wait_transaction+0x8b/0x140
 btrfs_commit_transaction+0x1151/0x1ac0
 transaction_kthread+0x243/0x2b0
 kthread+0x181/0x1b0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

Reported by Kernel Concurrency Sanitizer on:
CPU: 0 PID: 405 Comm: btrfs-transacti Tainted: G                T  6.6.0-Zen3 #3
Hardware name: To Be Filled By O.E.M. B450M Steel Legend/B450M Steel Legend, BIOS P8.01 03/14/2023

Some data on the machine:

 # inxi -bz
System:
  Kernel: 6.5.10-gentoo-Zen3 arch: x86_64 bits: 64 Console: pty pts/2
    Distro: Gentoo Base System release 2.14
Machine:
  Type: Desktop Mobo: ASRock model: B450M Steel Legend serial: <filter>
    UEFI: American Megatrends v: P8.01 date: 03/14/2023
CPU:
  Info: 16-core AMD Ryzen 9 5950X [MT MCP] speed (MHz): avg: 3626
    min/max: 550/5084
Graphics:
  Device-1: AMD Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
    driver: amdgpu v: kernel
  Device-2: AMD Turks PRO [Radeon HD 6570/7570/8550 / R5 230] driver: radeon
    v: kernel
  Display: x11 server: X.Org v: 21.1.8 driver: X: loaded: amdgpu
    unloaded: fbdev,modesetting,radeon dri: radeonsi gpu: amdgpu,radeon
    resolution: 2560x1600~60Hz
  API: OpenGL v: 4.5 Mesa 23.1.8 renderer: llvmpipe (LLVM 16.0.6 256 bits)
Network:
  Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet
    driver: r8169
Drives:
  Local Storage: total: 931.51 GiB used: 260.46 GiB (28.0%

Full kernel dmesg and kernel v6.6 .config attached.
dmesg_66-van_zen3.txt
config_66-van_zen3.txt

[ernsteiswuerfel]

The number of btrfs specific data races decreased on kernel v6.7-rc5, also there are no longer direct mentions in the ""BUG: KCSAN: data-race in ..." line. Some btrfs mentions prevails however like in the BUG: KCSAN: data-race in wq_worker_tick / wq_worker_tick and BUG: KCSAN: data-race in process_one_work / process_one_work races.

I enabled PROVE_LOCKING=y in the .config in order to get additional output via KCSAN_VERBOSE=y.

On v6.7-rc5 I get 3 times:

==================================================================
BUG: KCSAN: data-race in wq_worker_tick / wq_worker_tick

read to 0xffff9c9b5e37cda8 of 8 bytes by interrupt on cpu 21:
 wq_worker_tick+0x5f/0x250
 scheduler_tick+0x1a0/0x1c0
 update_process_times+0xbf/0xf0
 tick_nohz_highres_handler+0xbe/0x130
 __hrtimer_run_queues+0x3b0/0x5f0
 hrtimer_interrupt+0x1cb/0x3e0
 __sysvec_apic_timer_interrupt+0x9d/0x260
 sysvec_apic_timer_interrupt+0x80/0xc0
 asm_sysvec_apic_timer_interrupt+0x1a/0x20
 delay_halt_mwaitx+0x38/0x50
 delay_halt+0x3a/0x80
 kcsan_setup_watchpoint+0x2c6/0x690
 lzo1x_decompress_safe+0x216/0x9f0
 lzo_decompress_bio+0x360/0x4c0
 end_compressed_bio_read+0x100/0x150
 __btrfs_bio_end_io+0x6e/0xc0
 btrfs_orig_bbio_end_io+0x7c/0x170
 btrfs_check_read_bio+0x5c0/0x870
 btrfs_end_bio_work+0x61/0x90
 process_one_work+0x452/0x940
 worker_thread+0x3fb/0x730
 kthread+0x189/0x1c0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

2 locks held by kworker/u65:0/174:
 #0: ffff9c944931e938 ((wq_completion)btrfs-endio){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
 #1: ffffabc00073fe10 ((work_completion)(&bbio->end_io_work)){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
irq event stamp: 11485
hardirqs last  enabled at (11484): [<ffffffffb239c810>] _raw_spin_unlock_irqrestore+0x50/0x60
hardirqs last disabled at (11485): [<ffffffffb2385fee>] sysvec_apic_timer_interrupt+0xe/0xc0
softirqs last  enabled at (11151): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0
softirqs last disabled at (11146): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0

write to 0xffff9c9b5e37cda8 of 8 bytes by interrupt on cpu 4:
 wq_worker_tick+0x75/0x250
 scheduler_tick+0x1a0/0x1c0
 update_process_times+0xbf/0xf0
 tick_nohz_highres_handler+0xbe/0x130
 __hrtimer_run_queues+0x3b0/0x5f0
 hrtimer_interrupt+0x1cb/0x3e0
 __sysvec_apic_timer_interrupt+0x9d/0x260
 sysvec_apic_timer_interrupt+0x80/0xc0
 asm_sysvec_apic_timer_interrupt+0x1a/0x20
 lzo1x_decompress_safe+0x209/0x9f0
 lzo_decompress_bio+0x360/0x4c0
 end_compressed_bio_read+0x100/0x150
 __btrfs_bio_end_io+0x6e/0xc0
 btrfs_orig_bbio_end_io+0x7c/0x170
 btrfs_check_read_bio+0x5c0/0x870
 btrfs_end_bio_work+0x61/0x90
 process_one_work+0x452/0x940
 worker_thread+0x3fb/0x730
 kthread+0x189/0x1c0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

2 locks held by kworker/u65:5/283:
 #0: ffff9c944931e938 ((wq_completion)btrfs-endio){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
 #1: ffffabc001f03e10 ((work_completion)(&bbio->end_io_work)){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
irq event stamp: 8120
hardirqs last  enabled at (8119): [<ffffffffb239c810>] _raw_spin_unlock_irqrestore+0x50/0x60
hardirqs last disabled at (8120): [<ffffffffb2385fee>] sysvec_apic_timer_interrupt+0xe/0xc0
softirqs last  enabled at (7954): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0
softirqs last disabled at (7949): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0

value changed: 0x000000000010d577 -> 0x000000000010ef81

Reported by Kernel Concurrency Sanitizer on:
CPU: 4 PID: 283 Comm: kworker/u65:5 Tainted: G                T  6.7.0-rc5-Zen3 #2
Hardware name: To Be Filled By O.E.M. B550M Pro4/B550M Pro4, BIOS P3.20 09/27/2023
Workqueue: btrfs-endio btrfs_end_bio_work
==================================================================

and 2 times:

==================================================================
BUG: KCSAN: data-race in process_one_work / process_one_work

write to 0xffff9c9450ed49a0 of 8 bytes by task 198 on cpu 15:
 process_one_work+0x4b0/0x940
 worker_thread+0x3fb/0x730
 kthread+0x189/0x1c0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

2 locks held by kworker/u66:22/198:
 #0: ffff9c94488c2d38 ((wq_completion)btrfs-endio-write){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
 #1: ffffabc0007ffe10 ((work_completion)(&work->normal_work)){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
irq event stamp: 11993
hardirqs last  enabled at (11993): [<ffffffffb239c810>] _raw_spin_unlock_irqrestore+0x50/0x60
hardirqs last disabled at (11992): [<ffffffffb239c52c>] _raw_spin_lock_irqsave+0x6c/0x70
softirqs last  enabled at (10742): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0
softirqs last disabled at (10735): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0

read to 0xffff9c9450ed49a0 of 8 bytes by task 199 on cpu 29:
 process_one_work+0x49c/0x940
 worker_thread+0x3fb/0x730
 kthread+0x189/0x1c0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

2 locks held by kworker/u66:23/199:
 #0: ffff9c94488c2d38 ((wq_completion)btrfs-endio-write){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
 #1: ffffabc000807e10 ((work_completion)(&work->normal_work)){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
irq event stamp: 54984
hardirqs last  enabled at (54984): [<ffffffffb239c810>] _raw_spin_unlock_irqrestore+0x50/0x60
hardirqs last disabled at (54983): [<ffffffffb239c52c>] _raw_spin_lock_irqsave+0x6c/0x70
softirqs last  enabled at (53685): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0
softirqs last disabled at (53678): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0

Reported by Kernel Concurrency Sanitizer on:
CPU: 29 PID: 199 Comm: kworker/u66:23 Tainted: G                T  6.7.0-rc5-Zen3 #2
Hardware name: To Be Filled By O.E.M. B550M Pro4/B550M Pro4, BIOS P3.20 09/27/2023
Workqueue: btrfs-endio-write btrfs_work_helper
==================================================================

Also I get:

==================================================================
BUG: KCSAN: data-race in __hrtimer_run_queues / hrtimer_active

read to 0xffff9c9b5e21a508 of 8 bytes by interrupt on cpu 6:
 hrtimer_active+0x7d/0xf0
 task_tick_fair+0x2f/0x2e0
 scheduler_tick+0xd3/0x1c0
 update_process_times+0xbf/0xf0
 tick_nohz_highres_handler+0xbe/0x130
 __hrtimer_run_queues+0x3b0/0x5f0
 hrtimer_interrupt+0x1cb/0x3e0
 __sysvec_apic_timer_interrupt+0x9d/0x260
 sysvec_apic_timer_interrupt+0x80/0xc0
 asm_sysvec_apic_timer_interrupt+0x1a/0x20
 __tsan_read1+0xc4/0x1b0
 lzo1x_decompress_safe+0x37b/0x9f0
 lzo_decompress_bio+0x360/0x4c0
 end_compressed_bio_read+0x100/0x150
 __btrfs_bio_end_io+0x6e/0xc0
 btrfs_orig_bbio_end_io+0x7c/0x170
 btrfs_check_read_bio+0x5c0/0x870
 btrfs_end_bio_work+0x61/0x90
 process_one_work+0x452/0x940
 worker_thread+0x3fb/0x730
 kthread+0x189/0x1c0
 ret_from_fork+0x2f/0x50
 ret_from_fork_asm+0x11/0x20

3 locks held by kworker/u65:3/279:
 #0: ffff9c944931e938 ((wq_completion)btrfs-endio){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
 #1: ffffabc001edbe10 ((work_completion)(&bbio->end_io_work)){+.+.}-{0:0}, at: process_one_work+0x3b3/0x940
 #2: ffff9c9b5e528598 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x1b/0xa0
irq event stamp: 5000
hardirqs last  enabled at (4999): [<ffffffffb239c79d>] _raw_spin_unlock_irq+0x2d/0x50
hardirqs last disabled at (5000): [<ffffffffb2385fee>] sysvec_apic_timer_interrupt+0xe/0xc0
softirqs last  enabled at (4934): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0
softirqs last disabled at (4929): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0

write to 0xffff9c9b5e21a508 of 8 bytes by interrupt on cpu 0:
 __hrtimer_run_queues+0x26f/0x5f0
 hrtimer_interrupt+0x1cb/0x3e0
 __sysvec_apic_timer_interrupt+0x9d/0x260
 sysvec_apic_timer_interrupt+0x80/0xc0
 asm_sysvec_apic_timer_interrupt+0x1a/0x20
 _raw_spin_unlock_irqrestore+0x34/0x60
 ___slab_alloc+0xb73/0x1250
 kmem_cache_alloc+0x34e/0x390
 alloc_iova+0x4c/0x420
 alloc_iova_fast+0x2c0/0x4c0
 iommu_dma_alloc_iova+0x171/0x1d0
 iommu_dma_map_sg+0x469/0x6f0
 __dma_map_sg_attrs+0x7d/0x130
 dma_map_sgtable+0x4b/0x80
 nvme_map_data+0x1b0/0xf60
 nvme_queue_rqs+0x176/0x530
 __blk_mq_flush_plug_list+0x6b/0x90
 blk_mq_flush_plug_list+0xed7/0x1170
 blk_add_rq_to_plug+0x1a0/0x350
 blk_mq_submit_bio+0x78d/0xe90
 __submit_bio+0x38/0xf0
 submit_bio_noacct_nocheck+0x535/0x6b0
 submit_bio_noacct+0x144/0x710
 submit_bio+0x37/0xc0
 btrfs_submit_dev_bio+0x247/0x450
 __btrfs_submit_bio+0x2a4/0x310
 btrfs_submit_chunk+0x25e/0x960
 btrfs_submit_bio+0x1f/0x40
 submit_one_bio+0x5c/0x90
 extent_readahead+0x901/0x930
 btrfs_readahead+0x1a/0x30
 read_pages+0xdb/0x4a0
 page_cache_ra_unbounded+0x215/0x2d0
 page_cache_ra_order+0x33f/0x490
 filemap_fault+0x9b2/0x15e0
 __do_fault+0x78/0x2e0
 do_fault+0x4be/0xb50
 __handle_mm_fault+0x809/0xb90
 handle_mm_fault+0x2b7/0x5d0
 exc_page_fault+0x1a0/0xad0
 asm_exc_page_fault+0x26/0x30

3 locks held by flatpak/412:
 #0: ffff9c944068e980 (mapping.invalidate_lock){.+.+}-{3:3}, at: page_cache_ra_unbounded+0xba/0x2d0
 #1: ffffffffb2c83360 (rcu_read_lock){....}-{1:2}, at: blk_mq_flush_plug_list+0xea1/0x1170
 #2: ffff9c9b5e21a458 (hrtimer_bases.lock){-.-.}-{2:2}, at: hrtimer_interrupt+0x93/0x3e0
irq event stamp: 82853
hardirqs last  enabled at (82852): [<ffffffffb239c810>] _raw_spin_unlock_irqrestore+0x50/0x60
hardirqs last disabled at (82853): [<ffffffffb2385fee>] sysvec_apic_timer_interrupt+0xe/0xc0
softirqs last  enabled at (82509): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0
softirqs last disabled at (82498): [<ffffffffb116095e>] irq_exit_rcu+0x7e/0xa0

Reported by Kernel Concurrency Sanitizer on:
CPU: 0 PID: 412 Comm: flatpak Tainted: G                T  6.7.0-rc5-Zen3 #2
Hardware name: To Be Filled By O.E.M. B550M Pro4/B550M Pro4, BIOS P3.20 09/27/2023

So of the locks reported by KSCAN ((wq_completion)btrfs-endio) and ((wq_completion)btrfs-endio-write) are prominently mentioned in the dmesg.

Full kernel dmesg and kernel v6.7-rc5 .config attached.
config_67-rc5_zen3.txt
dmesg_67-rc5_zen3.txt