| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The CamelCase Node typedef is a 1997 leftover and hides that this is
a plain struct. Call it what it is: struct binfmt_misc_entry, matching
struct binfmt_misc that it hangs off of and the entry bit and flag
enums. Drop the typedef, switch the size computations in
create_entry() to sizeof(*e) and adjust the comments that still
referred to the old name.
No functional change.
Link: https://patch.msgid.link/20260710-work-binfmt_misc-locking-v3-9-a162f7cb58d6@kernel.org
Reviewed-by: Jori Koolstra <jkoolstra@xs4all.nl>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
The MISC_FMT_* behavior flags are macros using unsigned long literals
while the entry bit numbers right above them are now a proper enum.
Move the flags into an enum as well so every flags word constant is
declared in one form and shows up in debuginfo. (1U << N) keeps the
enumerators within unsigned int range which is well-defined for enum
constants and the values are unchanged when promoted to the unsigned
long flags word.
No functional change.
Link: https://patch.msgid.link/20260710-work-binfmt_misc-locking-v3-8-a162f7cb58d6@kernel.org
Reviewed-by: Jori Koolstra <jkoolstra@xs4all.nl>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Enabled and Magic are bit numbers in the flags word of an entry but
are declared as bare, unprefixed enumerators with implicit values in
a style that predates the git history. Give the enum a name, explicit
bit numbers and namespaced names and use BIT() instead of open-coding
the shifts when building the initial flags word in create_entry().
No functional change.
Link: https://patch.msgid.link/20260710-work-binfmt_misc-locking-v3-7-a162f7cb58d6@kernel.org
Reviewed-by: Jori Koolstra <jkoolstra@xs4all.nl>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
->enabled has always been read and written locklessly: every exec
reads it in load_misc_binary() while bm_status_write() or a concurrent
remount via bm_fill_super() may flip it. That is fine as it is an
independent boolean toggle but the accesses should be marked
accordingly for KCSAN. Annotate them with READ_ONCE()/WRITE_ONCE().
Link: https://patch.msgid.link/20260710-work-binfmt_misc-locking-v3-6-a162f7cb58d6@kernel.org
Reviewed-by: Jori Koolstra <jkoolstra@xs4all.nl>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Once binfmt_misc is loaded load_misc_binary() runs for every execve()
on the system since binfmt_misc registers at the head of the formats
list. Every exec therefore performs read_lock() and read_unlock() on
the entries_lock of the relevant binfmt_misc instance, i.e., two
atomic read-modify-writes on a shared cacheline. User namespaces
without their own binfmt_misc mount fall back to an ancestor's
instance so on container-heavy systems every exec on the machine
typically ends up hammering the cacheline of init_binfmt_misc. On
PREEMPT_RT the rwlock additionally turns the handler lookup into a
sleeping lock on the exec fast path.
The lock protects very little. Entries are immutable after publication
except for the Enabled bit which is already toggled locklessly via
set_bit()/clear_bit() and entry lifetime is already handled by the
users refcount via get_binfmt_handler()/put_binfmt_handler(). The read
lock's only remaining job is to make "the entry is still linked" and
"take a reference" atomic with respect to the unlink sites.
Switch the lookup to an RCU walk:
* Lookup walks the entry list under rcu_read_lock() and acquires a
reference via refcount_inc_not_zero(). The refcount can only drop to
zero after an entry has been unlinked so a failed increment means
the walk raced with an unlink. Restarting the search is bounded
because an unlinked entry cannot be found again.
* The unlink sites use hlist_del_init_rcu() which keeps the forward
pointer intact for concurrent walkers and preserves hlist_unhashed()
as the protection against double removal.
* The final put frees the entry via kfree_rcu() as a concurrent walker
may still dereference its flags, magic, mask, and inline strings.
They all live in the entry allocation itself and thus stay valid
until a grace period has elapsed. Closing the interpreter file stays
synchronous. It is only used with a reference already held and all
final puts run in process context.
* Writers remain serialized by the inode lock of the root dentry with
one exception. bm_evict_inode() called from generic_shutdown_super()
during umount unlinks entries without holding it. Keep a spinlock
around the unlink sites instead of relying on superblock lifetime
rules to make that exclusion implicit.
Handler removal semantics are unchanged. An exec that acquired a
reference just before its handler was unregistered already completes
with the removed handler today. The read lock never protected against
that, it only made the window smaller.
With this an exec that matches no binfmt_misc entry, the common case,
no longer writes to any shared cacheline at all.
Link: https://patch.msgid.link/20260710-work-binfmt_misc-locking-v3-5-a162f7cb58d6@kernel.org
Reviewed-by: Jori Koolstra <jkoolstra@xs4all.nl>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
The upcoming conversion of the handler lookup to RCU walks cannot use
list_del_init(): reinitializing the forward pointer of a removed entry
would make a concurrent lockless walker standing on that entry loop
back onto it indefinitely. The removal paths do rely on
reinitialization though because bm_{entry,status}_write() and
bm_evict_inode() need to detect whether an entry has already been
unlinked.
hlists support exactly this pattern: hlist_del_init_rcu() keeps the
forward pointer of the removed entry intact for concurrent walkers and
only zeroes ->pprev with hlist_unhashed() serving as the linked test.
Convert the entry list to an hlist now while keeping the rwlock so the
subsequent RCU conversion is a pure locking change. hlist_add_head()
inserts at the head just as list_add() did so lookup precedence
between registered handlers is unchanged.
Link: https://patch.msgid.link/20260710-work-binfmt_misc-locking-v3-4-a162f7cb58d6@kernel.org
Reviewed-by: Jori Koolstra <jkoolstra@xs4all.nl>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
cifs_try_adding_channels() takes a temporary reference to an interface
before dropping iface_lock. If cifs_ses_add_channel() fails, it drops
that reference and then increments iface->weight_fulfilled.
A concurrent interface list refresh can remove the list reference while
channel creation is in progress. In that case, the failure-path
kref_put() releases the last reference and frees iface. Updating
weight_fulfilled afterward then accesses freed memory.
Increment weight_fulfilled before dropping the temporary reference,
keeping iface alive for the final access.
Fixes: 6aac002bcfd5 ("cifs: failure to add channel on iface should bump up weight")
Cc: stable@vger.kernel.org
Signed-off-by: Shuangpeng Bai <shuangpeng.kernel@gmail.com>
Signed-off-by: Steve French <stfrench@microsoft.com>
|
|
Example:
1. mount -t cifs -o compress //localhost/test /mnt
2. mount | grep cifs
//localhost/test on /mnt type cifs (..., compress, ...)
Signed-off-by: ChenXiaoSong <chenxiaosong@kylinos.cn>
Signed-off-by: Steve French <stfrench@microsoft.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs fixes from Christian Brauner:
"binfmt_misc:
- Don't let an 'F' entry pin its own instance.
An entry registered with 'F' opens its interpreter at registration
time and holds that file until the entry is freed, so an entry
nobody removes by hand is only closed once the binfmt_misc
superblock is shut down.
If the interpreter lives on a mount that keeps that superblock
alive the two pin each other and the file is never closed. That's
reachable by pointing the interpreter at the instance itself or by
using the instance as an overlayfs lower layer, and once the mount
namespace is gone there's nothing left to unregister through
either.
- Restore write access when removing an entry.
Registering with the MISC_FMT_OPEN_FILE flag opens the interpreter
via open_exec() which denies write access for as long as the entry
exists, but removal only did filp_close() and never restored it.
The inode's i_writecount stayed permanently negative and opening
the interpreter for writing kept failing with ETXTBSY long after
the entry was gone.
- Use exe_file_deny_write_access() for the interpreter clone so both
sides base their decision on the same mode.
- Reject a flag character as the field delimiter. create_entry() pads
the buffer with the delimiter so the field parsers terminate even
on a truncated string, but check_special_flags() consumes flag
characters instead of scanning for the delimiter.
If the delimiter is itself a flag character the padding stops
acting as a terminator and the scan keeps reading past the end of
the allocation. Such a registration was always rejected, just only
after the out of bounds read has already happened.
- Don't leak the user namespace when the mount fails.
bm_get_tree() hands its reference to get_tree_keyed() and sget_fc()
moves it into sb->s_fs_info, but generic_shutdown_super() only
calls ->put_super() from inside the if (sb->s_root) branch and
bm_fill_super() can fail before either s_root or s_op is in place.
Drop the reference in ->kill_sb() instead, which runs
unconditionally.
netfs:
- Clear PG_private_2 on a copy-to-cache append failure.
- Handle a rolling buffer allocation failure in single-object
writeback and drop the extra folio reference
netfs_write_folio_single() took before the append.
- Release the previously batched readahead folios when
rolling_buffer_load_from_ra() fails in
netfs_prepare_read_iterator()
- Fix the folio_queue ENOMEM in writeback by adding a mempool and
passing gfp flags into the rolling buffer helpers.
iomap:
- Add a separate bio_set for iomap_split_ioend(). It can split bios
that already come from iomap_ioend_bioset and deadlock once that
bioset is exhausted.
afs:
- Set call->async for an asynchronous afs_fs_fetch_data() the way
afs_fs_fetch_data64() already does.
- Subtract subreq->transferred from subreq->len in
afs_fs_fetch_data() rather than adding it.
- Fix a UAF when sending a message"
* tag 'vfs-7.2-rc6.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
iomap: add a separate bio_set for iomap_split_ioend
binfmt_misc: don't leak the user namespace when the mount fails
binfmt_misc: reject a flag character as the field delimiter
binfmt_misc: use exe_file_deny_write_access() for the interpreter clone
binfmt_misc: restore write access when removing an entry
binfmt_misc: don't let an 'F' entry pin its own instance
netfs: Fix folio_queue ENOMEM in writeback by adding a mempool
netfs: release readahead folios on iterator preparation failure
netfs: handle single writeback rolling buffer allocation failure
netfs: clear PG_private_2 on copy-to-cache append failure
afs: Fix UAF when sending a message
afs: Fix afs_fs_fetch_data() to subtract transferred from len
afs: Fix afs_fs_fetch_data() to set call->async
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/linkinjeon/ntfs
Pull ntfs fixes from Namjae Jeon:
- Keep RECALL_ON_OPEN in inode flags when reloading them from
$FILE_NAME
- Check runlist reallocation sizes for negative values and overflow
- Drop stale page cache after shrinking non-resident attributes to
prevent writeback failures and data loss
* tag 'ntfs-for-7.2-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/linkinjeon/ntfs:
ntfs: drop stale page-cache when shrinking a non-resident attr
ntfs: harden runlist realloc size calculations
ntfs: preserve RECALL_ON_OPEN on WSL special-file reparse points
|
|
Pull smb server fixes from Steve French:
- Use memcmp() when comparing fixed-size binary ClientGUIDs, so
embedded NUL bytes are handled correctly
- Reject repeated SMB2 NEGOTIATE requests after dialect selection
This prevents preauth_info leaks, enforces the SMB2 protocol
requirements, and serializes negotiation state updates.
- Fix a use-after-free in __close_file_table_ids() by removing the
volatile file ID from the owning IDR before dropping the IDR
reference
* tag 'v7.2-rc5-smb3-server-fixes' of git://git.samba.org/ksmbd:
ksmbd: use memcmp() to compare ClientGUIDs
ksmbd: reject repeated SMB2 NEGOTIATE requests
ksmbd: fix use-after-free in __close_file_table_ids()
|
|
Casefolding filesystems can request a specific version of UTF-8 at
mount-time. utf8_parse_version then assembles the "major.minor.rev"
string into an unsigned int. There were two issues with the parser
logic: first, individual fields are read as signed int, allowing
negative numbers, second, an overflowed field will result in unexpected
results. Something like the below actually succeeds to mount using
utf8-12.1.0.
mount -t tmpfs -o casefold=utf8-12.0.256 none /mnt
[ 10.867859] tmpfs: Using encoding : utf8-12.1.0
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
|
|
utf8_normalize() was added in 2019 as part of
commit 9d53690f0d4e ("unicode: implement higher level API for string
handling")
but has remained unused.
(I think because the other higher level routines added by that patch
normalise as part of their operations)
Remove it.
Signed-off-by: Dr. David Alan Gilbert <linux@treblig.org>
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
|
|
A call to listxattr() with a buffer size of 0 returns the actual
size of the buffer needed for a subsequent call. On an NFSv4.2
mount this triggers the following oops:
[ 399.768687] BUG: kernel NULL pointer dereference, address: 0000000000000000
[ 399.768705] RIP: 0010:_copy_from_pages+0x44/0xe0
[ 399.768722] Call Trace:
[ 399.768723] nfs4_xattr_alloc_entry+0x1bf/0x1e0
[ 399.768730] nfs4_xattr_cache_set_list+0x43/0x1f0
[ 399.768731] nfs4_listxattr+0x21f/0x250
[ 399.768733] vfs_listxattr+0x55/0xa0
[ 399.768736] listxattr+0x23/0x160
[ 399.768737] path_listxattrat+0xba/0x1e0
[ 399.768739] do_syscall_64+0xe2/0x680
security_inode_listsecurity() (via the xattr_list_one() helper) now
decrements the remaining size even when the buffer pointer is NULL, so
in the size-query case, 'left' underflows to a huge size_t value. As a
result, nfs4_listxattr_nfs4_user() treats the NULL buffer as a real one,
leading to a NULL pointer dereference in _copy_from_pages().
security_inode_listsecurity() does not return the number of bytes
it added to the list, so the code derived it as
'size - error - left'. That is also wrong in the size-query case:
the generic_listxattr() contribution is only subtracted from 'left'
when a buffer is present. Thus, the query result comes up short by
exactly that contribution (e.g., "system.nfs4_acl" on a mount with
ACL support), and a caller that allocates the returned size gets
-ERANGE on the subsequent call.
Declare 'left' as ssize_t, use a scratch copy to measure security
hook consumption, and only decrement 'left' if a buffer is present.
Fixes: f71ece9712b7 ("security,fs,nfs,net: update security_inode_listsecurity() interface")
Suggested-by: Paul Moore <paul@paul-moore.com>
Signed-off-by: Achilles Gaikwad <achillesgaikwad@gmail.com>
Reviewed-by: Paul Moore <paul@paul-moore.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
|
|
Set BIO_COMPLETE_IN_TASK in __bh_submit() for write bios when the folio
has dropbehind set, so that buffer_head writeback completions get
deferred to task context where folio_end_dropbehind() can safely
invalidate folios.
Read completions are not deferred since dropbehind invalidation for
reads is handled synchronously by the reader.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Reviewed-by: Jan Kara <jack@suse.cz>
Link: https://patch.msgid.link/20260730-blk-dontcache-v7-4-3e8e6850068d@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Set BIO_COMPLETE_IN_TASK on iomap writeback bios when a dropbehind folio
is added. This ensures that bi_end_io runs in task context, where
folio_end_dropbehind() can safely invalidate folios.
With the bio layer now handling task-context deferral generically,
IOMAP_IOEND_DONTCACHE is no longer needed, as XFS no longer needs to
route DONTCACHE ioends through its completion workqueue. Remove the flag
and its NOMERGE entry.
Without the NOMERGE, regular I/Os that get merged with a dropbehind
folio will also have their completion deferred to task context.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20260730-blk-dontcache-v7-3-3e8e6850068d@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Move the atomic context detection logic from erofs's z_erofs_in_atomic()
into the block layer as bio_in_atomic(). This helper returns true when
the current context is unsafe for sleeping bio completion handlers (e.g.,
hard/soft IRQ, preempt-disabled).
The logic was originally added to erofs in commit c99fab6e80b7 ("erofs:
fix atomic context detection when !CONFIG_DEBUG_LOCK_ALLOC"). A
subsequent patch will use it in the block layer's bio completion
infrastructure, so move it to include/linux/bio.h where both subsystems
can share it.
Convert erofs to call the new bio_in_atomic() directly.
Suggested-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260730-blk-dontcache-v7-1-3e8e6850068d@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
[BUG]
When running generic/795 with 8K block size, 4K page size, the test
always fails, triggering some ASSERT()s related to folio size:
795 (241074): drop_caches: 3
assertion failed: IS_ALIGNED(start, blocksize) && IS_ALIGNED(end + 1, blocksize), in extent_io.c:1404 (blocksize=8192 root=262 ino=258 start=16826368 end=16830463 mapping min order=0)
------------[ cut here ]------------
kernel BUG at extent_io.c:1404!
Oops: invalid opcode: 0000 [#1] SMP
CPU: 8 UID: 0 PID: 241105 Comm: fsstress Tainted: G OE 7.2.0-rc5-custom+ #442 PREEMPT(full) f4bfb352566f3949f29c233ce6f735050a03b245
Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown 02/02/2022
RIP: 0010:assert_folio_range.cold+0x3d/0x3f [btrfs]
Call Trace:
<TASK>
btrfs_read_folio+0x9e/0x170 [btrfs 4cd1dd93b341b8ef766643f9512f4a86259567a3]
prepare_one_folio.constprop.0+0x104/0x2a0 [btrfs 4cd1dd93b341b8ef766643f9512f4a86259567a3]
btrfs_buffered_write+0x285/0xa50 [btrfs 4cd1dd93b341b8ef766643f9512f4a86259567a3]
btrfs_do_write_iter+0x1aa/0x210 [btrfs 4cd1dd93b341b8ef766643f9512f4a86259567a3]
iter_file_splice_write+0x31a/0x540
direct_splice_actor+0x53/0x170
splice_direct_to_actor+0xe9/0x240
do_splice_direct+0x76/0xb0
vfs_copy_file_range+0x1fd/0x630
__x64_sys_copy_file_range+0xf9/0x220
do_syscall_64+0xe1/0x790
entry_SYSCALL_64_after_hwframe+0x4b/0x53
</TASK>
---[ end trace 0000000000000000 ]---
The ASSERT() itself is added by a later patch.
The crash is triggered with that new debug patch, and without this fix.
[CAUSE]
In the above case, the start 16826368 is properly 8K aligned, but the
end (16830463 + 1) is not 8K aligned.
Furthermore the mapping's minimal folio order is 0, not the expected 1
for 8K block size with 4K page size.
So this means some inodes do not have btrfs_set_inode_mapping_order()
called on it.
The missing btrfs_set_inode_mapping_order() call happens for cached
inodes, through the following events:
- btrfs_create_new_inode() called for inode X
Which properly sets minimal folio order for the VFS inode.
- btrfs_update_inode() called for inode X
Which calls btrfs_delayed_update_inode() to create a delayed_node
into root->delayed_nodes xarray.
- Drop cache/memory pressure, evicting in-memory inode X
Which evicted the inode X, but delayed_node is still in
root->delayed_nodes for future reuse.
- btrfs_iget() for inode X called again
btrfs_iget()
|- btrfs_iget_locked()
| |- iget5_locked_rcu()
| Which creates a new vfs_inode for btrfs, whose mapping still
| has the minimal order as 0.
|
|- btrfs_read_locked_inode()
|- btrfs_fill_inode()
| |- btrfs_get_delayed_node()
| Which found out the previous node, and use that delayed
| node to initialize the new inode.
|
|- filled = true;
|- if (filled) goto cache_index;
Which skips the btrfs_update_inode_mapping_flags() and
btrfs_set_inode_mapping_order() calls.
So the inode still has minimal folio order set as 0, not
the required 1.
Thus later page cache read will get a folio whose size is smaller than
block size, as the mapping has its minimal folio order set as 0 not 1,
then trigger the ASSERT().
[FIX]
Move the btrfs_update_inode_mapping_flags() and
btrfs_set_inode_mapping_order() calls under cache_index label,
so that the mapping flags and minimal folio order is always set
no matter if we have a cached inode.
Assisted-by: LLM (analysis)
Fixes: ecde48a1a6b3 ("btrfs: expose per-inode stable writes flag")
Fixes: cc38d178ff33 ("btrfs: enable large data folio support under CONFIG_BTRFS_EXPERIMENTAL")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Btrfs relies on mapping_set_folio_order_range() to set the minimal
folio order for all its data inodes, but that function will be no-op if
transparent hugepage is not enabled.
Guard the bs > ps support behind CONFIG_TRANSPARENT_HUGEPAGE, just like
all other filesystems.
Fixes: 98077f7f2180 ("btrfs: enable experimental bs > ps support")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Local fuzzing of 6.12.94 has found the following memory leak:
Unreferenced object 0xffff888018050a80 (size 64):
comm "syz.0.17", pid 10297, jiffies 4294953601
hex dump (first 32 bytes):
00 10 00 00 00 00 00 00 01 00 00 00 00 00 00 00 ................
10 0a 05 18 80 88 ff ff 10 0a 05 18 80 88 ff ff ................
backtrace (crc a8a6fc29):
kmemleak_alloc_recursive include/linux/kmemleak.h:42 [inline]
slab_post_alloc_hook mm/slub.c:4152 [inline]
slab_alloc_node mm/slub.c:4197 [inline]
__kmalloc_cache_noprof+0x168/0x2c0 mm/slub.c:4358
kmalloc_noprof include/linux/slab.h:878 [inline]
extent_changeset_alloc fs/btrfs/extent_io.h:207 [inline]
qgroup_reserve_data+0x1c5/0x7d0 fs/btrfs/qgroup.c:4305
btrfs_qgroup_reserve_data+0x2e/0xb0 fs/btrfs/qgroup.c:4355
btrfs_do_encoded_write+0x92e/0x1040 fs/btrfs/inode.c:9746
btrfs_encoded_write fs/btrfs/file.c:1482 [inline]
btrfs_do_write_iter+0x280/0x610 fs/btrfs/file.c:1507
btrfs_ioctl_encoded_write+0x3d6/0x490 fs/btrfs/ioctl.c:4738
btrfs_ioctl+0x6f9/0xc90 fs/btrfs/ioctl.c:-1
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:906 [inline]
__se_sys_ioctl+0xf9/0x170 fs/ioctl.c:892
do_syscall_x64 arch/x86/entry/common.c:47 [inline]
do_syscall_64+0xbe/0x1a0 arch/x86/entry/common.c:78
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Unreferenced object 0xffff888018050a00 (size 64):
comm "syz.0.17", pid 10297, jiffies 4294953601
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 ff 0f 00 00 00 00 00 00 ................
90 0a 05 18 80 88 ff ff 90 0a 05 18 80 88 ff ff ................
backtrace (crc cb5c9580):
kmemleak_alloc_recursive include/linux/kmemleak.h:42 [inline]
slab_post_alloc_hook mm/slub.c:4152 [inline]
slab_alloc_node mm/slub.c:4197 [inline]
__kmalloc_cache_noprof+0x168/0x2c0 mm/slub.c:4358
kmalloc_noprof include/linux/slab.h:878 [inline]
kzalloc_noprof include/linux/slab.h:1014 [inline]
ulist_prealloc+0x9c/0x110 fs/btrfs/ulist.c:114
extent_changeset_prealloc fs/btrfs/extent_io.h:217 [inline]
__set_extent_bit+0x16b/0x1a70 fs/btrfs/extent-io-tree.c:1086
set_record_extent_bits+0x50/0x90 fs/btrfs/extent-io-tree.c:1821
qgroup_reserve_data+0x274/0x7d0 fs/btrfs/qgroup.c:4312
btrfs_qgroup_reserve_data+0x2e/0xb0 fs/btrfs/qgroup.c:4355
btrfs_do_encoded_write+0x92e/0x1040 fs/btrfs/inode.c:9746
btrfs_encoded_write fs/btrfs/file.c:1482 [inline]
btrfs_do_write_iter+0x280/0x610 fs/btrfs/file.c:1507
btrfs_ioctl_encoded_write+0x3d6/0x490 fs/btrfs/ioctl.c:4738
btrfs_ioctl+0x6f9/0xc90 fs/btrfs/ioctl.c:-1
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:906 [inline]
__se_sys_ioctl+0xf9/0x170 fs/ioctl.c:892
do_syscall_x64 arch/x86/entry/common.c:47 [inline]
do_syscall_64+0xbe/0x1a0 arch/x86/entry/common.c:78
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Fix this by freeing an extent changeset before returning from
btrfs_do_encoded_write().
Fixes: 7c0c7269f7b5 ("btrfs: add BTRFS_IOC_ENCODED_WRITE")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
For a crafted btrfs image, the following KASAN can be triggered when
reading an inline lzo compressed file extent:
BUG: KASAN: slab-out-of-bounds in lzo_decompress+0x57d/0x700
Read of size 4 at addr ffff888006f2e644 by task btrfs_lzo_inlin/77
Call Trace:
<TASK>
dump_stack_lvl+0x5b/0x70
print_report+0xd1/0x610
kasan_report+0xe0/0x110
__asan_report_load_n_noabort+0x13/0x20
lzo_decompress+0x57d/0x700
btrfs_decompress+0x140/0x1c0
uncompress_inline+0x147/0x1b0
btrfs_get_extent+0xb23/0x10a0
btrfs_do_readpage.constprop.0+0x538/0x1ac0
btrfs_readahead+0x32f/0x5f0
read_pages+0x16f/0x850
page_cache_ra_unbounded+0x296/0x490
do_page_cache_ra+0xd9/0x130
page_cache_sync_ra+0x3ee/0x6f0
filemap_get_pages+0x306/0x15c0
filemap_read+0x329/0xd00
btrfs_file_read_iter+0x1f8/0x2b0
vfs_read+0x4ef/0x720
ksys_read+0xf8/0x1d0
__x64_sys_read+0x71/0xb0
x64_sys_call+0x1ab0/0x1b70
do_syscall_64+0x61/0x470
entry_SYSCALL_64_after_hwframe+0x4b/0x53
</TASK>
[CAUSE]
For an inline lzo compressed file extent, there should always be one lzo
header, recording the total length of the compressed data, followed by
one segment header, recording the compressed lzo payload.
But if a crafted inline lzo compressed file extent contains only an lzo
header, without the segment header or payload, lzo_decompress() will
still try to read the segment header, causing a read beyond the item
boundary.
Furthermore if the inline lzo compressed file extent is the first item
of the leaf, it will be at the extent buffer boundary. The above
out-of-boundary read will go beyond the extent buffer boundary,
triggering the above KASAN report.
[FIX]
Validate the total length of the inlined lzo compressed file extent, to
make sure there is at least one LZO header and one segment header, and a
non-zero payload.
Fixes: a6fa6fae40ec ("btrfs: Add lzo compression support")
Assisted-by: Codex:gpt-5.5
Signed-off-by: David Lee <david.lee@trailofbits.com>
[ Rework the commit message to remove slop ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
There is a bug report that on 32bit systems (i686), btrfs crashes when
trying to do zstd compression:
BUG: unable to handle page fault for address: fffbc000
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
CPU: 0 UID: 0 PID: 61 Comm: kworker/u8:5 Tainted: G N 7.2.0-rc3-P3 #2 PREEMPTLAZY
Hardware name: LENOVO 2007F2G/2007F2G, BIOS 79ETE7WW (2.27 ) 03/21/2011
Workqueue: btrfs-delalloc btrfs_work_helper
EIP: ZSTD_compressStream2+0x221/0x5fc
Call Trace:
ZSTD_compressStream+0xd/0x48
zstd_compress_stream+0x8/0x10
zstd_compress_bio+0x20a/0x564
btrfs_compress_bio+0x94/0xc0
compress_file_range+0x20a/0x380
btrfs_work_helper+0xc1/0x1b4
process_scheduled_works+0x15f/0x204
worker_thread+0x10c/0x178
kthread+0xe1/0xe8
ret_from_fork+0x1d/0x14c
ret_from_fork_asm+0x12/0x18
entry_INT80_32+0xf0/0xf0
CR2: 00000000fffbc000
---[ end trace 0000000000000000 ]---
[CAUSE]
Inside zstd_compress_bio(), we assume the whole page cache folio can be
mapped in one go.
However that assumption is not true on systems with CONFIG_HIGHMEM, the
pages of the large folio can be in HIGHMEM, which needs to be mapped
before access.
Meanwhile zstd_compress_bio() only map the page of a large folio where
the start filepos is, the remaining pages are not mapped, and accessing
the remaining pages will trigger the above crash.
[FIX]
Do not enable large folios when the kernel has CONFIG_HIGHMEM enabled.
This is the same handling for bs > ps support.
Link: https://github.com/kdave/btrfs-progs/issues/1146
Reported-by: Erhard Furtner <erhard_f@mailbox.org>
Fixes: 9bce95edb1b4 ("btrfs: move large data folios out of experimental features")
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The bio-based drivers don't necessarily check the alignment split, and
stacking block drivers don't always handle a misalignment detected after
submitting the bio. Validate user vectors against the device's
dma_alignment as the bio is built from the iov_iter, rejecting
misaligned early with -EINVAL.
Cc: stable@vger.kernel.org
Fixes: 5ff3f74e145a ("block: simplify direct io validity check")
Fixes: 7eac33186957 ("iomap: simplify direct io validity check")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Keith Busch <kbusch@kernel.org>
Link: https://patch.msgid.link/20260720201057.1862857-6-kbusch@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Providing compatibility to various old-school Unix systems from the 1990s
was fun 25 years ago, but today it just attracts automated bug checkers
that have no point in something like this used for retro-computing.
As there as only one known users (and contributor) in the last 15 years,
drop it. Anyone who needs to access such a file system can stay on an
older kernel, or maintain the code out of tree as it doesn't require
anything fancy.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260729113643.2255419-2-hch@lst.de
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
fanotify_ioctl(FIONREAD) reports the number of bytes available to read
from the event queue. It currently accounts only FAN_EVENT_METADATA_LEN
for each queued event.
That underestimates events that carry additional information records, such
as FAN_REPORT_DFID_NAME events. A userspace program that uses FIONREAD to
size its read buffer can receive a length that is smaller than the next
event. Reading with that buffer then fails with -EINVAL, while a larger
buffer succeeds and reports a larger metadata.event_len.
Use fanotify_event_len() when summing queued events so FIONREAD includes
all info records.
Fixes: 5e469c830fdb ("fanotify: copy event fid info to user")
Signed-off-by: Yichong Chen <chenyichong@uniontech.com>
Link: https://patch.msgid.link/20260731021827.602479-1-chenyichong@uniontech.com
Signed-off-by: Jan Kara <jack@suse.cz>
|
|
fsnotify_pre_content() builds its file_range on the triggering task's
stack. fanotify_alloc_perm_event() saves a pointer to range.pos in the
heap-allocated permission event so copy_range_info_to_user() can report
the offset later.
The event reader can set the event state to FAN_EVENT_REPORTED and then
sleep while preparing the file descriptor. If a signal interrupts the
triggering task at that point, fanotify_get_response() changes the state
to FAN_EVENT_CANCELED and returns. This unwinds the file_range stack
frame while the reader still owns the event. The reader then dereferences
pevent->ppos and copies the stale stack value to userspace.
KASAN reported:
BUG: KASAN: use-after-free in fanotify_read+0x293e/0x2970
Read of size 8 at addr ffff88811434fc50 by task fanotify_inotif/95
Call Trace:
fanotify_read+0x293e/0x2970
vfs_read+0x177/0xa20
ksys_read+0xf7/0x1c0
do_syscall_64+0xf9/0x540
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Store the range position directly in the permission event and use
FANOTIFY_NO_RANGE when range information is unavailable. The event remains
alive until the reader finishes, so the reported offset no longer depends
on the triggering task's stack.
Fixes: 870499bc1d4d ("fanotify: report file range info with pre-content events")
Cc: stable@vger.kernel.org
Suggested-by: Jan Kara <jack@suse.cz>
Signed-off-by: Chengfeng Ye <nicoyip.dev@gmail.com>
Link: https://patch.msgid.link/20260730134316.2085087-1-nicoyip.dev@gmail.com
Signed-off-by: Jan Kara <jack@suse.cz>
|
|
Convert exfat iomap_ops to the new ->iomap_next() callback. Each
callback is generated with the DEFINE_IOMAP_ITER_NEXT() and
DEFINE_IOMAP_ITER_NEXT_END() macros, which wrap the iomap_iter_next()
helper to finish the previous mapping if needed and produce the next
one. No functional changes are intended.
Acked-by: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-20-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert fuse iomap_ops to the new ->iomap_next() callback. Each callback is
generated with the DEFINE_IOMAP_ITER_NEXT()/DEFINE_IOMAP_ITER_NEXT_END()
macros, which wrap the iomap_iter_next() helper to finish the previous
mapping if needed and produce the next one. No functional changes are
intended.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-19-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert hpfs iomap_ops to the new ->iomap_next() callback. The callback is
generated with the DEFINE_IOMAP_ITER_NEXT() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-18-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert gfs2 iomap_ops to the new ->iomap_next() callback. The callback is
generated with the DEFINE_IOMAP_ITER_NEXT_END() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-17-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert f2fs iomap_ops to the new ->iomap_next() callback. The callback is
generated with the DEFINE_IOMAP_ITER_NEXT() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-16-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert ext2 iomap_ops to the new ->iomap_next() callback. The callback is
generated with the DEFINE_IOMAP_ITER_NEXT_END() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-14-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert zonefs iomap_ops to the new ->iomap_next() callback. Each
callback is generated with the DEFINE_IOMAP_ITER_NEXT() macro, which
wraps the iomap_iter_next() helper to finish the previous mapping if
needed and produce the next one. No functional changes are intended.
Acked-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-13-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert erofs iomap_ops to the new ->iomap_next() callback. Each
callback is generated with the DEFINE_IOMAP_ITER_NEXT() and
DEFINE_IOMAP_ITER_NEXT_END() macros, which wrap the iomap_iter_next()
helper to finish the previous mapping if needed and produce the next
one. No functional changes are intended.
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-12-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert ext4 iomap_ops to the new ->iomap_next() callback. Each callback is
generated with the DEFINE_IOMAP_ITER_NEXT() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-11-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert ntfs iomap_ops to the new ->iomap_next() callback. Each callback is
generated with the DEFINE_IOMAP_ITER_NEXT()/DEFINE_IOMAP_ITER_NEXT_END()
macros, which wrap the iomap_iter_next() helper to finish the previous
mapping if needed and produce the next one. No functional changes are
intended.
Acked-by: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-10-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert ntfs3 iomap_ops to the new ->iomap_next() callback. The callback is
generated with the DEFINE_IOMAP_ITER_NEXT_END() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-9-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert btrfs iomap_ops to the new ->iomap_next() callback. The callback is
generated with the DEFINE_IOMAP_ITER_NEXT_END() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Acked-by: David Sterba <dsterba@suse.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-8-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Convert xfs iomap_ops to the new ->iomap_next() callback. Each callback is
generated with the DEFINE_IOMAP_ITER_NEXT()/DEFINE_IOMAP_ITER_NEXT_END()
macros, which wrap the iomap_iter_next() helper to finish the previous
mapping if needed and produce the next one. No functional changes are
intended.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-7-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Have one ->iomap_next() callback instead of ->iomap_begin() and
->iomap_end(). ->iomap_next() finishes the previous mapping if needed,
and produces the next mapping.
Collapsing to a single callback lets a performance-critical caller
inline its iteration loop and pass its ->iomap_next() function as a
compile-time constant, so the compiler can devirtualize that callback
into a direct call instead of an indirect call through a function
pointer.
iomap_iter() uses ->iomap_next() when the filesystem provides that
callback and otherwise falls back to the ->iomap_begin()/->iomap_end()
path, so filesystems can be converted one at a time.
Suggested-by: Christoph Hellwig <hch@lst.de>
Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-6-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
For non-blocking iocbs we should avoid blocking allocation where
possible, so switch to a GFP_NOWAIT allocation here.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Fengnan Chang <changfengnan@bytedance.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260729192737.3190206-5-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
The pending iomap_iter_next conversion creates performance issues for the
new simple direct I/O read fast path, because it assumes a model where
the iterator must be advanced at the end, which the direct I/O read fast
path tries to avoid.
Side step this by splitting the simple path from iomap_dio_rw, and
require the file systems to call into it explicitly, and pass only a
->begin callback. This allows to drop various checks for incompatible
features while creating a requirement for the file system to only call
the simple path for cases that it can handle.
As a side-benefit we can now inline the initial part of the simple
direct I/O read fast path and let the compiler convert the indirect call
to ->begin into a direct call.
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Fengnan Chang <changfengnan@bytedance.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260729192737.3190206-4-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
In preparation for changing iomap to use an in-iter (->iomap_next())
model, move the iomap_iter() logic out into the new iomap_iter_next()
helper function.
iomap_iter_next() is added as an inlined helper so it can be called
directly by ->iomap_next() implementations where the begin()/end()
callbacks can be direct calls.
The DEFINE_IOMAP_ITER_NEXT() and DEFINE_IOMAP_ITER_NEXT_END() macros are
also provided to generate the boilerplate ->iomap_next() wrapper
functions that simply forward to iomap_iter_next() with the appropriate
begin/end callbacks. DEFINE_IOMAP_ITER_NEXT() is for the common case
where there is no end() callback. DEFINE_IOMAP_ITER_NEXT_END() is for
the case where there is an explicit end() callback.
No functional change intended. The one would-be behavioral difference is
that on the iomap_end() error path (ret < 0 && !advanced), the old code
returned with iter.status left as the caller's last value whereas the
new code zeroes it, but this is not observable in practice as there are
no in-tree callers that read iter.status after the iteration loop.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Fengnan Chang <changfengnan@bytedance.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-3-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
A sashiko review of an unrelated patch points out that the folio
batch mechanism used for iomap zero range fails to release the batch
in a couple error scenarios. If either calls to ->iomap_end() or
->iomap_begin() fail, the direct return paths bypass the batch
cleanup.
The ->iomap_end() case is not a practical issue at the moment
because there is no user of the mechanism that returns an error from
this path. The ->iomap_begin() case is theoretically possible
because XFS can invoke the fill helper and error out at various
points thereafter. This subtly complicates things because XFS does
not transfer iomap_flags to the iomap data structure in the error
path.
To deal with both of these issues, first make sure to invoke the
cleanup helper in the error path for either fs callback. Second,
update the helper to clear the flag unconditionally and release the
batch so long as it is populated. This more clearly delineates the
purpose of the flag to control the I/O path and not necessarily the
status of the fbatch, so add a comment around this as well.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 395ed1ef0012 ("iomap: optional zero range dirty folio processing")
Signed-off-by: Brian Foster <bfoster@redhat.com>
Link: https://patch.msgid.link/20260729192737.3190206-2-joannelkoong@gmail.com
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
When fuse enables large folios, a large folio will be backed by
iomap_folio_state that keeps track of uptodate and dirty state in an
internal bitmap.
Fuse writethrough and notify store paths currently set folio uptodate
state with folio_mark_uptodate(), which touches only the folio-level
flag, but on an iomap-backed folio, that leaves the uptodate bitmap out
of sync.
Use the iomap_folio_mark_uptodate() helper to update both the folio
uptodate state and the iomap uptodate bitmap.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260707220450.1200943-4-joannelkoong@gmail.com
Acked-by: Miklos Szeredi <mszeredi@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Add an exported helper iomap_folio_mark_uptodate() to mark a folio as
uptodate and update its uptodate bitmap if the folio has iomap state
data attached.
This is needed because there are some filesystems (eg fuse) that have
paths outside of conventional iomap calls that need to mark a folio as
uptodate (eg writing server-pushed data directly into the page cache)
and need the iomap-internal uptodate bitmap to be in sync with the
uptodate state of the folio.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260707220450.1200943-3-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
In the writethrough path (fuse_send_write_pages()), if the write to the
server failed or was a short write, the uptodate flag on the folios are
cleared.
As explained by Matthew in [1], this is dangerous because the folio may
be mapped into userspace. The mm code has the invariant that a
non-uptodate folio must never be visible to userspace (to avoid
potentially leaking confidental information to userspace) and has checks
in place for this that if violated can bring down the whole machine.
Practically speaking, the effect of this change for the fuse
writethrough error path is that if an application does a write and then
the server fails to persist the data or only services a short write, the
page cache folio keeps the data the application wrote instead of being
reverted to the server's contents on the next read. The failure is still
reported to the application synchronously through the short count /
error return of the write() syscall. Folios that were only partially
written are unaffected since they were never marked uptodate in the
first place (fuse_fill_write_page() only marks a folio as uptodate if
the whole folio was written to).
[1] https://lore.kernel.org/linux-fsdevel/ajtPMgO65FA1TXhi@casper.infradead.org/
Suggested-by: Matthew Wilcox <willy@infradead.org>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260707220450.1200943-2-joannelkoong@gmail.com
Acked-by: Miklos Szeredi <mszeredi@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
When running 4K random read workloads on high-performance Gen5 NVMe
SSDs, the software overhead in the iomap direct I/O path
(__iomap_dio_rw) becomes a significant bottleneck.
Using io_uring with poll mode for a 4K randread test on a raw block
device:
taskset -c 30 ./t/io_uring -p1 -d512 -b4096 -s32 -c32 -F1 -B1 -R1 -X1
-n1 -P1 /dev/nvme10n1
Result: ~3.2M IOPS
Running the exact same workload on ext4 and XFS:
taskset -c 30 ./t/io_uring -p1 -d512 -b4096 -s32 -c32 -F1 -B1 -R1 -X1
-n1 -P1 /mnt/testfile
Result: ~1.92M IOPS
Profiling the ext4 workload reveals that a significant portion of CPU
time is spent on memory allocation and the iomap state machine
iteration:
5.33% [kernel] [k] __iomap_dio_rw
3.26% [kernel] [k] iomap_iter
2.37% [kernel] [k] iomap_dio_bio_iter
2.35% [kernel] [k] kfree
1.33% [kernel] [k] iomap_dio_complete
Introduce a simple dio path to reduce the overhead of iomap. It is
triggered when the request satisfies all of:
- a READ request whose I/O size is <= inode blocksize (fits in a single
block, no splits);
- no custom iomap_dio_ops (dops) registered by the filesystem;
- no caller-accumulated residual (done_before == 0);
- none of IOMAP_DIO_FORCE_WAIT / IOMAP_DIO_PARTIAL / IOMAP_DIO_BOUNCE
set, the range is within i_size, and the inode is not encrypted.
The bio is allocated from a dedicated bioset whose front_pad embeds
struct iomap_dio_simple, so the whole request lives in a single
cacheline-aligned allocation and no separate struct iomap_dio is
needed. Completion is handled inline from ->bi_end_io for the common
success case, and only punted to the s_dio_done_wq workqueue on error.
After this optimization, the heavy generic functions disappear from the
profile, replaced by a single streamlined execution path:
4.83% [kernel] [k] iomap_dio_simple
With this patch, 4K random read IOPS on ext4 increases from 1.92M to
2.19M in the original single-core io_uring poll-mode workload.
Below are the test results using fio:
fs workload qd simple=0 simple=1 gain
ext4 libaio 1 18,740 18,761 +0.11%
ext4 libaio 64 462,850 480,587 +3.83%
ext4 libaio 128 459,498 478,824 +4.21%
ext4 libaio 256 459,938 480,156 +4.40%
ext4 io_uring 1 18,836 18,880 +0.24%
ext4 io_uring 64 568,193 600,625 +5.71%
ext4 io_uring 128 570,998 602,148 +5.46%
ext4 io_uring 256 572,052 602,536 +5.33%
ext4 io_uring_poll 1 19,283 19,272 -0.06%
ext4 io_uring_poll 64 989,735 1,013,342 +2.39%
ext4 io_uring_poll 128 1,467,336 1,538,444 +4.85%
ext4 io_uring_poll 256 1,663,498 1,830,842 +10.06%
xfs libaio 1 18,764 18,776 +0.06%
xfs libaio 64 462,408 480,860 +3.99%
xfs libaio 128 461,280 480,819 +4.24%
xfs libaio 256 461,626 480,190 +4.02%
xfs io_uring 1 18,871 18,903 +0.17%
xfs io_uring 64 570,383 597,399 +4.74%
xfs io_uring 128 568,290 597,370 +5.12%
xfs io_uring 256 570,616 598,775 +4.93%
xfs io_uring_poll 1 19,211 19,315 +0.54%
xfs io_uring_poll 64 989,726 1,008,455 +1.89%
xfs io_uring_poll 128 1,430,426 1,513,064 +5.78%
xfs io_uring_poll 256 1,587,339 1,742,220 +9.76%
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Link: https://patch.msgid.link/20260701033253.46420-4-changfengnan@bytedance.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Change should_report_dio_fserror() to take an error code instead of the
full struct iomap_dio, decoupling it for reuse by the upcoming simple
direct I/O path.
No functional change.
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Link: https://patch.msgid.link/20260701033253.46420-3-changfengnan@bytedance.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Extract the alignment computation from iomap_dio_bio_iter() into a
standalone helper so the upcoming simple direct I/O path can reuse it
without requiring a struct iomap_dio.
No functional change.
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Link: https://patch.msgid.link/20260701033253.46420-2-changfengnan@bytedance.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
