From f74e6dff5440f662bced614b723a7d8f2b05919d Mon Sep 17 00:00:00 2001 From: Sergey Senozhatsky Date: Thu, 30 Jul 2026 16:51:46 +0900 Subject: Documentation: zram: correct algo parameters configuration documentation zram has always reset all previously set parameters for the given algorithm in comp_params_store(). Make documentation more clear and explicitly state that all relevant/necessary parameters should be set in one configuration write. Link: https://lore.kernel.org/20260730075158.1339787-1-senozhatsky@chromium.org Signed-off-by: Sergey Senozhatsky Cc: Jonathan Corbet Cc: Minchan Kim Signed-off-by: Andrew Morton --- Documentation/admin-guide/blockdev/zram.rst | 34 ++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst index 2f6bbfd991fe..148b7cf3b924 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -109,14 +109,41 @@ path to the `dict` along with other parameters:: #pass path to pre-trained zstd dictionary echo "algo=zstd dict=/etc/dictionary" > /sys/block/zram0/algorithm_params + #pass path to pre-trained zstd dictionary and compression level + echo "algo=zstd level=8 dict=/etc/dictionary" > \ + /sys/block/zram0/algorithm_params + #same, but using algorithm priority + echo "algo=zstd priority=1" > /sys/block/zram0/recomp_algorithm echo "priority=1 dict=/etc/dictionary" > \ /sys/block/zram0/algorithm_params - #pass path to pre-trained zstd dictionary and compression level +Each write to `algorithm_params` replaces the entire set of parameters of +the corresponding algorithm, parameters that are not listed in the write +are reset to their default values. Configure all of the parameters of an +algorithm in one write:: + + #WRONG: the second write resets level back to its default value + echo "algo=zstd level=8" > /sys/block/zram0/algorithm_params + echo "algo=zstd dict=/etc/dictionary" > /sys/block/zram0/algorithm_params + + #RIGHT echo "algo=zstd level=8 dict=/etc/dictionary" > \ /sys/block/zram0/algorithm_params +Select the compression algorithm before configuring its parameters. The +parameters of one algorithm are not necessarily valid for another one, so +changing the algorithm of a particular priority resets that priority's +parameters:: + + #WRONG: comp_algorithm write resets the previously configured level + echo "level=8" > /sys/block/zram0/algorithm_params + echo zstd > /sys/block/zram0/comp_algorithm + + #RIGHT + echo zstd > /sys/block/zram0/comp_algorithm + echo "algo=zstd level=8" > /sys/block/zram0/algorithm_params + Parameters are algorithm specific: not all algorithms support pre-trained dictionaries, not all algorithms support `level`. Furthermore, for certain algorithms `level` controls the compression level (the higher the value the @@ -124,6 +151,11 @@ better the compression ratio, it even can take negatives values for some algorithms), for other algorithms `level` is acceleration level (the higher the value the lower the compression ratio). +Parameters are handed over to the compression algorithm when the device is +initialised, hence invalid parameters (or parameters that the selected +algorithm does not support) are reported by the `disksize` write, and not +by the `algorithm_params` write that has configured them. + Set Disksize ============ -- cgit From 7a39f03bc9da3499c2423758f353ab72d23faa16 Mon Sep 17 00:00:00 2001 From: Pratyush Mallick Date: Fri, 31 Jul 2026 19:37:05 +0000 Subject: mm/page_reporting: add page_reporting_delay_ms module parameter Free page reporting currently hardcodes a 2-second interval between reports. This rigid delay cannot accommodate diverse guest workloads. This patch introduces a module parameter, page_reporting_delay_ms (default: 2000), allowing users to tune the reporting rate: - Lower values enable aggressive memory reclamation by returning unused pages to the host immediately. - Higher values help batch pages during spiky allocation/free churn, reducing hypercalls and nested page fault overheads. Setting the delay to 0 is safe and execution is strictly gated by: - reporting is only triggered by high-order page frees. - expensive hypercalls are bounded by a slot capacity watermark check before proceeding. Link: https://lore.kernel.org/20260731193705.2902728-1-pratmal@google.com Signed-off-by: Pratyush Mallick Reviewed-by: SJ Park Acked-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Cc: Anshuman Khandual Cc: Brendan Jackman Cc: Greg Thelen Cc: Johannes Weiner Cc: Jonathan Corbet Cc: Liam R. Howlett Cc: Michal Hocko Cc: Mike Rapoport Cc: SeongJae Park Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Zi Yan Signed-off-by: Andrew Morton --- Documentation/admin-guide/kernel-parameters.txt | 6 ++++++ mm/page_reporting.c | 28 ++++++++++++++++--------- 2 files changed, 24 insertions(+), 10 deletions(-) (limited to 'Documentation') diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index b5493a7f8f22..364c2dce8e70 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4810,6 +4810,12 @@ Kernel parameters Adjust the minimal page reporting order. The page reporting is disabled when it exceeds MAX_PAGE_ORDER. + page_reporting.page_reporting_delay_ms= + [KNL] Free page reporting delay in milliseconds + Format: + Adjust the delay in milliseconds between free page + reporting intervals. Default is 2000 (2 seconds). + panic= [KNL] Kernel behaviour on panic: delay timeout > 0: seconds before rebooting timeout = 0: wait forever diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 1cce8729696e..de587be17801 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -48,7 +48,11 @@ MODULE_PARM_DESC(page_reporting_order, "Set page reporting order"); */ EXPORT_SYMBOL_GPL(page_reporting_order); -#define PAGE_REPORTING_DELAY (2 * HZ) +static unsigned int page_reporting_delay_ms = 2 * MSEC_PER_SEC; +module_param(page_reporting_delay_ms, uint, 0644); +MODULE_PARM_DESC(page_reporting_delay_ms, + "Set page reporting delay in milliseconds"); + static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly; enum { @@ -57,6 +61,13 @@ enum { PAGE_REPORTING_ACTIVE }; +/* schedule work for page reporting */ +static void page_reporting_schedule_work(struct page_reporting_dev_info *prdev) +{ + queue_delayed_work(system_freezable_wq, &prdev->work, + msecs_to_jiffies(page_reporting_delay_ms)); +} + /* request page reporting */ static void __page_reporting_request(struct page_reporting_dev_info *prdev) @@ -77,12 +88,10 @@ __page_reporting_request(struct page_reporting_dev_info *prdev) return; /* - * Delay the start of work to allow a sizable queue to build. For - * now we are limiting this to running no more than once every - * couple of seconds. + * Delay the start of work to allow a sizable queue to build. + * We limit this based on page_reporting_delay_ms. */ - queue_delayed_work(system_freezable_wq, &prdev->work, - PAGE_REPORTING_DELAY); + page_reporting_schedule_work(prdev); } /* notify prdev of free page reporting request */ @@ -337,13 +346,12 @@ static void page_reporting_process(struct work_struct *work) err_out: /* * If the state has reverted back to requested then there may be - * additional pages to be processed. We will defer for 2s to allow - * more pages to accumulate. + * additional pages to be processed. We will defer by + * page_reporting_delay_ms to allow more pages to accumulate. */ state = atomic_cmpxchg(&prdev->state, state, PAGE_REPORTING_IDLE); if (state == PAGE_REPORTING_REQUESTED) - queue_delayed_work(system_freezable_wq, &prdev->work, - PAGE_REPORTING_DELAY); + page_reporting_schedule_work(prdev); } static DEFINE_MUTEX(page_reporting_mutex); -- cgit From c310a8932a3107c9bc8f01d473e9d085f8aa9c98 Mon Sep 17 00:00:00 2001 From: Eric Biggers Date: Mon, 3 Aug 2026 11:04:26 -0700 Subject: mm/swap: reject swapon() on filesystem-level encrypted files ext4 and f2fs don't prevent filesystem-level encrypted files from being set up directly as swap files. In this case, encryption is bypassed. No one should be doing this, vs. the methods of encrypted swap that actually do work (such as swapping to a dm-crypt device, or swapping to a loopback device on top of a filesystem-level encrypted file). Nevertheless, to prevent user error, make swapon() explicitly reject this case. Document this behavior in fscrypt.rst as well. Link: https://lore.kernel.org/20260803180426.3123-1-ebiggers@kernel.org Fixes: 9bd8212f981e ("ext4 crypto: add encryption policy and password salt support") Fixes: f424f664f0e8 ("f2fs crypto: add encryption policy and password salt support") Signed-off-by: Eric Biggers Reviewed-by: Baoquan He Reviewed-by: Muhammad Usama Anjum Reviewed-by: "Darrick J. Wong" Cc: Barry Song Cc: Chris Li Cc: Kairui Song Cc: Kemeng Shi Cc: Nhat Pham Cc: Signed-off-by: Andrew Morton --- Documentation/filesystems/fscrypt.rst | 4 ++++ mm/swapfile.c | 7 +++++++ 2 files changed, 11 insertions(+) (limited to 'Documentation') diff --git a/Documentation/filesystems/fscrypt.rst b/Documentation/filesystems/fscrypt.rst index c0dd35f1af12..e4882b73120e 100644 --- a/Documentation/filesystems/fscrypt.rst +++ b/Documentation/filesystems/fscrypt.rst @@ -1238,6 +1238,10 @@ astute users may notice some differences in behavior: - DAX (Direct Access) is not supported on encrypted files. +- Encrypted files cannot be used directly as swap files. To swap to + an encrypted file, set up a loopback device on top of it. + Alternatively, encrypted swap can use a dm-crypt device. + - The maximum length of an encrypted symlink is 2 bytes shorter than the maximum length of an unencrypted symlink. For example, on an EXT4 filesystem with a 4K block size, unencrypted symlinks can be up diff --git a/mm/swapfile.c b/mm/swapfile.c index 4e07d457e261..d7f749ad60c2 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3668,6 +3668,13 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) error = -EBUSY; goto bad_swap_unlock_inode; } + if (IS_ENCRYPTED(inode)) { + pr_warn_once( + "Filesystem-level encrypted swapfile '%s' is unsupported. Create a loop device over it, or use dm-crypt\n", + name->name); + error = -EINVAL; + goto bad_swap_unlock_inode; + } /* * The swap subsystem needs a major overhaul to support this. -- cgit From a44ab4bd1ec0432d686c25f9c160379ffa1e694c Mon Sep 17 00:00:00 2001 From: Hongfu Li Date: Wed, 5 Aug 2026 18:59:07 +0800 Subject: ksm: update comments and docs to reference folio->mapping The KSM code already stores and checks the stable node key via folio->mapping, but the comment in ksm_get_folio() and the reverse mapping documentation in ksm.rst still refer to page->mapping. This is a pure wording update to match the folio-based implementation. No functional change is intended. Link: https://lore.kernel.org/20260805105927.41987-1-hongfu.li@linux.dev Signed-off-by: Hongfu Li Acked-by: David Hildenbrand (Arm) Reviewed-by: Xu Xin Reviewed-by: Dongliang Mu Cc: Alex Shi Cc: Chengming Zhou Cc: David Hildenbrand Cc: Jonathan Corbet Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Yanteng Si Signed-off-by: Andrew Morton --- Documentation/mm/ksm.rst | 4 ++-- Documentation/translations/zh_CN/mm/ksm.rst | 4 ++-- mm/ksm.c | 7 +++---- 3 files changed, 7 insertions(+), 8 deletions(-) (limited to 'Documentation') diff --git a/Documentation/mm/ksm.rst b/Documentation/mm/ksm.rst index 2806e3e4a10e..2b4f72f1f953 100644 --- a/Documentation/mm/ksm.rst +++ b/Documentation/mm/ksm.rst @@ -24,13 +24,13 @@ tree. If a KSM page is shared between less than ``max_page_sharing`` VMAs, the node of the stable tree that represents such KSM page points to a -list of struct ksm_rmap_item and the ``page->mapping`` of the +list of struct ksm_rmap_item and the ``folio->mapping`` of the KSM page points to the stable tree node. When the sharing passes this threshold, KSM adds a second dimension to the stable tree. The tree node becomes a "chain" that links one or more "dups". Each "dup" keeps reverse mapping information for a KSM -page with ``page->mapping`` pointing to that "dup". +page with ``folio->mapping`` pointing to that "dup". Every "chain" and all "dups" linked into a "chain" enforce the invariant that they represent the same write protected memory content, diff --git a/Documentation/translations/zh_CN/mm/ksm.rst b/Documentation/translations/zh_CN/mm/ksm.rst index f0f458753d0c..822c7a289671 100644 --- a/Documentation/translations/zh_CN/mm/ksm.rst +++ b/Documentation/translations/zh_CN/mm/ksm.rst @@ -31,10 +31,10 @@ KSM维护着稳定树中的KSM页的逆映射信息。 当KSM页面的共享数小于 ``max_page_sharing`` 的虚拟内存区域(VMAs)时，则代表了 KSM页的稳定树其中的节点指向了一个ksm_rmap_item结构体类型的列表。同时，这个KSM页 -的 ``page->mapping`` 指向了该稳定树节点。 +的 ``folio->mapping`` 指向了该稳定树节点。 如果共享数超过了阈值，KSM将给稳定树添加第二个维度。稳定树就变成链接一个或多 -个稳定树"副本"的"链"。每个副本都保留KSM页的逆映射信息，其中 ``page->mapping`` +个稳定树"副本"的"链"。每个副本都保留KSM页的逆映射信息，其中 ``folio->mapping`` 指向该"副本"。 每个链以及链接到该链中的所有"副本"强制不变的是，它们代表了相同的写保护内存 diff --git a/mm/ksm.c b/mm/ksm.c index 14dd6a6e8e6d..49d48d1e0998 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -959,10 +959,9 @@ enum ksm_get_folio_flags { * seconds or even minutes: much too unresponsive. So instead we use a * "keyhole reference": access to the ksm page from the stable node peeps * out through its keyhole to see if that page still holds the right key, - * pointing back to this stable node. This relies on freeing a PageAnon - * page to reset its page->mapping to NULL, and relies on no other use of - * a page to put something that might look like our key in page->mapping. - * is on its way to being freed; but it is an anomaly to bear in mind. + * pointing back to this stable node. This relies on freeing an anon + * folio to reset its mapping to NULL, and relies on no other use of a + * folio to put something that might look like our key in its mapping. */ static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, enum ksm_get_folio_flags flags) -- cgit From afff109c2f8b35b88ea783d345c1067a311a57d8 Mon Sep 17 00:00:00 2001 From: Abhishek Bapat Date: Wed, 5 Aug 2026 17:29:52 +0000 Subject: alloc_tag: expose boot-time compression configuration Currently, userspace has limited visibility into the exact active runtime state of memory allocation profiling and its page extension compression ('sysctl.vm.mem_profiling={0|1|never}[,compressed]'). While reading the sysctl provides basic on/off status, it is currently impossible for userspace to natively determine whether page-tag compression was successfully enabled without scraping dmesg boot logs. Add a new read-only sysctl representing how compression was configured at boot time. Link: https://lore.kernel.org/c795f8089f82841e8a6e00d7ca286da2b23aeb7b.1785950530.git.abhishekbapat@google.com Signed-off-by: Abhishek Bapat Acked-by: Suren Baghdasaryan Cc: Hao Ge Signed-off-by: Andrew Morton --- Documentation/mm/allocation-profiling.rst | 11 +++++++++++ mm/alloc_tag.c | 6 ++++++ 2 files changed, 17 insertions(+) (limited to 'Documentation') diff --git a/Documentation/mm/allocation-profiling.rst b/Documentation/mm/allocation-profiling.rst index 5389d241176a..e928aa3e4e1e 100644 --- a/Documentation/mm/allocation-profiling.rst +++ b/Documentation/mm/allocation-profiling.rst @@ -43,6 +43,17 @@ sysctl: warnings produced by allocations made while profiling is disabled and freed when it's enabled. + /proc/sys/vm/mem_profiling_compressed + + 1: Page alloc tag compression is enabled. + + 0: Page alloc tag compression is disabled. + + This reflects a static boot-time configuration of how page allocation tags are + stored (in page flags when compression is enabled and in page_ext when disabled). + Toggling ``mem_profiling`` at runtime does not change the state of + ``mem_profiling_compressed``. + Runtime info: /proc/allocinfo diff --git a/mm/alloc_tag.c b/mm/alloc_tag.c index e93e7fec1f06..b60ee89704cc 100644 --- a/mm/alloc_tag.c +++ b/mm/alloc_tag.c @@ -961,6 +961,12 @@ static const struct ctl_table memory_allocation_profiling_sysctls[] = { .mode = 0644, .proc_handler = proc_mem_profiling_handler, }, + { + .procname = "mem_profiling_compressed", + .data = &mem_profiling_compressed, + .mode = 0444, + .proc_handler = proc_do_static_key, + }, }; static void __init sysctl_init(void) -- cgit From 34e0849142c317eed68f3a4818dd3808fb1c47db Mon Sep 17 00:00:00 2001 From: Sourav Panda Date: Fri, 7 Aug 2026 04:00:03 +0000 Subject: mm/hugetlb_cma: support percentage-based hugetlb_cma reservation Currently, hugetlb_cma reservation only supports absolute sizes (e.g., hugetlb_cma=2G or hugetlb_cma=0:1G,1:1G). This can be restrictive in heterogeneous environments or when deploying common kernel command lines across machines with different memory capacities. Add support for percentage-based hugetlb_cma reservation (e.g., hugetlb_cma=20% or hugetlb_cma=0:20%,1:10%). The percentage is calculated against the total memory (for global settings) or against the node-specific memory (for node-specific settings) using memblock APIs during early boot. Link: https://lore.kernel.org/20260807040003.2156630-1-souravpanda@google.com Signed-off-by: Sourav Panda Acked-by: Usama Arif Cc: David Hildenbrand Cc: David Rientjes Cc: Frank van der Linden Cc: Greg Thelen Cc: Muchun Song Cc: Oscar Salvador Cc: Suren Baghdasaryan Signed-off-by: Andrew Morton --- Documentation/admin-guide/kernel-parameters.txt | 10 +- mm/hugetlb_cma.c | 142 ++++++++++++++++++++++-- 2 files changed, 142 insertions(+), 10 deletions(-) (limited to 'Documentation') diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 364c2dce8e70..1af62cd16c9d 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2064,8 +2064,14 @@ Kernel parameters hugetlb_cma= [HW,CMA,EARLY] The size of a CMA area used for allocation of gigantic hugepages. Or using node format, the size of a CMA area per node can be specified. - Format: nn[KMGTPE] or (node format) - :nn[KMGTPE][,:nn[KMGTPE]] + The size can be an absolute value (e.g., 2G) or a + percentage of the total memory or node memory (e.g., 20%). + Percentage-derived sizes are rounded down to a multiple of + the architecture's gigantic hugepage size and may become + zero. + Format: nn[KMGTPE] or nn% or (node format) + :nn[KMGTPE][,:nn[KMGTPE]] or + :nn%[,:nn%] The size must be a multiple of the gigantic page size. When using node format, this applies to each per-node size. diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c index 4dfce68b354a..db0680e82847 100644 --- a/mm/hugetlb_cma.c +++ b/mm/hugetlb_cma.c @@ -9,6 +9,9 @@ #include #include +#include +#include +#include #include "internal.h" #include "hugetlb_cma.h" @@ -18,6 +21,28 @@ static unsigned long hugetlb_cma_size_in_node[MAX_NUMNODES] __initdata; static bool hugetlb_cma_only __ro_after_init; static unsigned long hugetlb_cma_size __ro_after_init; +static unsigned int hugetlb_cma_percent __initdata; +static unsigned int hugetlb_cma_percent_in_node[MAX_NUMNODES] __initdata; + +#ifdef CONFIG_NUMA +static phys_addr_t __init memblock_node_memory_size(int nid) +{ + struct memblock_region *reg; + phys_addr_t size = 0; + + for_each_mem_region(reg) { + if (reg->nid == nid) + size += reg->size; + } + return size; +} +#else +static phys_addr_t __init memblock_node_memory_size(int nid) +{ + return memblock_phys_mem_size(); +} +#endif + void hugetlb_cma_free_frozen_folio(struct folio *folio) { WARN_ON_ONCE(!cma_release_frozen(hugetlb_cma[folio_nid(folio)], @@ -90,14 +115,31 @@ static int __init cmdline_parse_hugetlb_cma(char *p) break; if (s[count] == ':') { + char *next; + if (tmp >= MAX_NUMNODES) break; nid = array_index_nospec(tmp, MAX_NUMNODES); + hugetlb_cma_size = 0; + hugetlb_cma_percent = 0; + s += count + 1; - tmp = memparse(s, &s); - hugetlb_cma_size_in_node[nid] = tmp; - hugetlb_cma_size += tmp; + tmp = memparse(s, &next); + if (*next == '%') { + if (tmp > 100) { + pr_warn("hugetlb_cma: invalid percentage %lu for node %d\n", + tmp, nid); + break; + } + hugetlb_cma_percent_in_node[nid] = tmp; + hugetlb_cma_size_in_node[nid] = 0; + s = next + 1; + } else { + hugetlb_cma_size_in_node[nid] = tmp; + hugetlb_cma_percent_in_node[nid] = 0; + s = next; + } /* * Skip the separator if have one, otherwise @@ -108,7 +150,28 @@ static int __init cmdline_parse_hugetlb_cma(char *p) else break; } else { - hugetlb_cma_size = memparse(p, &p); + char *next; + + tmp = memparse(p, &next); + if (*next == '%') { + if (tmp > 100) { + pr_warn("hugetlb_cma: invalid percentage %lu\n", tmp); + } else { + hugetlb_cma_percent = tmp; + hugetlb_cma_size = 0; + for (nid = 0; nid < MAX_NUMNODES; nid++) { + hugetlb_cma_size_in_node[nid] = 0; + hugetlb_cma_percent_in_node[nid] = 0; + } + } + } else { + hugetlb_cma_size = tmp; + hugetlb_cma_percent = 0; + for (nid = 0; nid < MAX_NUMNODES; nid++) { + hugetlb_cma_size_in_node[nid] = 0; + hugetlb_cma_percent_in_node[nid] = 0; + } + } break; } } @@ -134,8 +197,36 @@ void __init hugetlb_cma_reserve(void) { unsigned long size, reserved, per_node, order, gigantic_page_size; bool node_specific_cma_alloc = false; + bool has_node_specific_param = false; int nid; + for (nid = 0; nid < MAX_NUMNODES; nid++) { + if (hugetlb_cma_size_in_node[nid] || hugetlb_cma_percent_in_node[nid]) { + has_node_specific_param = true; + break; + } + } + + if (has_node_specific_param) { + hugetlb_cma_size = 0; + for (nid = 0; nid < MAX_NUMNODES; nid++) { + if (hugetlb_cma_percent_in_node[nid]) { + phys_addr_t node_gfp_mem = memblock_node_memory_size(nid); + u64 s; + + s = mul_u64_u32_div((u64)node_gfp_mem, + hugetlb_cma_percent_in_node[nid], + 100); + + hugetlb_cma_size_in_node[nid] = s; + } + hugetlb_cma_size += hugetlb_cma_size_in_node[nid]; + } + } else if (hugetlb_cma_percent) { + hugetlb_cma_size = mul_u64_u32_div((u64)memblock_phys_mem_size(), + hugetlb_cma_percent, 100); + } + if (!hugetlb_cma_size) return; @@ -154,6 +245,32 @@ void __init hugetlb_cma_reserve(void) VM_WARN_ON(order <= MAX_PAGE_ORDER); gigantic_page_size = PAGE_SIZE << order; + if (hugetlb_cma_percent) { + unsigned long orig_size = hugetlb_cma_size; + + hugetlb_cma_size = ALIGN_DOWN(hugetlb_cma_size, PAGE_SIZE << order); + if (orig_size && !hugetlb_cma_size) + pr_warn("hugetlb_cma: reservation size rounded down to 0 from %lu MiB (%u%%)\n", + orig_size / SZ_1M, hugetlb_cma_percent); + } else if (has_node_specific_param) { + hugetlb_cma_size = 0; + for (nid = 0; nid < MAX_NUMNODES; nid++) { + if (hugetlb_cma_percent_in_node[nid]) { + unsigned long orig_size = hugetlb_cma_size_in_node[nid]; + + hugetlb_cma_size_in_node[nid] = + ALIGN_DOWN(hugetlb_cma_size_in_node[nid], + PAGE_SIZE << order); + if (orig_size && !hugetlb_cma_size_in_node[nid]) + pr_warn("hugetlb_cma: reservation size rounded down to 0 from %lu MiB (%u%%) on node %d\n", + orig_size / SZ_1M, + hugetlb_cma_percent_in_node[nid], + nid); + } + hugetlb_cma_size += hugetlb_cma_size_in_node[nid]; + } + } + hugetlb_bootmem_set_nodes(); for (nid = 0; nid < MAX_NUMNODES; nid++) { @@ -194,8 +311,13 @@ void __init hugetlb_cma_reserve(void) per_node = DIV_ROUND_UP(hugetlb_cma_size, nodes_weight(hugetlb_bootmem_nodes)); per_node = round_up(per_node, gigantic_page_size); - pr_info("hugetlb_cma: reserve %lu MiB, up to %lu MiB per node\n", - hugetlb_cma_size / SZ_1M, per_node / SZ_1M); + if (hugetlb_cma_percent) + pr_info("hugetlb_cma: reserve %lu MiB (%u%%), up to %lu MiB per node\n", + hugetlb_cma_size / SZ_1M, hugetlb_cma_percent, + per_node / SZ_1M); + else + pr_info("hugetlb_cma: reserve %lu MiB, up to %lu MiB per node\n", + hugetlb_cma_size / SZ_1M, per_node / SZ_1M); } reserved = 0; @@ -230,8 +352,12 @@ void __init hugetlb_cma_reserve(void) } reserved += size; - pr_info("hugetlb_cma: reserved %lu MiB on node %d\n", - size / SZ_1M, nid); + if (hugetlb_cma_percent_in_node[nid]) + pr_info("hugetlb_cma: reserved %lu MiB (%u%%) on node %d\n", + size / SZ_1M, hugetlb_cma_percent_in_node[nid], nid); + else + pr_info("hugetlb_cma: reserved %lu MiB on node %d\n", + size / SZ_1M, nid); if (reserved >= hugetlb_cma_size) break; -- cgit From 1d581ab2348cdbb6d4d0a467382926b68e374ec9 Mon Sep 17 00:00:00 2001 From: Suren Baghdasaryan Date: Wed, 8 Jul 2026 18:01:23 +0000 Subject: alloc_tag: add ioctl to /proc/allocinfo Patch series "alloc_tag: introduce IOCTL-based filtering for MAP", v8. Currently, memory allocation profiling data is primarily exposed through /proc/allocinfo. While useful for manual inspection, this text-based interface poses challenges for production monitoring and large-scale analysis: 1. Userspace must parse large amounts of text to extract specific fields. 2. To find specific tags, userspace must read the entire dataset, requiring many context switches and high data copying. 3. The kernel currently aggregates per-CPU counters for every allocation size, even those the user intends to filter out immediately. This series introduces a new IOCTL-based binary interface for allocinfo that supports kernel-side filtering. By allowing the user to specify a filter mask, we significantly reduce the work performed in-kernel and the amount of data transferred to userspace. The IOCTL mechanism was chosen for allocinfo to address the per-CPU counter aggregation bottleneck. A traditional read() operation must report the total allocation count and sizes for every code tag in the system. Doing so requires iterating across all CPUs to sum their per-CPU counters for thousands of tags, which introduces substantial runtime overhead. The IOCTL interface allows userspace to push selective filtering criteria directly into the kernel before the per-CPU counter aggregation. The kernel aggregates per-CPU counters only for a small subset of tags that match the filter. This results in significant performance improvement. Beyond fast filtered retrieval, the IOCTL foundation allows introducing a context capture mechanism in the future to capture the context for specific allocations. Performance measurements were conducted on an Intel Xeon Platinum 8481C (224 CPUs) with caches dropped before each run. The IOCTL mechanism shows a ~20x performance improvement for filtered queries. The kernel avoids the expensive per-CPU counter aggregation (alloc_tag_read) for any tags that fail the initial string or location filters. Scenario 1: Specific File Filtering (arch/x86/events/rapl.c) 1. Traditional (cat /proc/allocinfo | grep): 22ms (sys) 2. IOCTL Interface: 1ms (sys) Scenario 2: Compound Filtering (Filename + Size) 1. Traditional: (cat ... | grep | awk): 21ms (sys) 2. IOCTL Interface: 1ms (sys) Scenario 3: Size-Based Filtering (min_size = 1MB) 1. Traditional: (cat ... | awk): 21ms (sys) 2. IOCTL Interface: 14ms (sys) This patch (of 6): Add the following ioctl commands for /proc/allocinfo file: ALLOCINFO_IOC_CONTENT_ID - gets content identifier which can be used to check whether the file content has changed specifically due to module load/unload. Every time a module is loaded / unloaded, the returned value will be different. By comparing the identifier value at the beginning and at the end of the content retrieval operation, users can validate retrieved information for consistency. ALLOCINFO_IOC_GET_AT - gets the record at the specified position. This is the position of a record in /proc/allocinfo. ALLOCINFO_IOC_GET_NEXT - gets the record next to the last retrieved one. If no records were previously retrieved, returns the first record. Note, function file and module names often have the same prefixes, therefore when filtering for them, we compare the last 64 characters to minimize the chances of name collisions. [akpm@linux-foundation.org: include compat.h, per Suren] Closes: https://lore.kernel.org/oe-kbuild-all/202607091820.qbjlGhKK-lkp@intel.com/ Link: https://lore.kernel.org/cover.1783532853.git.abhishekbapat@google.com Link: https://lore.kernel.org/15596de2607ef13e7c77c6d74763f4ae992ec475.1783532853.git.abhishekbapat@google.com Signed-off-by: Suren Baghdasaryan Signed-off-by: Abhishek Bapat Acked-by: Hao Ge Cc: Jonathan Corbet Cc: Kent Overstreet Cc: Sourav Panda Signed-off-by: Andrew Morton --- Documentation/mm/allocation-profiling.rst | 5 + Documentation/userspace-api/ioctl/ioctl-number.rst | 2 + MAINTAINERS | 1 + include/linux/codetag.h | 2 + include/uapi/linux/alloc_tag.h | 65 ++++++ lib/codetag.c | 18 ++ mm/alloc_tag.c | 239 ++++++++++++++++++++- 7 files changed, 330 insertions(+), 2 deletions(-) create mode 100644 include/uapi/linux/alloc_tag.h (limited to 'Documentation') diff --git a/Documentation/mm/allocation-profiling.rst b/Documentation/mm/allocation-profiling.rst index e928aa3e4e1e..b2ebcef8af6f 100644 --- a/Documentation/mm/allocation-profiling.rst +++ b/Documentation/mm/allocation-profiling.rst @@ -57,6 +57,11 @@ sysctl: Runtime info: /proc/allocinfo + Profiling data can be retrieved either by reading `/proc/allocinfo` directly as + text or programmatically via `ioctl()` calls defined in ``. + The ioctl interface supports structured binary data extraction as well as filtering + by module name, function, file, line number, accuracy, or allocation size limits. + Example output:: root@moria-kvm:~# sort -g /proc/allocinfo|tail|numfmt --to=iec diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 3f0ef1e27eb0..2fc53093752d 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -346,6 +346,8 @@ Code Seq# Include File Comments 0xA5 20-2F linux/surface_aggregator/dtx.h Microsoft Surface DTX driver +0xA6 00-0F uapi/linux/alloc_tag.h Memory allocation profiling + 0xAA 00-3F linux/uapi/linux/userfaultfd.h 0xAB 00-1F linux/nbd.h 0xAC 00-1F linux/raw.h diff --git a/MAINTAINERS b/MAINTAINERS index 06271e742d32..557e5fd32073 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -16940,6 +16940,7 @@ S: Maintained F: Documentation/mm/allocation-profiling.rst F: include/linux/alloc_tag.h F: include/linux/pgalloc_tag.h +F: include/uapi/linux/alloc_tag.h F: mm/alloc_tag.c MEMORY MANAGEMENT - BALLOON diff --git a/include/linux/codetag.h b/include/linux/codetag.h index ddae7484ca45..a25a085c2df1 100644 --- a/include/linux/codetag.h +++ b/include/linux/codetag.h @@ -77,6 +77,8 @@ struct codetag_iterator { void codetag_lock_module_list(struct codetag_type *cttype); bool codetag_trylock_module_list(struct codetag_type *cttype); void codetag_unlock_module_list(struct codetag_type *cttype); +unsigned long codetag_get_content_id(struct codetag_type *cttype); +unsigned int codetag_get_count(struct codetag_type *cttype); struct codetag_iterator codetag_get_ct_iter(struct codetag_type *cttype); struct codetag *codetag_next_ct(struct codetag_iterator *iter); diff --git a/include/uapi/linux/alloc_tag.h b/include/uapi/linux/alloc_tag.h new file mode 100644 index 000000000000..ee6a023cbaf4 --- /dev/null +++ b/include/uapi/linux/alloc_tag.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * alloc_tag IOCTL API definition + * + * Copyright (C) 2026 Google, LLC. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#ifndef _UAPI_ALLOC_TAG_H +#define _UAPI_ALLOC_TAG_H + +#include + +/* + * Function, file and module names often have the same prefixes, therefore + * when filtering by these criteria, we compare the last 64 characters to + * minimize the chances of name collisions + */ +#define ALLOCINFO_STR_SIZE 64 + +struct allocinfo_content_id { + __u64 id; +}; + +struct allocinfo_tag { + /* Longer names are trimmed */ + char modname[ALLOCINFO_STR_SIZE]; + char function[ALLOCINFO_STR_SIZE]; + char filename[ALLOCINFO_STR_SIZE]; + __u64 lineno; +}; + +/* The alignment ensures 32-bit compatible interfaces are not broken */ +struct allocinfo_counter { + __u64 bytes; + __u64 calls; + __u8 accurate; +} __attribute__((aligned(8))); + +struct allocinfo_tag_data { + struct allocinfo_tag tag; + struct allocinfo_counter counter; +}; + +struct allocinfo_get_at { + __u64 pos; /* input */ + struct allocinfo_tag_data data; +}; + +#define _ALLOCINFO_IOC_CONTENT_ID 0 +#define _ALLOCINFO_IOC_GET_AT 1 +#define _ALLOCINFO_IOC_GET_NEXT 2 + +#define ALLOCINFO_IOC_BASE 0xA6 +#define ALLOCINFO_IOC_CONTENT_ID _IOR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_CONTENT_ID, \ + struct allocinfo_content_id) +#define ALLOCINFO_IOC_GET_AT _IOWR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_GET_AT, \ + struct allocinfo_get_at) +#define ALLOCINFO_IOC_GET_NEXT _IOR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_GET_NEXT, \ + struct allocinfo_tag_data) + +#endif /* _UAPI_ALLOC_TAG_H */ diff --git a/lib/codetag.c b/lib/codetag.c index 4001a7ea6675..a9cda4c962a3 100644 --- a/lib/codetag.c +++ b/lib/codetag.c @@ -19,6 +19,8 @@ struct codetag_type { struct codetag_type_desc desc; /* generates unique sequence number for module load */ unsigned long next_mod_seq; + /* bumped on every module load and unload */ + unsigned long content_id; }; struct codetag_range { @@ -50,6 +52,20 @@ void codetag_unlock_module_list(struct codetag_type *cttype) up_read(&cttype->mod_lock); } +unsigned long codetag_get_content_id(struct codetag_type *cttype) +{ + lockdep_assert_held(&cttype->mod_lock); + + return cttype->content_id; +} + +unsigned int codetag_get_count(struct codetag_type *cttype) +{ + lockdep_assert_held(&cttype->mod_lock); + + return cttype->count; +} + struct codetag_iterator codetag_get_ct_iter(struct codetag_type *cttype) { struct codetag_iterator iter = { @@ -204,6 +220,7 @@ static int codetag_module_init(struct codetag_type *cttype, struct module *mod) down_write(&cttype->mod_lock); cmod->mod_seq = ++cttype->next_mod_seq; + ++cttype->content_id; mod_id = idr_alloc(&cttype->mod_idr, cmod, 0, 0, GFP_KERNEL); if (mod_id >= 0) { if (cttype->desc.module_load) { @@ -368,6 +385,7 @@ void codetag_unload_module(struct module *mod) cttype->count -= range_size(cttype, &cmod->range); idr_remove(&cttype->mod_idr, mod_id); kfree(cmod); + ++cttype->content_id; } up_write(&cttype->mod_lock); if (found && cttype->desc.free_section_mem) diff --git a/mm/alloc_tag.c b/mm/alloc_tag.c index b60ee89704cc..b2ac166880ac 100644 --- a/mm/alloc_tag.c +++ b/mm/alloc_tag.c @@ -5,6 +5,8 @@ #include #include #include +#include +#include #include #include #include @@ -14,6 +16,7 @@ #include #include #include +#include #include "internal.h" #include "page_alloc.h" @@ -59,6 +62,10 @@ struct allocinfo_private { struct codetag_iterator iter; struct codetag_iterator reported_iter; bool print_header; + /* ioctl uses a separate iterator not to interfere with reads */ + struct codetag_iterator ioctl_iter; + bool positioned; /* seq_open_private() sets to 0 */ + struct mutex ioctl_lock; }; static void *allocinfo_start(struct seq_file *m, loff_t *pos) @@ -142,6 +149,235 @@ static const struct seq_operations allocinfo_seq_op = { .show = allocinfo_show, }; +/* + * Initializes seq_file operations and allocates private state when opening + * the /proc/allocinfo procfs entry. + */ +static int allocinfo_open(struct inode *inode, struct file *file) +{ + int ret; + + ret = seq_open_private(file, &allocinfo_seq_op, + sizeof(struct allocinfo_private)); + if (!ret) { + struct seq_file *m = file->private_data; + struct allocinfo_private *priv = m->private; + + mutex_init(&priv->ioctl_lock); + } + return ret; +} + +/* + * Cleans up the seq_file state and frees up the private state allocated in + * allocinfo_open() when closing the /proc/allocinfo file descriptor. + */ +static int allocinfo_release(struct inode *inode, struct file *file) +{ + struct seq_file *m = file->private_data; + struct allocinfo_private *priv = m->private; + + mutex_destroy(&priv->ioctl_lock); + return seq_release_private(inode, file); +} + +/* + * Returns a pointer to the suffix of a string so that its length fits within + * ALLOCINFO_STR_SIZE, preserving the trailing characters. + * Function, file and module names often have the same prefixes, therefore + * when filtering by these criteria, we compare the last 64 characters to + * minimize the chances of name collisions + */ +static const char *allocinfo_str(const char *str) +{ + size_t len = strlen(str); + + /* Keep an extra space for the trailing NULL. */ + if (len >= ALLOCINFO_STR_SIZE) + str += (len - ALLOCINFO_STR_SIZE) + 1; + return str; +} + +/* Copy a string and trim from the beginning if it's too long */ +static void allocinfo_copy_str(char *dest, const char *src) +{ + strscpy_pad(dest, allocinfo_str(src), ALLOCINFO_STR_SIZE); +} + +/* + * Populates the UAPI allocinfo_tag_data structure with active runtime + * profiling counters extracted from the given kernel codetag. + */ +static void allocinfo_to_params(struct codetag *ct, + struct allocinfo_tag_data *data) +{ + struct alloc_tag *tag = ct_to_alloc_tag(ct); + struct alloc_tag_counters counter = alloc_tag_read(tag); + + if (ct->modname) + allocinfo_copy_str(data->tag.modname, ct->modname); + else + data->tag.modname[0] = '\0'; + allocinfo_copy_str(data->tag.function, ct->function); + allocinfo_copy_str(data->tag.filename, ct->filename); + data->tag.lineno = ct->lineno; + data->counter.bytes = counter.bytes; + data->counter.calls = counter.calls; + data->counter.accurate = !alloc_tag_is_inaccurate(tag); +} + +/* + * Retrieves the unique content ID representing the current allocation tag module + * layout, allowing userspace to detect if modules were loaded / unloaded. + */ +static int allocinfo_ioctl_get_content_id(struct seq_file *m, void __user *arg) +{ + struct allocinfo_content_id params; + + codetag_lock_module_list(alloc_tag_cttype); + params.id = codetag_get_content_id(alloc_tag_cttype); + codetag_unlock_module_list(alloc_tag_cttype); + if (copy_to_user(arg, ¶ms, sizeof(params))) + return -EFAULT; + + return 0; +} + +/* + * Seeks the ioctl iterator to the specified 0-indexed tag position, reads its + * profiling data and returns it to userspace. + */ +static int allocinfo_ioctl_get_at(struct seq_file *m, void __user *arg) +{ + struct allocinfo_private *priv; + struct codetag *ct; + __u64 pos; + struct allocinfo_get_at params = {0}; + + if (copy_from_user(¶ms, arg, sizeof(params))) + return -EFAULT; + + priv = m->private; + pos = params.pos; + + mutex_lock(&priv->ioctl_lock); + codetag_lock_module_list(alloc_tag_cttype); + + if (pos >= codetag_get_count(alloc_tag_cttype)) { + codetag_unlock_module_list(alloc_tag_cttype); + mutex_unlock(&priv->ioctl_lock); + return -ENOENT; + } + + /* Find the codetag */ + priv->ioctl_iter = codetag_get_ct_iter(alloc_tag_cttype); + ct = codetag_next_ct(&priv->ioctl_iter); + while (ct && pos--) + ct = codetag_next_ct(&priv->ioctl_iter); + if (ct) { + allocinfo_to_params(ct, ¶ms.data); + priv->positioned = true; + } + + codetag_unlock_module_list(alloc_tag_cttype); + mutex_unlock(&priv->ioctl_lock); + + if (!ct) + return -ENOENT; + + if (copy_to_user(arg, ¶ms, sizeof(params))) + return -EFAULT; + + return 0; +} + +/* + * Advances the ioctl iterator to the next allocation tag in the sequence and + * returns its profiling data to userspace. + */ +static int allocinfo_ioctl_get_next(struct seq_file *m, void __user *arg) +{ + struct allocinfo_private *priv; + struct codetag *ct; + struct allocinfo_tag_data params; + int ret = 0; + + memset(¶ms, 0, sizeof(params)); + priv = m->private; + + mutex_lock(&priv->ioctl_lock); + codetag_lock_module_list(alloc_tag_cttype); + + if (!priv->positioned) { + priv->ioctl_iter = codetag_get_ct_iter(alloc_tag_cttype); + priv->positioned = true; + } + + ct = codetag_next_ct(&priv->ioctl_iter); + if (ct) + allocinfo_to_params(ct, ¶ms); + + if (!ct) { + priv->positioned = false; + ret = -ENOENT; + } + codetag_unlock_module_list(alloc_tag_cttype); + mutex_unlock(&priv->ioctl_lock); + + if (ret == 0) { + if (copy_to_user(arg, ¶ms, sizeof(params))) + return -EFAULT; + } + return ret; +} + +/* + * Entry point ioctl function for /proc/allocinfo routing requests to fetch the + * layout content ID, seek to a specific tag, or read sequential tags. + */ +static long allocinfo_ioctl(struct file *file, unsigned int cmd, + unsigned long __arg) +{ + void __user *arg = (void __user *)__arg; + int ret; + + switch (cmd) { + case ALLOCINFO_IOC_CONTENT_ID: + ret = allocinfo_ioctl_get_content_id(file->private_data, arg); + break; + case ALLOCINFO_IOC_GET_AT: + ret = allocinfo_ioctl_get_at(file->private_data, arg); + break; + case ALLOCINFO_IOC_GET_NEXT: + ret = allocinfo_ioctl_get_next(file->private_data, arg); + break; + default: + ret = -ENOIOCTLCMD; + break; + } + + return ret; +} + +#ifdef CONFIG_COMPAT +static long allocinfo_compat_ioctl(struct file *file, unsigned int cmd, + unsigned long arg) +{ + return allocinfo_ioctl(file, cmd, (unsigned long)compat_ptr(arg)); +} +#endif + +static const struct proc_ops allocinfo_proc_ops = { + .proc_open = allocinfo_open, + .proc_read_iter = seq_read_iter, + .proc_lseek = seq_lseek, + .proc_release = allocinfo_release, + .proc_ioctl = allocinfo_ioctl, +#ifdef CONFIG_COMPAT + .proc_compat_ioctl = allocinfo_compat_ioctl, +#endif +}; + size_t alloc_tag_top_users(struct codetag_bytes *tags, size_t count, bool can_sleep) { struct codetag_iterator iter; @@ -999,8 +1235,7 @@ static int __init alloc_tag_init(void) return 0; } - if (!proc_create_seq_private(ALLOCINFO_FILE_NAME, 0400, NULL, &allocinfo_seq_op, - sizeof(struct allocinfo_private), NULL)) { + if (!proc_create(ALLOCINFO_FILE_NAME, 0400, NULL, &allocinfo_proc_ops)) { pr_err("Failed to create %s file\n", ALLOCINFO_FILE_NAME); shutdown_mem_profiling(false); return -ENOMEM; -- cgit From 0df74c11587941b35596d1e8990dcab06bdbfeb5 Mon Sep 17 00:00:00 2001 From: Christoph Hellwig Date: Mon, 13 Jul 2026 11:33:43 +0200 Subject: mm/swap: remove SWP_FS_OPS Provide a swap_fs_activate helper that directly sets up swap_fs_ops, and a flag in struct swap_ops to indicate of NOFS swapping is allowed. Link: https://lore.kernel.org/20260713093350.2154226-7-hch@lst.de Signed-off-by: Christoph Hellwig Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Chris Li Cc: Kairui Song Cc: Kemeng Shi Cc: Nhat Pham Cc: Youngjun Park Signed-off-by: Andrew Morton --- Documentation/filesystems/locking.rst | 5 +++-- Documentation/filesystems/vfs.rst | 4 ++-- fs/nfs/file.c | 4 +--- fs/smb/client/file.c | 4 +--- include/linux/swap.h | 6 +++++- mm/page_io.c | 10 +++++++++- mm/swap.h | 22 ++++++++++------------ mm/swapfile.c | 2 -- mm/vmscan.c | 15 +++++++-------- 9 files changed, 38 insertions(+), 34 deletions(-) (limited to 'Documentation') diff --git a/Documentation/filesystems/locking.rst b/Documentation/filesystems/locking.rst index 08d01bc62c31..1a50d41a39a1 100644 --- a/Documentation/filesystems/locking.rst +++ b/Documentation/filesystems/locking.rst @@ -355,13 +355,14 @@ should perform any validation and preparation necessary to ensure that writes can be performed with minimal memory allocation. It should call add_swap_extent(), or the helper iomap_swapfile_activate(), and return the number of extents added. If IO should be submitted through -->swap_rw(), it should set SWP_FS_OPS, otherwise IO will be submitted +->swap_rw(), it should call swap_fs_activate, otherwise IO will be submitted directly to the block device ``sis->bdev``. ->swap_deactivate() will be called in the sys_swapoff() path after ->swap_activate() returned success. -->swap_rw will be called for swap IO if SWP_FS_OPS was set by ->swap_activate(). +->swap_rw will be called for swap IO if swap_fs_activate was called by +->swap_activate(). file_lock_operations ==================== diff --git a/Documentation/filesystems/vfs.rst b/Documentation/filesystems/vfs.rst index 7c753148af88..e7677423a20f 100644 --- a/Documentation/filesystems/vfs.rst +++ b/Documentation/filesystems/vfs.rst @@ -977,7 +977,7 @@ cache in your filesystem. The following members are defined: can be performed with minimal memory allocation. It should call add_swap_extent(), or the helper iomap_swapfile_activate(), and return the number of extents added. If IO should be submitted - through ->swap_rw(), it should set SWP_FS_OPS, otherwise IO will + through ->swap_rw(), it should call swap_fs_activate, otherwise IO will be submitted directly to the block device ``sis->bdev``. ``swap_deactivate`` @@ -985,7 +985,7 @@ cache in your filesystem. The following members are defined: successful. ``swap_rw`` - Called to read or write swap pages when SWP_FS_OPS is set. + Called to read or write swap pages when swap_fs_activate was called. The File Object =============== diff --git a/fs/nfs/file.c b/fs/nfs/file.c index a0d8f1c1cf10..851d93a09988 100644 --- a/fs/nfs/file.c +++ b/fs/nfs/file.c @@ -597,7 +597,7 @@ static int nfs_swap_activate(struct swap_info_struct *sis, struct file *file, ret = rpc_clnt_swap_activate(clnt); if (ret) return ret; - ret = add_swap_extent(sis, 0, sis->max, 0); + ret = swap_fs_activate(sis); if (ret < 0) { rpc_clnt_swap_deactivate(clnt); return ret; @@ -607,8 +607,6 @@ static int nfs_swap_activate(struct swap_info_struct *sis, struct file *file, if (cl->rpc_ops->enable_swap) cl->rpc_ops->enable_swap(inode); - - sis->flags |= SWP_FS_OPS; return ret; } diff --git a/fs/smb/client/file.c b/fs/smb/client/file.c index b279a44be729..7f2924ce2881 100644 --- a/fs/smb/client/file.c +++ b/fs/smb/client/file.c @@ -3451,9 +3451,7 @@ static int cifs_swap_activate(struct swap_info_struct *sis, * but we could add call to grab a byte range lock to prevent others * from reading or writing the file */ - - sis->flags |= SWP_FS_OPS; - return add_swap_extent(sis, 0, sis->max, 0); + return swap_fs_activate(sis); } static void cifs_swap_deactivate(struct file *file) diff --git a/include/linux/swap.h b/include/linux/swap.h index 5979b1427368..8dd68733c955 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -202,7 +202,6 @@ enum { SWP_SOLIDSTATE = (1 << 4), /* blkdev seeks are cheap */ SWP_BLKDEV = (1 << 6), /* its a block device */ SWP_ACTIVATED = (1 << 7), /* set after swap_activate success */ - SWP_FS_OPS = (1 << 8), /* swapfile operations go through fs */ SWP_AREA_DISCARD = (1 << 9), /* single-time swap area discards */ SWP_PAGE_DISCARD = (1 << 10), /* freed swap page-cluster discards */ SWP_STABLE_WRITES = (1 << 11), /* no overwrite PG_writeback pages */ @@ -343,6 +342,7 @@ extern void __meminit kswapd_stop(int nid); #ifdef CONFIG_SWAP +int swap_fs_activate(struct swap_info_struct *sis); int add_swap_extent(struct swap_info_struct *sis, unsigned long start_page, unsigned long nr_pages, sector_t start_block); int generic_swapfile_activate(struct swap_info_struct *, struct file *, @@ -468,6 +468,10 @@ static inline bool folio_free_swap(struct folio *folio) return false; } +static inline int swap_fs_activate(struct swap_info_struct *sis) +{ + return -EINVAL; +} static inline int add_swap_extent(struct swap_info_struct *sis, unsigned long start_page, unsigned long nr_pages, sector_t start_block) diff --git a/mm/page_io.c b/mm/page_io.c index c36b44ffe947..cea438b66bce 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -686,12 +686,20 @@ static bool swap_fs_can_merge(struct folio *folio, struct folio *prev_folio, swap_dev_pos(prev_folio->swap) + prev_folio_size; } -const struct swap_ops swap_fs_ops = { +static const struct swap_ops swap_fs_ops = { + .flags = SWAP_OPS_F_REQUIRE_NOFS, .submit_write = swap_fs_submit_write, .submit_read = swap_fs_submit_read, .can_merge = swap_fs_can_merge, }; +int swap_fs_activate(struct swap_info_struct *sis) +{ + sis->ops = &swap_fs_ops; + return add_swap_extent(sis, 0, sis->max, 0); +} +EXPORT_SYMBOL_GPL(swap_fs_activate); + void swap_write_submit(struct swap_io_ctx *ctx) { if (!ctx->sio) diff --git a/mm/swap.h b/mm/swap.h index ffc36695d4ac..1a78578fd067 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -96,7 +96,17 @@ struct swap_io_ctx { struct swap_info_struct *sis; }; +/* + * SWAP_OPS_F_REQUIRE_NOFS: + * When set, all reclaim operations must operated as GFS_NOFS and not + * just GFP_NOIO, as GFP_NOIO allocations could recourse into the + * file system backing this swap file. + */ +#define SWAP_OPS_F_REQUIRE_NOFS (1U << 0) + struct swap_ops { + unsigned int flags; + bool (*can_merge)(struct folio *folio, struct folio *prev_folio, size_t prev_folio_size, int rw); void (*submit_write)(struct swap_io_ctx *ctx); @@ -347,11 +357,6 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t flag, unsigned long orders, void swap_update_readahead(struct folio *folio, struct vm_area_struct *vma, unsigned long addr); -static inline unsigned int folio_swap_flags(struct folio *folio) -{ - return __swap_entry_to_info(folio->swap)->flags; -} - #else /* CONFIG_SWAP */ static inline struct swap_cluster_info *swap_cluster_lock( struct swap_info_struct *si, pgoff_t offset, bool irq) @@ -482,16 +487,9 @@ static inline void __swap_cache_replace_folio(struct swap_cluster_info *ci, struct folio *old, struct folio *new) { } - -static inline unsigned int folio_swap_flags(struct folio *folio) -{ - return 0; -} - #endif /* CONFIG_SWAP */ extern const struct swap_ops swap_bdev_ops; -extern const struct swap_ops swap_fs_ops; int shmem_writeout(struct swap_io_ctx *ctx, struct folio *folio, struct list_head *folio_list); diff --git a/mm/swapfile.c b/mm/swapfile.c index ad623dae483b..dacef34a3ed7 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -2975,8 +2975,6 @@ static int setup_swap_extents(struct swap_info_struct *sis, ret = mapping->a_ops->swap_activate(sis, swap_file, span); if (ret < 0) return ret; - if (sis->flags & SWP_FS_OPS) - sis->ops = &swap_fs_ops; sis->flags |= SWP_ACTIVATED; return ret; } diff --git a/mm/vmscan.c b/mm/vmscan.c index 4742297693fe..3194da7dcc79 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1040,16 +1040,15 @@ static bool may_enter_fs(struct folio *folio, gfp_t gfp_mask) { if (gfp_mask & __GFP_FS) return true; - if (!folio_test_swapcache(folio) || !(gfp_mask & __GFP_IO)) - return false; /* - * We can "enter_fs" for swap-cache with only __GFP_IO - * providing this isn't SWP_FS_OPS. - * ->flags can be updated non-atomically, - * but that will never affect SWP_FS_OPS, so the data_race - * is safe. + * We can "enter_fs" for swap-cache with only __GFP_IO unless backed by + * a swapfile that requires GFP_NOFS I/O. */ - return !data_race(folio_swap_flags(folio) & SWP_FS_OPS); + if (folio_test_swapcache(folio) && (gfp_mask & __GFP_IO) && + !(__swap_entry_to_info(folio->swap)->ops->flags & + SWAP_OPS_F_REQUIRE_NOFS)) + return true; + return false; } /* -- cgit From e776db8e710165c2bc47566b62edcf36ff46bbdd Mon Sep 17 00:00:00 2001 From: Breno Leitao Date: Mon, 13 Jul 2026 04:48:05 -0700 Subject: mm: kmemleak: report leaks only after N consecutive unreferenced scans kmemleak reports an object the first scan it is found unreferenced. Its mark phase runs without stopping the rest of the kernel and without a write barrier, so a live object whose only reference is briefly invisible during a concurrent RCU update -- e.g. a VMA moved between maple tree nodes, or a page-cache xa_node -- can be seen as unreferenced for that one scan. Because an object is flagged as reported only once, such a transient race turns into a permanent false positive. Track how many consecutive scans each object has been seen unreferenced and only report it once that reaches min_unref_scans, a new module parameter. It defaults to 1, leaving the behaviour unchanged; setting it higher (e.g. 2) still reports a genuine leak, one scan later, while an object referenced again before the threshold restarts its run and is never reported. min_unref_scans can be set at boot with kmemleak.min_unref_scans= or at run-time via /sys/module/kmemleak/parameters/min_unref_scans. Link: https://lore.kernel.org/20260713-catalin_pto-v1-2-5b93b1131089@debian.org Signed-off-by: Breno Leitao Reviewed-by: Catalin Marinas Cc: David Hildenbrand Cc: Geert Uytterhoeven Cc: Jonathan Corbet Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Shuah Khan Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- Documentation/dev-tools/kmemleak.rst | 8 ++++++++ mm/kmemleak.c | 13 ++++++++++++- 2 files changed, 20 insertions(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/dev-tools/kmemleak.rst b/Documentation/dev-tools/kmemleak.rst index 7d784e03f3f9..a8a83bc69ceb 100644 --- a/Documentation/dev-tools/kmemleak.rst +++ b/Documentation/dev-tools/kmemleak.rst @@ -198,6 +198,14 @@ systems, because of pointers temporarily stored in CPU registers or stacks. Kmemleak defines MSECS_MIN_AGE (defaulting to 1000) representing the minimum age of an object to be reported as a memory leak. +The ``min_unref_scans`` module parameter (default 1) requires an object to +be seen unreferenced in that many consecutive scans before it is reported. +Keeping it at 1 preserves the historical behaviour; higher values filter +the transient false positives described above, at the cost of delaying +genuine reports by up to that many scans. It can be set at boot with +``kmemleak.min_unref_scans=`` or at run-time via +``/sys/module/kmemleak/parameters/min_unref_scans``. + Limitations and Drawbacks ------------------------- diff --git a/mm/kmemleak.c b/mm/kmemleak.c index 95bd8ccd3c5b..7afd08ed8546 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -151,6 +151,8 @@ struct kmemleak_object { int min_count; /* the total number of pointers found pointing to this object */ int count; + /* consecutive scans the object has been seen unreferenced */ + unsigned int unref_scans; /* checksum for detecting modified objects */ u32 checksum; depot_stack_handle_t trace_handle; @@ -234,6 +236,9 @@ static unsigned long max_percpu_addr; static struct task_struct *scan_thread; /* used to avoid reporting of recently allocated objects */ static unsigned long jiffies_min_age; +/* consecutive scans an object must stay unreferenced before reporting */ +static unsigned int min_unref_scans = 1; +module_param(min_unref_scans, uint, 0644); static unsigned long jiffies_last_scan; /* delay between automatic memory scannings */ static unsigned long jiffies_scan_wait; @@ -692,6 +697,7 @@ static struct kmemleak_object *__alloc_object(gfp_t gfp) object->excess_ref = 0; object->count = 0; /* white color initially */ object->checksum = ~0; + object->unref_scans = 0; object->del_state = 0; /* task information */ @@ -1890,6 +1896,9 @@ static int __kmemleak_scan(bool full) __paint_it(object, KMEMLEAK_BLACK); } + /* referenced last scan: restart the unreferenced run */ + if (!color_white(object)) + object->unref_scans = 0; /* reset the reference count (whiten the object) */ object->count = 0; if (full) @@ -2064,9 +2073,11 @@ static void kmemleak_scan(void) raw_spin_lock_irq(&object->lock); trace_handle = 0; dedup_print = false; + if (unreferenced_object(object) && (object->flags & OBJECT_SUSPECT) && - !(object->flags & OBJECT_REPORTED)) { + !(object->flags & OBJECT_REPORTED) && + ++object->unref_scans >= min_unref_scans) { object->flags |= OBJECT_REPORTED; if (kmemleak_verbose) { trace_handle = object->trace_handle; -- cgit From 09dde5e9bac0429fa565d3a9fcb0e80d0f668f19 Mon Sep 17 00:00:00 2001 From: Breno Leitao Date: Fri, 31 Jul 2026 03:13:05 -0700 Subject: Documentation: kmemleak: document the conditional min_unref_scans default min_unref_scans now defaults to 2 when CONFIG_DEBUG_KMEMLEAK_AUTO_SCAN and CONFIG_DEBUG_KMEMLEAK_VERBOSE are both enabled, but the documentation still states that the default is unconditionally 1. Link: https://lore.kernel.org/20260731-kmemleak_hardened-v2-2-7b9689ac77cb@debian.org Signed-off-by: Breno Leitao Acked-by: Catalin Marinas Cc: David Hildenbrand Cc: Jonathan Corbet Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Shuah Khan Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- Documentation/dev-tools/kmemleak.rst | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) (limited to 'Documentation') diff --git a/Documentation/dev-tools/kmemleak.rst b/Documentation/dev-tools/kmemleak.rst index a8a83bc69ceb..d1b690b17169 100644 --- a/Documentation/dev-tools/kmemleak.rst +++ b/Documentation/dev-tools/kmemleak.rst @@ -198,11 +198,13 @@ systems, because of pointers temporarily stored in CPU registers or stacks. Kmemleak defines MSECS_MIN_AGE (defaulting to 1000) representing the minimum age of an object to be reported as a memory leak. -The ``min_unref_scans`` module parameter (default 1) requires an object to -be seen unreferenced in that many consecutive scans before it is reported. -Keeping it at 1 preserves the historical behaviour; higher values filter -the transient false positives described above, at the cost of delaying -genuine reports by up to that many scans. It can be set at boot with +The ``min_unref_scans`` module parameter requires an object to be seen +unreferenced in that many consecutive scans before it is reported. It +defaults to 2 when CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled, where the +periodic scan thread confirms a leak on its own, and to 1 otherwise. A +value of 1 preserves the historical behaviour; higher values filter the +transient false positives described above, at the cost of delaying genuine +reports by up to that many scans. It can be set at boot with ``kmemleak.min_unref_scans=`` or at run-time via ``/sys/module/kmemleak/parameters/min_unref_scans``. -- cgit From 22779ae8175aad7c04827db44e934e53bf2bd2d4 Mon Sep 17 00:00:00 2001 From: Christoph Hellwig Date: Thu, 23 Jul 2026 07:46:06 +0200 Subject: mm/swap: move swap_ops into file systems for file system-based swap Currently swap to and from file systems goes through two indirect calls between the swap ops and the swap_rw method. Reduce this by directly providing the swap_ops from the file system. For this refactor swap_fs_submit into a swap_fs_prepare_rw helper that initializes the iov_iter on the callers stack so that file systems can call it directly, and use that to initialize file system specific ops in the NFS and SMB clients, which then get passed to swap_fs_activate. Link: https://lore.kernel.org/20260723054622.3460249-4-hch@lst.de Signed-off-by: Christoph Hellwig Acked-by: Chris Li Cc: Baoquan He Cc: Kairui Song Cc: Kairui Song Cc: Kemeng Shi Cc: Nhat Pham Cc: Steve French Cc: Usama Arif Signed-off-by: Andrew Morton --- Documentation/filesystems/locking.rst | 9 ++--- Documentation/filesystems/vfs.rst | 8 ++--- fs/nfs/direct.c | 20 ----------- fs/nfs/file.c | 42 ++++++++++++++++++++--- fs/smb/client/file.c | 63 ++++++++++++++++++++++------------- include/linux/fs.h | 1 - include/linux/nfs_fs.h | 1 - include/linux/swap.h | 6 ---- include/linux/swap_ops.h | 5 +++ mm/page_io.c | 34 ++++--------------- 10 files changed, 93 insertions(+), 96 deletions(-) (limited to 'Documentation') diff --git a/Documentation/filesystems/locking.rst b/Documentation/filesystems/locking.rst index 1a50d41a39a1..f58a8d7d5897 100644 --- a/Documentation/filesystems/locking.rst +++ b/Documentation/filesystems/locking.rst @@ -266,7 +266,6 @@ prototypes:: int (*error_remove_folio)(struct address_space *, struct folio *); int (*swap_activate)(struct swap_info_struct *sis, struct file *f, sector_t *span) int (*swap_deactivate)(struct file *); - int (*swap_rw)(struct kiocb *iocb, struct iov_iter *iter); locking rules: All except dirty_folio and free_folio may block @@ -291,7 +290,6 @@ is_partially_uptodate: yes error_remove_folio: yes swap_activate: no swap_deactivate: no -swap_rw: yes, unlocks ====================== ======================== ========= =============== ->write_begin(), ->write_end() and ->read_folio() may be called from @@ -355,15 +353,12 @@ should perform any validation and preparation necessary to ensure that writes can be performed with minimal memory allocation. It should call add_swap_extent(), or the helper iomap_swapfile_activate(), and return the number of extents added. If IO should be submitted through -->swap_rw(), it should call swap_fs_activate, otherwise IO will be submitted -directly to the block device ``sis->bdev``. +the file system it should call swap_fs_activate, otherwise IO will be +submitted directly to the block device ``sis->bdev``. ->swap_deactivate() will be called in the sys_swapoff() path after ->swap_activate() returned success. -->swap_rw will be called for swap IO if swap_fs_activate was called by -->swap_activate(). - file_lock_operations ==================== diff --git a/Documentation/filesystems/vfs.rst b/Documentation/filesystems/vfs.rst index e7677423a20f..c437a342d4f3 100644 --- a/Documentation/filesystems/vfs.rst +++ b/Documentation/filesystems/vfs.rst @@ -776,7 +776,6 @@ cache in your filesystem. The following members are defined: int (*error_remove_folio)(struct mapping *mapping, struct folio *); int (*swap_activate)(struct swap_info_struct *sis, struct file *f, sector_t *span) int (*swap_deactivate)(struct file *); - int (*swap_rw)(struct kiocb *iocb, struct iov_iter *iter); }; ``read_folio`` @@ -977,16 +976,13 @@ cache in your filesystem. The following members are defined: can be performed with minimal memory allocation. It should call add_swap_extent(), or the helper iomap_swapfile_activate(), and return the number of extents added. If IO should be submitted - through ->swap_rw(), it should call swap_fs_activate, otherwise IO will - be submitted directly to the block device ``sis->bdev``. + through the file system it should call swap_fs_activate, otherwise IO + will be submitted directly to the block device ``sis->bdev``. ``swap_deactivate`` Called during swapoff on files where swap_activate was successful. -``swap_rw`` - Called to read or write swap pages when swap_fs_activate was called. - The File Object =============== diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index e626c72495e6..ccafdc1ce64d 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -145,26 +145,6 @@ static void nfs_direct_file_adjust_size_locked(struct inode *inode, } } -/** - * nfs_swap_rw - NFS address space operation for swap I/O - * @iocb: target I/O control block - * @iter: I/O buffer - * - * Perform IO to the swap-file. This is much like direct IO. - */ -int nfs_swap_rw(struct kiocb *iocb, struct iov_iter *iter) -{ - ssize_t ret; - - if (iov_iter_rw(iter) == READ) - ret = nfs_file_direct_read(iocb, iter, true); - else - ret = nfs_file_direct_write(iocb, iter, true); - if (ret < 0) - return ret; - return 0; -} - static void nfs_direct_release_pages(struct page **pages, unsigned int npages) { unsigned int i; diff --git a/fs/nfs/file.c b/fs/nfs/file.c index 851d93a09988..e1bdd10b35f1 100644 --- a/fs/nfs/file.c +++ b/fs/nfs/file.c @@ -29,9 +29,8 @@ #include #include #include -#include #include - +#include #include #include @@ -575,6 +574,38 @@ static int nfs_launder_folio(struct folio *folio) return ret; } +#ifdef CONFIG_SWAP +static void nfs_swap_submit_write(struct swap_io_ctx *ctx) +{ + struct swap_iocb *sio = ctx->sio; + struct iov_iter iter; + int ret; + + swap_fs_prepare_rw(ctx, WRITE, &iter); + ret = nfs_file_direct_write(&sio->iocb, &iter, true); + if (ret != -EIOCBQUEUED) + sio->iocb.ki_complete(&sio->iocb, ret); +} + +static void nfs_swap_submit_read(struct swap_io_ctx *ctx) +{ + struct swap_iocb *sio = ctx->sio; + struct iov_iter iter; + int ret; + + swap_fs_prepare_rw(ctx, READ, &iter); + ret = nfs_file_direct_read(&sio->iocb, &iter, true); + if (ret != -EIOCBQUEUED) + sio->iocb.ki_complete(&sio->iocb, ret); +} + +static const struct swap_ops nfs_swap_ops = { + .flags = SWAP_OPS_F_REQUIRE_NOFS, + .submit_write = nfs_swap_submit_write, + .submit_read = nfs_swap_submit_read, + .can_merge = swap_fs_can_merge, +}; + static int nfs_swap_activate(struct swap_info_struct *sis, struct file *file, sector_t *span) { @@ -597,7 +628,7 @@ static int nfs_swap_activate(struct swap_info_struct *sis, struct file *file, ret = rpc_clnt_swap_activate(clnt); if (ret) return ret; - ret = swap_fs_activate(sis); + ret = swap_fs_activate(sis, &nfs_swap_ops); if (ret < 0) { rpc_clnt_swap_deactivate(clnt); return ret; @@ -620,6 +651,10 @@ static void nfs_swap_deactivate(struct file *file) if (cl->rpc_ops->disable_swap) cl->rpc_ops->disable_swap(file_inode(file)); } +#else +#define nfs_swap_activate NULL +#define nfs_swap_deactivate NULL +#endif /* CONFIG_SWAP */ const struct address_space_operations nfs_file_aops = { .read_folio = nfs_read_folio, @@ -636,7 +671,6 @@ const struct address_space_operations nfs_file_aops = { .error_remove_folio = generic_error_remove_folio, .swap_activate = nfs_swap_activate, .swap_deactivate = nfs_swap_deactivate, - .swap_rw = nfs_swap_rw, }; /* diff --git a/fs/smb/client/file.c b/fs/smb/client/file.c index 7f2924ce2881..ead69232ac1c 100644 --- a/fs/smb/client/file.c +++ b/fs/smb/client/file.c @@ -20,7 +20,7 @@ #include #include #include -#include +#include #include #include #include "cifsfs.h" @@ -3410,6 +3410,38 @@ out: cifs_done_oplock_break(cinode); } +#ifdef CONFIG_SWAP +static void cifs_swap_submit_write(struct swap_io_ctx *ctx) +{ + struct swap_iocb *sio = ctx->sio; + struct iov_iter iter; + int ret; + + swap_fs_prepare_rw(ctx, WRITE, &iter); + ret = netfs_unbuffered_write_iter_locked(&sio->iocb, &iter, NULL); + if (ret != -EIOCBQUEUED) + sio->iocb.ki_complete(&sio->iocb, ret); +} + +static void cifs_swap_submit_read(struct swap_io_ctx *ctx) +{ + struct swap_iocb *sio = ctx->sio; + struct iov_iter iter; + int ret; + + swap_fs_prepare_rw(ctx, READ, &iter); + ret = netfs_unbuffered_read_iter_locked(&sio->iocb, &iter); + if (ret != -EIOCBQUEUED) + sio->iocb.ki_complete(&sio->iocb, ret); +} + +static const struct swap_ops cifs_swap_ops = { + .flags = SWAP_OPS_F_REQUIRE_NOFS, + .submit_write = cifs_swap_submit_write, + .submit_read = cifs_swap_submit_read, + .can_merge = swap_fs_can_merge, +}; + static int cifs_swap_activate(struct swap_info_struct *sis, struct file *swap_file, sector_t *span) { @@ -3420,7 +3452,7 @@ static int cifs_swap_activate(struct swap_info_struct *sis, cifs_dbg(FYI, "swap activate\n"); - if (!swap_file->f_mapping->a_ops->swap_rw) + if (swap_file->f_mapping->a_ops != &cifs_addr_ops) /* Cannot support swap */ return -EINVAL; @@ -3451,7 +3483,7 @@ static int cifs_swap_activate(struct swap_info_struct *sis, * but we could add call to grab a byte range lock to prevent others * from reading or writing the file */ - return swap_fs_activate(sis); + return swap_fs_activate(sis, &cifs_swap_ops); } static void cifs_swap_deactivate(struct file *file) @@ -3467,26 +3499,10 @@ static void cifs_swap_deactivate(struct file *file) /* do we need to unpin (or unlock) the file */ } - -/** - * cifs_swap_rw - SMB3 address space operation for swap I/O - * @iocb: target I/O control block - * @iter: I/O buffer - * - * Perform IO to the swap-file. This is much like direct IO. - */ -static int cifs_swap_rw(struct kiocb *iocb, struct iov_iter *iter) -{ - ssize_t ret; - - if (iov_iter_rw(iter) == READ) - ret = netfs_unbuffered_read_iter_locked(iocb, iter); - else - ret = netfs_unbuffered_write_iter_locked(iocb, iter, NULL); - if (ret < 0) - return ret; - return 0; -} +#else +#define cifs_swap_activate NULL +#define cifs_swap_deactivate NULL +#endif /* CONFIG_SWAP */ const struct address_space_operations cifs_addr_ops = { .read_folio = netfs_read_folio, @@ -3503,7 +3519,6 @@ const struct address_space_operations cifs_addr_ops = { */ .swap_activate = cifs_swap_activate, .swap_deactivate = cifs_swap_deactivate, - .swap_rw = cifs_swap_rw, }; /* diff --git a/include/linux/fs.h b/include/linux/fs.h index 50ce731a2b78..87b5e9957c00 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -438,7 +438,6 @@ struct address_space_operations { int (*swap_activate)(struct swap_info_struct *sis, struct file *file, sector_t *span); void (*swap_deactivate)(struct file *file); - int (*swap_rw)(struct kiocb *iocb, struct iov_iter *iter); }; extern const struct address_space_operations empty_aops; diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h index ec17e602c979..764056498eba 100644 --- a/include/linux/nfs_fs.h +++ b/include/linux/nfs_fs.h @@ -548,7 +548,6 @@ static inline const struct cred *nfs_file_cred(struct file *file) /* * linux/fs/nfs/direct.c */ -int nfs_swap_rw(struct kiocb *iocb, struct iov_iter *iter); ssize_t nfs_file_direct_read(struct kiocb *iocb, struct iov_iter *iter, bool swap); ssize_t nfs_file_direct_write(struct kiocb *iocb, diff --git a/include/linux/swap.h b/include/linux/swap.h index 8dd68733c955..5658a1634b85 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -341,8 +341,6 @@ extern void __meminit kswapd_run(int nid); extern void __meminit kswapd_stop(int nid); #ifdef CONFIG_SWAP - -int swap_fs_activate(struct swap_info_struct *sis); int add_swap_extent(struct swap_info_struct *sis, unsigned long start_page, unsigned long nr_pages, sector_t start_block); int generic_swapfile_activate(struct swap_info_struct *, struct file *, @@ -468,10 +466,6 @@ static inline bool folio_free_swap(struct folio *folio) return false; } -static inline int swap_fs_activate(struct swap_info_struct *sis) -{ - return -EINVAL; -} static inline int add_swap_extent(struct swap_info_struct *sis, unsigned long start_page, unsigned long nr_pages, sector_t start_block) diff --git a/include/linux/swap_ops.h b/include/linux/swap_ops.h index e92b4f532604..57ac6c703f68 100644 --- a/include/linux/swap_ops.h +++ b/include/linux/swap_ops.h @@ -36,4 +36,9 @@ struct swap_ops { void (*submit_read)(struct swap_io_ctx *ctx); }; +void swap_fs_prepare_rw(struct swap_io_ctx *ctx, int rw, struct iov_iter *iter); +bool swap_fs_can_merge(struct folio *folio, struct folio *prev_folio, + size_t prev_folio_size, int rw); +int swap_fs_activate(struct swap_info_struct *sis, const struct swap_ops *ops); + #endif /* _MM_SWAP_OPS_H */ diff --git a/mm/page_io.c b/mm/page_io.c index e741e67d6592..88962571cb93 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -650,11 +650,9 @@ const struct swap_ops swap_bdev_ops = { .can_merge = swap_bdev_can_merge, }; -static void swap_fs_submit(struct swap_io_ctx *ctx, int rw) +void swap_fs_prepare_rw(struct swap_io_ctx *ctx, int rw, struct iov_iter *iter) { struct swap_iocb *sio = ctx->sio; - struct iov_iter iter; - int ret; init_sync_kiocb(&sio->iocb, ctx->sis->swap_file); sio->iocb.ki_pos = swap_dev_pos(bvec_folio(&sio->bvecs[0])->swap); @@ -663,40 +661,22 @@ static void swap_fs_submit(struct swap_io_ctx *ctx, int rw) else sio->iocb.ki_complete = swap_fs_read_complete; - iov_iter_bvec(&iter, rw == WRITE ? ITER_SOURCE : ITER_DEST, + iov_iter_bvec(iter, rw == WRITE ? ITER_SOURCE : ITER_DEST, sio->bvecs, sio->nr_bvecs, sio->len); - ret = sio->iocb.ki_filp->f_mapping->a_ops->swap_rw(&sio->iocb, &iter); - if (ret != -EIOCBQUEUED) - sio->iocb.ki_complete(&sio->iocb, ret); } +EXPORT_SYMBOL_GPL(swap_fs_prepare_rw); -static void swap_fs_submit_write(struct swap_io_ctx *ctx) -{ - swap_fs_submit(ctx, WRITE); -} - -static void swap_fs_submit_read(struct swap_io_ctx *ctx) -{ - swap_fs_submit(ctx, READ); -} - -static bool swap_fs_can_merge(struct folio *folio, struct folio *prev_folio, +bool swap_fs_can_merge(struct folio *folio, struct folio *prev_folio, size_t prev_folio_size, int rw) { return swap_dev_pos(folio->swap) == swap_dev_pos(prev_folio->swap) + prev_folio_size; } +EXPORT_SYMBOL_GPL(swap_fs_can_merge); -static const struct swap_ops swap_fs_ops = { - .flags = SWAP_OPS_F_REQUIRE_NOFS, - .submit_write = swap_fs_submit_write, - .submit_read = swap_fs_submit_read, - .can_merge = swap_fs_can_merge, -}; - -int swap_fs_activate(struct swap_info_struct *sis) +int swap_fs_activate(struct swap_info_struct *sis, const struct swap_ops *ops) { - sis->ops = &swap_fs_ops; + sis->ops = ops; return add_swap_extent(sis, 0, sis->max, 0); } EXPORT_SYMBOL_GPL(swap_fs_activate); -- cgit From 2a0be246e342e239ef91d28e2658b6bf508cc065 Mon Sep 17 00:00:00 2001 From: "Nico Pache (Red Hat)" Date: Tue, 11 Aug 2026 06:48:39 -0600 Subject: mm: Documentation: clarify where the mTHP stats live The note about khugepaged counters references /proc/vmstat for the PMD case, but never mentions where the mTHPs stats can be found (i.e.: /sys/kernel/mm/transparent_hugepage/hugepages-kB/stats/) Add a small addition to this section for clarity. Also fix a missing period while we are at it. Link: https://lore.kernel.org/20260811-khugepaged_pte_refactor-v4-7-ddac39d61c4a@linux.dev Signed-off-by: Nico Pache (Red Hat) Reviewed-by: Baolin Wang Suggested-by: Lorenzo Stoakes Acked-by: David Hildenbrand (Arm) Reviewed-by: Zi Yan Acked-by: Pedro Falcato Reviewed-by: Lorenzo Stoakes (ARM) Reviewed-by: Lance Yang Cc: Barry Song Cc: Dev Jain Cc: Jonathan Corbet Cc: Liam R. Howlett Cc: Michal Hocko Cc: Mike Rapoport Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Usama Arif Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- Documentation/admin-guide/mm/transhuge.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 16f37135ed80..b187d618452f 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -224,7 +224,7 @@ khugepaged will be automatically started when any THP size is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when all THP sizes are disabled (when both the per-size anon control and the -top-level control are "never") +top-level control are "never"). process THP controls -------------------- @@ -301,7 +301,9 @@ being replaced by a PMD mapping, or (2) physical pages replaced by one hugepage of various sizes (PMD-sized or mTHP). Each may happen independently, or together, depending on the type of memory and the failures that occur. As such, this value should be interpreted roughly as a sign of progress, -and counters in /proc/vmstat consulted for more accurate accounting):: +and counters in /proc/vmstat consulted for more accurate accounting. +Per-order mTHP collapse statistics are also available under +/sys/kernel/mm/transparent_hugepage/hugepages-kB/stats/):: /sys/kernel/mm/transparent_hugepage/khugepaged/pages_collapsed -- cgit From 73b5d07990a0e6ea9cdf2c07fa8fbc865d398c1d Mon Sep 17 00:00:00 2001 From: Song Hu Date: Wed, 12 Aug 2026 15:57:39 +0800 Subject: Docs/mm: fix outdated "radix tree" in page_migration Steps 7 and 9 of the migration description still say "radix tree", unlike steps 5 and 11 which already use "i_pages lock". The page cache moved to the XArray at mapping->i_pages long ago. Use "page cache tree" for the two remaining references. Link: https://lore.kernel.org/20260812075739.325441-1-husong@kylinos.cn Signed-off-by: Song Hu Cc: David Hildenbrand Cc: Jonathan Corbet Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Randy Dunlap Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Matthew Wilcox Cc: Jan Kara Signed-off-by: Andrew Morton --- Documentation/mm/page_migration.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'Documentation') diff --git a/Documentation/mm/page_migration.rst b/Documentation/mm/page_migration.rst index 34602b254aa6..5b8d50308db1 100644 --- a/Documentation/mm/page_migration.rst +++ b/Documentation/mm/page_migration.rst @@ -110,13 +110,13 @@ Steps: 6. The refcount of the page is examined and we back out if references remain. Otherwise, we know that we are the only one referencing this page. -7. The radix tree is checked and if it does not contain the pointer to this - page then we back out because someone else modified the radix tree. +7. The page cache tree is checked and if it does not contain the pointer to this + page then we back out because someone else modified the page cache tree. 8. The new page is prepped with some settings from the old page so that accesses to the new page will discover a page with the correct settings. -9. The radix tree is changed to point to the new page. +9. The page cache tree is changed to point to the new page. 10. The reference count of the old page is dropped because the address space reference is gone. A reference to the new page is established because -- cgit From 5e0b9b71bcf405a0390ea9efc853bd07186c65a0 Mon Sep 17 00:00:00 2001 From: "Liam R. Howlett (Oracle)" Date: Fri, 21 Aug 2026 15:26:12 -0400 Subject: maple_tree: documentation fix Don't include the word flag in the quotes with the actual flag. Link: https://lore.kernel.org/20260821192627.4085470-5-liam@infradead.org Signed-off-by: Liam R. Howlett (Oracle) Cc: Boqun Feng Cc: Breno Leitao Cc: Chris Mason Cc: Chuck Lever Cc: Ingo Molnar Cc: Jason Gunthorpe Cc: Joe Perches Cc: Peter Zijlstra Cc: Rik van Riel Cc: Waiman Long Cc: Will Deacon Signed-off-by: Andrew Morton --- Documentation/core-api/maple_tree.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/core-api/maple_tree.rst b/Documentation/core-api/maple_tree.rst index ccdd1615cf97..34964ec88d17 100644 --- a/Documentation/core-api/maple_tree.rst +++ b/Documentation/core-api/maple_tree.rst @@ -211,7 +211,7 @@ Advanced Locking The maple tree uses a spinlock by default, but external locks can be used for tree updates as well. To use an external lock, the tree must be initialized -with the ``MT_FLAGS_LOCK_EXTERN flag``, this is usually done with the +with the ``MT_FLAGS_LOCK_EXTERN`` flag, this is usually done with the MTREE_INIT_EXT() #define, which takes an external lock as an argument. Functions and structures -- cgit From ee2487d9ba6ccf1b10c55fbae2cb0526c7b775e3 Mon Sep 17 00:00:00 2001 From: "Liam R. Howlett (Oracle)" Date: Fri, 21 Aug 2026 15:26:23 -0400 Subject: maple_tree: document erase and allocations better During a discussion on the maple tree erase process and GFP flags, Jason suggested there be an amendment to the documentation to clarify the situation on allocations within the tree. The added text is an attempt to better explain that the tree may allocate, even when erasing, and provide some guidance on how to work around such issues. [akpm@linux-foundation.org: tweak mtree_erase() description, per Jason] Link: https://lore.kernel.org/all/20260617180419.GA231643@ziepe.ca/ Link: https://lore.kernel.org/20260821192627.4085470-16-liam@infradead.org Signed-off-by: Liam R. Howlett (Oracle) Suggested-by: Jason Gunthorpe Cc: Rik van Riel Cc: Boqun Feng Cc: Breno Leitao Cc: Chris Mason Cc: Chuck Lever Cc: Ingo Molnar Cc: Joe Perches Cc: Peter Zijlstra Cc: Waiman Long Cc: Will Deacon Signed-off-by: Andrew Morton --- Documentation/core-api/maple_tree.rst | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) (limited to 'Documentation') diff --git a/Documentation/core-api/maple_tree.rst b/Documentation/core-api/maple_tree.rst index 34964ec88d17..12bccfb6aac1 100644 --- a/Documentation/core-api/maple_tree.rst +++ b/Documentation/core-api/maple_tree.rst @@ -17,7 +17,8 @@ supports iterating over a range of entries and going to the previous or next entry in a cache-efficient manner. The tree can also be put into an RCU-safe mode of operation which allows reading and writing concurrently. Writers must synchronize on a lock, which can be the default spinlock, or the user can set -the lock to an external lock of a different type. +the lock to an external lock of a different type. Note that external locks may +interfere with allocations in a low memory situation. The Maple Tree maintains a small memory footprint and was designed to use modern processor cache efficiently. The majority of the users will be able to @@ -42,6 +43,15 @@ successful store operation within a given code segment when allocating cannot be done. Allocations of nodes are relatively small at around 256 bytes. +Since the maple tree uses internal nodes that are allocated and has rules on +data density, erasing an entry may cause allocations to occur. That is, +erasing an entry may consume memory. Users must take care to ensure that they +do not violate the larger system constraints on when and how memory is +allocated. Most situations are fine to allocate, but the pre-allocation +support is provided as a mechanism to avoid trickier situations. There is also +the possibility of using special entries and clean up the tree later, in +extreme circumstances. + .. _maple-tree-normal-api: Normal API @@ -63,7 +73,10 @@ success or an error code otherwise. mtree_store_range() works in the same way but takes a range. mtree_load() is used to retrieve the entry stored at a given index. You can use mtree_erase() to erase an entire range by only knowing one value within that range, or mtree_store() call with an entry of -NULL may be used to partially erase a range or many ranges at once. +NULL may be used to partially erase a range or many ranges at once. Note that +mtree_erase() may use GFP_KERNEL | __GFP_NOFAIL for allocations and cannot +fail. mtree_erase() can sleep, so it must not be called from an atomic +context. If you want to only store a new entry to a range (or index) if that range is currently ``NULL``, you can use mtree_insert_range() or mtree_insert() which @@ -163,7 +176,10 @@ You can use mas_erase() to erase an entire range by setting index and last of the maple state to the desired range to erase. This will erase the first range that is found in that range, set the maple state index and last as the range that was erased and return the entry that existed -at that location. +at that location. Note that mas_erase() may allocate with the GFP_KERNEL +__GFP_NOFAIL and cannot fail, but may sleep. If this is not okay, consider +using mas_store_gfp() and pass it a ``NULL``, +after setting up the correct range by walking to the entry. You can walk each entry within a range by using mas_for_each(). If you want to walk each element of the tree then ``0`` and ``ULONG_MAX`` may be used as -- cgit