| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The contended_release tracepoint needs to hook queued_spin_unlock(), but
architectures with a custom unlock define queued_spin_unlock() directly,
leaving no single generic place to add the tracing.
Introduce queued_spin_release() as the arch-overridable release
primitive and make queued_spin_unlock() a generic wrapper around it.
An architecture that only customizes the release can then override
queued_spin_release() and inherit the generic wrapper.
Rename the MIPS override to queued_spin_release() accordingly. x86
paravirt overrides queued_spin_unlock() directly and is left unchanged.
No functional change intended.
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Juergen Gross <jgross@suse.com>
Link: https://patch.msgid.link/b8daabae6469ad72cc784a911f6cc43a6d45df3a.1785778551.git.d@ilvokhin.com
|
|
queued_spin_lock_slowpath() and queued_spin_unlock() are dispatched
through pv_ops_lock via the paravirt-ops ALTERNATIVE machinery, which
picks the target (native inline store / hypervisor call) once at boot
and cannot change at runtime.
Convert both to static_call(). The site becomes a direct call patched in
place (one byte smaller), and on native the unlock still collapses to
the inline "movb $0, (%rdi)" store, so the fast path is unchanged.
Unlike the ALTERNATIVE mechanism, a static_call() target can also be
updated at runtime via static_call_update(). This is a prerequisite for
the contended_release tracepoint, which has to swap in a traced unlock
while the system is running.
[ ilvokhin: commit message; fix PARAVIRT_SPINLOCKS=n build; teach
__static_call_validate() about the inline unlock insn; make the
slowpath site module-safe: static_call_mod() +
EXPORT_STATIC_CALL_TRAMP(); pass @lock to the callee-save unlock,
fixing a boot hang under CALL_DEPTH_TRACKING. Boot tested native + KVM
PV guest. ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Co-developed-by: Dmitry Ilvokhin <d@ilvokhin.com>
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/all/20260603120811.GW3493090@noisy.programming.kicks-ass.net/
Link: https://patch.msgid.link/9a32ae399eb804a02a31af04dcabe7e7ee4f3fdf.1785778551.git.d@ilvokhin.com
|
|
KVM lets userspace configure the IMSIC guest, HART, group-width, and
group-shift fields independently. The individual bounds allow a layout
such as guest_bits=7, hart_bits=6, group_bits=1, and group_shift=24.
That layout places the group index bit at bit 24 while the lower
guest+HART field also extends through bit 24. The IMSIC address helpers
later consume these values as one address layout: aia_imsic_ppn() masks
the lower guest+HART field and the group field, while
aia_imsic_hart_index() combines the group and HART index. Allowing an
overlapping configuration therefore gives one address bit two different
meanings.
Reject group-index layouts that start before the end of the guest+HART
field before initializing the in-kernel irqchip.
Fixes: 89d01306e34d ("RISC-V: KVM: Implement device interface for AIA irqchip")
Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260718040000.005.6ede2881-caplitmus-rv@iscas.ac.cn
Signed-off-by: Anup Patel <anup@brainfault.org>
|
|
KVM memslots describe half-open ranges [base_gfn, base_gfn + npages). A
slot ending exactly at the guest physical-address limit has its final page
below that limit and is valid.
The current greater-than-or-equal comparison rejects a valid boundary case.
Use a strict greater-than comparison, matching equivalent KVM architecture
checks.
Fixes: 9d05c1fee837 ("RISC-V: KVM: Implement stage2 page table programming")
Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260718033646.002.31dc89f2-caplitmus-rv@iscas.ac.cn
Signed-off-by: Anup Patel <anup@brainfault.org>
|
|
Pull vkm fixes from Paolo Bonzini:
"s390:
- fix a lot of small bugs and races
x86:
- fix missing locking related to KVM_CAP_MOVE_ENC_CONTEXT_FROM
- warn on creating a new page table that is the child of an invalid
one, and limit damage before it's too late
- disable use of INVLPGA when NPT is enabled, because it doesn't seem
to flush TLBs correctly"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (26 commits)
KVM: x86/mmu: WARN and clear role.invalid when creating a child shadow page
KVM: SVM: Serialize accesses to the owner and mirror list with separate lock
KVM: SVM: make svm_flush_tlb_gva do a full asid flush if NPT enabled
KVM: s390: Fix cleanup in kvm_s390_pv_create_cpu()
KVM: s390: Fix ordering when adding to SCA
KVM: s390: Return -EINTR if a signal is pending while faulting-in
KVM: s390: Free the mmu cache when kvm_arch_vcpu_create() fails
KVM: s390: ucontrol: Add missing locking around gmap_remove_child()
KVM: s390: cmma: Fix dirty tracking when removing memslot
KVM: s390: Fix race in __do_essa()
KVM: s390: Fix leaking of PGM_ADDRESSING to userspace
KVM: s390: ucontrol: Fix sca_clear_ext_call()
KVM: s390: Fix overclearing ESCA in case of error
KVM: s390: Fix kvm_s390_vcpu_unsetup_cmma()
KVM: s390: Do not free SCA if it was not allocated
KVM: s390: Fix unlikely NULL gmap dereference
s390/vfio_ccw: Implement a crw lock
s390/vfio_ccw: Selectively expand io_mutex
s390/vfio_ccw: Move cp cleanup out of not operational
s390/vfio_ccw: Cancel existing workqueues
...
|
|
The existing kvm_riscv_gstage_wp_range() walks the entire [start, end)
range to apply write protection for dirty log clearing, even when the
provided mask has zero bits (i.e., many GFNs do not need protection).
This leads to unnecessary page table walks when the mask is sparse.
Replace the range-based approach with a new function
kvm_riscv_gstage_wp_pt_masked() that iterates only over the set bits in
the mask. For each set bit, it looks up the leaf PTE and applies write
protection. Once a huge page is encountered, the entire huge-page range
is processed in one go, and the corresponding bits in the mask are cleared
using bitmap_clear().
Performance was measured with KVM selftests dirty_log_perf_test on a
Spacemit k3 host with the following configuration:
- vCPUs: 2 (-v 2)
- Memory: 1GB (-b 1G)
- Iterations: 3 (-i 3)
- Write percentage varied via -w parameter to simulate different
dirty mask densities.
The following data shows the time spent in the clear-dirty-log phase
(i.e., the KVM_CLEAR_DIRTY_LOG ioctl) under each configuration.
+------------------+------------------+------------------+-------------+
| Write Percentage | Original (s) | Patched (s) | Improvement |
+------------------+------------------+------------------+-------------+
| 10% | 0.012905 | 0.009213 | +28.6% |
| 30% | 0.014217 | 0.010287 | +27.6% |
| 50% | 0.014606 | 0.011772 | +19.4% |
| 70% | 0.014735 | 0.013199 | +10.4% |
| 100% | 0.014759 | 0.015294 | -3.6% |
+------------------+------------------+------------------+-------------+
The performance improvement is most significant when the dirty mask is
sparse (low write percentage), which is common in real-world scenarios
with low to moderate memory write intensity. In the worst-case scenario
where the mask is fully set (100% write), the optimization introduces
a slight 3.6% overhead due to the additional bit operations, which is
acceptable given the substantial gains in common cases.
This change significantly reduces the number of page-table walks when
the dirty mask has many zero bits, improving the efficiency of
KVM_CLEAR_DIRTY_LOG and related ioctls.
Signed-off-by: Wang Yechao <wang.yechao255@zte.com.cn>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260717080654.877151-1-wang.yechao255@zte.com.cn
Signed-off-by: Anup Patel <anup@brainfault.org>
|
|
Same problem as on x86-64: add_exception_handler() decides whether an
instruction is a load by its class, and a load-acquire is of BPF_STX
class even though it reads from src_reg into dst_reg. As a result ...
if (BPF_CLASS(insn->code) != BPF_LDX)
dst_reg = DONT_CLEAR;
... drops the register to clear, and ...
if (BPF_CLASS(insn->code) == BPF_LDX)
arena_reg = bpf2a64[insn->src_reg];
else
arena_reg = bpf2a64[insn->dst_reg];
... hands ex_handler_bpf() the value register instead of the address
register. A load-acquire from an arena pointer that faults on an
unmapped page is therefore reported as a WRITE at a bogus address,
and dst_reg keeps its previous value instead of being cleared to 0.
Note that emit_atomic_ld_st() already picks src_reg as the address
for BPF_LOAD_ACQ, so only the exception table metadata was out of sync
with the emitted access.
Same as on x86-64, use bpf_atomic_is_load_acq() so a load-acquire takes
the load path.
Fixes: 9bb12368d539 ("bpf, arm64: Support load-acquire and store-release instructions")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Puranjay Mohan <puranjay@kernel.org>
Link: https://lore.kernel.org/bpf/20260806201047.333389-4-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
A load-acquire from an arena pointer is converted to BPF_PROBE_ATOMIC and
gets an exception table entry, but the entry is filled in as if it were a
store, since populate_extable() decides based on instruction class alone
and a load-acquire is of BPF_STX class:
if (BPF_CLASS(insn->code) == BPF_LDX) {
arena_reg = reg2pt_regs[src_reg];
fixup_reg = reg2pt_regs[dst_reg];
} else {
arena_reg = reg2pt_regs[dst_reg];
fixup_reg = DONT_CLEAR;
}
For a load-acquire dst_reg holds the loaded value and src_reg holds the
address, so both assignments in the else branch are wrong. On a fault
over an unmapped arena page ex_handler_bpf() then:
- computes the reported address from the value register instead
of the address register
- reports the access as a WRITE, since it derives the direction
from fixup_reg == DONT_CLEAR
- leaves dst_reg untouched, so the program continues with a stale
value instead of the 0 that BPF_PROBE_* loads deliver
The access itself is emitted correctly, emit_atomic_ld_st_index() uses
src_reg as the address, so this is a broken probe contract and a wrong
diagnostic rather than a memory safety issue.
Use bpf_atomic_is_load_acq() helper so a load-acquire takes the load path.
Fixes: 5341c9a4d833 ("bpf, x86: Support load-acquire and store-release instructions")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260806201047.333389-3-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
A load-acquire is the only BPF_STX class instruction that reads from
src_reg into dst_reg, that is, it has the operand roles of a BPF_LDX.
JIT code which tells loads from stores apart by instruction class alone
has to special case it, for example when deciding which register holds
the faulting address and which one to clear from an exception handler.
riscv64 already does so, open coded as a bare insn->imm test. Add a
bpf_atomic_is_load_acq() helper and convert riscv64 over to it, so that
the x86-64 and arm64 JITs can use the same helper in subsequent patches.
Unlike bpf_atomic_is_load_store(), which presumes that its argument is
already known to be a BPF_ATOMIC instruction, the new helper is called
from code which still sees all instruction classes, so it checks class
and mode itself.
Also, move bpf_atomic_is_load_store() to filter.h next to BPF_ATOMIC_OP,
so that both helpers stay together. No functional change intended.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260806201047.333389-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The linear aliases of the kernel text and rodata are also mapped
read-only in the linear map. Given that the contents of these regions
are mostly identical to the version in the loadable image, mapping them
read-only and leaving their contents visible is a reasonable hardening
measure.
Data and bss, however, are now also mapped read-only but the contents of
these regions are more likely to contain data that we'd rather not leak.
So let's unmap these entirely in the linear map when the kernel is
running normally.
When going into hibernation or waking up from it, these regions need to
be mapped, so map the region initially, and toggle the valid bit so
map/unmap the region as needed.
Doing so is required because pages covering the kernel image are marked
as PageReserved, and therefore disregarded for snapshotting by the
hibernate logic unless they are mapped.
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Liz Prucka <lizprucka@google.com>
Cc: Seth Jenkins <sethjenkins@google.com>
Cc: Kees Cook <kees@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
|
|
Explicitly clear role.invalid when deriving a child shadow page's role from
its parent to harden against bugs elsewhere in KVM, as violating KVM's
invariant that invalid pages are NOT on the list of active MMU pages leads
to use-after-free due to __kvm_mmu_prepare_zap_page() using list_add()
instead of list_move() when processing an invalid shadow page, i.e. makes a
bad situation far worse.
Yell loudly if the parent is invalid, as it means KVM has missed a validity
check, i.e. KVM is attempting to map memory using an invalid/obsolete root,
but continue on as the child is otherwise still a valid shadow page.
==================================================================
BUG: KASAN: slab-use-after-free in __kvm_mmu_get_shadow_page+0x1817/0x1860 [kvm]
Write of size 8 at addr ff11000153dd1368 by task repro/853
CPU: 1 UID: 1000 PID: 853 Comm: repro Not tainted 7.2.0-rc2-3aec122bdcaf-next-vm #5 PREEMPT
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
Call Trace:
<TASK>
dump_stack_lvl+0x4b/0x70
print_report+0x153/0x49c
kasan_report+0xbc/0xf0
__kvm_mmu_get_shadow_page+0x1817/0x1860 [kvm]
mmu_alloc_root+0x141/0x320 [kvm]
kvm_mmu_load+0x612/0x20f0 [kvm]
kvm_arch_vcpu_ioctl_run+0x3dd5/0x6150 [kvm]
kvm_vcpu_ioctl+0x5e4/0x10d0 [kvm]
__x64_sys_ioctl+0x131/0x1b0
do_syscall_64+0x67/0x5f0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
</TASK>
Allocated by task 853:
kasan_save_stack+0x20/0x40
kasan_save_track+0x14/0x30
__kasan_slab_alloc+0x5f/0x70
kmem_cache_alloc_noprof+0xfe/0x2e0
__kvm_mmu_topup_memory_cache+0x135/0x530 [kvm]
paging64_page_fault+0x318/0x1e30 [kvm]
kvm_mmu_do_page_fault+0x21d/0x630 [kvm]
kvm_mmu_page_fault+0x18c/0x17b0 [kvm]
kvm_arch_vcpu_ioctl_run+0x1f35/0x6150 [kvm]
kvm_vcpu_ioctl+0x5e4/0x10d0 [kvm]
__x64_sys_ioctl+0x131/0x1b0
do_syscall_64+0x67/0x5f0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
Freed by task 853:
kasan_save_stack+0x20/0x40
kasan_save_track+0x14/0x30
kasan_save_free_info+0x3b/0x60
__kasan_slab_free+0x43/0x70
kmem_cache_free+0xe2/0x400
kvm_mmu_commit_zap_page.part.0+0x1e2/0x310 [kvm]
kvm_mmu_free_roots+0x283/0x560 [kvm]
kvm_arch_vcpu_ioctl_run+0x33c8/0x6150 [kvm]
kvm_vcpu_ioctl+0x5e4/0x10d0 [kvm]
__x64_sys_ioctl+0x131/0x1b0
do_syscall_64+0x67/0x5f0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Fixes: a770f6f28b1a ("KVM: MMU: Inherit a shadow page's guest level count from vcpu setup")
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM fixes from Andrew Morton:
"17 hotfixes. 15 are cc:stable. 16 are for MM.
There's a patch series from Lorenzo "mm: fix UAF caused by race
between ptdump and vmap pgtable freeing" which addresses a quite old
bug in the ptdump code.
And another series also from Lorenzo which fixes a four year old bug
in the huge_zero_folio handling.
A series from SJ fixes a few possible divide-by-zero issues which
Sashiko sniffed out. And a series which fixes handling of the
commit_inputs parameters.
The remainder are singletons, please see their changelogs for details"
* tag 'mm-hotfixes-stable-2026-08-06-18-44' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm:
mm/damon: adjust isolated pages stat for DAMOS_MIGRATE_{HOT,COLD}
mm/damon/ops-common: putback folios on invalid migrate nid
mm/huge_memory: initialise workingset state before folio split
mm/page_table_check: skip special zero mappings
mm/damon/lru_sort: skip damon_call() if ctx has not started
mm/damon/reclaim: skip damon_call() if ctx has not started
mm/damon/lru_sort: error out for >10000 active_mem_bp
samples/damon/mtier: error out for zero quota goal target values
mailmap: map old addresses to Danila Tikhonov
mm/huge_memory: separate out CONFIG_PERSISTENT_HUGE_ZERO_FOLIO logic
mm/huge_memory: fix huge_zero_pfn race
MAINTAINERS: update address for Brendan Jackman
mm/filemap: __filemap_add_folio() restore index before retrying
microblaze: restore the page alignment of swapper_pg_dir
arm64: remove redundant concurrent ptdump UAF mitigation
mm/ptdump: always stabilise against page table freeing using init_mm
mm/vmalloc: acquire init_mm lock on huge vmap to avoid ptdump UAF
|
|
We are frequently hitting the following splat during the riscv bpf
selftests:
00000000026dc75a: expected (7c3ff297) but got (00100073)
------------[ ftrace bug ]------------
ftrace failed to modify
[<ffffffff03c44c1c>] bpf_kfunc_common_test+0x4/0x20 [bpf_testmod]
actual: e7:82:c2:ce
Updating ftrace call site to call a different ftrace function
ftrace record flags: 80100002
(2)
expected tramp: ffffffff80043904
------------[ cut here ]------------
WARNING: kernel/trace/ftrace.c:2278 at ftrace_bug+0x46e/0x4b0, CPU#1: test_progs/98
...
[<ffffffff80008f4e>] ftrace_bug+0x46e/0x4b0
[<ffffffff803d3e86>] ftrace_replace_code+0x16e/0x170
[<ffffffff803d42b6>] ftrace_modify_all_code+0x12e/0x1b8
[<ffffffff800430f4>] arch_ftrace_update_code+0x14/0x28
[<ffffffff803e0324>] ftrace_startup+0x14c/0x2a0
[<ffffffff803e133c>] ftrace_startup_subops+0x584/0x1050
[<ffffffff804500e6>] register_ftrace_graph+0x4e6/0x1018
[<ffffffff804cf9f6>] register_fprobe_ips+0xc66/0x12f8
[<ffffffff8049abe8>] bpf_kprobe_multi_link_attach+0x5d8/0xe68
[<ffffffff8050fcaa>] __sys_bpf+0x3d5a/0x47f0
[<ffffffff805107ee>] __riscv_sys_bpf+0xae/0x168
[<ffffffff80034d78>] syscall_handler+0x60/0x100
[<ffffffff8228b4f4>] do_trap_ecall_u+0x174/0x208
[<ffffffff822b69c4>] handle_exception+0x16c/0x178
After debugging, it can be triggered by similar commands below:
```
echo do_nanosleep > set_ftrace_filter
echo function > current_tracer
echo 'p do_nanosleep' > kprobe_events
echo 1 > events/kprobes/enable
echo 'f do_nanosleep' > dynamic_events
echo 1 > events/fprobes/enable
```
The reason is that attaching a kprobe to an ftrace-traced function entry
replaces its initial auipc insn with ebreak. When ftrace_modify_call
later runs, it expects auipc insn, so verification fails and triggers
ftrace_bug.
The expected auipc logic remains conceptually unchanged, and kprobe
single-stepping ensures normal execution. Therefore, if the first insn
is ebreak, bypassing the check to continue patching the jalr insn is
safe and avoids ftrace failures.
Fixes: b2137c3b6d7a ("riscv: ftrace: prepare ftrace for atomic code patching")
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Link: https://patch.msgid.link/20260802094929.3978390-1-pulehui@huaweicloud.com
[pjw@kernel.org: fixed reproducer in commit message]
Signed-off-by: Paul Walmsley <pjw@kernel.org>
|
|
On arm64, __early_set_fixmap(), __late_set_fixmap() and
__late_clear_fixmap() are all __set_fixmap(). Calling
early_ioremap_reset() changes nothing. So remove the call and the macros.
No functional change.
Link: https://lore.kernel.org/20260708170647.362562-4-ekffu200098@gmail.com
Signed-off-by: Sang-Heon Jeon <ekffu200098@gmail.com>
Acked-by: Will Deacon <will@kernel.org>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
__late_set_fixmap() and __late_clear_fixmap() are only used after
early_ioremap_reset() has been called. riscv never calls it because
__set_fixmap() works before and after paging_init(). So remove them.
No functional change.
Link: https://lore.kernel.org/20260708170647.362562-3-ekffu200098@gmail.com
Signed-off-by: Sang-Heon Jeon <ekffu200098@gmail.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
As part of the changes converting VMA flags from a system word size to a
bitmap, extend this change to generic_get_unmapped_area() and
generic_get_unmapped_area_topdown(), which also allows us to convert
stack_guard_placement() as well.
We retain arch_get_unmapped_area() and arch_get_unmapped_area_topdown()
as-is for now, using legacy_to_vma_flags() as necessary to do so.
No functional change intended.
Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-4-0fa2357d5431@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Dave Airlie <airlied@gmail.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jann Horn <jannh@google.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Nico Pache <npache@redhat.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The core do_mmap() function accepts a vm_flags_t parameter which it then
manipulates before passing to mmap_region() to do the heavy lifting of the
memory mapping.
Update do_mmap() to instead accept a vma_flags_t parameter, and adjust all
the logic within do_mmap() to manipulate this instead.
This is as part of the ongoing effort to convert VMA flags from a system
word size to a bitmap type which allows us to unrestrict the number of VMA
flags, as well as gain control over how VMA flag manipulation occurs.
We do not cascade these changes to all functions which accept vm_flags_t,
but rather use vma_flags_to_legacy() where necessary, specifically
deferring converting calc_vm_prot_bits(), calc_vm_flag_bits() and
__get_unmapped_area() to vma_flags_t.
Also utilise the new vma_flags_can_grow() predicate which correctly
handles the case of architectures without upward growing stacks.
As part of this change, introduce VMA_SHADOW_STACK so we can correctly
handle the case of the shadow stack not being defined.
No functional change intended.
Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-2-0fa2357d5431@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Dave Airlie <airlied@gmail.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jann Horn <jannh@google.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Nico Pache <npache@redhat.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Cross-merge networking fixes after downstream PR (net-7.2-rc7).
No conflicts, or adjacent changes.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Correct a few white-space issues, like missing space before bracket '{'
character or spurious space, which will be flagged by dt-check-style
("redundant-whitespace" warning). No functional changes.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Link: https://patch.msgid.link/20260806070245.40219-3-krzysztof.kozlowski@oss.qualcomm.com
Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
|
|
MDIO clock index changed from 62:3 to 81:0 to match proper clock
definition in the SoC device tree. Clock Id 81:0 belongs to ICSSG0
core clock, where as 62 belongs to EQEP2 device.
See: https://software-dl.ti.com/tisci/esd/latest/5_soc_doc/am64x/clocks.html
Fixes: c9087e3898a1d0 ("arm64: dts: ti: k3-am64-main: Add ICSSG nodes")
Signed-off-by: Meghana Malladi <m-malladi@ti.com>
Link: https://patch.msgid.link/20260706092229.82674-1-m-malladi@ti.com
Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
|
|
Interaction between KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM and
KVM_CAP_VM_COPY_ENC_CONTEXT_FROM can cause two separate issues:
- in sev_migrate_from(), when the destination KVM is a mirror, the mirror
entry is moved from the source's list to the owner's mirror_vms list,
without holding the owner's lock unlike other writers of the owner's
mirror list (sev_vm_copy_enc_context_from(), sev_vm_destroy()).
A concurrent COPY or destroy can race with sev_migrate_from() and
corrupt the list.
- In sev_vm_destroy(), the *owner* is still active and could receive
concurrently a KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM that causes
sev->enc_context_owner to change. In this case the incorrect VM
receives kvm_put_kvm().
The second issue needs particular care because the owner could disappear
altogether (even though the race window is impossibly small) between
reading it and locking it. There is thus no way to perform the checks
under the owner lock without putting struct kvm under SLAB_TYPESAFE_BY_RCU
(which would allow kvm_get_kvm_safe() under RCU critical section).
It is much simpler to just use a global lock, since the critical
sections are so small and the new lock is always a leaf lock.
Fixes: b2125513dfc0 ("KVM: SEV: Allow SEV intra-host migration of VM with mirrors")
Cc: stable@vger.kernel.org
Reported-by: Shen Yongchao <grayhat@foxmail.com>
Link: https://lore.kernel.org/kvm/tencent_625C0F42824E542C72B34733392AF2C49709@qq.com/
Link: https://lore.kernel.org/kvm/tencent_DDC4E4352EC91CAC05A9A8F4E55E8C96730A@qq.com/
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
|
|
https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux into HEAD
KVM: s390: Misc fixes for 7.2
Fix a bunch of small issues that came up during the previous round of
fixes.
They are mostly extremely unlikely races, but they should be fixed
nonetheless.
|
|
Red Hat is seeing multiple reports of Windows memory corruptions
(and consequent BSODs) with hv-tlbflush=on, on AMD processors only.
The crashes, while extremely rare, happen even with a stock configuration,
but with Driver Verifier enabled they can be detected after approximately
200 VM hours. In particular, Alexander Lougovski measured the following:
- on AMD Turin, 15 crashes in 3300 VM hours
- on AMD Milan, 2 crashes in 500 VM hours (there are fewer hours
here due to the host being smaller)
- on Intel Sapphire Rapids, 0 crashes in 8000 VM hours
- on AMD Turin with full TLB flush (not exactly this patch but
similar), no crashes in ~2 weeks of run time which should also
be ~7000 VM hours
For Turin, the microcode version was 0x0b002162, which (assuming
this is the same issue) should not be affected by the problem listed in
https://knowledge.broadcom.com/external/article/419026/bsod-on-virtual-machines-running-on-amd.html;
on the other hand that problem should not apply to earlier processors.
AMD has not provided any information or analysis yet, and when we asked
we didn't know yet that it reproduced on Milan as well.
As to the workload, Alexander threw more or less everything at the same
time at the VM:
- a full Windows Defender scan every 30 minutes
- a disk I/O job
- a loop doing repeated mmap of system files (mostly to hope that
it triggers some consistency check in the Windows memory manager)
- SQL Express 2022 + StressDB (1.6M rows), with the host doing queries
(75% write/25% read) via sqlcmd
Driver Verifier is able to detect BSODs more or less at the same time as
the pages are freed. They mostly happen in the Windows Defender filter
driver, but occasionally also in the networking stack (e.g., afd.sys)
or elsewhere in the filesystem stack (e.g., fltmgr.sys).
The flush is issued from kvm_hv_vcpu_flush_tlb(), which receives the
cross-CPU requests from the Hyper-V TLB flush hypercalls via a kfifo
and is invoked by the KVM_REQ_HV_TLB_FLUSH request. The mechanism is
the same for both Intel and AMD, and the handler for both vendors is
a simple INVVPID(ADDR)/INVLPGA instruction.
Because the request is handled on the destination CPU, there is a question
of what happens if the VM is migrated across physical CPUs. In that case,
the INVLPGA instruction would use a stale svm->vmcb->control.asid; but
if anything that might do an *unnecessary* flush (on an asid that's being
used for another VM) and then pre_svm_run() would force a full TLB rebuild.
So, for lack of better ideas, this patch forces a full ASID bump in
svm_flush_tlb_gva(). To avoid paying the price on Intel and also to
avoid unnecessary loops on AMD, the flush_tlb_gva op now returns whether
it did a full flush or not; kvm_hv_vcpu_flush_tlb() takes note and exits
its loops immediately. While there is an obvious performance impact,
about half of the benefit from Hyper-V tlbflush is preserved (10% vs. 20%
on the SQL Server workload).
kvm_mmu_invalidate_addr() is the only other caller of the flush_tlb_gva op.
The change would have a performance impact on every intercepted INVLPG and,
for nested SVM, on every L1 INVLPGA. For INVLPGA specifically, this covers
the same suspected issue but for nested hypervisors, so it is correct to
apply the workaround; for INVLPG on shadow paging, instead, the impact
would be stronger and, due to lack of data, for now the use of INVLPGA is
left in place in svm_flush_tlb_gva().
Analyzed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Analyzed-by: Alexander Lougovski <alougovs@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
|
|
When reserved-memory regions with the "no-map" property are not
page-aligned, the kernel may accidentally map them into the linear
mapping, contradicting the no-map semantics.
The root cause is a mismatch between /proc/iomem's address boundaries
and the actual page table mapping boundaries:
1. /proc/iomem derives its ranges from memblock via
memblock_region_reserved_base_pfn/memblock_region_reserved_end_pfn,
which perform PFN rounding so the displayed boundaries are
page-aligned. This gives the impression that the no-map region
occupies whole pages.
2. However, memblock_mark_nomap() splits memblock.memory regions at
exact byte boundaries (memblock_isolate_range preserves raw DT
base/size with no alignment). When for_each_mem_range iterates the
non-NOMAP regions adjacent to a no-map region, it returns start/end
values that are NOT page-aligned — they are the precise byte
boundaries from the memblock split.
3. These sub-page-aligned values are passed to
__create_pgd_mapping_locked(), which does:
phys &= PAGE_MASK;
addr = virt & PAGE_MASK;
end = PAGE_ALIGN(virt + size);
The downward rounding of phys via PAGE_MASK extends the mapped
range backward into the adjacent no-map region, effectively
including no-map memory in the linear mapping.
For example, with 64K pages, reserved_region@A2000000 (base=0xA2000000,
size=0x8000, no-map) causes for_each_mem_range to return
start=0xA2008000 for the next mappable region. After phys &= PAGE_MASK,
the actual mapping starts at 0xA2000000 — the entire no-map region is
incorrectly mapped.
Fix this by rounding the mappable range inward to PAGE_SIZE boundaries
before passing it to __map_memblock: start is rounded UP and end is
rounded DOWN. This ensures the mapped area never overlaps with adjacent
no-map regions. The cost is at most one page of unmapped gap at each
boundary, which is preferable to violating no-map semantics.
Signed-off-by: liulhong617 <liulhong617@gmail.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Overriding the idle mechanism might be useful for debugging and performance
testing. Add a cmdline parameter for it, similar to the existing idle=
parameter already present for the x86 and ppc architectures.
It is also useful on platforms where the WFI instruction misbehaves,
such as Apple Silicon SoCs. Generally, a misbehaving instruction should
be treated as an erratum and patched using the alternatives framework.
However, in the Apple Silicon case we need more flexibility because it is
difficult to detect whether the erratum applies. For example, Linux VMs
inside macOS have the same MIDR and may even seem like they're running
in EL2 in the case of NV, but should continue using WFI (it's trapped and
handled correctly by the hypervisor there). Thus, we prefer to
let the m1n1 bootloader add the idle=nop parameter[1].
Link[1]: https://lore.kernel.org/all/99b69262-e54b-424e-baa2-96ef7013b87a@kernel.org/
Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Most EL1 exceptions already call local_daif_mask() before returning,
with the exception of debug exception handlers which do not change
DAIF, and the IRQ/FIQ/Error handlers.
However, DAIF get masked in kernel_exit() in all cases when returning
from EL1 C handlers anyway.
Move this masking from assembly to C by calling local_daif_mask()
before irqentry_nmi_exit(). Unlike the raw DAIF masking helper,
local_daif_mask() invokes trace_hardirqs_off(), so it must execute
while RCU is still watching.
Remove the disable_daif assembly macro, as this was its only use.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
When we resume from cpu_suspend() context tracking, specially,
ct_idle_exit() performs IRQ save/restore sequence. It doesn't cause
any functional issues since we have masked all exceptions prior
suspend and have not restored them. However, in case of pseudo-NMI PMR
can be set by firmware to arbitrary value, thus IRQ save/restore
routines manipulates this arbitrary value. Again, it doesn't cause any
issues since PMR variant of IRQ save helper carries a
__pmr_irqs_disabled_flags() guard.
Going forward __pmr_irqs_disabled_flags() guard will be gone and we
will call __pmr_local_irq_disable() unconditionally - that would cause
warning in case CONFIG_ARM64_DEBUG_PRIORITY_MASKING is set.
Initialize PMR to a value known to Linux on resume until the normal
exception restore path restores the saved DAIF and PMR state.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Commit 77345ef70445 ("arm64: suspend: Use cpuidle context helpers
in cpu_suspend()") added cpuidle helpers to handle PMR manipulation
and restoration to ensure that the CPU receives interrupts when suspended
and pseudo-NMIs are enabled.
However, those helpers are called in between a pair of `local_daif_save()`
and `local_daif_restore()`, which already configure the PMR as expected.
Effectively, `arm_cpuidle_save_irq_context()` is a no-op here,
even when using pseudo-NMIs, and `arm_cpuidle_restore_irq_context()`
would not restore proper interrupt masking configuration early enough
if there were unexpected changes during suspend or resume.
(This can be observed with Trusted Firmware A (TF-A) at EL3 handling
suspend through PSCI. Even though it should not be the case, TF-A can
reset `ICC_PMR_EL1` during CPU_SUSPEND, thus resuming the kernel
with an inconsistent priority mask value on hardware implementing
more than the minimum number of priority levels, such as Morello.)
Thus : remove the cpuidle context helpers as they do not do anything,
but keep the comment mentioning the need for interrupts to reach the CPU
if we are using pseudo-NMIs.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Sashiko AI has reported that if swsusp_mte_save_tags() for some reason
fails we return from swsusp_arch_suspend() with DAIF being masked -
that is not what we'd expect. Restore the saved DAIF state before
returning from the error path.
Fixes: ee11f332af96 ("arm64: mte: Save tags when hibernating")
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The arm64 hibernate code manages the exception masking in an unsound
way, leading to potential crashes and/or warnings during resume.
When a hibernation image is saved in `swsusp_arch_suspend()`, all DAIF
exceptions are masked (by virtue of `local_daif_save()`), and the
suspended image is saved assuming that all DAIF exceptions will remain
masked when the image is restored.
When a hibernation image is resumed by `swsusp_arch_resume()`, only
interrupts are masked (by virtue of `local_irq_disable()` in
`resume_target_kernel()`). When pseudo-NMI is enabled the DAIF.IF bits
will be clear, and regardless of pseudo-NMI the DAIF.DA bits will be
clear.
This means that there are two problems:
(1) It is possible to take Debug, SError, or pseudo-NMI exceptions
during the resume process. This is unsafe, as during the resume
process both the old ane new kernels will tranisently be in an
inconsistent state, and swsusp_arch_suspend_exit() won't retain
an executable mapping of any exception vectors.
Any exception taken here will be fatal and silent.
(2) When re-entering the resumed kernel, some DAIF bits will be clear
unexpectedly. This permits Debug, SError, or pseudo-NMI exceptions
to be taken for a short period while the resumed kernel is not yet
in a consistent state.
This is detected by CONFIG_ARM64_DEBUG_PRIORITY_MASKING.
Avoid these issues by masking all DAIF exceptions during resume.
Fixes: 82869ac57b5d ("arm64: kernel: Add support for hibernate/suspend-to-disk")
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Masking DAIF around the write to MDSCR_EL1 doesn't do anything: we can
write to sysregs with interrupts unmasked, and writing to PSTATE is
not a context synchronization event so it does not synchronize it.
This is done in the context of a general interrupt handling cleanup,
so it does not address the missing context synchronization for the
MDSCR_EL1 write, staying consistent with the current state. This
should be addressed in a future patch.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Last user of INIT_PSTATE_EL2 has gone with ae4b7e38e9a9 ("arm64: Allow
sticky E2H when entering EL1"), so remove it.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Add ID_AA64MMFR2_EL1.BBM based BBML3 feature detection in
cpu_supports_bbml3() so that cpus with the feature would
not have to be added into MIDR based supports_bbml3_list.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
[will: Tidy up cpu_supports_bbml3()]
Signed-off-by: Will Deacon <will@kernel.org>
|
|
- As bbml2_noabort is functionally equivalent to bbml3,
rename cpu/system_supports_bbml2_noabort to
cpu/system_supports_bbml3.
The ARM64 capability name is also renamed accordingly.
- As BBML2_NOABORT or the equivalent BBML3 is the
kernel requirement for setting up linear map with
block/contpte mappings and not BBML2, replace all
bbml2 references with bbml3.
FEAT_BBML3, is introduced as part of 2025 Architecture Extensions.
https://developer.arm.com/documentation/109697/2026_03/2025-Architecture-Extensions
No functional changes are introduced with this patch.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Add BBM_3 definition for ID_AA64MMFR2_EL1 register.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Add below cpus to the midr list, which supports
BBML2_NOABORT.
Cortex A520(AE)
Cortex A715
Cortex A720(AE)
Cortex A725
Neoverse N3
C1-Nano
C1-Pro
C1-Ultra
C1-Premium
C1-Ultra and C1-Premium both suffer from erratum 3683289,
where Break-Before-Make must be followed to avoid a livelock.
For both CPUs, the erratum is fixed from r1p1.
Hence we do not enable BBML2_NOABORT for CPU revisions <= r1p0.
The relevant SDENs are:
* C1-Ultra: https://developer.arm.com/documentation/111077/9-00/
* C1-Premium: https://developer.arm.com/documentation/111078/9-00/
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Add cputype definitions for C1-Nano.
The definition can be found in C1-Nano TRM,
https://developer.arm.com/documentation/107753/0002
as part of MIDR_EL1 bit descriptions.
This is going to be used in the bbml3 support list.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Add cputype definitions for Cortex-A520AE.
The definition can be found in Cortex-A520AE TRM,
https://developer.arm.com/documentation/107726/0001/
as part of MIDR_EL1 bit descriptions.
This is going to be used in the bbml3 support list.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
pte_present() is used as the basis for both pmd_present() and pud_present().
It is currently implemented as a macro composed of pte_val() and
pte_present_invalid().
When pte_present() or its higher-level variants are used directly with
ptep_get() or pXdp_get(), for example:
pte_present(ptep_get(pte));
pmd_present(pmdp_get(pmd));
pud_present(pudp_get(pud));
the macro expansion causes the compiler to evaluate the argument twice,
resulting in redundant loads. For example, pte_present() expands to:
!pte_val(READ_ONCE(*pte) || pte_present_invalid(READ_ONCE(*pte))
A typical example is pud_free_pmd_page(), where the expansion of
pmd_present() generates:
...
/* pmd_present() (x20 = pmdp) */
1b88: f9400288 ldr x8, [x20] // read pmdp.
1b8c: f9000fa8 str x8, [x29, #0x18]
1b90: 3707fec8 tbnz w8, #0x0, 0x1b68 <pud_free_pmd_page+0xd0>
1b94: f9400288 ldr x8, [x20] // redundant read of pmdp.
1b98: 8a170109 and x9, x8, x23
1b9c: f9000fa8 str x8, [x29, #0x18]
1ba0: f120013f cmp x9, #0x800
1ba4: 54fffe20 b.eq 0x1b68 <pud_free_pmd_page+0xd0>
1ba8: 17fffff4 b 0x1b78 <pud_free_pmd_page+0xe0>
...
Convert pte_present() to static inline function so that prevent the
generation of redundant code and move pte_valid() and
pte_present_invalid() further up so the inline function can use them.
After this change, the generated code becomes:
...
/* pmd_present() (x20 = pmdp) */
1a30: f9400288 ldr x8, [x20]
1a34: 8a170109 and x9, x8, x23
1a38: f9000fa8 str x8, [x29, #0x18]
1a3c: f120013f cmp x9, #0x800
1a40: 54fffe80 b.eq 0x1a10 <pud_free_pmd_page+0xd0>
1a44: 3607fee8 tbz w8, #0x0, 0x1a20 <pud_free_pmd_page+0xe0>
1a48: 17fffff2 b 0x1a10 <pud_free_pmd_page+0xd0>
...
This eliminates the redundant load and also reduces code size at
call sites using this pattern. For example, pud_free_pmd_page() shrinks
from 7,500 bytes to 7,148 bytes, a reduction of approximately 4.7%.
Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
When simdisk support is built as a loadable module,
simdisk_param_set_filename() receives a pointer into module::args and
stores each filename pointer as is.
In preparation for removing module::args, update the simdisk.filename
parameter code to copy the provided string. This is somewhat complicated by
the fact that simdisk support can also be built-in, in which case the
parameters are parsed during early boot before slab is available. In that
case, the command line itself is preserved for the lifetime of the kernel,
so continue storing the incoming pointer directly.
Reviewed-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
CONFIG_XEN_PVHVM_SMP is referenced only on x86 in Xen specific code,
so it can be replaced with CONFIG_SMP.
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260805082137.1214967-5-jgross@suse.com>
|
|
On x86 CONFIG_XEN_PVHVM is now a synonym of CONFIG_XEN.
In Xen specific x86 code it can be just dropped, in non-Xen specific
x86 code it can be replaced with CONFIG_XEN.
In architecture independent code it is used only where CONFIG_XEN is
defined, so it can be replaced with CONFIG_X86 there.
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260805082137.1214967-3-jgross@suse.com>
|
|
CONFIG_XEN depends on CONFIG_X86_LOCAL_APIC already, so the dependency
of CONFIG_XEN_PVHVM on CONFIG_X86_LOCAL_APIC can be dropped.
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260805082137.1214967-2-jgross@suse.com>
|
|
Replace min_t() with the simpler min() macro since the values are
unsigned and compatible.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260804112011.59416-4-thorsten.blum@linux.dev
|
|
Replace min_t() with the simpler min() macro since the values are
unsigned and compatible.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260804112011.59416-3-thorsten.blum@linux.dev
|
|
Base enablement patch to register performance monitoring hardware
support for power12. Patch introduce the raw event encoding format,
defines the supported list of events, config fields for the event
attributes and their corresponding bit values which are exported via
sysfs.
Signed-off-by: Athira Rajeev <atrajeev@linux.ibm.com>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Tested-by: Praveen K Pandey <praveen@linux.ibm.com>
Reviewed-by: Nikhil Kumar Singh <nikhilks@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260806051120.3703698-5-mahesh@linux.ibm.com
|
|
PVR value of 0x0f000008 means we are arch v3.2 compliant (i.e.
Power12). This is used by phyp and kvm when booting as a pseries guest
to detect the presence of new Power12 features and to enable the
appropriate hwcap and facility bits.
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Tested-by: Praveen K Pandey <praveen@linux.ibm.com>
Reviewed-by: Nikhil Kumar Singh <nikhilks@linux.ibm.com>
Reviewed-by: Amit Machhiwal <amachhiw@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260806051120.3703698-4-mahesh@linux.ibm.com
|
|
Add CPU table entries for raw mode.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Tested-by: Praveen K Pandey <praveen@linux.ibm.com>
Reviewed-by: Nikhil Kumar Singh <nikhilks@linux.ibm.com>
Reviewed-by: Amit Machhiwal <amachhiw@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260806051120.3703698-3-mahesh@linux.ibm.com
|
|
Current Power system firmware caps the PVR list array size at 16 entries
during CAS (Client Architecture Support) negotiation. Passing more than
capped size to older firmware could cause an undefined behaviour and
breaks compatibility.
Future Power system firmware releases will lift this restriction and
support greater than 16 array entries.
Ensure that when running on Power11 or below hardware, the number
of PVR entries passed during CAS negotiation does not exceed the
firmware-imposed limit of 16.
In prom_send_capabilities(), compute start_index to skip the oldest
leading pvrs[] entries when running on Power11 or below hardware, so
that the pointer passed to ibm,client-architecture-support points to
ibm_architecture_vec.pvrs[start_index], presenting exactly 16 entries to
firmware.
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Tested-by: Praveen K Pandey <praveen@linux.ibm.com>
Reviewed-by: Nikhil Kumar Singh <nikhilks@linux.ibm.com>
Reviewed-by: Amit Machhiwal <amachhiw@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260806051120.3703698-2-mahesh@linux.ibm.com
|
