summaryrefslogtreecommitdiff
path: root/arch/arm64/kvm/emulate-nested.c
AgeCommit message (Collapse)AuthorFilesLines
2026-08-19Merge branch 'kvm-arm64/feat-nv3' into nextOliver Upton1-5/+13
* kvm-arm64/feat-nv3: (29 commits) : Support for FEAT_NV2p1 and FEAT_NV3, courtesy of Marc Zyngier : : Couple of accelerations for nested virtualization: : : - Elide traps for EL1 alias registers when hardware guarantees that : EL2-only bits are stateful (NV2p1) : : - Use TGE-aware hardware to avoid unnecesary ERET/TLBI traps that : remain 'in-context' (i.e. L1 kernel/userspace) arm64: Add override for ID_AA64MMFR4_EL1.NV_frac KVM: arm64: selftest: Add NVHCR_EL2 to get-reg-list KVM: arm64: Expose FEAT_NV3 to guests KVM: arm64: Add FEAT_NV3 detection KVM: arm64: Engage NV3 TLBI trap elision KVM: arm64: Engage NV3 ERET trap elision KVM: arm64: Add NVHCR_EL2 context switching KVM: arm64: Add routing for NVHCR_EL2 trap KVM: arm64: Add NVHCR_EL2 handling to the sysreg array KVM: arm64: Add sanitisation for NVHCR_EL2 KVM: arm64: Make HCR_EL2 a non-VNCR register KVM: arm64: Add kvm_has_nv{2,3}() predicates KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation KVM: arm64: Split NV-specific exit fixups from the non-NV handling arm64: Add ARM64_HAS_NV3 capability arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3 arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2 arm64: Add FEAT_NV2p1 detection KVM: arm64: Expose FEAT_NV2p1 to NV guests KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present ... Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-07-31KVM: arm64: Add routing for NVHCR_EL2 trapMarc Zyngier1-0/+13
NVHCR_EL2 accesses from EL1 are taken to EL2 when HCRX_EL2.NVTGE==0 and HCR_EL2.NV==1. Describe this in the exception routing tables. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://patch.msgid.link/20260730071022.296811-23-maz@kernel.org Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-07-31KVM: arm64: Drop __HCRX_EL2_* masksMarc Zyngier1-5/+0
The __HCRX_EL2_* masks are a leftover from a time where we didn't have much sanitisation for the system registers. Since we are now in a better place, rely on the existing checks to detect unhandled bits in HCRX_EL2. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://patch.msgid.link/20260730071022.296811-5-maz@kernel.org Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-06-22KVM: arm64: Set IL for nested SError injectionFuad Tabba1-1/+1
kvm_inject_nested_serror() constructs an SError syndrome without IL. The architecture mandates IL=1 for SError unconditionally. Fixes: 77ee70a07357 ("KVM: arm64: nv: Honor SError exception routing / masking") Signed-off-by: Fuad Tabba <tabba@google.com> Link: https://patch.msgid.link/20260618121643.4105064-7-tabba@google.com Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-06-22KVM: arm64: Set IL for injected FPAC exceptions during ERET emulationFuad Tabba1-1/+1
The FPAC syndrome constructed during nested ERET emulation does not set IL. For FPAC (EC=0x1C), IL reflects the instruction length. ERET and its authenticated variants are always A64 32-bit instructions, so IL must be 1. Fixes: 213b3d1ea161 ("KVM: arm64: nv: Handle ERETA[AB] instructions") Suggested-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Fuad Tabba <tabba@google.com> Link: https://patch.msgid.link/20260618121643.4105064-5-tabba@google.com Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-06-17KVM: arm64: nv: Fix PSTATE construction on illegal exception returnFuad Tabba1-8/+24
kvm_check_illegal_exception_return() sourced the flags {N,Z,C,V} and masks {D,A,I,F} of the resulting PSTATE from the current PSTATE, but R_VWJHB takes them from the SPSR being returned to and leaves PSTATE.{EL,SP,nRW} (and EXLOCK when FEAT_GCS) unchanged. PAN, ALLINT and PM were not applied at all. Build the PSTATE by taking those fields from the SPSR while preserving EL, SP, nRW and EXLOCK from the current PSTATE, then set IL. Fixes: 47f3a2fc765a ("KVM: arm64: nv: Support virtual EL2 exceptions") Suggested-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/all/86wlvxs5r0.wl-maz@kernel.org/ Signed-off-by: Fuad Tabba <tabba@google.com> Link: https://patch.msgid.link/20260617144907.2972095-1-tabba@google.com [maz: tidied things a bit] Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-06-17KVM: arm64: nv: Write ESR_EL2 for injected nested SError exceptionsFuad Tabba1-0/+1
kvm_inject_el2_exception() writes ESR_EL2 for synchronous exceptions but not for SError. enter_exception64() does not write ESR_ELx for any exception type, so the constructed syndrome is dropped. A guest L2 hypervisor taking a nested SError observes stale ESR_EL2. This affects both kvm_inject_nested_serror() and the EASE path in kvm_inject_nested_sea(). Write ESR_EL2 for except_type_serror, matching except_type_sync. Fixes: 77ee70a07357 ("KVM: arm64: nv: Honor SError exception routing / masking") Reported-by: sashiko <sashiko@sashiko.dev> Signed-off-by: Fuad Tabba <tabba@google.com> Link: https://patch.msgid.link/20260615131116.390977-1-tabba@google.com Signed-off-by: Marc Zyngier <maz@kernel.org> Cc: stable@vger.kernel.org
2026-06-12Merge branch kvm-arm64/vgic-v5-PPI-fixes into kvmarm-master/nextMarc Zyngier1-0/+8
* kvm-arm64/vgic-v5-PPI-fixes: : . : Substantial cleanup of the vgic-v5 PPI support. From the original : cover letter: : : "With the GICv5 PPi support merged in, it has become obvious that a few : things could be improved, both from the correctness and maintainability : angles." : . KVM: arm64: Fix arch timer interrupts for GICv3-on-GICv5 guests irqchip/gic-v5: Immediately exec priority drop following activate Documentation: KVM: Clarify that PMU_V3_IRQ IntID requirements for GICv5 Documentation: KVM: Fix typos in VGICv5 documentation KVM: arm64: selftests: Improve error handling for GICv5 PPI selftest KVM: arm64: selftests: Cleanup unused vars in GICv5 PPI selftest KVM: arm64: selftests: Add missing GIC CDEN to no-vgic-v5 selftest KVM: arm64: vgic-v5: Atomically assign bits to PPI DVI bitmap KVM: arm64: vgic-v5: Add missing trap handing for NV triage KVM: arm64: vgic-v5: Limit support to 64 PPIs KVM: arm64: vgic: Rationalise per-CPU irq accessor KVM: arm64: vgic-v5: Drop defensive checks from vgic_v5_ppi_queue_irq_unlock() KVM: arm64: vgic: Consolidate vgic_allocate_private_irqs_locked() KVM: arm64: vgic: Constify struct irq_ops usage KVM: arm64: vgic-v5: Drop pointless ARM64_HAS_GICV5_CPUIF check KVM: arm64: vgic-v5: Remove use of __assign_bit() with a constant KVM: arm64: vgic-v5: Move PPI caps into kvm_vgic_global_state KVM: arm64: vgic-v5: Add for_each_visible_v5_ppi() iterator Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-05-23KVM: arm64: vgic-v5: Add missing trap handing for NV triageSascha Bischoff1-0/+8
As things stand, there is no support for Nested Virt with GICv5 guests yet. However, this is coming and therefore we need to be able to correctly triage the traps when running with NV. Add the missing fgtreg lookups required for that to triage_sysreg_trap(). These are specific to the FGT regs added as part of GICv5: * ICH_HFGRTR_EL2 * ICH_HFGWTR_EL2 * ICH_HFGITR_EL2 Fixes: 9d6d9514c08f ("KVM: arm64: gic-v5: Support GICv5 FGTs & FGUs") Link: https://sashiko.dev/#/patchset/20260319154937.3619520-1-sascha.bischoff%40arm.com Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com> Reviewed-by: Joey Gouly <joey.gouly@arm.com> Link: https://lore.kernel.org/r/20260520091949.542365-11-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-05-21KVM: arm64: nv: Track L2 to L1 exception emulationMarc Zyngier1-0/+4
While we currently track that we are emulating a nested ERET from L1 to L2, we currently don't track the reverse direction (an exception going from L2 to L1). Add a new vcpu state flag for this purpose, which will see some use shortly. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://patch.msgid.link/20260520085036.541666-2-maz@kernel.org
2026-03-19KVM: arm64: gic-v5: Support GICv5 FGTs & FGUsSascha Bischoff1-0/+68
Extend the existing FGT/FGU infrastructure to include the GICv5 trap registers (ICH_HFGRTR_EL2, ICH_HFGWTR_EL2, ICH_HFGITR_EL2). This involves mapping the trap registers and their bits to the corresponding feature that introduces them (FEAT_GCIE for all, in this case), and mapping each trap bit to the system register/instruction controlled by it. As of this change, none of the GICv5 instructions or register accesses are being trapped. Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Link: https://patch.msgid.link/20260319154937.3619520-14-sascha.bischoff@arm.com Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-02-05Merge branch kvm-arm64/misc-6.20 into kvmarm-master/nextMarc Zyngier1-0/+1
* kvm-arm64/misc-6.20: : . : Misc KVM/arm64 changes for 6.20 : : - Trivial FPSIMD cleanups : : - Calculate hyp VA size only once, avoiding potential mapping issues when : VA bits is smaller than expected : : - Silence sparse warning for the HYP stack base : : - Fix error checking when handling FFA_VERSION : : - Add missing trap configuration for DBGWCR15_EL1 : : - Don't try to deal with nested S2 when NV isn't enabled for a guest : : - Various spelling fixes : . KVM: arm64: nv: Avoid NV stage-2 code when NV is not supported KVM: arm64: Fix various comments KVM: arm64: nv: Add trap config for DBGWCR<15>_EL1 KVM: arm64: Fix error checking for FFA_VERSION KVM: arm64: Fix missing <asm/stackpage/nvhe.h> include KVM: arm64: Calculate hyp VA size only once KVM: arm64: Remove ISB after writing FPEXC32_EL2 KVM: arm64: Shuffle KVM_HOST_DATA_FLAG_* indices KVM: arm64: Fix comment in fpsimd_lazy_switch_to_host() Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-02-05Merge branch kvm-arm64/resx into kvmarm-master/nextMarc Zyngier1-9/+1
* kvm-arm64/resx: : . : Add infrastructure to deal with the full gamut of RESx bits : for NV. As a result, it is now possible to have the expected : semantics for some bits such as SCTLR_EL2.SPAN. : . KVM: arm64: Add debugfs file dumping computed RESx values KVM: arm64: Add sanitisation to SCTLR_EL2 KVM: arm64: Remove all traces of HCR_EL2.MIOCNCE KVM: arm64: Remove all traces of FEAT_TME KVM: arm64: Simplify handling of full register invalid constraint KVM: arm64: Get rid of FIXED_VALUE altogether KVM: arm64: Simplify handling of HCR_EL2.E2H RESx KVM: arm64: Move RESx into individual register descriptors KVM: arm64: Add RES1_WHEN_E2Hx constraints as configuration flags KVM: arm64: Add REQUIRES_E2H1 constraint as configuration flags KVM: arm64: Simplify FIXED_VALUE handling KVM: arm64: Convert HCR_EL2.RW to AS_RES1 KVM: arm64: Correctly handle SCTLR_EL1 RES1 bits for unsupported features KVM: arm64: Allow RES1 bits to be inferred from configuration KVM: arm64: Inherit RESx bits from FGT register descriptors KVM: arm64: Extend unified RESx handling to runtime sanitisation KVM: arm64: Introduce data structure tracking both RES0 and RES1 bits KVM: arm64: Introduce standalone FGU computing primitive KVM: arm64: Remove duplicate configuration for SCTLR_EL1.{EE,E0E} arm64: Convert SCTLR_EL2 to sysreg infrastructure Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-02-05KVM: arm64: Extend unified RESx handling to runtime sanitisationMarc Zyngier1-9/+1
Add a new helper to retrieve the RESx values for a given system register, and use it for the runtime sanitisation. This results in slightly better code generation for a fairly hot path in the hypervisor, and additionally covers all sanitised registers in all conditions, not just the VNCR-based ones. Reviewed-by: Fuad Tabba <tabba@google.com> Tested-by: Fuad Tabba <tabba@google.com> Link: https://patch.msgid.link/20260202184329.2724080-6-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-01-30KVM: arm64: nv: Add trap config for DBGWCR<15>_EL1Zenghui Yu (Huawei)1-0/+1
Seems that it was missed when MDCR_EL2 was first added to the trap forwarding infrastructure. Add it back. Fixes: cb31632c4452 ("KVM: arm64: nv: Add trap forwarding for MDCR_EL2") Signed-off-by: Zenghui Yu (Huawei) <zenghui.yu@linux.dev> Link: https://patch.msgid.link/20260130094435.39942-1-zenghui.yu@linux.dev Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-01-23Merge branch kvm-arm64/feat_idst into kvmarm-master/nextMarc Zyngier1-0/+21
* kvm-arm64/feat_idst: : . : Add support for FEAT_IDST, allowing ID registers that are not implemented : to be reported as a normal trap rather than as an UNDEF exception. : . KVM: arm64: selftests: Add a test for FEAT_IDST KVM: arm64: pkvm: Report optional ID register traps with a 0x18 syndrome KVM: arm64: pkvm: Add a generic synchronous exception injection primitive KVM: arm64: Force trap of GMID_EL1 when the guest doesn't have MTE KVM: arm64: Handle CSSIDR2_EL1 and SMIDR_EL1 in a generic way KVM: arm64: Handle FEAT_IDST for sysregs without specific handlers KVM: arm64: Add a generic synchronous exception injection primitive KVM: arm64: Add trap routing for GMID_EL1 arm64: Repaint ID_AA64MMFR2_EL1.IDS description Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-01-22KVM: arm64: nv: Return correct RES0 bits for FGT registersZenghui Yu (Huawei)1-1/+1
We had extended the sysreg masking infrastructure to more general registers, instead of restricting it to VNCR-backed registers, since commit a0162020095e ("KVM: arm64: Extend masking facility to arbitrary registers"). Fix kvm_get_sysreg_res0() to reflect this fact. Note that we're sure that we only deal with FGT registers in kvm_get_sysreg_res0(), the if (sr < __VNCR_START__) is actually a never false, which should probably be removed later. Fixes: 69c19e047dfe ("KVM: arm64: Add TCR2_EL2 to the sysreg arrays") Signed-off-by: Zenghui Yu (Huawei) <zenghui.yu@linux.dev> Link: https://patch.msgid.link/20260121101631.41037-1-zenghui.yu@linux.dev Signed-off-by: Marc Zyngier <maz@kernel.org> Cc: stable@vger.kernel.org
2026-01-22KVM: arm64: Always populate FGT masks at boot timeMarc Zyngier1-9/+9
We currently only populate the FGT masks if the underlying HW does support FEAT_FGT. However, with the addition of the RES1 support for system registers, this results in a lot of noise at boot time, as reported by Nathan. That's because even if FGT isn't supported, we still check for the attribution of the bits to particular features, and not keeping the masks up-to-date leads to (fairly harmess) warnings. Given that we want these checks to be enforced even if the HW doesn't support FGT, enable the generation of FGT masks unconditionally (this is rather cheap anyway). Only the storage of the FGT configuration is avoided, which will save a tiny bit of memory on these machines. Reported-by: Nathan Chancellor <nathan@kernel.org> Tested-by: Nathan Chancellor <nathan@kernel.org> Fixes: c259d763e6b09 ("KVM: arm64: Account for RES1 bits in DECLARE_FEAT_MAP() and co") Link: https://lore.kernel.org/r/20260120211558.GA834868@ax162 Link: https://patch.msgid.link/20260122085153.535538-1-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-01-15KVM: arm64: Handle FEAT_IDST for sysregs without specific handlersMarc Zyngier1-0/+13
Add a bit of infrastrtcture to triage_sysreg_trap() to handle the case of registers falling into the Feature ID space that do not have a local handler. For these, we can directly apply the FEAT_IDST semantics and inject an EC=0x18 exception. Otherwise, an UNDEF will do. Reviewed-by: Joey Gouly <joey.gouly@arm.com> Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com> Link: https://patch.msgid.link/20260108173233.2911955-5-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-01-15KVM: arm64: Add trap routing for GMID_EL1Marc Zyngier1-0/+8
HCR_EL2.TID5 is currently ignored by the trap routing infrastructure. Wire it in the routing table so that GMID_EL1, the sole register trapped by this bit, is correctly handled in the NV case. Link: https://patch.msgid.link/20260108173233.2911955-3-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
2026-01-15KVM: arm64: Account for RES1 bits in DECLARE_FEAT_MAP() and coMarc Zyngier1-23/+30
None of the registers we manage in the feature dependency infrastructure so far has any RES1 bit. This is about to change, as VTCR_EL2 has its bit 31 being RES1. In order to not fail the consistency checks by not describing a bit, add RES1 bits to the set of immutable bits. This requires some extra surgery for the FGT handling, as we now need to track RES1 bits there as well. There are no RES1 FGT bits *yet*. Watch this space. Reviewed-by: Fuad Tabba <tabba@google.com> Tested-by: Sascha Bischoff <sascha.bischoff@arm.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Tested-by: Fuad Tabba <tabba@google.com> Link: https://patch.msgid.link/20251210173024.561160-5-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-09-20Merge branch kvm-arm64/misc-6.18 into kvmarm-master/nextMarc Zyngier1-0/+1
* kvm-arm64/misc-6.18: : . : . : Misc improvements and bug fixes: : : - Fix XN handling in the S2 page table dumper : (20250809135356.1003520-1-r09922117@csie.ntu.edu.tw) : : - Fix sanitity checks for huge mapping with pKVM running np guests : (20250815162655.121108-1-ben.horgan@arm.com) : : - Fix use of TRBE when KVM is disabled, and Linux running under : a lesser hypervisor (20250902-etm_crash-v2-1-aa9713a7306b@oss.qualcomm.com) : : - Fix out of date MTE-related comments (20250915155234.196288-1-alexandru.elisei@arm.com) : : - Fix PSCI BE support when running a NV guest (20250916161103.1040727-1-maz@kernel.org) : : - Fix page reference leak when refusing to map a page due to mismatched attributes : (20250917130737.2139403-1-tabba@google.com) : : - Add trap handling for PMSDSFR_EL1 : (20250901-james-perf-feat_spe_eft-v8-7-2e2738f24559@linaro.org) : : - Add advertisement from FEAT_LSFE (Large System Float Extension) : (20250918-arm64-lsfe-v4-1-0abc712101c7@kernel.org) : . KVM: arm64: Expose FEAT_LSFE to guests KVM: arm64: Add trap configs for PMSDSFR_EL1 KVM: arm64: Fix page leak in user_mem_abort() KVM: arm64: Fix kvm_vcpu_{set,is}_be() to deal with EL2 state KVM: arm64: Update stale comment for sanitise_mte_tags() KVM: arm64: Return early from trace helpers when KVM isn't available KVM: arm64: Fix debug checking for np-guests using huge mappings KVM: arm64: ptdump: Don't test PTE_VALID alongside other attributes Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-09-19KVM: arm64: Add trap configs for PMSDSFR_EL1James Clark1-0/+1
SPE data source filtering (SPE_FEAT_FDS) adds a new register PMSDSFR_EL1, add the trap configs for it. PMSNEVFR_EL1 was also missing its VNCR offset so add it along with PMSDSFR_EL1. Tested-by: Leo Yan <leo.yan@arm.com> Signed-off-by: James Clark <james.clark@linaro.org> Reviewed-by: Joey Gouly <joey.gouly@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-08-15KVM: arm64: Correctly populate FAR_EL2 on nested SEA injectionMarc Zyngier1-1/+1
vcpu_write_sys_reg()'s signature is not totally obvious, and it is rather easy to write something that looks correct, except that... Oh wait... Swap addr and FAR_EL2 to restore some sanity in the nested SEA department. Fixes: 9aba641b9ec2a ("KVM: arm64: nv: Respect exception routing rules for SEAs") Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250813163747.2591317-1-maz@kernel.org Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-07-08KVM: arm64: Route SEAs to the SError vector when EASE is setOliver Upton1-0/+4
One of the finest additions of FEAT_DoubleFault2 is the ability for software to request *synchronous* external aborts be taken to the SError vector, which of coure are *asynchronous* in nature. Opinions be damned, implement the architecture and send SEAs to the SError vector if EASE is set for the target context. Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250708172532.1699409-18-oliver.upton@linux.dev Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-07-08KVM: arm64: nv: Describe trap behavior of SCTLR2_EL1Oliver Upton1-0/+12
Add the complete trap description for SCTLR2_EL1, including FGT and the inverted HCRX bit. Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250708172532.1699409-11-oliver.upton@linux.dev Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-07-08KVM: arm64: nv: Honor SError exception routing / maskingOliver Upton1-0/+14
To date KVM has used HCR_EL2.VSE to track the state of a pending SError for the guest. With this bit set, hardware respects the EL1 exception routing / masking rules and injects the vSError when appropriate. This isn't correct for NV guests as hardware is oblivious to vEL2's intentions for SErrors. Better yet, with FEAT_NV2 the guest can change the routing behind our back as HCR_EL2 is redirected to memory. Cope with this mess by: - Using a flag (instead of HCR_EL2.VSE) to track the pending SError state when SErrors are unconditionally masked for the current context - Resampling the routing / masking of a pending SError on every guest entry/exit - Emulating exception entry when SError routing implies a translation regime change Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250708172532.1699409-7-oliver.upton@linux.dev Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-07-08KVM: arm64: nv: Respect exception routing rules for SEAsOliver Upton1-0/+10
Synchronous external aborts are taken to EL2 if ELIsInHost() or HCR_EL2.TEA=1. Rework the SEA injection plumbing to respect the imposed routing of the guest hypervisor and opportunistically rephrase things to make their function a bit more obvious. Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250708172532.1699409-6-oliver.upton@linux.dev Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-07-08KVM: arm64: Add helper to identify a nested contextMarc Zyngier1-7/+2
A common idiom in the KVM code is to check if we are currently dealing with a "nested" context, defined as having NV enabled, but being in the EL1&0 translation regime. This is usually expressed as: if (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu) ... ) which is a mouthful and a bit hard to read, specially when followed by additional conditions. Introduce a new helper that encapsulate these two terms, allowing the above to be written as if (is_nested_context(vcpu) ... ) which is both shorter and easier to read, and makes more obvious the potential for simplification on some code paths. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250708172532.1699409-4-oliver.upton@linux.dev Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-05-23Merge branch kvm-arm64/nv-nv into kvmarm-master/nextMarc Zyngier1-7/+0
* kvm-arm64/nv-nv: : . : Flick the switch on the NV support by adding the missing piece : in the form of the VNCR page management. From the cover letter: : : "This is probably the most interesting bit of the whole NV adventure. : So far, everything else has been a walk in the park, but this one is : where the real fun takes place. : : With FEAT_NV2, most of the NV support revolves around tricking a guest : into accessing memory while it tries to access system registers. The : hypervisor's job is to handle the context switch of the actual : registers with the state in memory as needed." : . KVM: arm64: nv: Release faulted-in VNCR page from mmu_lock critical section KVM: arm64: nv: Handle TLBI S1E2 for VNCR invalidation with mmu_lock held KVM: arm64: nv: Hold mmu_lock when invalidating VNCR SW-TLB before translating KVM: arm64: Document NV caps and vcpu flags KVM: arm64: Allow userspace to request KVM_ARM_VCPU_EL2* KVM: arm64: nv: Remove dead code from ERET handling KVM: arm64: nv: Plumb TLBI S1E2 into system instruction dispatch KVM: arm64: nv: Add S1 TLB invalidation primitive for VNCR_EL2 KVM: arm64: nv: Program host's VNCR_EL2 to the fixmap address KVM: arm64: nv: Handle VNCR_EL2 invalidation from MMU notifiers KVM: arm64: nv: Handle mapping of VNCR_EL2 at EL2 KVM: arm64: nv: Handle VNCR_EL2-triggered faults KVM: arm64: nv: Add userspace and guest handling of VNCR_EL2 KVM: arm64: nv: Add pseudo-TLB backing VNCR_EL2 KVM: arm64: nv: Don't adjust PSTATE.M when L2 is nesting KVM: arm64: nv: Move TLBI range decoding to a helper KVM: arm64: nv: Snapshot S1 ASID tagging information during walk KVM: arm64: nv: Extract translation helper from the AT code KVM: arm64: nv: Allocate VNCR page when required arm64: sysreg: Add layout for VNCR_EL2 Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-19KVM: arm64: Handle TSB CSYNC trapsMarc Zyngier1-0/+1
The architecture introduces a trap for TSB CSYNC that fits in the same EC as LS64 and PSB CSYNC. Let's deal with it in a similar way. It's not that we expect this to be useful any time soon anyway. Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-19KVM: arm64: Add FGT descriptors for FEAT_FGT2Marc Zyngier1-0/+83
Bulk addition of all the FGT2 traps reported with EC == 0x18, as described in the 2025-03 JSON drop. Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-19KVM: arm64: Allow sysreg ranges for FGT descriptorsMarc Zyngier1-81/+42
Just like we allow sysreg ranges for Coarse Grained Trap descriptors, allow them for Fine Grain Traps as well. This comes with a warning that not all ranges are suitable for this particular definition of ranges. Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-19KVM: arm64: Add trap routing for FEAT_FGT2 registersMarc Zyngier1-0/+12
Similarly to the FEAT_FGT registers, pick the correct FEAT_FGT2 register when a sysreg trap indicates they could be responsible for the exception. Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-19KVM: arm64: Add sanitisation for FEAT_FGT2 registersMarc Zyngier1-0/+22
Just like the FEAT_FGT registers, treat the FGT2 variant the same way. THis is a large update, but a fairly mechanical one. The config dependencies are extracted from the 2025-03 JSON drop. Reviewed-by: Joey Gouly <joey.gouly@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-19KVM: arm64: nv: Remove dead code from ERET handlingMarc Zyngier1-7/+0
Cleanly, this code cannot trigger, since we filter this from the caller. Drop it. Reviewed-by: Oliver Upton <oliver.upton@linux.dev> Link: https://lore.kernel.org/r/20250514103501.2225951-16-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-10KVM: arm64: Handle PSB CSYNC trapsMarc Zyngier1-0/+1
The architecture introduces a trap for PSB CSYNC that fits in the same EC as LS64. Let's deal with it in a similar way as LS64. It's not that we expect this to be useful any time soon anyway. Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-10KVM: arm64: Use KVM-specific HCRX_EL2 RES0 maskMarc Zyngier1-0/+5
We do not have a computed table for HCRX_EL2, so statically define the bits we know about. A warning will fire if the architecture grows bits that are not handled yet. Reviewed-by: Joey Gouly <joey.gouly@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-06KVM: arm64: Add description of FGT bits leading to EC!=0x18Marc Zyngier1-6/+30
The current FTP tables are only concerned with the bits generating ESR_ELx.EC==0x18. However, we want an exhaustive view of what KVM really knows about. So let's add another small table that provides that extra information. Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-06KVM: arm64: Compute FGT masks from KVM's own FGT tablesMarc Zyngier1-0/+106
In the process of decoupling KVM's view of the FGT bits from the wider architectural state, use KVM's own FGT tables to build a synthetic view of what is actually known. This allows for some checking along the way. Reviewed-by: Joey Gouly <joey.gouly@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-06KVM: arm64: Simplify handling of negative FGT bitsMarc Zyngier1-37/+12
check_fgt_bit() and triage_sysreg_trap() implement the same thing twice for no good reason. We have to lookup the FGT register twice, as we don't communicate it. Similarly, we extract the register value at the wrong spot. Reorganise the code in a more logical way so that things are done at the correct location, removing a lot of duplication. Reviewed-by: Joey Gouly <joey.gouly@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-06KVM: arm64: Tighten handling of unknown FGT groupsMarc Zyngier1-2/+3
triage_sysreg_trap() assumes that it knows all the possible values for FGT groups, which won't be the case as we start adding more FGT registers (unless we add everything in one go, which is obviously undesirable). At the same time, it doesn't offer much in terms of debugging info when things go wrong. Turn the "__NR_FGT_GROUP_IDS__" case into a default, covering any unhandled value, and give the kernel hacker a bit of a clue about what's wrong (system register and full trap descriptor). Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-05-06arm64: sysreg: Replace HFGxTR_EL2 with HFG{R,W}TR_EL2Marc Zyngier1-77/+77
Treating HFGRTR_EL2 and HFGWTR_EL2 identically was a mistake. It makes things hard to reason about, has the potential to introduce bugs by giving a meaning to bits that are really reserved, and is in general a bad description of the architecture. Given that #defines are cheap, let's describe both registers as intended by the architecture, and repaint all the existing uses. Yes, this is painful. The registers themselves are generated from the JSON file in an automated way. Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-03-19Merge branch 'kvm-arm64/pmu-fixes' into kvmarm/nextOliver Upton1-2/+4
* kvm-arm64/pmu-fixes: : vPMU fixes for 6.15 courtesy of Akihiko Odaki : : Various fixes to KVM's vPMU implementation, notably ensuring : userspace-directed changes to the PMCs are reflected in the backing perf : events. KVM: arm64: PMU: Reload when resetting KVM: arm64: PMU: Reload when user modifies registers KVM: arm64: PMU: Fix SET_ONE_REG for vPMC regs KVM: arm64: PMU: Assume PMU presence in pmu-emul.c KVM: arm64: PMU: Set raw values from user to PM{C,I}NTEN{SET,CLR}, PMOVS{SET,CLR} Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-03-17KVM: arm64: PMU: Assume PMU presence in pmu-emul.cAkihiko Odaki1-2/+4
Many functions in pmu-emul.c checks kvm_vcpu_has_pmu(vcpu). A favorable interpretation is defensive programming, but it also has downsides: - It is confusing as it implies these functions are called without PMU although most of them are called only when a PMU is present. - It makes semantics of functions fuzzy. For example, calling kvm_pmu_disable_counter_mask() without PMU may result in no-op as there are no enabled counters, but it's unclear what kvm_pmu_get_counter_value() returns when there is no PMU. - It allows callers without checking kvm_vcpu_has_pmu(vcpu), but it is often wrong to call these functions without PMU. - It is error-prone to duplicate kvm_vcpu_has_pmu(vcpu) checks into multiple functions. Many functions are called for system registers, and the system register infrastructure already employs less error-prone, comprehensive checks. Check kvm_vcpu_has_pmu(vcpu) in callers of these functions instead, and remove the obsolete checks from pmu-emul.c. The only exceptions are the functions that implement ioctls as they have definitive semantics even when the PMU is not present. Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250315-pmc-v5-2-ecee87dab216@daynix.com Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-03-03KVM: arm64: nv: Request vPE doorbell upon nested ERET to L2Oliver Upton1-0/+2
Running an L2 guest with GICv4 enabled goes absolutely nowhere, and gets into a vicious cycle of nested ERET followed by nested exception entry into the L1. When KVM does a put on a runnable vCPU, it marks the vPE as nonresident but does not request a doorbell IRQ. Behind the scenes in the ITS driver's view of the vCPU, its_vpe::pending_last gets set to true to indicate that context is still runnable. This comes to a head when doing the nested ERET into L2. The vPE doesn't get scheduled on the redistributor as it is exclusively part of the L1's VGIC context. kvm_vgic_vcpu_pending_irq() returns true because the vPE appears runnable, and KVM does a nested exception entry into the L1 before L2 ever gets off the ground. This issue can be papered over by requesting a doorbell IRQ when descheduling a vPE as part of a nested ERET. KVM needs this anyway to kick the vCPU out of the L2 when an IRQ becomes pending for the L1. Link: https://lore.kernel.org/r/20240823212703.3576061-4-oliver.upton@linux.dev Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250225172930.1850838-13-maz@kernel.org Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-03-03arm64: sysreg: Add layout for ICH_HCR_EL2Marc Zyngier1-8/+8
The ICH_HCR_EL2-related macros are missing a number of control bits that we are about to handle. Take this opportunity to fully describe the layout of that register as part of the automatic generation infrastructure. This results in a bit of churn, unfortunately. Reviewed-by: Andre Przywara <andre.przywara@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20250225172930.1850838-2-maz@kernel.org Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
2025-01-17Merge branch kvm-arm64/nv-timers into kvmarm-master/nextMarc Zyngier1-2/+56
* kvm-arm64/nv-timers: : . : Nested Virt support for the EL2 timers. From the initial cover letter: : : "Here's another batch of NV-related patches, this time bringing in most : of the timer support for EL2 as well as nested guests. : : The code is pretty convoluted for a bunch of reasons: : : - FEAT_NV2 breaks the timer semantics by redirecting HW controls to : memory, meaning that a guest could setup a timer and never see it : firing until the next exit : : - We go try hard to reflect the timer state in memory, but that's not : great. : : - With FEAT_ECV, we can finally correctly emulate the virtual timer, : but this emulation is pretty costly : : - As a way to make things suck less, we handle timer reads as early as : possible, and only defer writes to the normal trap handling : : - Finally, some implementations are badly broken, and require some : hand-holding, irrespective of NV support. So we try and reuse the NV : infrastructure to make them usable. This could be further optimised, : but I'm running out of patience for this sort of HW. : : [...]" : . KVM: arm64: nv: Fix doc header layout for timers KVM: arm64: nv: Document EL2 timer API KVM: arm64: Work around x1e's CNTVOFF_EL2 bogosity KVM: arm64: nv: Sanitise CNTHCTL_EL2 KVM: arm64: nv: Propagate CNTHCTL_EL2.EL1NV{P,V}CT bits KVM: arm64: nv: Add trap routing for CNTHCTL_EL2.EL1{NVPCT,NVVCT,TVT,TVCT} KVM: arm64: Handle counter access early in non-HYP context KVM: arm64: nv: Accelerate EL0 counter accesses from hypervisor context KVM: arm64: nv: Accelerate EL0 timer read accesses when FEAT_ECV in use KVM: arm64: nv: Use FEAT_ECV to trap access to EL0 timers KVM: arm64: nv: Publish emulated timer interrupt state in the in-memory state KVM: arm64: nv: Sync nested timer state with FEAT_NV2 KVM: arm64: nv: Add handling of EL2-specific timer registers Signed-off-by: Marc Zyngier <maz@kernel.org>
2025-01-12Merge branch kvm-arm64/pkvm-fixed-features-6.14 into kvmarm-master/nextMarc Zyngier1-1/+1
* kvm-arm64/pkvm-fixed-features-6.14: (24 commits) : . : Complete rework of the pKVM handling of features, catching up : with the rest of the code deals with it these days. : Patches courtesy of Fuad Tabba. From the cover letter: : : "This patch series uses the vm's feature id registers to track the : supported features, a framework similar to nested virt to set the : trap values, and removes the need to store cptr_el2 per vcpu in : favor of setting its value when traps are activated, as VHE mode : does." : : This branch drags the arm64/for-next/cpufeature branch to solve : ugly conflicts in -next. : . KVM: arm64: Fix FEAT_MTE in pKVM KVM: arm64: Use kvm_vcpu_has_feature() directly for struct kvm KVM: arm64: Convert the SVE guest vcpu flag to a vm flag KVM: arm64: Remove PtrAuth guest vcpu flag KVM: arm64: Fix the value of the CPTR_EL2 RES1 bitmask for nVHE KVM: arm64: Refactor kvm_reset_cptr_el2() KVM: arm64: Calculate cptr_el2 traps on activating traps KVM: arm64: Remove redundant setting of HCR_EL2 trap bit KVM: arm64: Remove fixed_config.h header KVM: arm64: Rework specifying restricted features for protected VMs KVM: arm64: Set protected VM traps based on its view of feature registers KVM: arm64: Fix RAS trapping in pKVM for protected VMs KVM: arm64: Initialize feature id registers for protected VMs KVM: arm64: Use KVM extension checks for allowed protected VM capabilities KVM: arm64: Remove KVM_ARM_VCPU_POWER_OFF from protected VMs allowed features in pKVM KVM: arm64: Move checking protected vcpu features to a separate function KVM: arm64: Group setting traps for protected VMs by control register KVM: arm64: Consolidate allowed and restricted VM feature checks arm64/sysreg: Get rid of CPACR_ELx SysregFields arm64/sysreg: Convert *_EL12 accessors to Mapping ... Signed-off-by: Marc Zyngier <maz@kernel.org> # Conflicts: # arch/arm64/kvm/fpsimd.c # arch/arm64/kvm/hyp/nvhe/pkvm.c
2025-01-02KVM: arm64: nv: Add trap routing for CNTHCTL_EL2.EL1{NVPCT,NVVCT,TVT,TVCT}Marc Zyngier1-2/+56
For completeness, fun, and cerebral meltdown, add the virtualisation related traps to the counter and timers. Acked-by: Oliver Upton <oliver.upton@linux.dev> Link: https://lore.kernel.org/r/20241217142321.763801-9-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
generated by cgit (git 2.54.0.dirty) at 2026-09-28 05:18:43 +0000