| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Add a new set of predicates indicating whether VM is capable of
NV2, NV3, and is in a nested NV3 context.
This is going to become useful as we start dealing with a mix of
behaviours (NV2, NV3, NV2 on NV3...).
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-19-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Expose the FEAT_NV3 control bits to the sanitisation code so that
KVM stops moaning about the unattributed bits.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-18-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
In order to facilitate further changes, move the NV handling of
early fixups in its own helper. This also makes the code slightly
simpler to parse.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-17-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
As a bunch of KVM code is going to depend on FEAT_NV3 being detected
on the host, add a new capability that will describe it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-16-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
FEAT_NV3 introduces 4 new HCRX_EL2 control bits. Describe them
in the sysreg file.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-15-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
FEAT_NV3 introduces a new register that contains the HCR_EL2 value
exposed to a NV guest. As such, it has the exact same layout as
HCR_EL2.
Describe NVHCR_EL2 as a mapping to HCR_EL2.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-14-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Add the necessary NV2p1 probing to the cpufeature infrastructure.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-13-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Since NV2p1 is reducing the number of traps, it is valuable to expose
it to NV guests. Do so.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-12-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
With NV2p1, it is no longer necessary to use the split approach
where bits of CNTHCTL_EL2 cannot be accessed via CNTKCTL_EL1,
and we can treat the CNTKCTL_EL1 accessor as if it was "normal".
Key the special casing on FEAT_NV2P1 not being implemented.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-11-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
With FEAT_NV2P1, it is no longer necessary to trap CPTR_EL2 accesses
via CPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful.
Prevent such trapping and context switch CPACTR_EL1 in NV contexts
when NV2P1 is present.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-10-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
As we're about to deal with FEAT_NV2P1, add a new capability that
will be used to key any support for it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-9-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
We currently avoid using the ERET fast path if the guest has HCR_EL2.NV
set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.
Similarly, evaluating HFGITR_EL2.ERET makes little sense, as
this only applies to the nested context, while the ERET fast-path
is purely for the benefit of L1.
Replace these bizarre checks with is_nested_ctxt() which makes a lot
more sense: if we are running an L2, the ERET trap must go to L1.
Fixes: dd0717a998f77 ("KVM: arm64: nv: Fast-track 'InHost' exception returns")
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-8-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
It may not be obvious unless you look at it closely, but CPTR_EL2
is treated very differently from other registers. It is one the
registers that, despite looking very similar between EL1 and EL2
when E2H==1, have RES0 bits that get in the way.
Make it clear that CPTR_EL2 is odd by classifying it as SR_LOC_SPECIAL,
just like CNTHCTL_EL2 (and for the same reasons). This makes it
possible to use vcpu_read_sys_reg() with it, and will be necessary
once we support FEAT_NV2P1.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-7-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
HCRX_EL2.SRMASKEn is a new bit enabling FEAT_SRMASK for a guest.
We don't plan to support it any time soon, but it doesn't hurt to
actively document it, specially as we are going to add more bits
we actually care about.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-6-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The __HCRX_EL2_* masks are a leftover from a time where we didn't
have much sanitisation for the system registers. Since we are now
in a better place, rely on the existing checks to detect unhandled
bits in HCRX_EL2.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-5-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The way we merge the guest-provided HCRX_EL2 value with the host's
is bonkers. We try to make it look like the FGT registers by using
positive and negative polarities for traps, but most of these bits
are not strictly about trapping, as they actively change the way
some architectural state is managed.
It would be far better to deal with these bits like we do for
HCR_EL2, by enumerating the list of bits we don't allow the guest
to override. This is simplified by the fact that HCRX_EL2 only
affects EL1, and not EL2.
Re-jig the HCRX_EL2 handling with a macro that list the bits excluded
from the merge (TMEA, PTTWI, EnIDCP128).
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-4-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
ID_AA64MMFR4_EL1 has gained a few fields and enum values in the past
few months, so resync its definition with the 2026-03 JSON release.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-3-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The sysreg file is using the Mapping or Fields qualifiers to indicate
that a given encoding is only a mapping to a particular register (or
an instance of a more generic register definition).
As a result, we don't output any definition, and instead expect
the canonical definitions to be used.
This works rather well for individual fields, but creates problems
for macros that refer to more generic classes of bits such as RESx.
Relax the above rule by emitting the RESx and UNKN values for Mapping
and Fields qualifiers as well.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-2-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Add support for additional 32-bit RGB pixel formats (BGRA32, RGBA32,
BGRX32, RGBX32, ARGB2101010).
Formats with alpha channel (BGRA32, RGBA32) only support capture as
ISI ignores alpha bits when reading from memory.
Signed-off-by: Robert Chiras <robert.chiras@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Guoniu Zhou <guoniu.zhou@oss.nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Loic Poulain <loic.poulain@oss.qualcomm.com>
Link: https://patch.msgid.link/20260728-isi-v5-4-1d22ab91602a@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
Add support for 16-bit raw Bayer formats (SBGGR16, SGBRG16, SGRBG16,
SRGGB16) to both the pipeline subdev and video capture interface.
These formats are used by image sensors that output 16-bit raw data,
enabling the ISI to capture full dynamic range from such sensors.
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@oss.nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Guoniu Zhou <guoniu.zhou@oss.nxp.com>
Reviewed-by: Loic Poulain <loic.poulain@oss.qualcomm.com>
Link: https://patch.msgid.link/20260728-isi-v5-3-1d22ab91602a@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
multiplexed streams
The ISI crossbar needs to properly enable multiple streams from different
virtual channels on the same input pad. Currently only the first stream
gets enabled in hardware, subsequent streams are silently ignored.
The driver uses a single enable_count per input to track the input state.
When enable_count is non-zero, the code assumes the input is already active
and skips calling v4l2_subdev_enable_streams() for additional streams:
Call 1: enable_streams(stream 0)
-> enable_count == 0, enable gasket and stream 0 in hardware
-> enable_count = 1
Call 2: enable_streams(stream 1)
-> enable_count == 1, skip hardware enable
-> enable_count = 2
-> stream 1 never gets enabled
Similarly on disable, when enable_count reaches zero, ALL streams are
disabled regardless of which streams are actually still active.
Implement per-stream state tracking by storing the input index and stream
mask in the mxc_isi_pipe structure. On enable, record which input and
stream the pipe is receiving. On disable, clear the pipe's record and
check if any other pipe is still using the same input stream before
actually disabling it.
Signed-off-by: Guoniu Zhou <guoniu.zhou@oss.nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260728-isi-v5-2-1d22ab91602a@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
Use BIT_ULL() instead of BIT() for u64 stream masks to avoid incorrect
results on 32-bit architectures when stream IDs are 32 or greater.
Fixes: cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")
Cc: stable@vger.kernel.org
Signed-off-by: Guoniu Zhou <guoniu.zhou@oss.nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Loic Poulain <loic.poulain@oss.qualcomm.com>
Link: https://patch.msgid.link/20260728-isi-v5-1-1d22ab91602a@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
Fix the ISI input format for the color map V4L2_PIX_FMT_XBGR32 in
memory-to-memory mode.
Fixes: cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")
Cc: stable@vger.kernel.org
Signed-off-by: Guoniu Zhou <guoniu.zhou@oss.nxp.com>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260720-isi-v2-5-45845bc5d4fa@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
Add validation to enforce hardware constraints that were previously
missing in the crossbar routing configuration:
- SOURCE stream must be 0 (ISI pipes are hardcoded to stream 0)
This check complements the existing memory input validation and ensures
that all routing configurations respect hardware limitations.
Signed-off-by: Guoniu Zhou <guoniu.zhou@oss.nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Link: https://patch.msgid.link/20260720-isi-v2-2-45845bc5d4fa@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
The crossbar routing validation has a critical bug where it validates
the wrong routing table, allowing userspace to bypass validation entirely.
The __mxc_isi_crossbar_set_routing() function is called to validate and
apply a new routing table from userspace. However, the validation loop
iterates over state->routing (the currently active routing table) instead
of the routing parameter (the new table being validated):
for_each_active_route(&state->routing, route) {
This means userspace can submit any invalid routing configuration and it
will pass validation as long as the currently active routing is valid.
This is a security issue as it allows userspace to configure routes that
violate hardware constraints, potentially causing undefined hardware
behavior.
Fix by validating the routing table that will actually be applied.
Fixes: cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")
Cc: stable@vger.kernel.org
Signed-off-by: Guoniu Zhou <guoniu.zhou@oss.nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260720-isi-v2-1-45845bc5d4fa@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
The ISI supports different numbers of virtual channels depending on the
platform. i.MX95 supports 8 virtual channels, and i.MX8QXP/QM support 4
virtual channels. They are used in multiple camera use cases, such as
surround view. Other platforms (such as i.MX8/MN/MP/ULP/91/93) don't
support virtual channels, and the VC_ID bits are marked as read-only.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Guoniu Zhou <guoniu.zhou@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Link: https://patch.msgid.link/20260521-isi_vc-v5-2-a38eb4fcd58e@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
Implement the get_frame_desc pad operation for the crossbar subdev using
the v4l2_subdev_get_frame_desc_passthrough() helper. This allows the
crossbar to properly propagate frame descriptors from its sink pads to
its source pads, which is necessary for proper stream configuration in
multiplexed streams scenarios.
Signed-off-by: Guoniu.zhou <guoniu.zhou@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Link: https://patch.msgid.link/20260521-isi_vc-v5-1-a38eb4fcd58e@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
The total scaling factor including bi-linear downscaling and decimation
is clamped to 16. The bilinear factor calculation therefore produceds
values guaranteed not to exceed the maximum factor of 2.0. The clamping
is unneeded, drop it.
Note that the ISI_DOWNSCALE_THRESHOLD value of 0x4000 is incorrect, as
that would be a factor of 4.0. This was inconsequential given that the
computed factor never exceeded 0x2000.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Guoniu Zhou <guoniu.zhou@nxp.com>
Link: https://patch.msgid.link/20260520202738.86782-1-laurent.pinchart@ideasonboard.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
The i.MX8ULP variant does not require the fsl,blk-ctrl property. Add
fsl,imx8ulp-isi to the exception list alongside fsl,imx91-isi.
Fixes: 288517a3c6c9 ("dt-bindings: media: nxp,imx8-isi: Add i.MX8ULP ISI compatible string")
Cc: stable@vger.kernel.org
Signed-off-by: Guoniu Zhou <guoniu.zhou@oss.nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://patch.msgid.link/20260424-csi2_imx8ulp-v12-1-da148eabc035@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
list
Add Frank Li as a reviewer and the i.MX mailing list for the i.MX8 ISI
driver. This helps ensure patches receive review by the NXP i.MX
maintainers.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Link: https://patch.msgid.link/20260630163456.3317624-1-Frank.Li@oss.nxp.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
Add a test for the two stage-2 granularity changes dirty logging forces
at fault time, asserting the guest completes with no KVM_RUN error. The
first scenario collapses a page into a hugetlb-backed block: it writes
under logging, re-write-protects the page via GET_DIRTY_LOG, then writes
again with logging off. The second splits blocks: it faults in several
non-executable 2M blocks, enables logging, then executes in each block so
an execute permission fault splits it. It is skipped when CTR_EL0.DIC is
set, since mappings are then executable and no execute fault occurs.
Both paths make the fault handler allocate under mmu_lock, so a backend
that fails to stage that memory returns a KVM_RUN error or crashes the
host. The property holds on any host. On a pKVM host, where a
non-protected guest uses the pkvm_pgtable_*() backend, it also guards
that backend's fault-time staging.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-8-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Under pKVM the stage-2 walker resolves to pkvm_pgtable_stage2_split(), a
WARN_ON_ONCE(1) stub, yet KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE is still
enabled and reported for non-protected guests: the capability check keys
on the per-VM protected state while the walker dispatch keys on the
host-global mode. Enabling the cap and then dirty-logging the guest
reaches the stub, splatting a userspace-reachable WARN.
Reject the capability, and stop reporting a chunk size and the
supported block sizes, for every VM once pKVM is enabled, keyed on the
host-global mode like the split dispatch. Gating only protected VMs
would leave the non-protected guests that reach the stub still able to
enable it. Userspace decides whether eager splitting is available from
the block-size bitmap (QEMU falls back to no eager splitting when it
reads 0), so leaving it advertised steers an explicit request into the
enable failure instead of the fallback.
Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-7-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
pkvm_pgtable_stage2_map() wraps the __pkvm_host_share_guest() and
__pkvm_host_donate_guest() return in WARN_ON(), but those hypercalls
fail for reasons that are not EL1 invariant violations: -EINVAL for a
pfn that is not memblock RAM (check_range_allowed_memory() rejects a
device page mapped into a non-protected guest) and -ENOMEM under
memcache pressure. Both are reachable from a guest fault, so the WARN
splats on host input.
Return the error without warning. The unshare and write-protect WARNs
stay, since a failure there does signal a broken EL1 invariant.
Fixes: 3669ddd8fa8b5 ("KVM: arm64: Add a range to pkvm_mappings")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-6-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
pkvm_pgtable_stage2_flush() cleans the D-cache for every mapping in the
range even on hardware with stage-2 Force Write-Back, where FWB keeps
guest memory coherent to the PoC and the maintenance is unnecessary. The
generic kvm_pgtable_stage2_flush() returns early in that case, but the
pKVM MMU does not, so it needlessly cleans the whole range on, e.g.,
every set/way trap.
Return early when FWB is enabled, matching the generic walker.
Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-5-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Dirty logging forces new stage-2 mappings to page size but does not
always split an existing block first (eager splitting is best effort
and off by default). A non-write permission fault on such a block, an
instruction fetch, still needs a page-table page to split it, but the
top-up is gated on write faults.
With the cache empty, kvm_mmu_memory_cache_alloc() hits its
guest-triggerable WARN_ON(!nobjs) and falls back to a GFP_ATOMIC
allocation under mmu_lock, with a BUG_ON() if that fails.
Top up the memcache for any permission fault while dirty logging is
active.
Fixes: 6f745f1bb5bf ("KVM: arm64: Convert user_mem_abort() to generic page-table API")
Link: https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/
Signed-off-by: Bradley Morgan <include@grrlz.net>
[tabba: reword the commit message for the generic, non-pKVM failure mode]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-4-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
A permission fault normally only relaxes a leaf, so user_mem_abort()
skips the memcache top-up. Under pKVM such a fault can instead collapse
pages into a block. That needs a fresh pkvm_mapping object, and without
it cache->mapping is NULL, so pkvm_pgtable_stage2_map() dereferences NULL
and faults the host under mmu_lock. Staging only the object is not
enough: the hypervisor requires kvm_mmu_cache_min_pages in the memcache
even for the allocation-free install, so under memcache pressure the
collapse returns -ENOMEM and trips the WARN_ON(ret) in
pkvm_pgtable_stage2_map() where a non-pKVM guest succeeds.
Top up the full memcache for pKVM permission faults so both the mapping
object and the min-pages are staged before mmu_lock.
Fixes: db14091d8f75 ("KVM: arm64: Stage-2 huge mappings for np-guests")
Reported-by: Bradley Morgan <include@grrlz.net>
Link: https://lore.kernel.org/all/20260623161545.EA08E1F000E9@smtp.kernel.org/
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-3-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The pKVM flush path walks its own pkvm_mappings list and cleans the
data cache for every mapping, unlike the generic stage-2 walker it
shadows, which skips non-cacheable leaves. Cleaning the cacheable
alias of a non-cacheable mapping is pointless and can corrupt a
device endpoint. Record whether a mapping is non-cacheable in spare
bits of nr_pages and skip cache maintenance for it.
Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Bradley Morgan <include@grrlz.net>
[tabba: use Marc's anonymous bitfield in place of the open-coded mask and helpers]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-2-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
pKVM copies a non-protected guest's register context between the host
and the hypervisor on every world switch, even when the host never
inspects it. Defer the copy: on entry, flush the host context into the
hyp vCPU only when the host marked it dirty (PKVM_HOST_STATE_DIRTY); on
exit, leave it in the hyp vCPU and copy it back only when the host needs
it, via a __pkvm_vcpu_sync_state hypercall or at vcpu put. A protected
guest's context is copied as before, since lazy sync only helps where
the host is trusted to see the guest's registers.
PC and PSTATE are the exception, copied back on every exit: the kvm_exit
tracepoint reports the guest's real exit PC, and
vcpu_mode_is_bad_32bit(), the SError-masking checks and the host
software-step state machine all read PSTATE before the next on-demand
sync. Without it an ARM_EXCEPTION_IRQ exit syncs nothing, so
flush_debug_state() pushes a stale PSTATE.SS back on the next entry and
single-stepping runs an extra instruction before the step is reported.
The host needs the full context when it is about to read it (trap
handling) or write it (the SError injection that writes ESR_EL1). Sync
both from handle_exit_early(), which runs non-preemptible so the loaded
hyp vCPU is stable without a preempt guard.
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260729131823.2021516-9-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
pKVM performs its own world switch for protected VMs but has no
primitives to move the per-vCPU VGIC state between the host and
hypervisor vCPU contexts.
Add flush_hyp_vgic_state() and sync_hyp_vgic_state(). Flush copies
vgic_hcr, the in-use list registers and used_lrs from the host into the
hyp vCPU and pins vgic_sre to a fixed value; sync copies vgic_hcr,
vgic_vmcr and the in-use list registers back. The active priority
registers are handled separately by the save/restore-aprs path.
Bound used_lrs by hyp_gicv3_nr_lr, the cached implemented-LR count,
instead of reading ICH_VTR_EL2 on each entry. That clamps the
host-supplied value and avoids a per-entry sysreg read that is costly
under NV.
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Co-developed-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260729131823.2021516-8-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The host passes a vgic_v3_cpu_if pointer to the __vgic_v3_save_aprs and
__vgic_v3_restore_vmcr_aprs hypercalls, which EL2 dereferences
wholesale. That exposes the host's full VGIC emulation state to the
hypervisor, against pKVM's isolation goals.
Recover the host vCPU from the supplied cpu_if via container_of() and
copy only vgic_vmcr and the active priority registers between EL2's
hyp-side state and the host vCPU, so EL2 no longer dereferences the
host's vgic_v3_cpu_if directly.
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Co-developed-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260729131823.2021516-7-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The nVHE hypervisor repeatedly resolves a host vCPU into the EL2
address space and validates that the loaded hyp vCPU matches it, with
that logic open-coded in each handler.
Add __get_host_hyp_vcpus() and the get_host_hyp_vcpus() macro, which
translate the host vCPU into the hypervisor's address space and, when
pKVM is enabled, also return the loaded hyp vCPU if it matches. If pKVM
is enabled but the loaded hyp vCPU does not correspond to the requested
host vCPU, both the host and hyp vCPU are returned as NULL. Convert
handle___kvm_vcpu_run() to use it.
No functional change intended.
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Co-developed-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260729131823.2021516-6-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Move kvm_psci_valid_affinity() and kvm_psci_narrow_to_32bit() from
psci.c to include/kvm/arm_psci.h, and move psci_affinity_mask() there
too, renaming it kvm_psci_affinity_mask() now that it is no longer
file-local. A follow-up series handles some protected-guest PSCI calls
at EL2 using these helpers.
No functional change intended.
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260729131823.2021516-5-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Pull the reusable pieces out of kvm_reset_vcpu(): expose the reset
PSTATE values in kvm_arm.h, and split the core register reset and the
PSCI-driven reset into kvm_reset_vcpu_core() and kvm_reset_vcpu_psci().
A follow-up series reuses these to reset protected vCPUs at EL2.
No functional change intended.
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260729131823.2021516-4-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The vcpu_{read,write}_sys_reg() accessors abstract where a system
register lives, but their VHE implementation cannot be linked into the
nVHE hypervisor, so exception.c open-codes has_vhe() wrappers instead.
Redirect the accessors to the raw context accessors for nVHE hyp builds,
where registers are always in memory, and drop the local wrappers. This
lets hyp code reuse helpers built on the canonical accessors.
No functional change intended.
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260729131823.2021516-3-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Extract the vCPU MPIDR computation embedded in reset_mpidr() into a
kvm_calculate_mpidr() inline in sys_regs.h, so it can be computed
without duplicating the logic. A follow-up series reuses it to reset
protected vCPUs at EL2.
No functional change intended.
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260729131823.2021516-2-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
SWIOTLB_FORCE has no remaining in-tree users. Forced bouncing is now
controlled through the swiotlb=force command line option via
swiotlb_force_bounce.
Remove the unused flag and simplify the force_bounce initialization.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-24-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
Check the return value when converting swiotlb pools between encrypted and
decrypted mappings. If the default pool cannot be decrypted after early
initialization, mark the pool fully used so it cannot satisfy future bounce
allocations.
For late initialization, return the `set_memory_decrypted()` failure. For
restricted DMA pools, fail device initialization if the reserved pool
cannot be decrypted.
This prevents swiotlb from using pools whose encryption attributes do not
match their metadata, and avoids returning pages with uncertain encryption
state back to the allocator.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-23-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
swiotlb_dyn_free() is used after removing a dynamic swiotlb pool from
RCU-protected lists. It can call swiotlb_free_tlb(), which may need to
restore the encryption state of an unencrypted pool with
set_memory_encrypted() before freeing the pages.
RCU callbacks run in atomic context, but set_memory_encrypted() is not
guaranteed to be atomic-safe on all architectures. For example, page
attribute updates may allocate page tables or take sleeping locks.
Use queue_rcu_work() for dynamic pool freeing instead. This keeps the RCU
grace period before freeing a published pool, while running the actual pool
teardown from workqueue context. Use the same helper for the transient-pool
error path, since that path may also be reached from atomic DMA mapping
context.
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-22-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
ret in dma_direct_alloc() and dma_direct_alloc_pages() holds the returned
CPU mapping, not a generic return value. Rename it to cpu_addr and update
the remaining uses to match.
This makes the allocation paths easier to follow and keeps the local naming
consistent with what the variable actually represents.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-21-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
Make the dma-direct helpers derive the DMA address encoding from
__DMA_ATTR_ALLOC_CC_SHARED instead of implicitly relying on
force_dma_unencrypted() inside phys_to_dma_direct()
Pass an explicit unencrypted/decrypted state into phys_to_dma_direct(),
make the alloc paths return DMA addresses that match the requested buffer
encryption state. Also only call dma_set_decrypted() when
__DMA_ATTR_ALLOC_CC_SHARED is actually set.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Jiri Pirko <jiri@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Link: https://lore.kernel.org/r/20260717180442.110954-20-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
