| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
Pull dmaengine updates from Vinod Koul:
"Core:
- New API to combine configuration and preparation and users
New hardware support:
- Mediatek MT8189 SoC uart dma support
Updates:
- Designware dma driver flatten desc structures and simplify code,
interrupt-path groundwork changes, first part of PCI EP DMA support
- Updates to zynqmp_dma with runtime PM and device removal
improvments
- Xilinx dma optimizations for AXIDMA and MCDMA channel management"
* tag 'dmaengine-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (73 commits)
dmaengine: dw-edma: Mark emulated IRQ as level-triggered
dmaengine: idxd: assign all engines to group 0 in IAA defaults
dmaengine: qcom_hidma: remove conditional return with no effect
dmaengine: qcom-bam-dma: fix autosuspend cleanup during removal
dmaengine: fsl-edma: tracing: no ptr dereference during log output
dmaengine: dw-edma: Program endpoint function numbers
dmaengine: dw-edma-pcie: Add chip flags to match data
dmaengine: dw-edma-pcie: Handle optional data blocks
dmaengine: dw-edma-pcie: Factor out descriptor block address lookup
dmaengine: dw-edma-pcie: Add register offset match flag
dmaengine: dw-edma-pcie: Add platform ops to match data
dmaengine: dw-edma-pcie: Rename vsec_data to dma_data
dmaengine: dw-edma-pcie: Add capability match data
dmaengine: dw-edma-pcie: Track non-LL mode in DMA data
dmaengine: dw-edma: Add partial channel ownership mode
dmaengine: dw-edma: Initialize IRQ data before requesting IRQs
dmaengine: dw-edma: Add core quiesce operations
dmaengine: dw-edma: Add per-channel interrupt routing control
dmaengine: dw-edma: Factor out HDMA interrupt setup helper
dmaengine: dw-edma: Defer channel IRQ handling to workqueue
...
|
|
The IAA device defaults only assigned engine 0 to group 0, leaving
engines 1 through max_engines-1 unassigned (group_id = -1). This means
that by default only a single engine processed descriptors, limiting
throughput to one engine's capacity.
Assign all available engines to group 0 so that the full hardware
parallelism is used out of the box without requiring manual
accel-config setup.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://patch.msgid.link/20260805-iaa-crypto-fixes-zswap-v2-1-55c02694f499@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The failed_dev_add and failed_dev_name paths drop the file-device
reference while wq->wq_lock is still held. If put_device(fdev) drops the
last reference, idxd_file_dev_release() runs synchronously and tries to
take wq->wq_lock again, deadlocking.
Those paths also fall through into the later ctx cleanup labels even
though idxd_file_dev_release() owns that cleanup and frees ctx. This can
make idxd_xa_pasid_remove(ctx) and kfree(ctx) operate on a freed context.
Move idxd_wq_get() before file-device setup can fail, since the release
callback always calls idxd_wq_put(). Then unlock wq->wq_lock before
put_device(fdev) and return directly from the file-device setup failure
path, leaving ctx cleanup to the release callback.
Fixes: e6fd6d7e5f0fe ("dmaengine: idxd: add a device to represent the file opened")
Signed-off-by: Yuho Choi <dbgh9129@gmail.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260525141550.1385581-1-dbgh9129@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The release callbacks for wq, engine, and group devices
(idxd_conf_wq_release, idxd_conf_engine_release,
idxd_conf_group_release) each call kfree() on the enclosing struct.
The setup error paths and cleanup functions also call kfree()
explicitly after put_device(), producing a double free whenever
put_device() drops the reference count to zero and fires the release.
In the setup functions, device_initialize() is called before
device_add(), so the reference count is exactly 1 at the error sites.
put_device() unconditionally fires the release, which frees the struct;
the subsequent explicit kfree() then operates on freed memory.
For idxd_setup_wqs(), the wq release callback also owns opcap_bmap
and wqcfg. The error unwind additionally freed those fields explicitly
before calling put_device(), causing further double frees on both.
Remove the redundant explicit kfree() calls from all setup error paths
and cleanup functions for wq, engine, and group structs, delegating
sole ownership of those allocations to the release callbacks.
Fixes: 7c5dd23e57c1 ("dmaengine: idxd: fix wq conf_dev 'struct device' lifetime")
Fixes: 75b911309060 ("dmaengine: idxd: fix engine conf_dev lifetime")
Fixes: defe49f96012 ("dmaengine: idxd: fix group conf_dev lifetime")
Signed-off-by: Yuho Choi <dbgh9129@gmail.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260415205452.67155-1-dbgh9129@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The first device on a PCI root bus determines whether the host bridge is
whitelisted for P2PDMA. All Intel Xeon chips since Ice Lake (ICX, 2021)
expose a device with ID 0x09a2 as first device. It is loosely associated
with the IOMMU. All these Xeon chips support P2PDMA, so since the addition
of the device with commit feaea1fe8b36 ("PCI/P2PDMA: Add Intel 3rd Gen
Intel Xeon Scalable Processors to whitelist"), P2PDMA has been allowed on
all new Xeons without the need to amend the whitelist:
Xeons with Performance Cores:
Sapphire Rapids (SPR, 2023)
Emerald Rapids (EMR, 2023)
Granite Rapids (GNR, 2024)
Diamond Rapids (DMR, 2026)
Xeons with Efficiency Cores:
Sierra Forest (SRF, 2024)
Clearwater Forest (CWF, 2026)
However these Xeons also expose accelerators as first device on a root bus
of its own:
QuickAssist Technology (QAT, crypto & compression accelerator)
Data Streaming Accelerator (DSA, dma engine)
In-Memory Analytics Accelerator (IAA, compression accelerator)
Whitelist them for P2PDMA as well. Move their Device ID macros from the
accelerator drivers to <linux/pci_ids.h> for reuse by P2PDMA code.
Unfortunately the Device IDs vary across Xeon generations as additional
features were added to the accelerators. This currently necessitates an
amendment for each new Xeon chip.
For future chips, this need shall be avoided by an ongoing effort to extend
ACPI HMAT with PCIe P2PDMA characteristics (latency, bandwidth, ordering
constraints). The PCI core will be able look up in this BIOS-provided ACPI
table whether P2PDMA is supported, instead of relying on a whitelist that
needs to be amended continuously.
Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> # QAT
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/6aac4922b5fe7070b11874427a9285e42ddd05a4.1780585518.git.lukas@wunner.de
|
|
At the end of this function, d is the traversal cursor of flist, but the
code completes found instead. This can lead to issues such as NULL pointer
dereferences, double completion, or descriptor leaks.
Fix this by completing d instead of found in the final
list_for_each_entry_safe() loop.
Fixes: aa8d18becc0c ("dmaengine: idxd: add callback support for iaa crypto")
Signed-off-by: Tuo Li <islituo@gmail.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://patch.msgid.link/20260106032428.162445-1-islituo@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
During the device remove process, the device is reset, causing the
configuration registers to go back to their default state, which is
zero. As the driver is checking if the event log support was enabled
before deallocating, it will fail if a reset happened before.
Do not check if the support was enabled, the check for 'idxd->evl'
being valid (only allocated if the HW capability is available) is
enough.
Fixes: 244da66cda35 ("dmaengine: idxd: setup event log configuration")
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-10-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
It can happen that when the cdev .release() is called, the driver
already called ida_destroy(). Move ida_free() to the _del() path.
We see with DEBUG_KOBJECT_RELEASE enabled and forcing an early PCI
unbind.
Fixes: 04922b7445a1 ("dmaengine: idxd: fix cdev setup and free device lifetime issues")
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-9-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
idxd_wq_disable_cleanup() which is called from the reset path for a
workqueue, sets the wq type to NONE, which for other parts of the
driver mean that the wq is empty (all its resources were released).
Only set the wq type to NONE after its resources are released.
Fixes: da32b28c95a7 ("dmaengine: idxd: cleanup workqueue config after disabling")
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-8-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The workqueue associated with an DSA/IAA device is not released when
the object is freed.
Fixes: 47c16ac27d4c ("dmaengine: idxd: fix idxd conf_dev 'struct device' lifetime")
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-7-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
When the dmaengine "core" asks the driver to synchronize, send a Drain
operation to the device workqueue, which will wait for the already
submitted operations to finish.
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-6-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
When used as a dmaengine, the DMA "core" might ask the driver to
terminate all pending requests, when that happens, flush all pending
descriptors.
In this context, flush means removing the requests from the pending
lists, so even if they are completed after, the user is not notified.
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-5-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
When a Function Level Reset (FLR) happens, terminate the pending
descriptors that were issued by in-kernel users and disable the
interrupts associated with those. They will be re-enabled after FLR
finishes.
idxd_wq_flush_desc() is declared on idxd.h because it's going to be
used in by the DMA backend in a future patch.
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-4-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
In the case that the first Function Level Reset (FLR) concludes
correctly, but in the second FLR the scratch area for the saved
configuration cannot be allocated, it's possible for a invalid memory
access to happen.
Always set the deallocated scratch area to NULL after FLR completes.
Fixes: 98d187a98903 ("dmaengine: idxd: Enable Function Level Reset (FLR) for halt")
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-3-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
If reporting errors to the event log is not supported by the hardware,
and an error that causes Function Level Reset (FLR) is received, the
driver will try to restore the event log even if it was not allocated.
Also, only try to free the event log if it was properly allocated.
Fixes: 6078a315aec1 ("dmaengine: idxd: Add idxd_device_config_save() and idxd_device_config_restore() helpers")
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-2-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Move the check for IDXD_FLAG_CONFIGURABLE and the locking to "inside"
idxd_device_config(), as this is common to all callers, and the one
that wasn't holding the lock was an error (that was causing the
lockdep warning).
Suggested-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260121-idxd-fix-flr-on-kernel-queues-v3-v3-1-7ed70658a9d1@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <kees@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
Pull dmaengine updates from Vinod Koul:
"Core:
- Add Frank Li as susbstem reviewer to help with reviews
New Support:
- Mediatek support for Dimensity 6300 and 9200 controller
- Qualcomm Kaanapali and Glymur GPI DMA engine
- Synopsis DW AXI Agilex5
- Renesas RZ/V2N SoC
- Atmel microchip lan9691-dma
- Tegra ADMA tegra264
Updates:
- sg_nents_for_dma() helper use in subsystem
- pm_runtime_mark_last_busy() redundant call update for subsystem
- Residue support for xilinx AXIDMA driver
- Intel Max SGL Size Support and capabilities for DSA3.0
- AXI dma larger than 32bits address support"
* tag 'dmaengine-7.0-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (64 commits)
dmaengine: add Frank Li as reviewer
dt-bindings: dma: qcom,gpi: Update max interrupts lines to 16
dmaengine: fsl-edma: don't explicitly disable clocks in .remove()
dmaengine: xilinx: xdma: use sg_nents_for_dma() helper
dmaengine: sh: use sg_nents_for_dma() helper
dmaengine: sa11x0: use sg_nents_for_dma() helper
dmaengine: qcom: bam_dma: use sg_nents_for_dma() helper
dmaengine: qcom: adm: use sg_nents_for_dma() helper
dmaengine: pxa-dma: use sg_nents_for_dma() helper
dmaengine: lgm: use sg_nents_for_dma() helper
dmaengine: k3dma: use sg_nents_for_dma() helper
dmaengine: dw-axi-dmac: use sg_nents_for_dma() helper
dmaengine: bcm2835-dma: use sg_nents_for_dma() helper
dmaengine: axi-dmac: use sg_nents_for_dma() helper
dmaengine: altera-msgdma: use sg_nents_for_dma() helper
scatterlist: introduce sg_nents_for_dma() helper
dmaengine: idxd: Add Max SGL Size Support for DSA3.0
dmaengine: idxd: Expose DSA3.0 capabilities through sysfs
dmaengine: sh: rz-dmac: Make channel irq local
dmaengine: pl08x: Fix comment stating the difference between PL080 and PL081
...
|
|
Certain DSA 3.0 opcodes, such as Gather copy and Gather reduce, require max
SGL configured for workqueues prior to supporting these opcodes.
Configure the maximum scatter-gather list (SGL) size for workqueues during
setup on the supported HW. Application can then properly handle the SGL
size without explicitly setting it.
Signed-off-by: Yi Sun <yi.sun@intel.com>
Co-developed-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Tested-by: Yi Lai <yi1.lai@intel.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260107-idxd-yi-sun-dsa3-sgl-size-v2-2-dbef8f559e48@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Introduce sysfs interfaces for 3 new Data Streaming Accelerator (DSA)
capability registers (dsacap0-2) to enable userspace awareness of hardware
features in DSA version 3 and later devices.
Userspace components (e.g. configure libraries, workload Apps) require this
information to:
1. Select optimal data transfer strategies based on SGL capabilities
2. Enable hardware-specific optimizations for floating-point operations
3. Configure memory operations with proper numerical handling
4. Verify compute operation compatibility before submitting jobs
The output format is <dsacap2>,<dsacap1>,<dsacap0>, where each DSA
capability value is a 64-bit hexadecimal number, separated by commas.
The ordering follows the DSA 3.0 specification layout:
Offset: 0x190 0x188 0x180
Reg: dsacap2 dsacap1 dsacap0
Example:
cat /sys/bus/dsa/devices/dsa0/dsacaps
000000000000f18d,0014000e000007aa,00fa01ff01ff03ff
According to the DSA 3.0 specification, there are 15 fields defined for
the three dsacap registers. However, there's no need to define all
register structures unless a use case requires them. At this point,
support for the Scatter-Gather List (SGL) located in dsacap0 is necessary,
so only dsacap0 is defined accordingly.
For reference, the DSA 3.0 specification is available at:
Link: https://software.intel.com/content/www/us/en/develop/articles/intel-data-streaming-accelerator-architecture-specification.html
Signed-off-by: Yi Sun <yi.sun@intel.com>
Co-developed-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Tested-by: Yi Lai <yi1.lai@intel.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260107-idxd-yi-sun-dsa3-sgl-size-v2-1-dbef8f559e48@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Make sure to drop the reference taken when looking up the idxd device as
part of the compat bind and unbind sysfs interface.
Fixes: 6e7f3ee97bbe ("dmaengine: idxd: move dsa_drv support to compatible mode")
Cc: stable@vger.kernel.org # 5.15
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Johan Hovold <johan@kernel.org>
Link: https://patch.msgid.link/20251117161258.10679-7-johan@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
There's an errata[1], for the Disable WQ command that it
does not guaranteee that address translations are drained. If WQ
configuration is updated, pending address translations can use an
updated WQ configuration, resulting an invalid translation response
that is cached in the device translation cache.
Replace the Disable WQ command with a Drain WQ command followed by a
Reset WQ command, this guarantees that all ATS translations are
drained from the device before changing WQ configuration.
[1] https://cdrdv2.intel.com/v1/dl/getcontent/843306 ("Intel DSA May
Cause Invalid Translation Caching")
Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
Pull dmaengine updates from Vinod Koul:
"A couple of new device support and small driver updates for this
round.
New support:
- Intel idxd Wildcat Lake family support
- SpacemiT K1 PDMA controller support
- Renesas RZ/G3E family support
Updates:
- Xilinx shutdown support and dma client properties update
- Designware edma callback_result support"
* tag 'dmaengine-6.18-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine:
dt-bindings: dma: rz-dmac: Document RZ/G3E family of SoCs
dmaengine: dw-edma: Set status for callback_result
dmaengine: mv_xor: match alloc_wc and free_wc
dmaengine: mmp_pdma: Add SpacemiT K1 PDMA support with 64-bit addressing
dmaengine: mmp_pdma: Add operations structure for controller abstraction
dmaengine: mmp_pdma: Add reset controller support
dmaengine: mmp_pdma: Add clock support
dt-bindings: dma: Add SpacemiT K1 PDMA controller
dt-bindings: dmaengine: xilinx_dma: Remove DMA client properties
dmaengine: Fix dma_async_tx_descriptor->tx_submit documentation
dmaengine: xilinx_dma: Support descriptor setup from dma_vecs
dmaengine: sh: setup_xref error handling
dmaengine: Replace zero-length array with flexible-array
dmaengine: ppc4xx: Remove space before newline
dmaengine: idxd: Add a new IAA device ID for Wildcat Lake family platforms
dmaengine: idxd: Replace memset(0) + strscpy() with strscpy_pad()
dt-bindings: dma: nvidia,tegra20-apbdma: Add undocumented compatibles and "clock-names"
dmaengine: zynqmp_dma: Add shutdown operation support
|
|
Pull VFIO updates from Alex Williamson:
- Use fdinfo to expose the sysfs path of a device represented by a vfio
device file (Alex Mastro)
- Mark vfio-fsl-mc, vfio-amba, and the reset functions for
vfio-platform for removal as these are either orphaned or believed to
be unused (Alex Williamson)
- Add reviewers for vfio-platform to save it from also being marked for
removal (Mostafa Saleh, Pranjal Shrivastava)
- VFIO selftests, including basic sanity testing and minimal userspace
drivers for testing against real hardware. This is also expected to
provide integration with KVM selftests for KVM-VFIO interfaces (David
Matlack, Josh Hilke)
- Fix drivers/cdx and vfio/cdx to build without CONFIG_GENERIC_MSI_IRQ
(Nipun Gupta)
- Fix reference leak in hisi_acc (Miaoqian Lin)
- Use consistent return for unsupported device feature (Alex Mastro)
- Unwind using the correct memory free callback in vfio/pds (Zilin
Guan)
- Use IRQ_DISABLE_LAZY flag to improve handling of pre-PCI2.3 INTx and
resolve stalled interrupt on ppc64 (Timothy Pearson)
- Enable GB300 in nvgrace-gpu vfio-pci variant driver (Tushar Dave)
- Misc:
- Drop unnecessary ternary conversion in vfio/pci (Xichao Zhao)
- Grammatical fix in nvgrace-gpu (Morduan Zang)
- Update Shameer's email address (Shameer Kolothum)
- Fix document build warning (Alex Williamson)
* tag 'vfio-v6.18-rc1' of https://github.com/awilliam/linux-vfio: (48 commits)
vfio/nvgrace-gpu: Add GB300 SKU to the devid table
vfio/pci: Fix INTx handling on legacy non-PCI 2.3 devices
vfio/pds: replace bitmap_free with vfree
vfio: return -ENOTTY for unsupported device feature
hisi_acc_vfio_pci: Fix reference leak in hisi_acc_vfio_debug_init
vfio/platform: Mark reset drivers for removal
vfio/amba: Mark for removal
MAINTAINERS: Add myself as VFIO-platform reviewer
MAINTAINERS: Add myself as VFIO-platform reviewer
docs: proc.rst: Fix VFIO Device title formatting
vfio: selftests: Fix .gitignore for already tracked files
vfio/cdx: update driver to build without CONFIG_GENERIC_MSI_IRQ
cdx: don't select CONFIG_GENERIC_MSI_IRQ
MAINTAINERS: Update Shameer Kolothum's email address
vfio: selftests: Add a script to help with running VFIO selftests
vfio: selftests: Make iommufd the default iommu_mode
vfio: selftests: Add iommufd mode
vfio: selftests: Add iommufd_compat_type1{,v2} modes
vfio: selftests: Add vfio_type1v2_mode
vfio: selftests: Replicate tests across all iommu_modes
...
|
|
Allow drivers/dma/idxd/registers.h to be included from userspace in
tools/ by adjusting the include path to uapi/linux/idxd.h if __KERNEL__
is not defined.
A subsequent commit will use registers.h to implement a userspace driver
for Intel DSA devices in tools/testing/selftests/vfio.
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
Signed-off-by: David Matlack <dmatlack@google.com>
Link: https://lore.kernel.org/r/20250822212518.4156428-22-dmatlack@google.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
|
|
A new IAA device ID, 0xfd2d, is introduced across all Wildcat Lake
family platforms. Add the device ID to the IDXD driver.
Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250801215936.188555-1-vinicius.gomes@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The clean up in idxd_setup_wqs() has had a couple bugs because the error
handling is a bit subtle. It's simpler to just re-write it in a cleaner
way. The issues here are:
1) If "idxd->max_wqs" is <= 0 then we call put_device(conf_dev) when
"conf_dev" hasn't been initialized.
2) If kzalloc_node() fails then again "conf_dev" is invalid. It's
either uninitialized or it points to the "conf_dev" from the
previous iteration so it leads to a double free.
It's better to free partial loop iterations within the loop and then
the unwinding at the end can handle whole loop iterations. I also
renamed the labels to describe what the goto does and not where the goto
was located.
Fixes: 3fd2f4bc010c ("dmaengine: idxd: fix memory leak in error handling path of idxd_setup_wqs")
Reported-by: Colin Ian King <colin.i.king@gmail.com>
Closes: https://lore.kernel.org/all/20250811095836.1642093-1-colin.i.king@gmail.com/
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/aJnJW3iYTDDCj9sk@stanley.mountain
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Replace memset(0) followed by strscpy() with strscpy_pad() to improve
idxd_load_iaa_device_defaults(). This avoids zeroing the memory before
copying the strings and ensures the destination buffers are only written
to once, simplifying the code and improving efficiency.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250810225858.2953-2-thorsten.blum@linux.dev
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
A recent refactor introduced a misplaced put_device() call, resulting in a
reference count underflow during module unload.
There is no need to add additional put_device() calls for idxd groups,
engines, or workqueues. Although the commit claims: "Note, this also
fixes the missing put_device() for idxd groups, engines, and wqs."
It appears no such omission actually existed. The required cleanup is
already handled by the call chain:
idxd_unregister_devices() -> device_unregister() -> put_device()
Extend idxd_cleanup() to handle the remaining necessary cleanup and
remove idxd_cleanup_internals(), which duplicates deallocation logic
for idxd, engines, groups, and workqueues. Memory management is also
properly handled through the Linux device model.
Fixes: a409e919ca32 ("dmaengine: idxd: Refactor remove call with idxd_cleanup() helper")
Signed-off-by: Yi Sun <yi.sun@intel.com>
Tested-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://lore.kernel.org/r/20250729150313.1934101-3-yi.sun@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The call to idxd_free() introduces a duplicate put_device() leading to a
reference count underflow:
refcount_t: underflow; use-after-free.
WARNING: CPU: 15 PID: 4428 at lib/refcount.c:28 refcount_warn_saturate+0xbe/0x110
...
Call Trace:
<TASK>
idxd_remove+0xe4/0x120 [idxd]
pci_device_remove+0x3f/0xb0
device_release_driver_internal+0x197/0x200
driver_detach+0x48/0x90
bus_remove_driver+0x74/0xf0
pci_unregister_driver+0x2e/0xb0
idxd_exit_module+0x34/0x7a0 [idxd]
__do_sys_delete_module.constprop.0+0x183/0x280
do_syscall_64+0x54/0xd70
entry_SYSCALL_64_after_hwframe+0x76/0x7e
The idxd_unregister_devices() which is invoked at the very beginning of
idxd_remove(), already takes care of the necessary put_device() through the
following call path:
idxd_unregister_devices() -> device_unregister() -> put_device()
In addition, when CONFIG_DEBUG_KOBJECT_RELEASE is enabled, put_device() may
trigger asynchronous cleanup via schedule_delayed_work(). If idxd_free() is
called immediately after, it can result in a use-after-free.
Remove the improper idxd_free() to avoid both the refcount underflow and
potential memory corruption during module unload.
Fixes: d5449ff1b04d ("dmaengine: idxd: Add missing idxd cleanup to fix memory leak in remove call")
Signed-off-by: Yi Sun <yi.sun@intel.com>
Tested-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://lore.kernel.org/r/20250729150313.1934101-2-yi.sun@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The __packed attribute introduces potential unaligned memory accesses
and endianness portability issues. Instead of relying on compiler-specific
packing, it's much better to explicitly fill structure gaps using padding
fields, ensuring natural alignment.
Since all previously __packed structures already enforce proper alignment
through manual padding, the __packed qualifiers are unnecessary and can be
safely removed.
Signed-off-by: Yi Sun <yi.sun@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Link: https://lore.kernel.org/r/20250404053614.3096769-1-yi.sun@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Deletes the second initialization as the value stored to 'dev' during
its initialization (struct device *dev = &idxd->pdev->dev;) is
sufficient.
../drivers/dma/idxd/init.c:988:17: warning: Value stored to 'dev' during
its initialization is never read [deadcode.DeadStores]
988 | struct device *dev = &idxd->pdev->dev;
| ^~~ ~~~~~~~~~~~~~~~~
Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250521231331.889204-1-anil.s.keshavamurthy@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
Pull dmaengine updates from Vinod Koul:
"A fairly small update for the dmaengine subsystem. This has a new ARM
dmaengine driver and couple of new device support and few driver
changes:
New support:
- Renesas RZ/V2H(P) dma support for r9a09g057
- Arm DMA-350 driver
- Tegra Tegra264 ADMA support
Updates:
- AMD ptdma driver code removal and optimizations
- Freescale edma error interrupt handler support"
* tag 'dmaengine-6.16-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (27 commits)
dmaengine: idxd: Remove unused pointer and macro
arm64: dts: renesas: r9a09g057: Add DMAC nodes
dmaengine: sh: rz-dmac: Add RZ/V2H(P) support
dmaengine: sh: rz-dmac: Allow for multiple DMACs
irqchip/renesas-rzv2h: Add rzv2h_icu_register_dma_req()
dt-bindings: dma: rz-dmac: Document RZ/V2H(P) family of SoCs
dt-bindings: dma: rz-dmac: Restrict properties for RZ/A1H
dmaengine: idxd: Narrow the restriction on BATCH to ver. 1 only
dmaengine: ti: Add NULL check in udma_probe()
fsldma: Set correct dma_mask based on hw capability
dmaengine: idxd: Check availability of workqueue allocated by idxd wq driver before using
dmaengine: xilinx_dma: Set dma_device directions
dmaengine: tegra210-adma: Add Tegra264 support
dt-bindings: Document Tegra264 ADMA support
dmaengine: dw-edma: Add HDMA NATIVE map check
dmaegnine: fsl-edma: add edma error interrupt handler
dt-bindings: dma: fsl-edma: increase maxItems of interrupts and interrupt-names
dmaengine: ARM_DMA350 should depend on ARM/ARM64
dt-bindings: dma: qcom,bam: Document dma-coherent property
dmaengine: Add Arm DMA-350 driver
...
|
|
'arm/smmu/bindings', 'fsl/pamu', 'mediatek', 'renesas/ipmmu', 's390', 'intel/vt-d', 'amd/amd-vi' and 'core' into next
|
|
The pointer 'extern struct kmem_cache *idxd_desc_pool' and the macro
'#define IDXD_ALLOCATED_BATCH_SIZE 128U' were introduced in commit
bfe1d56091c1 ("dmaengine: idxd: Init and probe for Intel data
accelerators") but they were never used.
Signed-off-by: Eder Zulian <ezulian@redhat.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250408173829.892317-1-ezulian@redhat.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Allow BATCH operations to be submitted and the capability to be
exposed for DSA version 2 (or later) devices.
DSA version 2 devices allow safe submission of BATCH operations.
Signed-off-by: Anil Keshavamurthy <anil.s.keshavamurthy@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reported-by: Yi Sun <yi.sun@intel.com>
Tested-by: Yi Sun <yi.sun@intel.com>
Link: https://lore.kernel.org/r/20250312221511.277954-1-vinicius.gomes@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The fix to block access from different address space did not return a
correct value for ->poll() change. kernel test bot reported that a
return value of type __poll_t is expected rather than int. Fix to return
POLLNVAL to indicate invalid request.
Fixes: 8dfa57aabff6 ("dmaengine: idxd: Fix allowing write() from different address spaces")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202505081851.rwD7jVxg-lkp@intel.com/
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250508170548.2747425-1-dave.jiang@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
before using
Running IDXD workloads in a container with the /dev directory mounted can
trigger a call trace or even a kernel panic when the parent process of the
container is terminated.
This issue occurs because, under certain configurations, Docker does not
properly propagate the mount replica back to the original mount point.
In this case, when the user driver detaches, the WQ is destroyed but it
still calls destroy_workqueue() attempting to completes all pending work.
It's necessary to check wq->wq and skip the drain if it no longer exists.
Signed-off-by: Yi Sun <yi.sun@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Link: https://lore.kernel.org/r/20250509000304.1402863-1-yi.sun@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The idxd_cleanup() helper cleans up perfmon, interrupts, internals and
so on. Refactor remove call with the idxd_cleanup() helper to avoid code
duplication. Note, this also fixes the missing put_device() for idxd
groups, enginces and wqs.
Fixes: bfe1d56091c1 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Cc: stable@vger.kernel.org
Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250404120217.48772-10-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The remove call stack is missing idxd cleanup to free bitmap, ida and
the idxd_device. Call idxd_free() helper routines to make sure we exit
gracefully.
Fixes: bfe1d56091c1 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Cc: stable@vger.kernel.org
Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250404120217.48772-9-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Memory allocated for idxd is not freed if an error occurs during
idxd_pci_probe(). To fix it, free the allocated memory in the reverse
order of allocation before exiting the function in case of an error.
Fixes: bfe1d56091c1 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Cc: stable@vger.kernel.org
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Link: https://lore.kernel.org/r/20250404120217.48772-8-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Memory allocated for idxd is not freed if an error occurs during
idxd_alloc(). To fix it, free the allocated memory in the reverse order
of allocation before exiting the function in case of an error.
Fixes: a8563a33a5e2 ("dmanegine: idxd: reformat opcap output to match bitmap_parse() input")
Cc: stable@vger.kernel.org
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Link: https://lore.kernel.org/r/20250404120217.48772-7-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The idxd_cleanup_internals() function only decreases the reference count
of groups, engines, and wqs but is missing the step to release memory
resources.
To fix this, use the cleanup helper to properly release the memory
resources.
Fixes: ddf742d4f3f1 ("dmaengine: idxd: Add missing cleanup for early error out in probe call")
Cc: stable@vger.kernel.org
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250404120217.48772-6-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The idxd_setup_internals() is missing some cleanup when things fail in
the middle.
Add the appropriate cleanup routines:
- cleanup groups
- cleanup enginces
- cleanup wqs
to make sure it exits gracefully.
Fixes: defe49f96012 ("dmaengine: idxd: fix group conf_dev lifetime")
Cc: stable@vger.kernel.org
Suggested-by: Fenghua Yu <fenghuay@nvidia.com>
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20250404120217.48772-5-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Memory allocated for groups is not freed if an error occurs during
idxd_setup_groups(). To fix it, free the allocated memory in the reverse
order of allocation before exiting the function in case of an error.
Fixes: defe49f96012 ("dmaengine: idxd: fix group conf_dev lifetime")
Cc: stable@vger.kernel.org
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Link: https://lore.kernel.org/r/20250404120217.48772-4-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Memory allocated for engines is not freed if an error occurs during
idxd_setup_engines(). To fix it, free the allocated memory in the
reverse order of allocation before exiting the function in case of an
error.
Fixes: 75b911309060 ("dmaengine: idxd: fix engine conf_dev lifetime")
Cc: stable@vger.kernel.org
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Link: https://lore.kernel.org/r/20250404120217.48772-3-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Memory allocated for wqs is not freed if an error occurs during
idxd_setup_wqs(). To fix it, free the allocated memory in the reverse
order of allocation before exiting the function in case of an error.
Fixes: 7c5dd23e57c1 ("dmaengine: idxd: fix wq conf_dev 'struct device' lifetime")
Fixes: 700af3a0a26c ("dmaengine: idxd: add 'struct idxd_dev' as wrapper for conf_dev")
Fixes: de5819b99489 ("dmaengine: idxd: track enabled workqueues in bitmap")
Fixes: b0325aefd398 ("dmaengine: idxd: add WQ operation cap restriction support")
Cc: stable@vger.kernel.org
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Link: https://lore.kernel.org/r/20250404120217.48772-2-xueshuai@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
The IOMMU_DEV_FEAT_IOPF implementation in the iommu driver is just a no-op.
It will also be removed from the iommu driver in the subsequent patch.
Remove it to avoid dead code.
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Acked-by: Vinod Koul <vkoul@kernel.org>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Yi Liu <yi.l.liu@intel.com>
Tested-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Link: https://lore.kernel.org/r/20250418080130.1844424-6-baolu.lu@linux.intel.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
|
|
None of the drivers implement anything here anymore, remove the dead code.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Yi Liu <yi.l.liu@intel.com>
Tested-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Link: https://lore.kernel.org/r/20250418080130.1844424-3-baolu.lu@linux.intel.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
|
