I'm bringing up the DMA EP on a PM50052 (Gen5 PFX, 52 lanes, on a C-Payne CP-0550 board) with this driver. I'm stuck on something that looks like the engine silently dropping the whole write phase of a copy.
Setup:
- 2x Jetson Orin NX hosts (arm64), kernel 5.15.148 (L4T r36.4), one RC per partition (Gen4 x4)
- 4-partition config. NTB between the two host partitions works fine (
ntb_transport/ntb_netdev, ~5.5 Gbit/s TCP)
- driver at master (
79ef03c) with 2 small patches:
- driver binds, 12 channels register, SQ fetch / CE writeback / MSI-X all work
Issue:
dmatest: dma1chan0-copy0: summary 20 tests, 20 failures 1954.64 iops 851055 KB/s (0)
- all 20 transfers complete, no error CE, no channel pause
- but still 20 failures in dmatest where the destination buffer is untouched
- engine reports success without having written anything
I believe this isn't host-side: I fetched the switch's own USP TLP counters before and after a run using the ChipLink tool.
- non-posted: about +50000 between before and after a run.
- posted: only about +15 events added. Meaning it wrote almost nothing
- completions: stayed at 0
WRIMM behaves the same: completes ok, the immediate data never lands.
Ruled out so far:
- SMMU faults: none (the control path uses the same translation and works)
- CPU cache coherency: forced
dev->dma_coherent = false for explicit maintenance, no change
- partition-specific config: identical behavior from both hosts/partitions
- firmware build: started on the board's
6.7.FF.49, then flashed stock MR3 6.9.0.056, identical on both
What I can't rule out is the configuration file:
- the MR3 release notes contain this fix entry: "PPU Header credit overflow: Config change cause DMA related TLPs routed to PPU causing credit overflow. Fix: Corrected the configuration value." That sounds kind of like what I'm seeing (DMA data TLPs disappearing inside the switch)
- our config was migrated with ChipLink from the original 6.7 config. If that fix was a corrected config default, the migration would have carried the old value forward
- the configs shipped in the 6.9 bundle all have
Enable DMA EP = 0, so there's no reference for what a correct DMA-enabled config should look like
Questions:
- Which configuration value is the one referred to in that release-note fix, and what should it be set to? Is it visible/editable in ChipLink?
- Does the config editor migration path (6.7 to 6.9) preserve the old value?
- Do you have a known-good Gen5 PFX config with DMA EPs enabled that I could diff against?
Happy to share more logs if needed...
I'm bringing up the DMA EP on a PM50052 (Gen5 PFX, 52 lanes, on a C-Payne CP-0550 board) with this driver. I'm stuck on something that looks like the engine silently dropping the whole write phase of a copy.
Setup:
ntb_transport/ntb_netdev, ~5.5 Gbit/s TCP)79ef03c) with 2 small patches:1f18:0102id added, class-matched so it only binds the 0x0880 functioncopy_align = DMAENGINE_ALIGN_8_BYTESfix from PM40100 PFX failing with "Paused, a reserved opcode was detected" error under Linux 5.4.x kernel #15Issue:
I believe this isn't host-side: I fetched the switch's own USP TLP counters before and after a run using the ChipLink tool.
WRIMM behaves the same: completes ok, the immediate data never lands.
Ruled out so far:
dev->dma_coherent = falsefor explicit maintenance, no change6.7.FF.49, then flashed stock MR36.9.0.056, identical on bothWhat I can't rule out is the configuration file:
Enable DMA EP = 0, so there's no reference for what a correct DMA-enabled config should look likeQuestions:
Happy to share more logs if needed...