Skip to content

PM50052 (Gen5 PFX): address-mode MEMCPY completes without error but the destination is never written #21

Description

@Gillefranc

I'm bringing up the DMA EP on a PM50052 (Gen5 PFX, 52 lanes, on a C-Payne CP-0550 board) with this driver. I'm stuck on something that looks like the engine silently dropping the whole write phase of a copy.

Setup:

  • 2x Jetson Orin NX hosts (arm64), kernel 5.15.148 (L4T r36.4), one RC per partition (Gen4 x4)
  • 4-partition config. NTB between the two host partitions works fine (ntb_transport/ntb_netdev, ~5.5 Gbit/s TCP)
  • driver at master (79ef03c) with 2 small patches:
  • driver binds, 12 channels register, SQ fetch / CE writeback / MSI-X all work

Issue:

dmatest: dma1chan0-copy0: summary 20 tests, 20 failures 1954.64 iops 851055 KB/s (0)
  • all 20 transfers complete, no error CE, no channel pause
  • but still 20 failures in dmatest where the destination buffer is untouched
  • engine reports success without having written anything

I believe this isn't host-side: I fetched the switch's own USP TLP counters before and after a run using the ChipLink tool.

  • non-posted: about +50000 between before and after a run.
  • posted: only about +15 events added. Meaning it wrote almost nothing
  • completions: stayed at 0

WRIMM behaves the same: completes ok, the immediate data never lands.

Ruled out so far:

  • SMMU faults: none (the control path uses the same translation and works)
  • CPU cache coherency: forced dev->dma_coherent = false for explicit maintenance, no change
  • partition-specific config: identical behavior from both hosts/partitions
  • firmware build: started on the board's 6.7.FF.49, then flashed stock MR3 6.9.0.056, identical on both

What I can't rule out is the configuration file:

  • the MR3 release notes contain this fix entry: "PPU Header credit overflow: Config change cause DMA related TLPs routed to PPU causing credit overflow. Fix: Corrected the configuration value." That sounds kind of like what I'm seeing (DMA data TLPs disappearing inside the switch)
  • our config was migrated with ChipLink from the original 6.7 config. If that fix was a corrected config default, the migration would have carried the old value forward
  • the configs shipped in the 6.9 bundle all have Enable DMA EP = 0, so there's no reference for what a correct DMA-enabled config should look like

Questions:

  1. Which configuration value is the one referred to in that release-note fix, and what should it be set to? Is it visible/editable in ChipLink?
  2. Does the config editor migration path (6.7 to 6.9) preserve the old value?
  3. Do you have a known-good Gen5 PFX config with DMA EPs enabled that I could diff against?

Happy to share more logs if needed...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions