feat(networking): add DA-pending block lookup coordinator

[perfogic]

What was wrong?

Gossip blocks whose parent was known locally but still pending data availability could not be processed and were effectively dropped. Re-transmission was unreliable because the original arrival had already populated the seen cache.

Fixes ReamLabs#1484.

How was it fixed?

For detail about what block lookup is: https://hackmd.io/@perfogic/B1OxLsgLGl
State machine can be viewed at: https://www.danielpham.me/ream-beacon-spec.html

• Added a bounded block lookup coordinator that stages one entry per block root, containing its fully validated block and data columns.
• Validated deferred gossip objects completely at arrival and released them after their dependency became available without re-running gossip validation.
• Rechecked mutable release conditions such as finality, current slot, parent availability, and ancestry before import.
• Added block import and pending-availability notifications so gossip, RPC, and range-sync imports can advance staged work.
• Added deterministic eviction, data-availability retention pruning, and a no-progress timeout.
• Added end-to-end coverage for wrong proposers, seen-cache interaction, deferred block import, quarantined columns, finality changes, and failed imports.
• Kept unknown-parent lookup, recursive descendants, and retry policy out of scope for Recover unknown-parent blocks after parent lookup ReamLabs/ream#1532.

To-Do

• I have read CONTRIBUTING.md.
• This PR title follows conventional commits.

[Copilot]

Pull request overview

Adds coordinated handling for gossip blocks and columns blocked on parent data availability.

Changes:

• Adds bounded lookup coordination, pruning, and sequential imports.
• Adds typed block-import outcomes and notifications.
• Expands validation and end-to-end coverage.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
testing/mock-execution-engine/src/block_generator.rs	Adds deterministic blob fixtures.
testing/gossip-validation/tests/validate_block.rs	Tests dependency-aware validation.
crates/rpc/beacon/src/handlers/block.rs	Handles pending-availability outcomes.
crates/networking/syncer/src/lib.rs	Exposes block lookups.
crates/networking/syncer/src/block_range/mod.rs	Stops descendants after pending blocks.
crates/networking/syncer/src/block_lookups/mod.rs	Implements the coordinator.
crates/networking/manager/src/service.rs	Integrates coordinator and notifications.
crates/networking/manager/src/lib.rs	Exposes manager integration.
crates/networking/manager/src/gossipsub/validate/result.rs	Adds dependency validation results.
crates/networking/manager/src/gossipsub/validate/data_column_sidecar.rs	Defers validated columns.
crates/networking/manager/src/gossipsub/validate/beacon_block.rs	Defers validated blocks.
crates/networking/manager/src/gossipsub/handle.rs	Stages deferred gossip objects.
crates/networking/manager/src/block_lookup.rs	Adds worker and release checks.
crates/common/fork_choice/beacon/src/store.rs	Updates DA completion handling.
crates/common/fork_choice/beacon/src/handlers.rs	Separates DA insertion and completion.
crates/common/data_availability/src/lib.rs	Exports DA status.
crates/common/data_availability/src/checker.rs	Adds explicit DA lifecycle APIs.
crates/common/chain/beacon/src/beacon_chain.rs	Adds outcomes and import broadcasts.
bin/ream/src/tests/block_lookup_tests.rs	Adds lookup integration tests.
bin/ream/src/main.rs	Registers tests and expands finality coverage.

Suppressed comments (2)

crates/networking/manager/src/gossipsub/validate/beacon_block.rs:94

• The exact parent state is still at the parent slot when it is passed to signature verification, and verify_block_header_signature derives the domain from that state's current epoch. For a child in the first slot of a new fork, this uses the parent's old fork version and rejects a valid signature. Advance a cloned parent state to the block slot before verification (and reuse it for the proposer check), or verify with a domain explicitly derived for the block epoch.

    // Looking up the parent above does not classify the message. Validation still checks the
    // signature before returning unknown-parent, as required by the gossip specification.
    let state = parent.as_ref().map_or(&head_state, |parent| &parent.state);
    match validate_beacon_block(beacon_chain, cached_db, block, state, parent.as_ref()).await? {

crates/networking/manager/src/gossipsub/validate/data_column_sidecar.rs:150

• This verifies against the unadvanced parent state, while verify_block_header_signature uses the state's current epoch for the proposer domain. A sidecar for the first block of a new fork therefore gets checked with the previous fork version and is rejected. Advance the parent state to header.slot before this check, or verify using the header epoch explicitly.

    if !matches!(
        signature_state.verify_block_header_signature(&data_column_sidecar.signed_block_header),
        Ok(true)
    ) {
        return Ok(ValidationResult::Reject(
            "Invalid proposer signature on data column sidecar's block header".to_string(),

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

crates/networking/manager/src/gossipsub/handle.rs:193

• ParentPendingAvailability represents a block or column that has passed the full gossip checks, but mapping it to Ignore tells gossipsub not to propagate it. Because validation has also populated the seen cache, retransmission will be ignored and this node never forwards the object after its dependency imports, so downstream peers can permanently miss valid blocks/columns. Report these deferred-but-valid objects as Accept while still staging local import.

        DependencyValidationResult::Ignore(_)
        | DependencyValidationResult::ParentPendingAvailability { .. } => MessageAcceptance::Ignore,

[Copilot]

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 24 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

crates/networking/syncer/src/block_range/mod.rs:278

• Blocks outside the DA retention window intentionally have no blob sidecars fetched (block_cache.rs:223-233), but every block with commitments enters this branch and build_data_columns_from_blob_sidecars requires the full set. Range sync will therefore fail on historical Deneb/Electra/Fulu blocks with commitments once their sidecars are outside retention. Gate column construction with the same DA-retention predicate used by BlockCache, and process expired blocks without requiring blobs.

                } else {

[Copilot]

Pull request overview

Copilot reviewed 23 out of 24 changed files in this pull request and generated no new comments.