Skip to content

Durable emitter - error handling#2061

Draft
tarcisiozf wants to merge 3 commits into
batch-chip-protofrom
durable-emitter/error-handling
Draft

Durable emitter - error handling#2061
tarcisiozf wants to merge 3 commits into
batch-chip-protofrom
durable-emitter/error-handling

Conversation

@tarcisiozf
Copy link
Copy Markdown

Requires

Supports

Copilot AI review requested due to automatic review settings May 14, 2026 19:00
@tarcisiozf tarcisiozf requested review from a team as code owners May 14, 2026 19:00
@tarcisiozf tarcisiozf changed the base branch from main to batch-chip-proto May 14, 2026 19:00
@github-actions
Copy link
Copy Markdown

👋 tarcisiozf, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

Comment thread go.mod
sigs.k8s.io/yaml v1.4.0
)

replace github.com/smartcontractkit/chainlink-common/pkg/chipingress => ./pkg/chipingress
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

waiting for #2031 to be merged

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 14, 2026

⚠️ API Diff Results - github.com/smartcontractkit/chainlink-common

⚠️ Breaking Changes (1)

pkg/beholder.DurableEventStore (1)
  • MarkFailedBatch — ➕ Added

📄 View full apidiff report

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR attempts to add partial PublishBatch error handling for the durable emitter by enabling chipingress partial-success responses and recording failed event batches in the durable store.

Changes:

  • Adds MarkFailedBatch to the durable event store interface and metrics wrapper.
  • Updates typed batch publishing to request partial success and process per-event errors.
  • Points the root module at the local pkg/chipingress module.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
pkg/beholder/durable_event_store.go Adds failure-recording API support and metrics instrumentation.
pkg/beholder/durable_emitter.go Enables typed PublishBatch partial-success handling and records partial failures.
pkg/beholder/durable_emitter_test.go Adds a no-op MarkFailedBatch implementation to the in-memory test store.
go.mod Adds a local replace for the chipingress module.
go.sum Removes chipingress module checksums now hidden by the local replace.
Comments suppressed due to low confidence (2)

pkg/beholder/durable_emitter.go:804

  • After recording per-event failures, this path still returns nil, so flushBatch treats the whole PublishBatch as successful and calls MarkDeliveredBatch for every ID in the batch. Any events present in response.Results with errors will be marked delivered/deleted instead of remaining pending for retransmit, which defeats the failure tracking added here.
	if len(groupByError) > 0 {
		if err := d.reportPartialFailures(ctx, groupByError); err != nil {
			return fmt.Errorf("publish errors reporting failed: %w", err)
		}
	}

	return nil

pkg/beholder/durable_emitter.go:795

  • If the server returns an errored result for an event ID that is not in the batch, continuing here lets flushBatch mark the entire batch delivered even though the response could not be reconciled to stored rows. Treat an unknown errored event_id as a publish failure for the batch so the original rows remain pending instead of risking data loss.
		id, exists := mapEventIDtoTableId[result.EventId]
		if !exists {
			d.log.Warnw("PublishBatch returned invalid event", "event_id", result.EventId)
			continue
		}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/beholder/durable_emitter.go
Comment thread pkg/beholder/durable_emitter.go Outdated
Comment thread go.mod
Comment thread pkg/beholder/durable_emitter.go
Comment thread pkg/beholder/durable_event_store.go
Comment thread pkg/beholder/durable_emitter.go Outdated
Comment thread pkg/beholder/durable_emitter.go Outdated
Comment thread pkg/beholder/durable_emitter.go
tarcisiozf and others added 2 commits May 14, 2026 16:08
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@tarcisiozf tarcisiozf marked this pull request as draft May 14, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants