Skip to content

[registry] Fix unordered completion logs during concurrent model generation - #1051

Open
KumarNirupam1 wants to merge 6 commits into
meshery:masterfrom
KumarNirupam1:fix/registary-generate-ordered-logs
Open

[registry] Fix unordered completion logs during concurrent model generation#1051
KumarNirupam1 wants to merge 6 commits into
meshery:masterfrom
KumarNirupam1:fix/registary-generate-ordered-logs

Conversation

@KumarNirupam1

@KumarNirupam1 KumarNirupam1 commented Jul 3, 2026

Copy link
Copy Markdown

Description

Fixes #1050
Relates to meshery/meshery#11036

When mesheryctl registry generate runs over many models, the terminal output can show mismatched log lines — for example:

Current model: jx-app-flagger
  extracted 11 components for Jenkins Operator (jenkins-operator)
Current model: jx-app-kubeless
  extracted 3 components for Jx App Flagger (jx-app-flagger)
Current model: k0s-homelab
  extracted 5 components for JSpolicy (jspolicy)
Current model: k3s

The model name on the first line does not match the model in the second line. This was reported in meshery/meshery#11036, but the root cause is in meshkit, not mesheryctl.

What was going wrong

InvokeGenerationFromSheetWithOptions processes up to 20 models in parallel (semaphore.NewWeighted(20)). On each successful completion, the code logged two separate lines:

  1. Current model: <name>
  2. Extracted N components for <display name> (<name>)

Those log calls run inside concurrent goroutines with no synchronization on stdout. When many models finish around the same time, the lines get interleaved and no longer belong to the same model.

The same two-line pattern existed in GenerateDefsForCoreRegistrant (the meshery registrant path), including the "no change in components" case.

Generation itself was fine — this is a logging/UX issue only.

What I changed

In registry/model.go, I merged each two-line completion log into one log line using Log.Info(fmt.Sprintf(...)):

  • ArtifactHub / GitHub path (InvokeGenerationFromSheetWithOptions):
    Current model: jenkins-operator — extracted 11 components for Jenkins Operator (jenkins-operator)

  • Meshery registrant path (GenerateDefsForCoreRegistrant):
    Same format for extracted components, plus a single line for the no-change case:
    Current model: <name> — no change in components for <display name> (<name>)

Single-line logs like Model already exists: ... were left unchanged.

No new helper functions, no concurrency changes, no changes to generation logic or file paths.

Why this approach

Combining model name and component count into one Log.Info call means each completion is one write to stdout. That prevents goroutines from splitting "Current model" and "extracted" across different lines. Keeping the change inline matches the existing Log.Info(fmt.Sprintf(...)) style already used elsewhere in this file.

Testing

Ran locally:

go test ./registry/... -count=1 -v
go build ./...

All registry tests passed.

Note: A single-model run (e.g. --model flyte) usually will not show this bug — it needs multiple models completing in parallel.

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

Summary by CodeRabbit

  • Bug Fixes

    • Improved model-generation log messages by consolidating relevant model details into a single, clearer line.
    • Ensured completion logs consistently include extracted-component counts and model status information.
  • Tests

    • Added coverage verifying accurate, non-duplicated logging when multiple models are processed concurrently.

Signed-off-by: Kumar Nirupam <kumar.nirupam24@gmail.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors logging statements in registry/model.go to combine multiple log calls into single formatted log statements using fmt.Sprintf inside Log.Info. The reviewer suggests using Log.Infof directly instead of nesting fmt.Sprintf inside Log.Info to make the code more idiomatic in Go.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread registry/model.go
Comment thread registry/model.go
Comment thread registry/model.go
Comment thread registry/model.go
@KumarNirupam1

KumarNirupam1 commented Jul 4, 2026

Copy link
Copy Markdown
Author

@YASHMAHAKAL @aabidsofi19 kindly review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes confusing, interleaved completion output during concurrent mesheryctl registry generate runs by ensuring each model’s “completion” message is emitted as a single log line (so goroutines can’t split related messages across multiple lines on stdout).

Changes:

  • Consolidated the two-line “Current model …” + “Extracted … components …” completion logs into a single Log.Info(...) call in the spreadsheet-driven generation path.
  • Applied the same single-line completion logging pattern in the Meshery registrant generation path, including the “no change in components” case.

@yi-nuo426

Copy link
Copy Markdown
Contributor

Yay! @KumarNirupam1 be sure to demo this on this week's meeting.

@KumarNirupam1

KumarNirupam1 commented Jul 7, 2026

Copy link
Copy Markdown
Author

Yay! @KumarNirupam1 be sure to demo this on this week's meeting.

okay 👍

@KumarNirupam1

Copy link
Copy Markdown
Author
Screenshot 2026-07-10 200438

Update -
here is the intended logs how it suppose to render and not mismatch in two seperate line.
I have presented this issue on developers meet to the community

@fitzergerald fitzergerald added the diffray-review Trigger diffray code review label Jul 21, 2026
@diffray diffray Bot added the diffray-review-failed diffray review status: failed label Jul 21, 2026
@leecalcote leecalcote added diffray-review Trigger diffray code review and removed diffray-review Trigger diffray code review diffray-review-failed diffray review status: failed labels Jul 21, 2026
@diffray diffray Bot added the diffray-review-failed diffray review status: failed label Jul 21, 2026

@Maanvi212006 Maanvi212006 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified this locally, not just from reading the diff.

  1. Wrote a quick script simulating 20 concurrent goroutines logging completions — old way (two separate calls) gave 120 interleaved/mismatched lines across 50 trials, new way (one combined call) gave 0. Fix works.
  2. Tests pass, build is clean.

Two things to flag:

  1. No test covers this specific fix — add one that checks for split lines under concurrent load.
  2. +1 to Gemini's suggestion — use Log.Infof(...) instead of Log.Info(fmt.Sprintf(...)).

Otherwise good fix, correctly scoped.

Signed-off-by: Kumar Nirupam <kumar.nirupam24@gmail.com>
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 345bb6ba-77cf-446b-9cb7-0c6621cfa496

📥 Commits

Reviewing files that changed from the base of the PR and between de5c753 and 9a84114.

📒 Files selected for processing (1)
  • registry/model_generation_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • registry/model_generation_test.go

📝 Walkthrough

Walkthrough

The change consolidates model-generation logs into single formatted messages. A concurrent test verifies one complete log line per model with the extracted-component count.

Changes

Model-generation logging

Layer / File(s) Summary
Formatted logging and concurrency validation
registry/model.go, registry/model_generation_test.go
Generation paths log model details, component counts, display names, and versions in formatted messages. The test verifies one complete log line per model during concurrent execution.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

  • meshery/meshkit#1050 — The logging changes and concurrency test address mixed completion messages during concurrent model generation.

Suggested reviewers: maanvi212006, yi-nuo426

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the registry log-ordering fix during concurrent model generation, which matches the main change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@KumarNirupam1

KumarNirupam1 commented Aug 1, 2026

Copy link
Copy Markdown
Author

Hi @Maanvi212006, thanks for the review.

  • I've added the regression test you asked for in meshkit - registry/model_generation_test.go, ``TestConcurrentCompletionLogsAreSingleLine. It redirects the logger to abuffer using the existing SetupLogger pattern, runs many model completions concurrently with a sync.WaitGroup, and asserts that every completion is logged exactly once as a single self-contained line (model name + extracted count on the same line).

It passes, including with -race.

  • Log.Info(fmt.Sprintf(...)) style already used elsewhere in this file. so that was used and gemmni suggestions were not accepted.

The full end-to-end concurrent run needs real spreadsheets/ArtifactHub/registrant APIs plus networkaccess, so that level of test isn't feasible in meshkit's unit tests. Once this PR is merged andmeshery bumps meshkit, I'll add the E2E test in the meshery repo for meshery/meshery#11036, asmentioned in the PR description.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@registry/model_generation_test.go`:
- Around line 287-290: Capture the existing package logger in origLog before
assigning the test logger in the setup around Log and SetupLogger. Keep the
deferred cleanup restoring origLog after the test so later tests retain the
logger state that existed before this test.
- Around line 299-326: Update the test around the concurrent logging block to
exercise the production paths in registry/model.go rather than emitting
completion messages directly from the test goroutines. Assert exactly one
complete log line for each model and its corresponding componentCounts[i] value,
and cover both ArtifactHub/GitHub and Meshery paths, including the no-change
branch near the existing production logic; alternatively, test the shared
formatter used by those paths.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b8898115-2d0d-477f-adc7-dc855cbc39a7

📥 Commits

Reviewing files that changed from the base of the PR and between be688f5 and de5c753.

📒 Files selected for processing (2)
  • registry/model.go
  • registry/model_generation_test.go

Comment thread registry/model_generation_test.go
Comment thread registry/model_generation_test.go
Signed-off-by: Kumar Nirupam <kumar.nirupam24@gmail.com>

@Maanvi212006 Maanvi212006 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffray-review Trigger diffray code review diffray-review-failed diffray review status: failed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[registry] Completion logs get mixed up during concurrent model generation

6 participants