Skip to content

build: split package execution stages - #2176

Closed
zhouguangyuan0718 wants to merge 7 commits into
xgo-dev:mainfrom
zhouguangyuan0718:agent/parallel-build-pr2
Closed

build: split package execution stages#2176
zhouguangyuan0718 wants to merge 7 commits into
xgo-dev:mainfrom
zhouguangyuan0718:agent/parallel-build-pr2

Conversation

@zhouguangyuan0718

Copy link
Copy Markdown
Contributor

Summary

  • split the serial package pipeline into preflight, execution, and finalization stages
  • keep cache lookup outside LLVM module work and keep LLVM backend execution serial
  • centralize archive publication and cache metadata finalization

Validation

  • GOCACHE=/private/tmp/llgo-pr2-gocache go test ./internal/build ./internal/goflags ./cmd/internal/flags -count=1

Dependency

This PR includes and depends on #2174 and #2175. Merge/rebase it after those PRs; it will then contain only this stage split.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

This is a well-structured refactor that cleanly separates the package build pipeline into preflight/execute/finalize stages and defers SSA building into a bounded worker pool, in preparation for parallel compilation. The concurrency boundary is drawn correctly: ssa.Package.Build() is documented as thread-safe and is the only work parallelized, while the mutating fixSSAOrder pass and the LLVM backend remain serial. effectiveDependencies correctly folds alt-package imports into fingerprinting, and the -p flag path validates and applies atomically. Good test coverage on the new helpers.

Two findings are noted inline. Security and documentation-accuracy passes found no actionable issues.

Minor (no inline): packageBuildResult fields spec, cacheHit, and archiveFile (internal/build/package_build.go) are populated by packageBuildResultFor but read only in tests; production code consumes only needRuntime/needPyInit. The comments frame these as scaffolding for a future scheduler PR, so this is acceptable, but reviewers should confirm the forward-looking fields are intended to land ahead of their consumer.

"gcflags",
"gccgoflags",
"ldflags",
"p",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding p to argumentListFlagNames conflates two different semantics. This list is needed by normalizeBuildFlags/argumentListBuildFlag (normalize.go) so that a space-separated -p 4 normalizes to -p=4 — that part is correct. But the same list also drives wholeLineValueFlag (flagfile.go:65-81), which treats the entire rest of a flags-file line as one unsplit value. That behavior is intended for flags whose value is an argument list (gcflags, ldflags, toolexec), not for a scalar integer flag.

Consequence: in a flags file, a line like -p=4 -tags=foo is captured as the single flag -p=4 -tags=foo (the -tags=foo is swallowed into -p's value) instead of being split into two flags.

Consider using a separate set for scalar flags that accept a detached value vs. flags whose value is an argument list, or excluding p from the whole-line path. At minimum, add a test covering -p combined with another flag on one flags-file line.

Comment thread internal/build/build.go Outdated
unique := make([]ssaBuildEntry, 0, len(entries))
seen := make(map[*ssa.Package]bool, len(entries))
for _, entry := range entries {
if entry.pkg == nil || seen[entry.pkg] {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dedup keeps the first-seen entry per *ssa.Package and drops later duplicates. Entries are concatenated as altEntries (fixOrder:false), then pkgEntries (fixOrder:true), then depEntries (see the append(append(altEntries, pkgEntries...), depEntries...) call). If the same *ssa.Package ever appeared in both an alt entry and a normal entry, the fixOrder:false alt entry would win and fixSSAOrder would be silently skipped for that package.

Today alt packages are distinct *ssa.Package instances (they carry altPkgPathPrefix and registerSSAPkgs skips those paths), so no real collision occurs — hence low severity. As cheap future-proofing, OR the fixOrder flag when merging a duplicate rather than first-wins, so a missing ordering fix can't surface as a hard-to-diagnose failure after a future change.

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@zhouguangyuan0718
zhouguangyuan0718 force-pushed the agent/parallel-build-pr2 branch 2 times, most recently from 6c9c543 to 475f24b Compare July 26, 2026 08:14
@zhouguangyuan0718
zhouguangyuan0718 force-pushed the agent/parallel-build-pr2 branch from 475f24b to 6b7aa0b Compare July 27, 2026 02:47
@zhouguangyuan0718
zhouguangyuan0718 force-pushed the agent/parallel-build-pr2 branch from 6b7aa0b to 7126afd Compare July 27, 2026 04:35
@zhouguangyuan0718

Copy link
Copy Markdown
Contributor Author

This PR has been absorbed into #2175 during the parallel-build stack consolidation. The retained PR now contains the explicit build units and preflight/execute/finalize stage split.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant