feat: rewrite go-runner to use overlay files

[not-matthias]

Changes in this PR:

• Rewrote go-runner
• Removed hugo (takes a long time to run in CI) and caddy (crashes in one benchmark)

Fixes #46

[Copilot]

Pull request overview

This PR rewrites the go-runner implementation to use overlay files instead of the previous templating approach. The main changes include removing large testing packages (hugo and caddy) from the test suite and restructuring the runner's architecture.

Key changes:

• Rewrote go-runner to use an overlay-based architecture for handling Go benchmark files
• Removed hugo and caddy test projects (hugo: slow CI builds, caddy: benchmark crashes)
• Deleted multiple quicktest and slogassert package files and their associated tests
• Simplified test utilities and CLI argument handling

Reviewed changes

Copilot reviewed 111 out of 362 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
go-runner/src/runner/overlay/templates.rs	Added new template system for overlay files with placeholders for instrument hooks and profile directories
go-runner/tests/utils.rs	Simplified test utilities, removing package-specific helpers in favor of CLI-based approach
go-runner/tests/pkg_arg.rs	Updated tests to use simplified CLI interface, removed dry_run parameter
go-runner/tests/error_file.rs	Migrated from args-based to CLI-based test execution
go.mod	Removed dependencies for testing frameworks (go-cmp, kr/pretty)
pkg/quicktest/, pkg/slogassert/	Removed entire testing utility packages
go-runner/src/utils.rs	Deleted utility functions for directory copying and git operations
go-runner/src/snapshots/*	Updated benchmark result snapshots to reflect new package ordering
go-runner/src/main.rs	Fixed profile_dir type conversion in main runner call
go-runner/src/builder/snapshots/*	Removed mixed import styles test snapshot

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codspeed-hq]

Merging this PR will improve performance by 18.67%

⚡ 1 improved benchmark
✅ 34 untouched benchmarks
⏩ 14 skipped benchmarks¹

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	bench_collect_results[10000000, 25]	6.1 s	5.2 s	+18.67%

---

_{Comparing cod-1672-improve-stability-of-codspeed-go-integration (3e1c66c) with main (4df4d46)}

Footnotes

1. 14 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[art049]

lgtm overall, nice to see deletions.

[art049]

To avoid conflict with modcache, you could run each of those test in a temp directory, just like we do for integration tests in codspeed-rust
This will also allow concurrency in runs

[GuillaumeLagrange]

lgtm, apart from the modcache issue, where Arthur's suggestion should do fine

[GuillaumeLagrange]

Since this is integration tests, shouldnt we just fail here ?

[not-matthias]

Actually, I don't think we can remove this. Or at least it will not be as clean as just those few lines.

The main reason we do it like this is because some projects only work with 1.25+ which then would break the integration test for 1.24. In order to fix it, we'd have to add an additional check in ci.yml where we exclude tests 1.25-only tests from running with GO1.24.

With the can_build_project we could just skip it and always run with all the go versions (which seems a bit simpler). I could add a comment explaining why we do this, then we wouldn't need to handle each case separately. Wdyt?

[Copilot]

Pull request overview

Copilot reviewed 89 out of 367 changed files in this pull request and generated no new comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[GuillaumeLagrange]

lgtm