-
Notifications
You must be signed in to change notification settings - Fork 66
feat: add --batch-size flag to push refs in batches #173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Add support for pushing refs in smaller batches to avoid server-side limits and timeouts when syncing large repositories with many tags/branches. - Add --batch-size flag (default 0 = no batching, original behavior) - Add References() method to GitRepository interface - Implement collectRefs() and pushRefsInBatches() helpers - Add MinBatchSize validation (must be 0 or >= 10) This addresses issues where repositories with 1000+ refs fail to sync to GHES with 'command error on refs/heads/<branch>: failed' errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds a --batch-size flag to the push and sync commands to enable pushing Git refs in smaller batches, helping avoid server-side limits when syncing repositories with many branches and tags.
Changes:
- Added
--batch-sizeflag with validation (0 for no batching, or minimum 10) - Implemented batching logic to collect and push refs in configurable batch sizes
- Added comprehensive unit tests for the new functionality
- Pinned tool dependencies to specific versions for Go 1.21 compatibility
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/push.go | Added batch-size flag, validation, collectRefs() function, and pushRefsInBatches() function to handle batched pushing |
| src/push_test.go | Added comprehensive tests for batch size validation, ref collection, and batched pushing functionality with mock implementations |
| src/git.go | Extended GitRepository interface with References() method and added (unused) RefInfo struct |
| src/git_test.go | Added tests verifying mock implementations satisfy the GitRepository and GitRemote interfaces |
| script/bootstrap | Pinned tool dependencies from @master to specific versions (v0.16.0, v1.6.0) for Go 1.21 compatibility |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
cd1a3b1 to
854c013
Compare
- Remove unused RefInfo struct from git.go - Remove redundant pushedAny variable tracking in pushRefsInBatches - Remove incomplete TestPushRefsInBatches_PartialUpToDate test (already covered by existing test case)
Summary
This PR adds a
--batch-sizeflag to thepushandsynccommands that allows pushing refs in smaller batches instead of all at once. This helps avoid server-side limits when syncing large repositories with many branches and tags.Problem
When syncing repositories with a large number of refs (branches + tags), the push operation can fail due to server-side limits on the number of refs that can be pushed in a single operation.
Solution
--batch-sizeflag (default: 0, meaning no batching for backward compatibility)+refs/heads/main:refs/heads/maininstead of wildcardsChanges
References()method toGitRepositoryinterface--batch-sizeflag with validation (must be 0 or >= 10)collectRefs()function to gather branch and tag refspushRefsInBatches()function to push refs in smaller batchesUsage
Testing