Fix local CI: Dynamic task lists and aarch64 support #2431
+485
−74
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Detect host OS and architecture using platform module - Fix FileNotFoundError on macOS when reading /etc/os-release - Add platform flags for amd64 emulation on macOS ARM64 - Inject --platform linux/amd64 into podman build/run on macOS with ARM64 - Gracefully handle missing /etc/os-release on non-Linux systems
…tion - Add --native-arch flag to use native host architecture (ARM64 on macOS) - Add --emulate flag to force amd64 emulation even on ARM64 hosts - Default behavior: Use native arch on macOS, amd64 on Linux - Detect and inject appropriate --platform flags for podman - ARM64 mode allows much faster builds on macOS by avoiding emulation
- Detect git worktree references in .git file - Initialize fresh git repo in container for worktrees - Allows ci-run.py to work from git worktree branches
- No need for full git init in container - Create minimal .git/config to satisfy Makepkgs checks - Avoids dependency on git being installed in container
- Remove .git file from worktree before replacing with directory - Prevents 'File exists' error when creating minimal .git directory
- pcpbuild user doesn't have permission to remove .git file - Use sudo to remove and replace with minimal .git directory
- Created from x86_64 package list to support ARM64 builds on macOS - Allows qa/admin/list-packages to work with native ARM64 containers
- Overview of changes and architecture - Usage guide and workflow - Troubleshooting section - Future enhancement roadmap - References to related documentation
- Add --quick flag to run against multiple platforms in one command - Implement platform loading with priority hierarchy: 1. CLI arguments (comma or space-separated) 2. PCP_CI_QUICK_PLATFORMS environment variable 3. .pcp-ci-quick config file (line-separated) - Leverage existing --until parameter to stop at desired phase - Helpful error messages guide users on platform specification - Quick mode continues through all platforms instead of stopping on errors - Supports all existing flags (--native-arch, --emulate) Examples: # Build-only on 2 platforms python3 build/ci/ci-run.py --quick ubuntu2404-container,fedora43-container reproduce --until build # Full pipeline on platforms from config file python3 build/ci/ci-run.py --quick reproduce # Via environment variable export PCP_CI_QUICK_PLATFORMS='ubuntu2404 fedora43' python3 build/ci/ci-run.py --quick reproduce --until qa_sanity
- Add macOS architecture support section - Document --native-arch and --emulate flags - Include quick start examples for common workflows - Add troubleshooting for Rosetta issues - LOCAL_CI_IMPLEMENTATION.md removed (content integrated)
The git worktree detection and handling added in earlier commits is not needed. The container build process works fine without special handling for worktree branches. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Remove LOCAL_CI_IMPLEMENTATION.md (too verbose, content moved) - Add macOS architecture support section - Document --native-arch and --emulate flags - Include quick start examples for common workflows - Add troubleshooting for Rosetta issues 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Code Quality Improvements: - Remove superfluous comments from simple helper functions - Rename load_quick_platforms() to resolve_platforms_to_run() - Extract string parsing logic into _parse_platform_string() helper - Improve error handling for config file reads with warnings - Refactor ContainerRunner init into platform-specific methods - Add clarifying comment about platform_flags in setup() - Extract main() logic into helper functions for readability Documentation Improvements: - Reorganize build/ci/README.md: move Quick Start section earlier - Expand Supported Platforms list from 3 to 9 actively tested platforms - Reference .github/workflows/ci.yml as authoritative source - Label Troubleshooting section as macOS-specific - Document additional available platforms (Debian, Fedora Rawhide) Package List Cleanup: - Delete redundant Ubuntu+24.04+aarch64 file (identical to x86_64 version)
…cal CI - Fix reproduce command to use dynamically discovered tasks from platform config instead of hardcoded list - This enables --until parameter to work with qa_sanity and other tasks - Add Ubuntu 24.04 'any' architecture package list for aarch64 compatibility - Fix container setup to replace worktree pointer .git file with directory so Makepkgs can detect git repo - Add validation for unknown task names with helpful error messages 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
On Linux (GitHub Actions), .git is already a real directory and should be preserved. Only on macOS with worktrees is .git a file pointer that needs replacement. Check if .git is already a directory before attempting to recreate it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Instead of trying to preserve or work around .git worktree pointers or incomplete .git directories, reinitialize the repository as a fresh git repo using 'git init' if the current .git is not a valid git repository. This ensures Makepkgs can run all git commands (status, checkout, archive) regardless of the source being a worktree, submodule, or incomplete copy. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Replace the git init band-aid fix with proper copying of the full .git directory structure. Makepkgs requires a valid git repository for: - git status (checking for local changes) - git checkout (restoring VERSION.pcp) - git archive / git ls-files (creating source tarball) Use podman cp with /. syntax to ensure hidden files (.git) are copied. This works properly with clean checkouts (non-worktree).
Contributor
Author
Update: Git Directory Copying Fix VerifiedBuild test successful! ✅ The latest commit () replaces the Reinitialized existing Git repository in /Users/psmith/dev/pcp/pcp-local-ci/.git/ band-aid fix with a proper solution: copying the complete directory structure to the container. What was fixed:
Test Results:
This fix is compatible with clean git checkouts (not worktrees). The previous worktree-specific issues should now be resolved. |
Contributor
Author
Update: Git Directory Copying Fix VerifiedBuild test successful! ✅ The latest commit replaces the git init band-aid fix with a proper solution: copying the complete .git directory structure to the container. What was fixed:
Test Results:
This fix is compatible with clean git checkouts (not worktrees). |
The Debian+12+aarch64 package list was created in commit 991ffa5 (Aug 2025) and had systemd-dev added in 9d7d113 (Sep 2025). However, systemd-dev doesn't exist on Debian 12 - it's only in backports. The x86_64 file was subsequently fixed in two steps: - fc1fb36: marked as not4ci - 68eebb9: changed to just 'systemd' The aarch64 file was overlooked during these fixes because there's no GitHub Actions CI for Debian 12 aarch64. This was discovered during local CI testing on macOS ARM64. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes several issues identified during local CI testing on macOS ARM64:
Changes
build/ci/ci-run.py:
qa/admin/package-lists/Ubuntu+24.04+any:
Testing
Local testing on macOS M4 Pro ARM64 with --quick mode and --until qa_sanity parameter.
Full QA test run validation in progress to confirm end-to-end functionality.