Skip to content

IaC Extractor: Windows Support#237

Open
dilanbhalla wants to merge 7 commits into
advanced-security:mainfrom
dilanbhalla:dilan/iac-windows-extraction
Open

IaC Extractor: Windows Support#237
dilanbhalla wants to merge 7 commits into
advanced-security:mainfrom
dilanbhalla:dilan/iac-windows-extraction

Conversation

@dilanbhalla
Copy link
Copy Markdown

@dilanbhalla dilanbhalla commented Nov 3, 2025

  • tested extraction and tests locally (most tests pass, handful of failures probably worth further investigation/debugging)
  • attempted to update pipelines to build/test for windows as well

Copilot AI review requested due to automatic review settings November 3, 2025 05:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds Windows support to the CodeQL IAC extractor by creating PowerShell equivalents of existing bash scripts and enabling Windows in CI/CD workflows.

  • Adds PowerShell scripts for Windows: create-extractor-pack.ps1, run-tests.ps1, and install-extractor.ps1
  • Updates CI/CD workflows to include Windows platform in the build matrix
  • Updates extractor configuration to support additional file extensions (.tfvars, .bicep) and changes language identifier from "hcl" to "iac"

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tools/qltest.cmd Added .tfvars and .bicep file extensions; changed language from "hcl" to "iac"
tools/index-files.cmd Added "extract" subcommand to extractor invocation
scripts/run-tests.ps1 New PowerShell script for running tests on Windows
scripts/install-extractor.ps1 New PowerShell script for installing extractor on Windows
scripts/create-extractor-pack.ps1 Rewritten PowerShell script with improved structure and error handling
.github/workflows/publish.yml Added Windows to build matrix and fixed typo in "Download"
.github/workflows/build.yml Added Windows to test matrix with platform-specific build and test steps
Comments suppressed due to low confidence (1)

.github/workflows/build.yml:63

  • Corrected spelling of 'extensions' to 'extension'. The command should be 'gh extension install' not 'gh extensions install'.
          gh extensions install github/gh-codeql

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/create-extractor-pack.ps1 Outdated
Comment thread scripts/create-extractor-pack.ps1 Outdated
Comment thread scripts/create-extractor-pack.ps1 Outdated
Comment thread scripts/install-extractor.ps1 Outdated
Dilan Bhalla and others added 2 commits May 27, 2026 10:10
Review fixes:
- install-extractor.ps1: replace fragile backtick-continued gh release
  download invocation with array splatting; add LASTEXITCODE checks
  (fixes aegilops syntax-error report).
- create-extractor-pack.ps1: normalize indentation to 4 spaces; fix
  catch alignment; add LASTEXITCODE checks for gh/cargo/git/tar
  (fixes 3 Copilot review comments).
- run-tests.ps1: add LASTEXITCODE check after codeql pack install.
- build.yml: fix typo 'gh extensions install' -> 'gh extension install'.

Sanity-check fixes during refresh:
- publish.yml: install Rust toolchain on Windows runner too (was
  previously skipped, leaving build to rely on preinstalled toolchain).
- tools/qltest.cmd: mirror tools/qltest.sh by running a second
  'database index-files' pass for .yml/.yaml/.json under --language=yaml,
  and restore .bicep extension. Without this, library tests for
  YAML/OpenAPI/CloudFormation/ARM had no source files extracted on
  Windows.
- docs/workflows.md: document Windows PowerShell install/build/test
  scripts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 27, 2026 17:22
@dilanbhalla dilanbhalla force-pushed the dilan/iac-windows-extraction branch from 22f90b9 to e0b732f Compare May 27, 2026 17:22
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.

@github-actions github-actions Bot added the docs label May 27, 2026
Dilan Bhalla and others added 2 commits May 27, 2026 10:26
The previous 'Download Extracter' step used bash syntax (set -e, chmod,
backslash continuations) but ran on all OSes including windows-latest,
which executed it with pwsh and produced a ParserError.

Also set fail-fast: false on the matrix so a failure on one OS does
not cancel running tests on the other.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PowerShell try/catch does not catch non-zero exit codes from native
commands (only PowerShell exceptions). The previous logic in
create-extractor-pack.ps1 and run-tests.ps1 assumed gh codeql would
throw when the extension wasn't installed; it doesn't, so the scripts
would proceed to invoke gh codeql without ever installing the
extension, causing failures later (e.g. codeql pack install).

Switch to checking LASTEXITCODE explicitly, which works for both
PowerShell 5.1 and 7.x.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 27, 2026 17:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.

Dilan Bhalla and others added 2 commits May 27, 2026 10:41
…from CFN/ECS tests

Two fixes for Windows CI:

1. build.yml paths-filter now triggers rebuild for tools/** and scripts/create-extractor-pack.* changes. Previously these changes caused CI to download the prebuilt extractor release containing the OLD qltest.cmd (--language=hcl), so the fixed qltest.cmd never reached test runs.

2. Removed empty .tf marker files from CloudFormation/ECS query tests. On Windows, mixing .tf and .yml in the same dir triggers a containerparent INVALID_KEY conflict (C:/ vs C: drive root normalization) between the iac and yaml extractors. The markers are 0-byte and contribute nothing to extraction.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The github/codeql repo (pulled in as a Cargo git dep) contains paths >260 chars. Without core.longpaths=true, git on Windows fails with 'path too long' (class=Filesystem 30) during 'cargo fetch'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 27, 2026 17:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.

Branch protection requires 'tests (library-tests)' and 'tests (queries-tests)', but the matrix emits OS-qualified names like 'tests (ubuntu-latest, library-tests)'. Two new aggregate jobs needs: tests and re-emit the bare names so required checks are satisfied without changing branch protection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants