Skip to content

hq: read-only query CLI for HCL2 files#277

Open
kkozik-amplify wants to merge 1 commit intorelease/8.xfrom
hq
Open

hq: read-only query CLI for HCL2 files#277
kkozik-amplify wants to merge 1 commit intorelease/8.xfrom
hq

Conversation

@kkozik-amplify
Copy link
Copy Markdown
Collaborator

@kkozik-amplify kkozik-amplify commented Mar 12, 2026

Summary

Adds hq, a read-only query CLI for HCL2 files, and the hcl2.query Python API it's built on.

Query API (hcl2/query/)

A typed view layer over the LarkElement IR tree, enabling structural navigation without serializing to dicts.

  • NodeView hierarchy — typed wrappers for every IR node: DocumentView, BodyView, BlockView, AttributeView, TupleView, ObjectView, ForTupleView, ForObjectView, FunctionCallView, ConditionalView. Each exposes semantic accessors (.block_type, .name_labels, .value, .elements, etc.)
  • Path resolution — structural path parser supporting label depth (resource.aws_instance.main), type filters (resource:aws_instance), wildcards ([*]), and select() predicates
  • Pipeline execution — pipe operator chains path stages, builtins, and select stages (resource[*] | .tags | keys)
  • Predicate engine — recursive-descent parser with boolean logic (and, or, not), comparisons (==, !=, >, <), and string functions (contains, test, has, startswith, endswith)
  • Builtinskeys, values, length, type, flatten, unique, sort, reverse, first, last, min, max, group_by, sort_by, to_entries, from_entries
  • Object construction — jq-style {key: .path} syntax for reshaping results
  • Structural diff — compare two HCL documents and report added/removed/changed elements
  • Introspection--describe shows view type and available accessors; --schema dumps the full view hierarchy
  • Hybrid/eval modesstructural::eval for mixing path queries with Python attribute access; -e for pure eval with AST-validated safe expressions
  • hcl2.walk — generic tree walker for LarkElement trees

CLI (cli/hq.py)

  • Distinct exit codes: 0 (success), 1 (no results), 2 (parse error), 3 (query error), 4 (I/O error)
  • Output modes: --json, --ndjson, --value, --raw
  • --with-location for file + line metadata in JSON output
  • --with-comments for comment extraction
  • __file__ provenance in multi-file JSON results
  • Multiple FILE arguments with built-in glob expansion
  • Compact JSON for non-TTY, pretty-printed for TTY
  • Optional queries (? suffix) — exit 0 on empty results
  • --diff for structural comparison between two files
  • Multiprocessing for directory queries

Docs & tests

  • 4 new doc files: querying API, advanced API, hq reference, hq examples
  • Unit tests for every query module (14 test files) + 1044-line CLI test suite

Test plan

  • python -m unittest discover -s test -p "test_*.py" — all tests pass
  • Exit codes: echo "" | hq 'x' - exits 1; hq 'x' nonexistent.tf exits 4
  • hq --json 'resource[*]' dir/ includes __file__ in each result
  • hq --ndjson 'resource[*]' dir/ produces one compact JSON per line
  • hq --json --with-location 'resource[*]' file.tf includes __line__ and __end_line__
  • hq 'query' file1.tf file2.tf dir/ works with multiple paths

🤖 Generated with Claude Code

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an initial HCL2 “query” layer (typed view facades + structural path/pipeline/predicate system) and a new hq CLI for jq-like querying, introspection, and structural diffs.

Changes:

  • Introduces hcl2.query package (views, path parsing, resolver, pipeline, predicates, builtins, introspection, diff utilities).
  • Adds hq CLI entrypoint and registers it in pyproject.toml, with docs updates and extensive unit tests.
  • Adds generic IR tree-walking helpers (hcl2.walk) and improves HCL reconstruction for subtrees (reconstruct_fragment).

Reviewed changes

Copilot reviewed 52 out of 53 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
test/unit/test_walk.py New tests for generic tree walking and find helpers.
test/unit/test_api.py Adds coverage for new hcl2.api.query() convenience entrypoint.
test/unit/rules/test_whitespace.py Updates comment serialization expectations and adds multiline block-comment test.
test/unit/query/init.py Test package marker for query tests.
test/unit/query/test_safe_eval.py Tests for AST validation and restricted eval behavior.
test/unit/query/test_resolver.py Tests for structural resolver (wildcards, recursion, type filters, skip-labels).
test/unit/query/test_predicate.py Tests for predicate tokenizer/parser/evaluator (select(), any/all/has, string funcs, postfix not).
test/unit/query/test_pipeline.py Tests for pipeline splitting/classification/execution and construct stage.
test/unit/query/test_path.py Tests for structural path parsing including recursion, type filters, skip-labels, optional suffix.
test/unit/query/test_introspect.py Tests for --describe/--schema output structure.
test/unit/query/test_functions.py Tests for FunctionCallView facade.
test/unit/query/test_for_exprs.py Tests for for-expression view facades.
test/unit/query/test_expressions.py Tests for ConditionalView and resolver/pipeline integration.
test/unit/query/test_diff.py Tests for structural diff utilities.
test/unit/query/test_containers.py Tests for TupleView/ObjectView behaviors.
test/unit/query/test_builtins.py Tests for built-in pipeline transforms (keys/values/length).
test/unit/query/test_body.py Tests for DocumentView/BodyView parsing and lookup helpers.
test/unit/query/test_blocks.py Tests for BlockView label handling and body delegation.
test/unit/query/test_base.py Tests for NodeView base behaviors and view registry dispatch.
test/unit/query/test_attributes.py Tests for AttributeView name/value/value_node and serialization.
test/unit/cli/test_hq.py CLI behavior tests (modes, output flags, optional, schema/describe, diff).
pyproject.toml Registers hq console script and packages hcl2.query.
pylintrc Disables/adjusts pylint rules to accommodate new query modules.
hcl2/walk.py Adds generic walk/find/ancestors helpers over IR nodes.
hcl2/rules/whitespace.py Changes comment extraction format to object list {value: ...} and adds block-comment handling.
hcl2/rules/base.py Ensures BodyRule.serialize returns a plain dict (not defaultdict).
hcl2/reconstructor.py Adds reconstruct_fragment() for subtree reconstruction without newline normalization.
hcl2/query/safe_eval.py Adds AST-validated restricted eval for hq eval/hybrid modes.
hcl2/query/resolver.py Implements structural path resolver with recursion/type filters/predicates.
hcl2/query/predicate.py Implements predicate tokenizer/parser/evaluator (no Python eval).
hcl2/query/pipeline.py Adds pipeline splitting/classification/execution, plus construct stage support.
hcl2/query/path.py Parses structural paths including recursion, type qualifiers, select(), skip-labels.
hcl2/query/introspect.py Adds schema/describe builders for CLI introspection.
hcl2/query/functions.py Adds FunctionCallView facade.
hcl2/query/for_exprs.py Adds ForTupleView/ForObjectView facades.
hcl2/query/expressions.py Adds ConditionalView facade.
hcl2/query/diff.py Adds structural diff + JSON/text formatting.
hcl2/query/containers.py Adds TupleView/ObjectView facades.
hcl2/query/builtins.py Adds built-in transforms used by pipeline stages.
hcl2/query/body.py Adds DocumentView/BodyView facades and parsing helpers.
hcl2/query/blocks.py Adds BlockView facade for labels/body delegation.
hcl2/query/attributes.py Adds AttributeView facade for name/value access.
hcl2/query/_base.py Adds NodeView base, registry, view factory, and common helpers.
hcl2/query/init.py Public exports for query system.
hcl2/api.py Adds query(source) convenience function returning DocumentView.
hcl2/init.py Re-exports query from api module.
docs/04_hq.md Adds full hq CLI reference and query language documentation.
docs/03_advanced_api.md Updates advanced docs to match new doc structure and references.
docs/02_querying.md Adds Python query API guide for view facades and walking.
docs/01_getting_started.md Reworks getting started guide; documents new comment format and hcl2.query().
README.md Updates documentation links and mentions the hq CLI tool.
CLAUDE.md Updates architecture map and docs guidance to include query/hq components.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@kkozik-amplify kkozik-amplify changed the title draft hcl2 tree query system and hq cli hq: read-only query CLI for HCL2 files Mar 30, 2026
@kkozik-amplify kkozik-amplify marked this pull request as ready for review March 30, 2026 18:57
@kkozik-amplify kkozik-amplify requested a review from a team as a code owner March 30, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants