Follow Makefile includes when completing make targets - #275
Conversation
adb42f0 to
1c60a56
Compare
|
/oz-review |
|
I'm re-reviewing this pull request in response to a review request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR changes the make command completion generator to recursively print Makefile include contents before the existing target parser runs.
Concerns
- The new include follower opens paths supplied by the repository's Makefile without bounding them to the project, so completion can read and parse local files outside the repo.
Security
- A malicious repository can add an absolute or parent-traversal include path and cause tab completion to read and parse local files before the user runs
make. Restrict recursive include following to project-local files or skip paths that resolve outside the working tree.
Verdict
Found: 0 critical, 1 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
| /// Known limitation: include paths that rely on globbing (`include dir/*.mk`) or make variables | ||
| /// (`include $(VAR)`) are not expanded, since resolving them safely would require either a shell | ||
| /// (injection risk) or evaluating the Makefile. | ||
| const LIST_TARGETS_COMMAND: &str = r##"out=$(awk 'function process(f, line,rest,n,i,parts){if(f in seen)return;seen[f]=1;while((getline line<f)>0){print line;if(match(line,/^[ \t]*[-s]?include[ \t]+/)){rest=substr(line,RLENGTH+1);n=split(rest,parts,/[ \t]+/);for(i=1;i<=n;i++)if(parts[i]!="")process(parts[i])}}close(f)}BEGIN{for(i=1;i<ARGC;i++)process(ARGV[i]);exit}' [Mm]akefile 2>/dev/null) && printf '%s\n' "$out" || cat [Mm]akefile 2>/dev/null"##; |
There was a problem hiding this comment.
.. traversals). Resolve include paths and skip anything outside the working tree before getline.
1c60a56 to
7028708
Compare
|
Thanks @lucieleblanc / Oz — addressed the security concern (the one important finding). Fix: the recursive include-follower now confines itself to the project subtree. A new This is a safe boundary because GNU make resolves Added a regression test ( Local /oz-review |
|
I'm re-reviewing this pull request in response to a review request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR replaces make target completion input with an awk-based include expander and adds end-to-end tests for included makefiles and defensive fallback cases.
Concerns
- The include expander does not fully enforce the stated project-subtree boundary: the path check is lexical, and tab-indented recipe lines can still be mistaken for include directives.
Security
- A hostile repository can use a relative symlink or an include-looking recipe line to make tab completion read files outside the intended Makefile include graph before the user runs
make.
Verdict
Found: 0 critical, 1 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
| /// (injection risk) or evaluating the Makefile. Absolute, home-relative, and `..`-escaping | ||
| /// includes are intentionally not followed (see the security boundary above); their targets are | ||
| /// simply not surfaced in completion. | ||
| const LIST_TARGETS_COMMAND: &str = r##"out=$(awk 'function safe(p){return p!~/^\//&&p!~/^~/&&p!~/(^|\/)\.\.(\/|$)/}function process(f, line,rest,n,i,parts){if(f in seen)return;seen[f]=1;while((getline line<f)>0){print line;if(match(line,/^[ \t]*[-s]?include[ \t]+/)){rest=substr(line,RLENGTH+1);n=split(rest,parts,/[ \t]+/);for(i=1;i<=n;i++)if(parts[i]!=""&&safe(parts[i]))process(parts[i])}}close(f)}BEGIN{for(i=1;i<ARGC;i++)process(ARGV[i]);exit}' [Mm]akefile 2>/dev/null) && printf '%s\n' "$out" || cat [Mm]akefile 2>/dev/null"##; |
There was a problem hiding this comment.
safe() is only lexical, so a project-local symlink can point outside the tree, and the regex accepts tab-prefixed recipe lines as include directives. Reject symlinks/out-of-tree realpaths and avoid treating recipe lines as directives before calling getline.
7028708 to
dd79529
Compare
|
Addressed both points from the latest review. 1. Recipe-line over-match. The include-directive match was 2. Symlink escape (lexical check insufficient). Correct — visit(){ rp=$(realpath -- "$1" 2>/dev/null)||return 0;
case "$rp" in "$root"|"$root"/*) ;; *) return 0;; esac; # symlink-resolved containment
…cat -- "$rp"; recurse into safe() include paths… }awk now only extracts include paths (never opens them); every path reaches the shell as a quoted argument, never interpolated, so there is still no command-injection surface. If Tests (6 in the make module, all green): nested includes + missing optional include (descriptions survive), directory include, absolute + /oz-review |
|
I'm re-reviewing this pull request in response to a review request. You can view the conversation on Warp. I reviewed this pull request and requested human review from: Comment Powered by Oz |
Oz no longer requests changes for this pull request after the latest automated review.
There was a problem hiding this comment.
Overview
This updates the make target generator to expand project-local included Makefiles while preserving a containment boundary for untrusted repositories, and adds end-to-end tests covering nested includes, invalid directory includes, path traversal, recipe-indented directives, and symlink escapes.
Concerns
- Include operand parsing currently does not strip trailing Make comments before word-splitting, so comment words can be followed as extra include paths if matching files exist.
Verdict
Found: 0 critical, 0 important, 1 suggestions
Approve with nits
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
| /// safely would require a shell (injection risk) or evaluating the Makefile. Absolute, | ||
| /// home-relative, `..`-escaping, and symlink-escaping includes are intentionally not followed (see | ||
| /// the security boundary above); their targets are simply not surfaced in completion. | ||
| const LIST_TARGETS_COMMAND: &str = r##"root=$(pwd -P)||exit 0;command -v realpath >/dev/null 2>&1||{ cat [Mm]akefile 2>/dev/null;exit 0;};seen="|";visit(){ rp=$(realpath -- "$1" 2>/dev/null)||return 0;case "$rp" in "$root"|"$root"/*) ;; *) return 0;; esac;case "$seen" in *"|$rp|"*) return 0;; esac;seen="$seen$rp|";cat -- "$rp" 2>/dev/null;set -f;for inc in $(awk 'function safe(p){return p!~/^\//&&p!~/^~/&&p!~/(^|\/)\.\.(\/|$)/} /^ *[-s]?include[ \t]+/{match($0,/^ *[-s]?include[ \t]+/);rest=substr($0,RLENGTH+1);n=split(rest,parts,/[ \t]+/);for(i=1;i<=n;i++)if(parts[i]!=""&&safe(parts[i]))print parts[i]}' "$rp" 2>/dev/null);do set +f;visit "$inc";set -f;done;set +f;};for f in [Mm]akefile;do [ -e "$f" ]&&visit "$f";done"##; |
There was a problem hiding this comment.
💡 [SUGGESTION] Strip trailing Make comments before splitting include operands; otherwise include common.mk # comment treats # and comment words as additional include paths when matching files exist, which can surface targets make would not load.
The `make` `list_targets` generator ran `cat [Mm]akefile`, which reads only the top-level Makefile, so targets defined in files pulled in via `include`/`-include`/`sinclude` were never suggested (warpdotdev/warp#11705). Replace the command with a POSIX-sh `visit()` walker over the include graph: it `cat`s a file, uses a small awk program to extract that file's include paths, and recurses into each. awk only extracts paths (it never opens them) and every path is passed to the shell as a quoted argument, never interpolated, so a Makefile cannot inject commands. The extractor only matches `include` directives indented with spaces, not a leading tab -- a tab-indented line is a make recipe (a shell command), so a tab-indented `include`-looking line is deliberately not followed -- and strips a trailing `# comment` before splitting so comment words are not mistaken for include paths. Confine include-following to the project subtree so a hostile repository cannot make tab-completion read files outside the working tree (e.g. `include /etc/passwd`) before the user runs `make`, using two layers: - a lexical `safe()` guard that skips absolute, home-relative, and `..`-escaping include paths; and - `realpath`, which canonicalizes each remaining candidate so it is followed only when it resolves under `$(pwd -P)`. This stops escapes a lexical check cannot, e.g. an in-tree symlink (`evil -> /etc`) used as `include evil/passwd`. GNU make resolves include paths relative to its working directory, so legitimate project-local includes are unaffected. A `seen` set of canonical paths guards against include cycles. If `realpath` is unavailable the command falls back to `cat [Mm]akefile` (top-level only) rather than following includes unsafely. Glob (`include dir/*.mk`), variable (`include $(VAR)`), and whitespace-containing include paths are intentionally left unresolved. Add end-to-end tests run against temp projects: nested includes plus a missing optional include (asserting `##` descriptions survive and a trailing-comment operand is not followed), a directory include, an absolute and a `..`-escaping include pointing at real out-of-tree sentinels, a tab-indented (recipe) `include`-looking line, and an in-tree symlink resolving out of tree -- asserting only in-tree targets surface in each.
dd79529 to
0de986a
Compare
|
Addressed the trailing-comment nit. The include extractor now strips a trailing 6 make-module tests + /oz-review |
|
I'm re-reviewing this pull request in response to a review request. You can view the conversation on Warp. I reviewed this pull request and requested human review from: Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR updates the make target generator to follow include, -include, and sinclude directives with a shell/awk walker, while constraining traversal to files that canonicalize under the current directory. It also adds end-to-end tests for nested includes, optional missing includes, directory includes, path escape attempts, recipe-indented include-like lines, and symlink escapes.
Concerns
- Non-blocking: the command currently concatenates each visited makefile with no guaranteed separator, so a file without a final newline can merge with the next included file's first line and corrupt target parsing.
Verdict
Found: 0 critical, 0 important, 1 suggestions
Approve with nits
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
| /// safely would require a shell (injection risk) or evaluating the Makefile. Absolute, | ||
| /// home-relative, `..`-escaping, and symlink-escaping includes are intentionally not followed (see | ||
| /// the security boundary above); their targets are simply not surfaced in completion. | ||
| const LIST_TARGETS_COMMAND: &str = r##"root=$(pwd -P)||exit 0;command -v realpath >/dev/null 2>&1||{ cat [Mm]akefile 2>/dev/null;exit 0;};seen="|";visit(){ rp=$(realpath -- "$1" 2>/dev/null)||return 0;case "$rp" in "$root"|"$root"/*) ;; *) return 0;; esac;case "$seen" in *"|$rp|"*) return 0;; esac;seen="$seen$rp|";cat -- "$rp" 2>/dev/null;set -f;for inc in $(awk 'function safe(p){return p!~/^\//&&p!~/^~/&&p!~/(^|\/)\.\.(\/|$)/} /^ *[-s]?include[ \t]+/{match($0,/^ *[-s]?include[ \t]+/);rest=substr($0,RLENGTH+1);sub(/#.*/,"",rest);n=split(rest,parts,/[ \t]+/);for(i=1;i<=n;i++)if(parts[i]!=""&&safe(parts[i]))print parts[i]}' "$rp" 2>/dev/null);do set +f;visit "$inc";set -f;done;set +f;};for f in [Mm]akefile;do [ -e "$f" ]&&visit "$f";done"##; |
There was a problem hiding this comment.
💡 [SUGGESTION] Add an explicit newline separator after each cat; if a makefile lacks a trailing newline, the next included file starts on the same output line and list_targets_post_process can miss or corrupt that first target.
`visit()` cat'd each makefile in the include graph back-to-back with no separator, so a file without a trailing newline merged its last line into the first line of the next included file -- a target at the top of an included file was swallowed into the preceding recipe line and dropped from completion. Emit an explicit newline (`echo`) after each `cat`. The extra blank line between files is inert to the target parser. Add a regression test: an include reached from a newline-less root Makefile whose last line is a recipe must still surface its leading target.
|
Thanks @oz-for-oss — addressed the last suggestion (the missing separator between concatenated makefiles). Fix: Regression test: /oz-review |
|
I ran into an unexpected error while working on this. Powered by Oz |
|
@lucieleblanc — this one's been waiting on human review for a while; I think it's ready whenever you have a moment. Where it landed:
(I re-triggered Oz to re-confirm, but it hit an internal error on its end — happy to re-run it if you'd like a fresh automated pass before merging.) Thanks! |
|
Gentle follow-up @lucieleblanc 🙂 — no rush, just flagging in case this slipped off the radar. #275 has been green and ready for a few weeks now: Oz approved-with-nits, that final nit addressed, full suite passing ( |
|
Re-triggering Oz, because the last attempt never actually produced a review. On Jun 19 I pushed Nothing about the change has moved since, and the gap it fixes is still open on Where the four Oz findings landed, for whoever picks this up:
Full suite green: 65 passed, /oz-review |
|
I'm re-reviewing this pull request in response to a review request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR updates the make target completion generator to follow explicit Makefile include directives while bounding traversal to files that canonicalize under the working tree. It also adds end-to-end coverage for nested includes, optional/missing includes, directory includes, path escape attempts, recipe-indented include-looking lines, symlink escapes, and newline-less file concatenation.
Concerns
- No blocking concerns found in the annotated diff.
Verdict
Found: 0 critical, 0 important, 0 suggestions
Approve
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
|
Oz came back clean on the re-trigger: 0 critical, 0 important, 0 suggestions — Approve. So the review state now matches the code, which it hadn't since June. One thing I noticed comparing that run to the earlier ones, and it may explain why this PR has been quiet rather than anyone ignoring it: this run ended with "no human review was requested for this pull request", whereas the Jun 6 runs ended with "requested human review from No action needed from me that I can see, but happy to rebase, re-run anything, or split it up if that would help. |
Summary
Fixes warpdotdev/warp#11705.
makeTab-completion only suggested targets defined directly in the top-level Makefile; targets pulled in throughincludedirectives were invisible (other terminals show them, since GNU make treats included files as one unified ruleset).Root cause
The
makelist_targetsgenerator (command-signatures/src/generators/make.rs) rancat [Mm]akefile, which reads only the root Makefile and never followsinclude/-include/sincludedirectives.Fix
Replace the command with a single self-contained
awkprogram that echoes the root Makefile and recurses into included files, with aseenset guarding against include cycles. Design choices:sh/bash/zshdiffer in word-splitting and this sidesteps that entirely.getline < path, never interpolated into a shell command, so a malicious Makefile can't inject commands.cat [Mm]akefile. Anincludepointing at a real directory (an invalid Makefile thatmakeitself rejects) aborts some awk builds with an i/o error — the fallback keeps top-level targets working instead of returning nothing.The Rust
post_process(target +##description parsing) is unchanged.Tests
Two end-to-end tests run the actual generator command via
sh -cagainst temp projects:test_list_targets_command_follows_includes— targets split across a nested include + a missing optional-include; also asserts a##description defined in an included file survives.test_list_targets_command_survives_directory_include— a directoryincludestill succeeds and falls back to top-level targets.cargo fmt --check,cargo clippy --all-targets --all-features -- -D warnings, andcargo testall pass locally (61 tests).Scope / known limitations
Include paths that rely on globbing (
include dir/*.mk) or make variables (include $(VAR)) are intentionally left unresolved — expanding them safely would require either a shell (injection risk) or evaluating the Makefile. Explicit file paths (the reported case and the common case) and arbitrarily nested includes are handled.