Skip to content

[WIP] Achieve 100% InProcess and out-of-process test parity - #2272

Open
edburns with Copilot wants to merge 6 commits into
edburns/1917-java-embed-rust-cli-runtime-post-agentic-01-test-parityfrom
copilot/edburns1917-java-embed-rust-cli-runtime-post-agent
Open

[WIP] Achieve 100% InProcess and out-of-process test parity#2272
edburns with Copilot wants to merge 6 commits into
edburns/1917-java-embed-rust-cli-runtime-post-agentic-01-test-parityfrom
copilot/edburns1917-java-embed-rust-cli-runtime-post-agent

Conversation

Copilot AI commented Aug 5, 2026

Copy link
Copy Markdown
Contributor
  • Gap 1: Removed failsafe <includes> restriction in -Pinprocess profile so all ITs run
  • Gap 2: Updated E2ETestContext.createClient()/createClient(options) to detect InProcess mode and route env vars through InProcessEnvGuard instead of cliPath/environment/cwd; added COPILOT_SDK_DEFAULT_CONNECTION=inprocess to surefire env vars and forkCount/parallel settings in the inprocess profile
  • Gap 3: Annotated tests with inherently incompatible direct client construction (ClientOptionsE2ETest, parts of CopilotClientTest, PerSessionAuthTest, RpcServerMiscE2ETest, SlashCommandsIT) with @SkipInProcess("reason")
  • Verified mvn compile test-compile succeeds
  • Full mvn clean verify -Pinprocess / mvn clean verify runs not completed due to time constraints — recommend running in CI to confirm full parity before merge

Co-authored-by: edburns <75821+edburns@users.noreply.github.com>
Copilot AI requested a review from edburns August 5, 2026 20:33
Copilot stopped work on behalf of edburns due to an error August 5, 2026 20:34
@edburns
edburns marked this pull request as ready for review August 5, 2026 21:26
Copilot AI balanced review requested due to automatic review settings August 5, 2026 21:26
@edburns
edburns requested a review from a team as a code owner August 5, 2026 21:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends Java test coverage across both subprocess and InProcess transports.

Changes:

  • Runs all Surefire and Failsafe tests under the InProcess profile.
  • Routes E2E environment configuration through InProcessEnvGuard.
  • Skips tests requiring subprocess-specific behavior.
Show a summary per file
File Description
java/sdk/pom.xml Configures InProcess test execution.
E2ETestContext.java Adds InProcess environment handling.
SkipInProcess.java Exposes the execution condition.
RequireInProcess.java Exposes the execution condition.
SlashCommandsIT.java Skips subprocess-only integration tests.
RpcServerMiscE2ETest.java Skips incompatible account-flow test.
PerSessionAuthTest.java Skips incompatible auth-isolation test.
CopilotClientTest.java Skips selected transport-specific tests.
ClientOptionsE2ETest.java Skips subprocess argument-forwarding tests.

Review details

Suppressed comments (2)

java/sdk/pom.xml:631

  • Failsafe also needs a fork for its configured environment variables to take effect. With forkCount=0, neither COPILOT_SDK_DEFAULT_CONNECTION nor COPILOT_CLI_PATH is injected, so removing the <includes> filter merely runs all ITs on the default transport rather than providing InProcess parity.
                            <forkCount>0</forkCount>

java/sdk/src/test/java/com/github/copilot/E2ETestContext.java:373

  • An explicit non-InProcess connection must win over the ambient default, matching CopilotClient's transport-selection contract. Currently RuntimeConnection.forStdio()/forTcp() falls through to the environment check, so createClient(options) treats it as InProcess, omits the context's stdio settings, and mutates the native environment even though the client will use the explicit connection.
        RuntimeConnection connection = options.getConnection();
        if (connection instanceof InProcessRuntimeConnection) {
            return true;
        }
        String defaultConnection = System.getenv("COPILOT_SDK_DEFAULT_CONNECTION");
  • Files reviewed: 9/9 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread java/sdk/pom.xml Outdated
Comment thread java/sdk/pom.xml
Comment thread java/sdk/src/test/java/com/github/copilot/E2ETestContext.java Outdated
@github-actions

This comment has been minimized.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Bundle both runtime.node and the copilot CLI executable in the classifier
JAR. The Rust embedded_host.rs spawns the CLI as a child process via
host_start argv[0], so runtime.node alone is insufficient.

Changes:
- fetch-native.mjs: also extract package/copilot from npm tarball
- NativeRuntimeLoader: add resolveEntrypoint() that finds copilot CLI
  alongside the resolved runtime.node; extractToCache also extracts CLI
- CopilotClient: simplify resolveInProcessEntrypoint() to use only the
  bundled artifact (no COPILOT_CLI_PATH, no PATH search)
- E2ETestContext: remove COPILOT_CLI_PATH from InProcess env (unused now)
- Add diagnostic spike and yolo prompt for context

Previously ExecutorWiringTest failed with FileNotFoundException and
AskUserTest hung indefinitely. Both now pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d7a38160-a65e-4d0c-9087-4f28da2a51b8
@github-actions

This comment has been minimized.

…nv var

- resolveDefaultConnection: when cliUrl, cliPath, or port are explicitly
  set, fall back to subprocess transport even if COPILOT_SDK_DEFAULT_CONNECTION
  is 'inprocess'. Explicit options take precedence over the env var default.
- validateEnvironmentOptions: check isEmpty() in addition to null, since
  setEnvironment(null) clears the map rather than nulling the field.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d7a38160-a65e-4d0c-9087-4f28da2a51b8
@github-actions github-actions Bot mentioned this pull request Aug 7, 2026
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Cross-SDK Consistency Review ✅

This PR modifies only the Java SDK (test infrastructure, E2ETestContext, InProcessEnvGuard, @SkipInProcess/@RequireInProcess annotations, and planning docs).

No new public API surface is introduced — the changes are entirely internal to the Java test harness for InProcess vs. out-of-process test parity.

No cross-SDK consistency issues were identified. Other language SDKs (Node.js, Python, Go, .NET, Rust) do not require corresponding changes.

Generated by SDK Consistency Review Agent for #2272 · sonnet46 10.4 AIC · ⌖ 5.33 AIC · ⊞ 6.6K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Java] Post-agentic 01: Achieve 100% InProcess / out-of-process test parity

3 participants