Skip to content

Conversation

@brianduff
Copy link

Problem

rust_test targets don't support Bazel's native test sharding. When shard_count is set on a rust_test:

  • Without --incompatible_check_sharding_support: all tests run N times (once per shard)
  • With --incompatible_check_sharding_support: the test fails because Rust's libtest harness doesn't read TEST_SHARD_INDEX/TEST_TOTAL_SHARDS environment variables

Solution

This PR adds opt-in sharding support via a new experimental_enable_sharding attribute. When enabled, tests are executed via a wrapper script that:

  1. Detects sharding via TEST_TOTAL_SHARDS environment variable
  2. Touches TEST_SHARD_STATUS_FILE to advertise sharding support to Bazel
  3. Enumerates tests using libtest's --list --format terse flag
  4. Partitions tests by index % TEST_TOTAL_SHARDS == TEST_SHARD_INDEX
  5. Runs only the tests assigned to the current shard using --exact

Usage

rust_test(
    name = "my_test",
    srcs = ["my_test.rs"],
    shard_count = 4,
    experimental_enable_sharding = True,
)

@brianduff brianduff marked this pull request as draft December 12, 2025 21:46
Copy link
Collaborator

@illicitonion illicitonion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This generally looks good to me (though I don't have enough batch skills to really verify the .bat file). One thought on provider organisation :)


# Get the test binary from CrateInfo - it's the output of the compiled test
crate_info_provider = None
for p in providers:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do something like edit rustc_compile_action to return a dict of provider types to providers, and then do a key lookup, rather than sniffing an ordered list.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

much better - done.

@brianduff brianduff force-pushed the push-tusnxzxumrtz branch 2 times, most recently from f6ec6ee to a7f30a5 Compare December 16, 2025 17:47
@brianduff brianduff force-pushed the push-tusnxzxumrtz branch 8 times, most recently from 47f7781 to c23d0dc Compare December 16, 2025 18:56
@brianduff brianduff marked this pull request as ready for review December 16, 2025 19:09
@brianduff
Copy link
Author

Remaining test breakages appear to be pre-existing issues unrelated to this change.

The windows .bat file was a pain to test, but was able to get it to work eventually. I also have a branch where I use a small rust util to launch the tests - lemme know if we should explore that either here or as a followup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants