This guide covers how to write tests for Gradle plugins using the gradle-plugin-testing framework.
- Quick Start
- Core Concepts
- File Operations
- Maven Repository Testing
- Git Repository Testing
- Executing Gradle Builds
- Assertions
- Error Prone Checks
import com.palantir.gradle.testing.junit.GradlePluginTests;
import com.palantir.gradle.testing.execution.GradleInvoker;
import com.palantir.gradle.testing.project.RootProject;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static com.palantir.gradle.testing.assertion.GradlePluginTestAssertions.assertThat;
@GradlePluginTests
class MyPluginTest {
@BeforeEach
void setup(RootProject project) {
project.buildGradle().plugins().add("my-plugin");
project.buildGradle().append("""
myPlugin {
enabled = true
}
""");
}
@Test
void plugin_applies_successfully(GradleInvoker gradle, RootProject project) {
assertThat(gradle.withArgs("tasks").buildsSuccessfully())
.output()
.contains("myTask");
}
}Add the gradle-idea-language-injector plugin to your root build.gradle:
plugins {
id 'com.palantir.idea-language-injector' version '<version>'
}This plugin automatically generates IntelliJ IDEA language injection configurations, enabling syntax highlighting for string literals inside IntelliJ.
Mark test classes with @GradlePluginTests to enable Gradle plugin testing.
Each test gets an isolated project directory under build/gradle-plugin-testing for debugging purposes.
Tests can request these parameters in any order:
GradleInvoker- Executes Gradle buildsRootProject- Gradle root project that will always be named "root". To use a different project name, callrootProject.settingsGradle().rootProjectName("custom-name")SubProject- Gradle subproject for the root project. The parameter name will be used as the project name exactly. For example,SubProject apiServicecreates a subproject namedapiServiceIncludedBuild- An included build (composite build) for the root project. The parameter name will be used as the build name exactly. For example,IncludedBuild myLibcreates an included build namedmyLibwith its ownsettings.gradleandbuild.gradleMavenRepo- Maven repository for publishing test artifacts. See Maven Repository TestingGit- Git repository initialised against the root project directory. See Git Repository Testing
These parameters can be used in the constructor for a test class or in @Test, @BeforeEach, @AfterEach, @BeforeAll, or @AfterAll methods.
Example with subprojects:
@Test
void multi_project_build(SubProject api, SubProject server) {
// Projects "api" and "server" are automatically created and included as sub-projects to the "root" project.
api.buildGradle().plugins().add("java-library");
server.buildGradle().plugins().add("application");
server.buildGradle().append("""
dependencies {
implementation project(':api')
}
""");
}Creating subprojects programmatically:
@Test
void create_nested_subprojects_manually(SubProject api) {
// Nested subprojects
SubProject nestedProject = api.subproject("nested");
}Example with included builds (composite builds):
@Test
void composite_build(GradleInvoker gradle, RootProject rootProject, IncludedBuild sharedLib) {
// "sharedLib" is automatically created as an included build with its own settings.gradle
sharedLib.buildGradle().plugins().add("java-library");
rootProject.buildGradle().plugins().add("java");
rootProject.buildGradle().append("""
dependencies {
implementation 'com.example:sharedLib'
}
""");
gradle.withArgs("tasks").buildsSuccessfully();
}Creating included builds programmatically:
@Test
void create_nested_included_build(IncludedBuild outerLib) {
outerLib.buildGradle().plugins().add("java-library");
// Included builds can contain their own included builds
IncludedBuild innerLib = outerLib.includedBuild("inner-lib");
innerLib.buildGradle().plugins().add("java-library");
// Included builds can also have their own subprojects
SubProject sub = outerLib.subproject("sub-module");
sub.buildGradle().plugins().add("java-library");
}The framework automatically runs each test against multiple Gradle versions.
Configure which Gradle versions to test against using the gradleTestUtils extension:
gradleTestUtils {
gradleVersions = ['8.10', '8.14.3']
}If not specified, tests run against the default version (currently 8.14.3).
See Resolution of Gradle versions to test against for more details.
Use @AdditionallyRunWithGradle to add extra Gradle versions for a specific test class or individual test methods.
Note: This annotation is intended for exceptional cases where a specific test needs additional versions. For configuring Gradle versions across your entire test suite, prefer setting versions in the
gradle/gradle-test-versions.ymlfile.
@GradlePluginTests
@AdditionallyRunWithGradle({"7.6.5", "8.0"})
class CompatibilityTest {
@Test
void works_on_older_gradle_versions(GradleInvoker gradle, RootProject project) {
// This test runs against the globally configured versions PLUS 7.6.5 and 8.0
}
@Test
@AdditionallyRunWithGradle("8.5")
void test_specific_version(GradleInvoker gradle, RootProject project) {
// This test runs against globally configured versions PLUS 7.6.5, 8.0 (from class), and 8.5 (from method)
}
}The versions from @AdditionallyRunWithGradle are merged with the globally configured versions. When applied to both a class and a method, all versions are combined. Duplicate versions are automatically deduplicated.
Use @ParameterizedByGradleVersion to inject different String values based on the Gradle version under test.
Conditions in when are evaluated in order, and the first matching condition is used. If no condition matches, otherwiseString provides the fallback value. Conditions must be ordered by ascending version (lowest lessThan value first).
@Test
@ParameterizedByGradleVersion(
when = @WhenVersion(lessThan = "8.0", stringValue = "legacyStyle"),
otherwiseString = "newStyle")
void test(GradleInvoker gradle, RootProject project, @InjectByGradleVersion String configOption) {
project.buildGradle().append("myPlugin.style = '%s'", configOption);
}With multiple conditions:
@Test
@ParameterizedByGradleVersion(
when = {
@WhenVersion(lessThan = "8.0", stringValue = "legacy"),
@WhenVersion(lessThan = "9.0", stringValue = "8.x")
},
otherwiseString = "modern")
void test(GradleInvoker gradle, RootProject project, @InjectByGradleVersion String generation) {
project.buildGradle().append("myPlugin.generation = '%s'", generation);
}Conditions are evaluated in order. For Gradle 7.6, generation is "legacy" (matches first condition). For Gradle 8.5, generation is "8.x" (doesn't match first, matches second). For Gradle 9.0+, generation is "modern" (no condition matches, uses fallback).
With multiple parameters:
@Test
@ParameterizedByGradleVersion(
name = "style",
when = @WhenVersion(lessThan = "8.0", stringValue = "old"),
otherwiseString = "new")
@ParameterizedByGradleVersion(
name = "format",
when = @WhenVersion(lessThan = "9.0", stringValue = "classic"),
otherwiseString = "modern")
void test(GradleInvoker gradle, @InjectByGradleVersion String style, @InjectByGradleVersion String format) {
project.buildGradle().append("myPlugin { style = '%s'; format = '%s' }", style, format);
}In @BeforeEach:
private String configStyle;
@BeforeEach
@ParameterizedByGradleVersion(
when = @WhenVersion(lessThan = "8.0", stringValue = "legacy"),
otherwiseString = "modern")
void setup(RootProject project, @InjectByGradleVersion String style) {
this.configStyle = style;
project.buildGradle().plugins().add("my-plugin");
}
@Test
void test_uses_correct_style(GradleInvoker gradle, RootProject project) {
project.buildGradle().append("myPlugin.style = '%s'", configStyle);
gradle.withArgs("build").buildsSuccessfully();
}Requirements:
- The receiving parameter must be annotated with
@InjectByGradleVersion - Must contain an
otherwiseStringto catch the general case - Conditions must be ordered by ascending
lessThanversion (lowest first) - When using multiple
@ParameterizedByGradleVersionannotations, each must have anamematching its parameter
All project files inherit from ProjectFile<T> with these methods:
overwrite(String text)- Replace entire file contentappend(String text)- Add text to end of fileprepend(String text)- Add text to start of fileappendLine(String line)- Add line with newlineprependLine(String line)- Add line at startedit(FileEditor editor)- Transform file content with callbacktext()- Read file contentcreateEmpty()- Create an empty fileassertThat()- AssertJ path assertions
All methods support String.format() syntax for dynamic values. The varargs overload provides better IDE support with syntax highlighting and is enforced by the GradleTestStringFormatting Error Prone check.
String version = "1.0.0";
project.buildGradle().append("version = '%s'", version);Reading file contents:
String content = project.buildGradle().text();Creating empty files:
project.file("versions.lock").createEmpty();Configure build.gradle for your projects:
@Test
void configure_build_file(RootProject project) {
project.buildGradle().append("""
group = 'com.example'
version = '1.0.0'
repositories {
mavenCentral()
}
dependencies {
implementation 'com.google.guava:guava:32.1.0-jre'
}
""");
}
@Test
void configure_subproject_build(SubProject api) {
api.buildGradle().append("""
dependencies {
api 'org.slf4j:slf4j-api:2.0.9'
}
""");
}Configure settings.gradle, the root project name defaults to root and can only be changed using the rootProjectName method on the settings file.
@Test
void configure_settings(RootProject project) {
project.settingsGradle()
.rootProjectName("my-service")
.include("api")
.include("impl");
}Use the structured plugins() API to add plugins:
Important: Always use the plugins() API instead of manually writing plugin blocks in append() or overwrite() calls. The plugins() API ensures correct positioning after buildscript {} blocks and prevents duplicate plugin entries.
@Test
void add_plugins(RootProject project) {
// Add plugins individually
project.buildGradle()
.plugins()
.add("java")
.add("application");
// Add without applying
project.buildGradle()
.plugins()
.addWithoutApply("com.palantir.baseline");
}When adding multiple plugins, chain the .add() calls rather than calling .plugins() multiple times:
// Preferred - chain the calls
rootProject.buildGradle().plugins()
.add("com.palantir.failure-reports")
.add("java");
// Avoid - unnecessary repetition
rootProject.buildGradle().plugins().add("com.palantir.failure-reports");
rootProject.buildGradle().plugins().add("java");The gradlePluginForTesting configuration is automatically created by the gradle-plugin-testing plugin. Use it to make external Gradle plugins available in your test's Gradle runtime:
dependencies {
gradlePluginForTesting 'com.palantir.baseline:gradle-baseline-java'
gradlePluginForTesting 'com.palantir.sls-packaging:gradle-sls-packaging'
}The plugin is then available in your tests using the standard .plugins().add() API:
@Test
void test_with_external_plugin(GradleInvoker gradle, RootProject project) {
project.buildGradle().plugins().add("com.palantir.sls-asset-distribution");
gradle.withArgs("tasks").buildsSuccessfully();
}Note: If you forget to add a plugin to gradlePluginForTesting, your test will fail with an error like:
Plugin [id: 'com.palantir.sls-asset-distribution'] was not found in any of the following sources:
- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Gradle TestKit (classpath: ...)
- Plugin Repositories (plugin dependency must include a version number for this source)
When using com.palantir.consistent-versions, dependencies in gradlePluginForTesting are automatically included in the [Test dependencies] section of versions.lock.
Create custom Gradle files beyond build.gradle and settings.gradle:
import com.palantir.gradle.testing.files.gradle.GradleFile;
@Test
void create_custom_gradle_files(RootProject project) {
// Create custom Gradle files
GradleFile gradleFile = project.gradleFile("dependencies.gradle").overwrite("""
dependencies {
implementation 'com.google.guava:guava:32.1.0-jre'
}
""");
// Reference from build.gradle
project.buildGradle().append("apply from: 'dependencies.gradle'");
}Or use a helper method to setup a standard build file used across multiple tests:
Note: only use this pattern if you cannot setup the build gradle file in a
@BeforeEach/@BeforeAll
import com.palantir.gradle.testing.files.gradle.GradleFile;
GradleFile standardBuildFile(RootProject project) {
rootProject
.buildGradle()
.plugins()
.add("com.palantir.jdks.latest")
.add("java-library");
// Return the GradleFile instance for further configuration in tests
return rootProject.buildGradle().append("""
repositories {
mavenCentral()
}
""");
}
@Test
void check_jdk_17(RootProject project) {
standardBuildFile(project).append("""
javaVersions {
libraryTarget = 17
}
""");
//...
}Write and manipulate Java source. The writeClass() method from JavaSrcDir (accessed via project.mainSourceSet().java() or project.sourceSet("name").java()) automatically parses the Java source code to extract the package and class name, then creates the file at the correct path.
@Test
void create_java_class(RootProject project) {
// Write a complete class
project.mainSourceSet().java().writeClass("""
package com.example;
public class Calculator {
public int add(int a, int b) {
return a + b;
}
}
""");
// Access by class name
project.mainSourceSet()
.java()
.fileByClassName("com.example.Calculator")
.edit(text -> text.replace("add", "sum"));
// Write to test sources
project.testSourceSet().java().writeClass("""
package com.example;
import org.junit.jupiter.api.Test;
class CalculatorTest {
@Test
void adds_numbers() {}
}
""");
// Access custom source sets
project.sourceSet("integrationTest")
.java()
.writeClass("""
package com.example;
class IntegrationTest {}
""");
}To add key-value pairs to properties files such as gradle.properties or custom files like versions.props, use the setProperty(key, value) method:
@Test
void set_gradle_properties(RootProject project) {
// Use `gradlePropertiesFile()` for 'gradle.properties' file
project.gradlePropertiesFile()
.setProperty("org.gradle.parallel", "true")
.setProperty("org.gradle.caching", "true");
}
@Test
void create_versions_props(RootProject project) {
// Use propertiesFile() for arbitrary properties files
project.propertiesFile("versions.props")
.setProperty("com.google.guava:*", "32.1.0-jre")
.setProperty("org.slf4j:*", "2.0.9");
}Create and manipulate YAML files:
@Test
void create_yaml_files(RootProject project) {
project
.yamlFile("application.yml")
.overwrite("""
server:
port: 8080
database:
url: jdbc:postgresql://localhost:5432/mydb
""");
}Create any file in the project:
@Test
void create_arbitrary_files(RootProject project) {
project.file("README.md").overwrite("# My Project");
project.file(".gitignore").append("*.log\n");
project.file("config.json").overwrite("""
{
"version": "1.0.0"
}
""");
}Create and work with directories:
@Test
void create_directories(RootProject project) {
// Create arbitrary directory structures
project.directory("scripts")
.file("deploy.sh")
.overwrite("#!/bin/bash\necho 'Deploying...'");
// Create nested directories explicitly
project.directory("docs/api/v1/README.md")
.createDirectories();
}Access Gradle's build output directory:
@Test
void access_build_output(GradleInvoker gradle, RootProject project) {
project.buildGradle().plugins().add("java");
project.mainSourceSet().java().writeClass("""
package com.example;
public class Main {}
""");
gradle.withArgs("build").buildsSuccessfully();
// Access compiled classes
project.buildDir()
.file("classes/java/main/com/example/Main.class")
.assertThat()
.exists();
// Access JAR files
project.buildDir()
.file("libs/root.jar")
.assertThat()
.exists();
}Note: buildDir() returns a Directory, so you can use all directory methods like file(), directory(), etc.
Use MavenRepo to publish test artifacts to a local Maven repository for testing plugins that resolve external dependencies.
@Test
void publish_and_resolve_artifacts(MavenRepo repo, RootProject root, GradleInvoker gradle) {
// Publish simple artifacts
repo.publish(MavenArtifact.of("com.example:library:1.0.0"));
// Publish artifacts with dependencies
repo.publish(
MavenArtifact.of("com.example:base:1.0.0"),
MavenArtifact.builder()
.coordinate("com.example:advanced:2.0.0")
.addDependency("com.example:base:1.0.0")
.addDependency("com.google.guava:guava:32.1.0-jre")
.build()
);
root.buildGradle().plugins().add("java-library");
// Configure build to use the repository
root.buildGradle().withMavenRepo(repo);
root.buildGradle().append("""
dependencies {
implementation 'com.example:advanced:2.0.0'
}
""");
gradle.withArgs("build").buildsSuccessfully();
}The MavenRepo instance is shared across all test methods in a class. Artifacts published in lifecycle methods (eg. @BeforeEach) remain available for subsequent tests.
Request a Git parameter to set up a git repository in a test project for plugins that read git state (versioning, release, changelog, etc.). It shells out to the git binary on PATH.
The repository is initialised against the root project directory on first injection per test method.
import com.palantir.gradle.testing.git.Git;
@Test
void resolves_version_from_tag(Git git, GradleInvoker gradle) {
git.commit("initial");
git.tag("1.0.0");
assertThat(gradle.withArgs("printVersion").buildsSuccessfully())
.output().contains("1.0.0");
}Auto-initialisation runs git init, disables gpg signing (commit.gpgsign, tag.gpgsign, tag.forcesignannotated), and sets a fixed user.email / user.name, so tests are hermetic regardless of the host's git config.
Common helpers:
git.commit(message)— creates an empty commitgit.commit(message, env)— creates an empty commit with environment variables (e.g.GIT_AUTHOR_DATE)git.tag(name)— creates a lightweight tag atHEAD
For other git operations:
git.run("checkout", "-b", "feature");
git.run(Map.of("GIT_AUTHOR_DATE", "2020-01-02T03:04:05+00:00"), "commit", "--allow-empty", "-m", "dated");Use GradleInvoker to run builds:
import com.palantir.gradle.testing.execution.InvocationResult;
@Test
void successful_build(GradleInvoker gradle, RootProject project) {
project.buildGradle().plugins().add("java");
// Run and expect success
InvocationResult result = gradle.withArgs("build").buildsSuccessfully();
// Run and expect failure
InvocationResult failure = gradle.withArgs("brokenTask").buildsWithFailure();
}For more complex logic (eg. adding both arguments and testing environment variables), use gradle.with(Options):
import com.palantir.gradle.testing.execution.Options;
@Test
void build_with_options(GradleInvoker gradle, RootProject project) {
project.buildGradle().plugins().add("java");
gradle.with(Options.builder()
.addArgs("build", "--info")
.putTestingEnvironmentVariables("FOO", "test_value")
.build())
.buildsSuccessfully();
}When configuration cache is enabled (getConfigurationCacheEnabled() == true), each GradleInvoker invocation will automatically run twice:
- The first run uses
--configuration-cacheand verifies that the configuration cache is properly stored. - The second run uses
--configuration-cache --dry-runand verifies that the configuration cache entry is successfully reused.
If configuration cache issues are detected, the build will fail with an UnexpectedConfigurationCacheFailure.
For tests or test classes that are incompatible with configuration cache, use the @DisabledConfigurationCache annotation:
// Disable configuration cache for a specific test method
@Test
@DisabledConfigurationCache("task abc is incompatible with configuration cache")
void incompatible_configuration_cache_build(GradleInvoker gradle, RootProject project) {Or
// Disable for an entire test class
@GradlePluginTests
@DisabledConfigurationCache("tasks abc, xyz are incompatible with configuration cache")
class PluginIncompatibleWithConfigCache {Use gradle-utils:environment-variables which provides a testing-friendly way to access environment variables via Gradle properties.
In your plugin, use EnvironmentVariables to read environment variables:
// In your plugin code
@Nested
abstract EnvironmentVariables getEnvironmentVariables();
public void someMethod() {
String value = getEnvironmentVariables().envVarOrFromTestingProperty("FOO").get();
}In your tests, pass extra testing environment variables via the Options builder. These will be automatically configured for use with EnvironmentVariables:
@Test
void plugin_reads_environment_variable(GradleInvoker gradle, RootProject project) {
project.buildGradle().plugins().add("my-plugin");
gradle.with(Options.builder()
.addArgs("myTask")
.putTestingEnvironmentVariables("FOO", "TEST_VALUE")
.build())
.buildsSuccessfully();
}By default, tests download Gradle distributions from services.gradle.org. To use a custom distribution server, set gradleDistributionBaseUrl on the extension:
gradleTestUtils {
gradleDistributionBaseUrl = 'https://example.com/gradle-distributions'
}When set, tests will download Gradle from {baseUrl}/gradle-{version}-bin.zip instead of the default server.
Import the assertion entry point:
import static com.palantir.gradle.testing.assertion.GradlePluginTestAssertions.assertThat;@Test
void task_assertions(GradleInvoker gradle, RootProject project) {
project.buildGradle().plugins().add("java");
InvocationResult result = gradle.withArgs("build").buildsSuccessfully();
// Check task succeeded
assertThat(result).task(":compileJava").succeeded();
// Check task was up-to-date
InvocationResult secondRun = gradle.withArgs("build").buildsSuccessfully();
assertThat(secondRun).task(":compileJava").upToDate();
// Check task failed
InvocationResult failure = gradle.withArgs("failingTask").buildsWithFailure();
assertThat(failure).task(":failingTask").failed();
// Check task was skipped
assertThat(result).task(":skippedTask").skipped();
// Check task had no source files
assertThat(result).task(":compileTestJava").noSource();
// Check task result was from build cache
assertThat(result).task(":compileJava").fromCache();
// Check task was not executed
assertThat(result).task(":nonExistentTask").notOnTaskGraph();
// Check specific outcome via outcome()
assertThat(result).task(":compileJava").outcome().isEqualTo(TaskOutcome.FROM_CACHE);
}@Test
void output_assertions(GradleInvoker gradle, RootProject project) {
project.buildGradle().append("println 'Configuration phase'");
InvocationResult result = gradle.withArgs("tasks").buildsSuccessfully();
// Check output contains text
assertThat(result).output().contains("Configuration phase");
// Use full AssertJ string assertions
assertThat(result).output()
.containsIgnoringCase("BUILD SUCCESSFUL")
.doesNotContain("deprecated");
}@Test
void file_assertions(GradleInvoker gradle, RootProject project) {
project.buildGradle().plugins().add("java");
project.mainSourceSet().java().writeClass("""
package com.example;
public class Main {}
""");
gradle.withArgs("build").buildsSuccessfully();
// Assert files exist
project.buildDir().file("classes/java/main/com/example/Main.class")
.assertThat().exists();
// Assert exact file contents
project.buildGradle().assertThat().hasContent("""
plugins {
id 'java'
}
""");
// Use AssertJ string assertions on file content
project.buildGradle().assertThat().content()
.contains("plugins")
.doesNotContain("application");
}String Comparisons with Whitespace: Avoid isEqualToIgnoringWhitespace() for trimming - it normalizes ALL whitespace (treating consecutive spaces as single spaces). To match Groovy's text.trim() == expected behavior, use:
assertThat(file.text().trim()).isEqualTo(expected);Assertion Descriptions: Use .as() to provide context for assertions instead of comments:
// Good - description appears in test failure messages
externalDepsFile.assertThat().as("we generate the correct config").exists();
// Avoid - comment doesn't appear in failure messages
// we generate the correct config
externalDepsFile.assertThat().exists();When migrating Spock tests, convert then: block labels to .as() calls, but keep when: block labels as comments since they describe actions, not assertions.
The gradle-plugin-testing plugin ships with custom Error Prone checks that are automatically enabled when the net.ltgt.errorprone plugin is applied to your project.
These checks serve two purposes:
-
Enforcing Best Practices - The checks guide developers toward using the framework's structured APIs correctly, preventing common mistakes.
-
Enabling Automated Migrations - Many of the checks are patchable, meaning Error Prone can automatically fix violations. This allows the framework to evolve over time while automatically migrating existing test code to new APIs or patterns. When upgrading the testing framework, your tests can be automatically updated without manual intervention.
To enable these checks, apply the Error Prone plugin to your project:
plugins {
id 'net.ltgt.errorprone' version '<version>'
}The framework's Error Prone checks will be automatically registered and applied to your test code, catching issues at compile time and offering automated fixes where possible.