WIP: Builtin/opaque dependencies by adamgemmell · Pull Request #16675 · rust-lang/cargo

adamgemmell · 2026-02-25T17:40:41Z

This WIP PR intends to evaluate an approach to rust-lang/rfcs#3875 by adding support for builtin and opaque dependencies and lets the current -Zbuild-std implementation use them. I'm working on this now as it's a fairly wide-reaching change and was worried that some necessary changes might invalidate other work on the build-std RFCs.

A builtin dependency is one which Cargo innately knows how to find, such as one included with the toolchain. The RFC above adds a new builtin package source for standard library dependencies. An opaque dependency isn't yet fully agreed upon, but is a possible future feature that "hides" transitive dependencies from the main resolve in order to reduce rebuilds.

This PR does this by:

injecting a default set of builtin dependencies to every package in dep_cache (which every call to activate in the resolver goes through), which are identified as opaque.
The registry, when queried for a package satisfying one of these dependencies, returns a dummy Summary representing a builtin package. This package has no dependencies, because the dependency injected is opaque. This allows the resolve to succeed as normal without modifying the resolver itself. The intended behaviour, though untested, is to make use of all the patching/replacement logic already in the registry query.
Instead of the current -Zbuild-std unit generation step which attaches UnitDeps to each root of the std resolve, the main unit generation loop replaces the dummy Summarys with roots from the separate std resolve.

This PR mostly works end to end for various packages I've tried, though some (like tokio) fail right now, maybe because it's a workspace. Removing the hardcoded hacks will be needed to check what tests break. -Zbuild-std-features still works as I haven't touched the main std resolve and take Units from that.

What would be great is some feedback on whether this approach seems sustainable or risks breaking too many things. Do you have any thoughts at this stage @ehuss @epage?

davidtwco · 2026-02-26T17:38:40Z

We aren't expecting that this will land prior to those RFCs being approved (unless Cargo want it to), the intent with this is just to get some feedback on the general direction for the implementation, we're happy to adjust as much or as little as required :) @adamgemmell will also be away for a couple of weeks starting next week, so we might not respond to feedback immediately.

epage · 2026-02-26T18:00:05Z

crates/cargo-util-schemas/src/core/source_kind.rs

    /// A directory-based registry.
    Directory,
+    /// Package sources distributed with the rust toolchain
+    Builtin,


https://github.com/rust-lang/cargo/blob/master/crates/cargo-util-schemas/src/core/package_id_spec.rs is at least one other place that would need updating

This will impact the unique identifier for the packages from this source in cargo's json output when compiling, cargo metadata, cargo <cmd> -p, etc

I'll modify there too, and add a note to check the stdout in various use cases. The RFCs often make notes on what the output of various commands will be. Note that builtin doesn't actually appear in Units - they're all Path dependencies by that point.

An interesting point on cargo metadata is that we decided that we have an unresolved question regarding if deps of builtins should be shown on output, which will be a little hard here as they're not attached until unit generation.

epage · 2026-02-26T18:01:56Z

src/cargo/core/source_id.rs

+                    .url
+                    .to_file_path()
+                    .expect("builtin sources should not be remote");
+                Ok(Box::new(PathSource::new(&path, self, gctx)))


Will using a PathSsource directly like this work?

This particularly could have interesting design questions

epage · 2026-02-26T18:10:41Z

src/cargo/core/resolver/dep_cache.rs

+        None
+    } else {
+        Some(builtins.iter())
+    };


I assume this is for implicit builtins. Is there a reason you chose to do this here?

Source replacements happen in dep_cache (see query) so it seemed simpler just to put this here for now. After digging a little more, it might just be the deprecated [replace] section though, in which case I can probably lift this out of the cache to a more appropriate place

epage · 2026-02-26T18:16:02Z

src/cargo/core/dependency.rs

+        }
+    }
+
+    pub fn new_injected_builtin(name: InternedString) -> Dependency {


what do you see as the role of this compared to the other news?

The difference is that opaque is true (making a SourceId was also more complicated in a previous iteration of this patch). public should also be true now that I look closer.

epage · 2026-02-26T18:18:28Z

src/cargo/core/registry.rs

                )
            })?;

+        if dep.is_opaque() {


I'd like to find a way to ask the source for the opaque variant of the summary. The tricky thing will be that we need to work with both variants.

epage · 2026-02-26T18:20:20Z

src/cargo/core/source_id.rs

+        // Injecting builtins earlier (somewhere with access to RustcTargetData) is needed instead of this
+        let home = std::env::var("HOME").expect("HOME is set");
+        let path = format!(
+            "file://{home}/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/"


The URL in source ids is public facing. I think we'll need something more generic and then a new Source

I definitely tried something more generic but found that it had to be valid at various points. Will experiment more with this.

epage · 2026-02-26T18:20:49Z

src/cargo/core/source_id.rs

+        );
+        if name == "compiler_builtins" {
+            Self::from_url(&format!(
+                "builtin+{path}compiler-builtins/compiler-builtins"


why the special case?

It's the only direct dependency that isn't directly in the library folder. We need a better way of discovering packages for sure.

epage · 2026-02-26T18:21:35Z

src/cargo/ops/fix/mod.rs

            has_dev_units,
            crate::core::resolver::features::ForceAllTargets::No,
            dry_run,
+            true,


not really a fan of bool constants being used in parameter lists. Makes it a lot harder to figure what what is going on here

epage · 2026-02-26T18:22:06Z

src/cargo/ops/resolve.rs

    keep_previous: Option<Keep<'_>>,
    specs: &[PackageIdSpec],
    register_patches: bool,
+    inject_builtins: bool,


Why do we need to tunnel this through? Not thrilled with this

This bool or the required logic behind it could probably be packaged into the workspace or it's gctx

Not a great solution and will need reworking, but good enough to unblock some tests for now

adamgemmell

Thank you for leaving feedback. My latest set of changes does a bit of tidying and works towards getting the tests passing, but doesn't address these comments yet or otherwise make progress towards removing hacks.

View changes since this review

adamgemmell · 2026-03-18T17:20:22Z

src/cargo/core/resolver/dep_cache.rs

+        None
+    } else {
+        Some(builtins.iter())
+    };


Source replacements happen in dep_cache (see query) so it seemed simpler just to put this here for now. After digging a little more, it might just be the deprecated [replace] section though, in which case I can probably lift this out of the cache to a more appropriate place

adamgemmell · 2026-03-18T17:24:10Z

src/cargo/core/dependency.rs

+        }
+    }
+
+    pub fn new_injected_builtin(name: InternedString) -> Dependency {


The difference is that opaque is true (making a SourceId was also more complicated in a previous iteration of this patch). public should also be true now that I look closer.

adamgemmell · 2026-03-18T17:32:31Z

src/cargo/core/source_id.rs

+        // Injecting builtins earlier (somewhere with access to RustcTargetData) is needed instead of this
+        let home = std::env::var("HOME").expect("HOME is set");
+        let path = format!(
+            "file://{home}/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/"


I definitely tried something more generic but found that it had to be valid at various points. Will experiment more with this.

adamgemmell · 2026-03-18T17:32:39Z

src/cargo/core/source_id.rs

+        );
+        if name == "compiler_builtins" {
+            Self::from_url(&format!(
+                "builtin+{path}compiler-builtins/compiler-builtins"


It's the only direct dependency that isn't directly in the library folder. We need a better way of discovering packages for sure.

adamgemmell · 2026-03-19T10:40:20Z

src/cargo/ops/resolve.rs

    keep_previous: Option<Keep<'_>>,
    specs: &[PackageIdSpec],
    register_patches: bool,
+    inject_builtins: bool,


This bool or the required logic behind it could probably be packaged into the workspace or it's gctx

epage reviewed Feb 26, 2026

View reviewed changes

biscuitrescue mentioned this pull request Mar 3, 2026

fix: ignore implicit std dependencies in unused-crate-dependencies lint #16677

Merged

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-author Status: The marked PR is awaiting some action (such as code changes) from the PR author. label Mar 6, 2026

adamgemmell added 7 commits March 19, 2026 12:33

Add new builtin SourceKind

7c80afb

Inject builtin deps & Summarys while resolving

f2a5690

Unit generation changes for opaque dependencies

05903b1

Allow overriding src root for build-std tests

7948ac0

Inject test Unit

a890490

Enable injecting deps depending on -Zbuild-std args

be19f77

Not a great solution and will need reworking, but good enough to unblock some tests for now

Move compiler-builtins in mock-std to match the real one

ab7d8b0

adamgemmell force-pushed the dev/adagem01/opaque branch from 16dc1ab to ab7d8b0 Compare March 19, 2026 14:20

adamgemmell commented Mar 19, 2026

View reviewed changes

Conversation

adamgemmell commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidtwco commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

adamgemmell left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

adamgemmell commented Feb 25, 2026 •

edited

Loading

davidtwco commented Feb 26, 2026 •

edited

Loading

adamgemmell left a comment •

edited by rustbot

Loading