[API Proposal] Value expansions in Microsoft.Extensions.Configuration

[rosebyte]

Today every value in IConfiguration is a literal string. The only way to share a value across keys is to duplicate it in multiple sections, which is error prone, or to read multiple keys at the binding site and pick one in user code, which needs complex custom binding logic. This proposal lets users refer to a common configuration value from other configuration keys, which both displays the intent in configuration (saying "the value here should be whatever My:Beloved:Key is") and lets users drop the error-prone, tedious configuration-file preprocessing they now have to do with Mustache or T4 (both great engines themselves otherwise).

Common scenarios that require duplication or call-site logic today:

• A connection string assembled from two different sources (typically a username and cloud or region information).
• A value that lives under one canonical key but needs to surface under several legacy keys for backward compat.
• A "primary, fall back to secondary" pattern (Azure region overrides, feature-flag tiers, environment overrides).

All three collapse to "this key's value comes from another key", a cross-cutting concern the configuration system is in the right place to handle once, generically, instead of being re-implemented at every binding site.

API Proposal

For the first iteration we want to keep a minimal API profile we can evolve later as the community starts using the feature and we see what direction would bring the highest value. The public surface of this minimal viable stage has just one new public method:

 namespace Microsoft.Extensions.Configuration;
 
 public static class ConfigurationBuilderExtensions
 {
     /// <summary>
     /// Allows expansions for the configuration produced by this builder.
     /// </summary>
     /// <remarks>
     /// When allowed, values of the form <c>ref(key1, key2, ...)</c> are resolved to the first
     /// listed key whose own value is non-null. All failure modes return the raw value verbatim.
     /// </remarks>
     public static IConfigurationBuilder AllowExpansions(this IConfigurationBuilder builder, bool allow = true);
 }

The opt-in mechanism is builder-scoped so we always allow or disallow expansions in all configuration sources within the builder. The idea is that even if we later add more fine-grained options to scan or skip particular configuration providers, this method will still have its value as the kill-switch for the whole feature.

For the value syntax, the first iteration offers a deliberately limited form: the value of one key can refer to one or more other keys.

ref( <key> [, <key>]* )

• Whole-string match: the value must start with exactly ref( and end with ). Otherwise it's a literal.
• Keys are comma-separated; whitespace around each key is trimmed.
• Empty keys (after trim), nested parens in the body, or any other deviation → literal.
• Resolution recurses: a referenced value that is itself a ref(...) is also resolved.
• Cycles → throws an exception (users wouldn't benefit from verbatim anyway as the problem is structural here).
• Escape: a value beginning with \ref( returns the rest verbatim with the leading \ stripped. Designed to be forward-compatible with later prefixes (\format(, …). Terminating characters in keys (,, , )) can be escaped by \ too.
• Relative keys: a referenced key may start with one or more ..: segments, each walking one level up the colon-separated path of the key currently being resolved. Walking past the root → verbatim.

We expect transitive references, so the target key itself can be a reference and the engine will follow the chain until it reaches a value or detects a cycle (in which case it leaves the verbatim string in place). A grace window of 32 nested resolutions avoids per-call allocation for the realistic short-chain case; beyond that a HashSet<string> is allocated and any cycle terminates the affected branch verbatim. A misconfigured reference is observable as the literal value at the consumer (e.g., a connection-string library will
fail when handed "ref(Foo)" — which is the right place for the error, with the right context).

ConfigurationManager honors the same flag — toggling AllowExpansions at runtime swaps the engine without rebuilding sources.

Sample

var config = new ConfigurationBuilder()
    .AddInMemoryCollection(new Dictionary<string, string?>
    {
        ["Db:Primary"]   = "Server=primary;Database=app",
        ["Db:Secondary"] = "Server=secondary;Database=app",

        // Fallback chain: use Primary if set, else Secondary.
        ["Db:Active"]    = "ref(Db:Primary, Db:Secondary)",

        // Aliasing: legacy key forwards to canonical one.
        ["ConnectionStrings:Default"] = "ref(Db:Active)",

        // Escape: a literal value that happens to start with "ref(".
        ["Docs:Sample"]  = @"\ref(NotAReference)",
    })
    .AllowExpansions()
    .Build();

config["Db:Active"];                     // "Server=primary;Database=app"
config["ConnectionStrings:Default"];     // "Server=primary;Database=app"
config["Docs:Sample"];                   // "ref(NotAReference)"

If Db:Primary is removed at runtime (provider reload), Db:Active automatically resolves to Db:Secondary on the next read — the engine subscribes to provider reload tokens and invalidates its cache.

Alternatives

There are two ideas we discussed and decided not to implement now.

Always-on (or opt-out only)

The first is to make the mechanism either always on or, at minimum, opt-out only. We prefer opt-in because there are millions of .NET applications that could be broken by collision with any sensible syntax we can offer, and it feels better to think about the
ergonomics of value syntax than about its deliberate obscurity to avoid collisions. We should, however, update the ASP.NET template (and similar templates) to turn this on by default in the host builder — for new services there is no risk of collisions, and
features using expansions will just work out of the box for users, most notably for beginners making their first web server experiment with .NET.

Sigil-prefixed bare key

The second is an alternative reference value syntax with a bare $-prefixed key path, optionally namespaced as $ref::

$path:to:key
$ref:path:to:key

This "sigil-prefixed bare key" syntax has been proposed by the Azure team and is worth taking seriously. It looks lighter at the call site:

$Db:Primary       // sigil form
ref(Db:Primary)   // proposed form

We considered it and rejected it for phase 1 for these reasons:

1. It's a dead-end grammar. $key has no opening or closing delimiter, which means there is nowhere to put additional arguments without breaking everyone. The ref(...) form already has a parenthesized argument list, so ref(key, default), format(...),
   env(...), etc. compose in a purely additive manner.
2. Ambiguity with literal $. Configuration values frequently contain $ (PowerShell snippets, jq expressions, env-var-style placeholders consumed by other systems, currency in localized text). Disambiguating these against references requires either a
   heuristic (fragile) or a doubling rule ($$) that breaks every existing value containing $$. ref(...) only collides with values that literally start with the four characters ref(, an extraordinarily rare prefix in real config.
3. Bare-key syntax forces a key-character whitelist. $Db:Primary has to terminate somewhere. Either the lexer hardcodes a key-character set (which leaks into how providers can name keys), or it uses delimiters, effectively mimicking ref() syntax with
   higher cognitive load. Configuration keys today are opaque strings; the resolution layer should not constrain them.
4. Inline interpolation pressure. Sigil syntax invites "prefix $foo suffix" interpolation, which sounds nice but immediately raises escaping and quoting questions that have no obvious answer for arbitrary string-typed values. The ref(...) form is
   whole-string-only, which is honest about what phase 1 supports and leaves a clean path to add interpolation later (as format(...) or similar) without changing what ref(...) means.

Risks

• Surprise. A user who pastes ref(...)-shaped text into a value, having forgotten AllowExpansions() is on, will see resolution. Mitigated by opt-in default and the \ref( escape — and in most cases the reference won't resolve to anything, leaving the
  exact same value in place.
• Order of evaluation. Keys resolve lazily on read. A value mutated after binding is re-resolved on the next access; binders that capture once are unaffected.
• Performance. The engine adds one cache lookup on the hot path and a parser pass on cache miss. The first 32 nested resolutions cost no tracking allocation; that depth is already deep enough to be a code smell, so paying for a HashSet past it is
  acceptable.

Out of scope

• Defaults inside the expression: ref(key, "literal-fallback").
• Format/transform expressions: format(...), env(...), etc.
• Cross-IConfiguration references.
• Configurable syntax / custom resolvers.

[cincuranet]

Great work @rosebyte. Glad to see this starting to get shape.

Although I don't have a very strong opinion, my mild preference would be to throw in case of walking past root or cycle or non-existing reference. OTOH, if we see strong demand for this behavior, we can add a switch in the future that would turn on throwing.

[rosebyte]

Thank you! Throwing is definitely possible and if we see value to have it there either as the default or as an option, I'm fine with it.

EDIT: in the case of missing target or going beyond root I would actually prefer returning null to be consistent with the current configuration behaviour where we return null on missing keys (which is basically the situation - we know key A should have the same value as key B and key B has no value).

[svick]

Using quotes for escaping now means that this syntax can't be used for defaults in the future (e.g. the example of ref(key, "literal-fallback") from the Out of scope section). I think both escaping and defaults are useful and that their syntaxes should be considered so that they don't clash, even if it won't be implemented in this first version.

[m-nash]

Agree with @cincuranet on throwing for cycle detection. I think it's safe to assume we are in a bad state at that point and can surface the error immediately vs falling back to a literal which will almost never be the right answer and may cause the error to happen much later at runtime of the application.

[rosebyte]

@svick, you're right, the quotes do not feel like the right choice in this context. Instead, we could escape the characters using backslashes (n - 1), which would compose well with the ref(...) part, as we already escape it this way. What do you think? (I've updated the proposal.)

[rosebyte]

@m-nash, I agree that cycles are a kind of error throwing makes sense (I edit the proposal).