Skip to content

Commit 7f696f7

Browse files
jsuerethjsuereth
committed
merge conflcits.
1 parent de53cb0 commit 7f696f7

File tree

3 files changed

+209
-346
lines changed

3 files changed

+209
-346
lines changed

‎CHANGELOG.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@ release.
3535

3636
### OTEPs
3737

38+
- Support multiple Resources within an SDK.
39+
([#4665](https://github.com/open-telemetry/opentelemetry-specification/pull/4665))
40+
3841
## v1.52.0 (2025-12-12)
3942

4043
### Context

‎oteps/entities/4665-multiple-resource-in-sdk.md‎

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
# Multiple Resource in SDK
2+
3+
Allow multiple `Resource` instances per SDK.
4+
5+
## Motivation
6+
7+
OpenTelemetry needs to address two fundamental problems:
8+
9+
- Reporting data against "mutable" or "changing" entities, where currently an
10+
SDK is allowed a single `Resource`, whose lifetime must match the lifetime of
11+
the SDK itself.
12+
- Providing true multi-tenant capabilities, where, e.g. metrics about one tenant
13+
will be implicitly separated from metrics about another tenant.
14+
15+
The first problem is well outlined in (not accepted) [OTEP 4316](https://github.com/open-telemetry/opentelemetry-specification/pull/4316).
16+
Fundamentally, while we need an immutable identity, the reality is that `Resource`
17+
in today's OpenTelemetry usage is not strong enough to support key use cases. For example,
18+
OpenTelemetry JS, in the node.js environment, cannot guarantee that all identifying
19+
attributes for Resource are discovered prior to SDK startup, leading to an "eventual identity" situation
20+
that must be addressed in the Specification. Additionally, our Client/Browser SIG has been
21+
trying to model the notion of "User Session" which has a much shorter lifespan than the SDK itself, so
22+
requiring a single identity that is both immutable and matches the SDK lifetime prevents any good mechanism of reporting user session.
23+
24+
However, [OTEP 4316](https://github.com/open-telemetry/opentelemetry-specification/pull/4316) explores
25+
relaxing the immutability restriction vs. providing a new mechanism. During prototyping,
26+
initially this seemed to be easily accomplished, but ran into major complications both in interactions
27+
with OpAmp (where a stable identity for the SDK is desired), and in designing a Metrics SDK, where
28+
changes in Resource mean a dynamic and divergent storage strategy, without a priori knowledge of whether these resource mutations are
29+
relevant to the metric or not.
30+
31+
Additionally, today when reporting data from one "process" about multiple resources, the only recourse available is to instantiate
32+
multiple SDKs and define different resources in each SDK. This absolute separation can be highly problematic with the notion of
33+
"built-in" instrumentation, where libraries (e.g. gRPC) come with an out-of-the-box OpenTelemetry support and it's unclear how
34+
to ensure this instrumentation is use correctly in the presence of multiple SDKs.
35+
36+
## Explanation
37+
38+
We proposes these new fundamental concepts in OpenTelemetry:
39+
40+
- `Resource` *remains immutable*
41+
- Building on [OTEP 264](0264-resource-and-entities.md), identifying attributes
42+
are clearly outlined in Resource going forward, addressing unclear real world usage of Resource attributes ([e,g, identifying attributes in OpAMP](https://github.com/open-telemetry/opamp-spec/blob/main/specification.md#agentdescriptionidentifying_attributes)).
43+
- SDK will be given an explicit initialization stage where `Resource` is not in a complete state, addressing OpenTelemetry JS concerns around async resource detection.
44+
- The SDK will be identified by a single `Resource` provided during SDK startup.
45+
- ResourceDetection will be expanded, as described in [OTEP 264](0264-resource-and-entities.md).
46+
- An explicit section about SDK initialization will be created.
47+
- Signal Providers in the SDK will allow "specialization" of the default SDK
48+
resource. We create new `{Signal}Provider` instances by providing a new
49+
`Entity` on the existing provider.
50+
- This will construct a new `Resource` specific to that provider.
51+
- The new provider will re-use all configuraiton (e.g. export pipeline)
52+
defined from the base provider.
53+
54+
## Internal details
55+
56+
TODO - introduction.
57+
58+
### API Details
59+
60+
Previously, every `{Signal}Provider` API defined a single
61+
`Get a {Signal}` operation. These will be expanded with a new
62+
`For Entity` operation, which will construct a new `{SignalProvider}`
63+
API component for reporting against a specific `Entity`.
64+
65+
#### For Entity
66+
67+
This API MUST accept the following parameters:
68+
69+
* `entities`: Specifies the `Entity` set to associate with
70+
emitted telemetry.
71+
72+
Any `entities` provided which conflict with those entities already provided in
73+
the SDK `Resource` represent an *override* of identity. The SDK MUST resolve the
74+
conflict without causing a fatal error.
75+
76+
The set of `Entity` provided to these operations MUST only include one `Entity`
77+
per `type`.
78+
79+
#### Entity
80+
81+
An `Entity` is a collection of the following values:
82+
83+
- `type`: A string describing the class of the Entity.
84+
- `id`: An attribute collection that identifies an instance of the Entity.
85+
- (optional) `description`: An attribute collection that describes the instance
86+
of the Entity.
87+
- (optional) `schema_url`: Specifies the Schema URL that should be recorded for
88+
this Entity.
89+
90+
An `Entity` is uniquely identified by the combination of its `type` and `id`.
91+
92+
`schema_url` defines version of the schema used to describe an `Entity`. If
93+
two entities exist with the same identity and different `schema_url`s, they
94+
MUST be considered in conflict with each other.
95+
96+
### SDK Details
97+
98+
TODO
99+
100+
## Trade-offs and mitigations
101+
102+
The primary trade-offs to make here are around "breaking changes" and subtle
103+
differences in generated telemetry for code leveraging Entity vs. code which
104+
does not. We need give room for consumers of OTLP (vendors, databases, collector)
105+
time to support the new semantics of Entity prior to data showing up which would
106+
not be correctly interpreted without understanding these new semantics. As such,
107+
primarily:
108+
109+
- `Entity`, as defined in OTLP, is an opt-in construct. `Resource` should be
110+
usable as an identity independent of `Entity`.
111+
- Consumers should now expect SDKs reporting multiple resources in the same
112+
batch. Theoretically, this SHOULD already be supported due to how OTLP is
113+
deisgned to allow aggregation / batching of data at any point.
114+
115+
## Prior art and alternatives
116+
117+
OpenCensus previously allowed contextual tags to be specified dynamically and
118+
used everywhere metric measurements were reported. Users were then required to
119+
select which of these were useful to them via the definition of "Views".
120+
OpenTelemetry has aimed for a simpler solution where every metric has an
121+
implicit View definition, and we leverage metric advice to allow sending
122+
attributes than is naturally used when reporting the metric.
123+
124+
As called out in the description, [OTEP 4316](https://github.com/open-telemetry/opentelemetry-specification/pull/4316)
125+
proposes making resource fully mutable, which comes with its own set of tradeoffs.
126+
127+
Today, Semantic Conventions already defines `Entity` and uses it to group and
128+
report `Resource` attributes cohesively. Additionally, Semantic convention only
129+
models "entity associations", that is requiring a signal (e.g. a metric, event
130+
or span) to be attached to an entity. For example, the `system.cpu.time` metric
131+
is expected to be associated with a `host` entity. This association makes no
132+
assumption about whether that is through `Resource` or some other mechanism,
133+
and can therefore be extended to support `InstrumentationScope` based entities.
134+
135+
## Open questions
136+
137+
Adding entity in InstrumentationScope has a lot of implications that must be
138+
resolved.
139+
140+
### What are the SDK safeguards against high-cardinality Entities?
141+
142+
As seen in [Issue #3062](https://github.com/open-telemetry/opentelemetry-specification/issues/3062),
143+
systems observing multiple tenants need to ensure that tenants which are only observed briefly do not
144+
continue to consume resources (particularly memory) for long periods of time. There needs to be
145+
some level of control (either direct or implicit) in allowing new "Scope with Entity" to be created.
146+
147+
### What happens when an entity already exists within Resource?
148+
149+
Should we consider this a failure or a feature?
150+
151+
We currently consider this a feature. Upon conflict, the *new* Entity would be
152+
used in the resulting `Resource` reported for a new `{SignalProvider}`.
153+
154+
The SDK needs some form of stable identity for itself, however when reporting
155+
Telemetry, it may be recording data on behalf of some other system.
156+
157+
### Are descriptive attributes allowed to change for Resource?
158+
159+
Its not clear how resource immutability is kept or what is meant by immutable.
160+
Will the resource emitted on the first export be the same as the one emitted for
161+
the entire lifetime of the process? Are descriptive attributes on entities
162+
attached to resource still allowed to change? What about attaching new entities
163+
to that resource?
164+
165+
For now:
166+
167+
- The set of entities reported on Resource becomes locked.
168+
All identifying attributes are also locked.
169+
- Whether we want to allow descriptive attributes to change - this can be
170+
determined or evolve over time. Until the ecosystem around OTLP is leveraging
171+
the "identity" attributes of Entity for Resource, we should not allow
172+
mutation of descriptive attributes.
173+
174+
### What is the expected impact on Collector components?
175+
176+
There should be *no* impact on collector components beyond those defined in
177+
[OTEP 4316](https://github.com/open-telemetry/opentelemetry-specification/pull/4316).
178+
179+
### How do we guide developers on when to use `For Entity`?
180+
181+
We will have clear guidance on the `For Entity` methods
182+
183+
## Prototypes
184+
185+
TODO: **In Progress**
186+
187+
## Future possibilities
188+
189+
This proposal brings strong multi-tenant capabilities to the OpenTelemetry SDK. One possibility
190+
is to improve the interaction between dynamic `Context` and signals, e.g. allowing
191+
some level of interaction `Context` and attributes / entities.
192+
193+
For example, rather than a lexical scope:
194+
195+
```js
196+
const myMeterProvider = globalMeterProvider.forEntity(getCurrentSession())
197+
doSomething(myMeterProvider)
198+
```
199+
200+
We could allow runtime scope:
201+
202+
```js
203+
const ctx = api.context.active();
204+
api.context.with(ctx.setValue(CURRENT_ENTITY_KEY, getCurrentSession()))
205+
doSomething()
206+
```

0 commit comments

Comments
 (0)