Skip to content

Commit 9f80317

Browse files
jsuerethdyladansmithkamphaus
authored
Create initial Entities data model specification. (#4442)
Adds an initial cut at the Entity DataModel specification from OTEP 256. ## Changes - Adds information to Resource readme. - Creates an initial Resource DataModel with content from Entities SIG discussions on purpose and usage or Resource. - Creates an `entities` diretory for storing Entities DataModel. See open-telemetry/opentelemetry-proto#635 for related protocol change. Prototypes: - java: open-telemetry/opentelemetry-java#6855 - collector: open-telemetry/opentelemetry-collector#11958 - go: open-telemetry/opentelemetry-go#5918 --------- Co-authored-by: Daniel Dyla <[email protected]> Co-authored-by: Nathan L Smith <[email protected]> Co-authored-by: Christophe Kamphaus <[email protected]>
1 parent 26e99d4 commit 9f80317

File tree

5 files changed

+409
-0
lines changed

5 files changed

+409
-0
lines changed

‎CHANGELOG.md

+3
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ release.
3434

3535
### Resource
3636

37+
- Add Datamodel for Entities
38+
([#4442](https://github.com/open-telemetry/opentelemetry-specification/pull/4442))
39+
3740
### Profiles
3841

3942
### OpenTelemetry Protocol

‎specification/entities/README.md

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
<!--- Hugo front matter used to generate the website version of this page:
2+
path_base_for_github_subdir:
3+
from: tmp/otel/specification/entities/_index.md
4+
to: entities/README.md
5+
--->
6+
7+
# Entities
8+
9+
<details>
10+
<summary>Table of Contents</summary>
11+
12+
<!-- toc -->
13+
14+
- [Overview](#overview)
15+
- [Specifications](#specifications)
16+
17+
<!-- tocstop -->
18+
19+
</details>
20+
21+
## Overview
22+
23+
Entity represents an object of interest associated with produced telemetry:
24+
traces, metrics, logs, profiles etc.
25+
26+
## Specifications
27+
28+
- [Data Model](./data-model.md)

‎specification/entities/data-model.md

+209
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# Entity Data Model
2+
3+
**Status**: [Development](../document-status.md)
4+
5+
<details>
6+
<summary>Table of Contents</summary>
7+
8+
<!-- toc -->
9+
10+
- [Minimally Sufficient Identity](#minimally-sufficient-identity)
11+
- [Repeatable Identity](#repeatable-identity)
12+
- [Examples of Entities](#examples-of-entities)
13+
14+
<!-- tocstop -->
15+
16+
</details>
17+
18+
Entity represents an object of interest associated with produced telemetry:
19+
traces, metrics, profiles, or logs.
20+
21+
For example, telemetry produced using an OpenTelemetry SDK is normally
22+
associated with a `service` entity. Similarly, OpenTelemetry defines system
23+
metrics for a `host`. The `host` is the entity we want to associate metrics with
24+
in this case.
25+
26+
Entities may be also associated with produced telemetry indirectly.
27+
For example a service that produces
28+
telemetry is also related to a process in which the service runs, so we say that
29+
the `service` entity is related to the `process` entity. The process normally
30+
also runs on a host, so we say that the `process` entity is related to the
31+
`host` entity.
32+
33+
> Note: Entity relationship modelling will be refined in future specification
34+
> work.
35+
36+
The data model below defines a logical model for an entity (irrespective of the
37+
physical format and encoding of how entity data is recorded).
38+
39+
<table>
40+
<tr>
41+
<td><strong>Field</strong>
42+
</td>
43+
<td><strong>Type</strong>
44+
</td>
45+
<td><strong>Description</strong>
46+
</td>
47+
</tr>
48+
<tr>
49+
<td>Type
50+
</td>
51+
<td>string
52+
</td>
53+
<td>Defines the type of the entity. MUST not change during the
54+
lifetime of the entity. For example: "service" or "host". This field is
55+
required and MUST not be empty for valid entities.
56+
</td>
57+
</tr>
58+
<tr>
59+
<td>Id
60+
</td>
61+
<td>map&lt;string, standard attribute value&gt;
62+
</td>
63+
<td>Attributes that identify the entity.
64+
<p>
65+
MUST not change during the lifetime of the entity. The Id must contain
66+
at least one attribute.
67+
<p>
68+
Follows OpenTelemetry <a
69+
href="../../specification/common/README.md#standard-attribute">Standard
70+
attribute definition</a>. SHOULD follow OpenTelemetry <a
71+
href="https://github.com/open-telemetry/semantic-conventions">semantic
72+
conventions</a> for attributes.
73+
</td>
74+
</tr>
75+
<tr>
76+
<td>Description
77+
</td>
78+
<td>map&lt;string, any&gt;
79+
</td>
80+
<td>Descriptive (non-identifying) attributes of the entity.
81+
<p>
82+
MAY change over the lifetime of the entity. MAY be empty. These
83+
attributes are not part of entity's identity.
84+
<p>
85+
Follows <a
86+
href="../../specification/logs/data-model.md#type-any">any</a>
87+
value definition in the OpenTelemetry spec. Arbitrary deep nesting of values
88+
for arrays and maps is allowed.
89+
<p>
90+
SHOULD follow OpenTelemetry <a
91+
href="https://github.com/open-telemetry/semantic-conventions">semantic
92+
conventions</a> for attributes.
93+
</td>
94+
</tr>
95+
</table>
96+
97+
## Minimally Sufficient Identity
98+
99+
Commonly, a number of attributes of an entity are readily available for the telemetry
100+
producer to compose an Id from. Of the available attributes the entity Id should
101+
include the minimal set of attributes that is sufficient for uniquely identifying
102+
that entity. For example a Process on a host can be uniquely identified by
103+
(`process.pid`,`process.start_time`) attributes. Adding for example `process.executable.name` attribute to the Id is unnecessary and violates the
104+
Minimally Sufficient Identity rule.
105+
106+
## Repeatable Identity
107+
108+
The identifying attributes for entity SHOULD be values that can be repeatably
109+
obtained by observers of that entity. For example, a `process` entity SHOULD
110+
have the same identity (and be recognized as the same process), regardless of whether
111+
the identity was generated from the process itself, e.g. via SDK, or by an
112+
OpenTelemetry Collector running on the same host, or by some other system
113+
describing the process.
114+
115+
> Aside: There are many ways to accomplish repeatable identifying attributes
116+
> across multiple observers. While many successful systems rely on pushing down
117+
> identity from a central registry or knowledge store, OpenTelemetry must
118+
> support all possible scenarios.
119+
120+
## Examples of Entities
121+
122+
_This section is non-normative and is present only for the purposes of
123+
demonstrating the data model._
124+
125+
Here are examples of entities, the typical identifying attributes they
126+
have and some examples of descriptive attributes that may be
127+
associated with the entity.
128+
129+
_Note: These examples MAY diverge from semantic conventions._
130+
131+
<table>
132+
<tr>
133+
<td><strong>Entity</strong>
134+
</td>
135+
<td><strong>Entity Type</strong>
136+
</td>
137+
<td><strong>Identifying Attributes</strong>
138+
</td>
139+
<td><strong>Descriptive Attributes</strong>
140+
</td>
141+
</tr>
142+
<tr>
143+
<td>Container
144+
</td>
145+
<td><pre>container</pre>
146+
</td>
147+
<td>container.id
148+
</td>
149+
<td>container.image.id<br/>
150+
container.image.name<br/>
151+
container.image.tag.{key}<br/>
152+
container.label.{key}<br/>
153+
container.name<br/>
154+
container.runtime<br/>
155+
oci.manifest.digest<br/>
156+
container.command<br/>
157+
</td>
158+
</tr>
159+
<tr>
160+
<td>Host
161+
</td>
162+
<td><pre>host</pre>
163+
</td>
164+
<td>host.id
165+
</td>
166+
<td>host.arch<br/>
167+
host.name<br/>
168+
host.type<br/>
169+
host.image.id<br/>
170+
host.image.name<br/>
171+
host.image.version<br/>
172+
host.type
173+
</td>
174+
</tr>
175+
<tr>
176+
<td>Kubernetes Node
177+
</td>
178+
<td><pre>k8s.node</pre>
179+
</td>
180+
<td>k8s.node.uid
181+
</td>
182+
<td>k8s.node.name
183+
</td>
184+
</tr>
185+
<tr>
186+
<td>Kubernetes Pod
187+
</td>
188+
<td><pre>k8s.pod</pre>
189+
</td>
190+
<td>k8s.pod.uid
191+
</td>
192+
<td>k8s.pod.name<br/>
193+
k8s.pod.label.{key}<br/>
194+
k8s.pod.annotation.{key}<br/>
195+
</td>
196+
</tr>
197+
<tr>
198+
<td>Service Instance
199+
</td>
200+
<td><pre>service.instance</pre>
201+
</td>
202+
<td>service.instance.id<br/>
203+
service.name<br/>
204+
service.namesapce
205+
</td>
206+
<td>service.version
207+
</td>
208+
</tr>
209+
</table>

‎specification/resource/README.md

+95
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,98 @@ path_base_for_github_subdir:
55
--->
66

77
# Resource
8+
9+
<details>
10+
<summary>Table of Contents</summary>
11+
12+
<!-- toc -->
13+
14+
- [Overview](#overview)
15+
* [Identity](#identity)
16+
* [Navigation](#navigation)
17+
* [Telescoping](#telescoping)
18+
- [Specifications](#specifications)
19+
20+
<!-- tocstop -->
21+
22+
</details>
23+
24+
## Overview
25+
26+
A Resource is a representation of the entity producing telemetry.
27+
Within OpenTelemetry, all signals are associated with a Resource, enabling
28+
contextual correlation of data from the same source. For example, if I see
29+
a high latency in a span I need to check the metrics for the same entity that
30+
produced that Span during the time when the latency was observed.
31+
32+
Resource provides two important aspects for observability:
33+
34+
- It MUST identify an entity that is producing telemetry.
35+
- It SHOULD allow users to determine where that entity resides within their infrastructure.
36+
37+
### Identity
38+
39+
Resource provides a natural way to understand "what" produced an effect and
40+
evaluate other signals of that same source. This is done through attaching the
41+
same set of identifying attributes on all telemetry produced in an
42+
OpenTelemetry SDK.
43+
44+
Resource identity provides a natural pivot point for observability signals, a
45+
key type of correlation in OpenTelemetry.
46+
47+
### Navigation
48+
49+
Implicit in the design of Resource and attributes is ensuring users are able to
50+
navigate their infrastructure, tools, UIs, etc. to find the *same* entity that
51+
telemetry is reporting against. For example, in practice we could see Resource
52+
including more than on entity, like:
53+
54+
- A process
55+
- A container
56+
- A kubernetes pod name
57+
- A namespace
58+
- A deployment
59+
60+
By including identifying attributes of each of these, we can help users navigate
61+
through their `kubectl` or Kubernetes UIs to find the specific process
62+
generating telemetry. This is as important as being able to uniquely identify
63+
one process from another.
64+
65+
> Aside: Observability signals SHOULD be actionable. Knowing a process is
66+
> struggling is not as useful as being able to scale up a deployment to take
67+
> load off the struggling process.
68+
69+
If the only thing important to Resource was identity, we could simply use UUIDs.
70+
However, this would rely on some other, easily accessible, system to provide
71+
human-friendly understanding for these UUIDs. OpenTelemetry provides a model
72+
where a full UUID-only solution could be chosen, but defaults to a *blended*
73+
approach, where resource provides both Identity and Navigation.
74+
75+
This leads to the next concept: Telescoping identity to the needs of a system.
76+
77+
### Telescoping
78+
79+
Within OpenTelemetry, we want to give users the flexibility to decide what
80+
information needs to be sent *with* observability signals and what information
81+
can be later joined. We call this "telescoping identity" where users can decide
82+
how *small* or *large* the size of an OpenTelemetry resource will be on the wire
83+
(and correspondingly, how large data points are when stored, depending on
84+
storage solution).
85+
86+
For example, in the extreme, OpenTelemery could synthesize a UUID for every
87+
system which produces telemetry. All identifying attributes for Resource and
88+
Entity could be sent via a side channel with known relationships to this UUID.
89+
While this would optimise the runtime generation and sending of telemetry, it
90+
comes at the cost of downstream storage systems needing to join data back
91+
together either at ingestion time or query time. For high performance use cases,
92+
e.g. alerting, these joins can be expensive.
93+
94+
In practice, users control Resource identity via the configuration of Resource
95+
Detection within SDKs and the collector. Users wishing for minimal identity will
96+
limit their resource detection just to a `service.instance.id`, for example.
97+
Some users highly customize resource detection with many concepts being appended.
98+
99+
## Specifications
100+
101+
- [Data Model](./data-model.md)
102+
- [Resource SDK](./sdk.md)

0 commit comments

Comments
 (0)