You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/features/communication/abi_compatible_data_types/index.rst
+11-17Lines changed: 11 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,9 +41,9 @@ To activate this feature, use the following feature flag:
41
41
Abstract
42
42
========
43
43
44
-
This feature request defines a set of ABI-compatible data types and a runtime type description format to support zero-copy inter-process communication between C++17 and Rust 1.88 processes using the same endianness. It ensures consistent type layouts across languages by requiring fixed-size, statically allocated types without absolute pointers or language-specific metadata.
44
+
This feature request defines a set of ABI-compatible data types and a runtime type metadata format to support zero-copy inter-process communication between C++17 and Rust 1.88 processes using the same endianness. It ensures consistent type layouts across languages by requiring fixed-size, statically allocated types without absolute pointers or language-specific metadata.
45
45
46
-
The specification covers primitive types, structs, enums, arrays, and introduces ABI-stable representations for vectors, strings, options, and results. An optional runtime-readable type description enables processes to interpret shared memory without compile-time access to type definitions.
46
+
The specification covers primitive types, structs, enums, arrays, and introduces ABI-stable representations for vectors, strings, options, and results. An optional runtime-readable type structure metadata enables processes to interpret shared memory without compile-time access to type definitions.
47
47
48
48
49
49
Motivation
@@ -53,7 +53,7 @@ This feature request addresses specific challenges in achieving type compatibili
53
53
54
54
1. **ABI Compatibility**: Processes implemented in different programming languages (C++17 and Rust 1.88) must interpret a shared memory location consistently as the same native type, provided both have compile-time access to the type definition. This scenario eliminates serialization overhead and allows direct memory access.
55
55
56
-
2. **Type Description**: It should be possible to record arbitrary data streams, and convert or analyze them at a later time and/or on a different system, without having to recompile the conversion or analysis tools for that particular data format. A machine-readable description of the format, including any user-defined data types, should be available on request during runtime. In addition, this description could potentially be used by gateway processes to perform relatively simple but generic transformations between different data representations.
56
+
2. **Type Structure Metadata**: It should be possible to record arbitrary data streams, and convert or analyze them at a later time and/or on a different system, without having to recompile the conversion or analysis tools for that particular data format. A machine-readable description of the format, including any user-defined data types, should be available on request during runtime. In addition, this structure description could potentially be used by gateway processes to perform relatively simple but generic transformations between different data representations.
57
57
58
58
59
59
ABI Compatibility
@@ -388,14 +388,7 @@ Language Conformance Summary
388
388
Type Description
389
389
----------------
390
390
391
-
To address the scenarios outlined in the motivation, a clearly defined type description mechanism is required. The type description provides sufficient information during runtime, enabling a process without compile-time access to type definitions to correctly interpret a given memory location according to the previously established ABI rules.
392
-
393
-
The goals are:
394
-
395
-
* Enable interpretation of shared memory content without compile-time access to type definitions.
396
-
* Support all ABI-compatible data types previously defined.
397
-
* Include versioning to manage schema evolution and compatibility.
398
-
* Allow easy generation and parsing by tooling in both C++ and Rust.
391
+
To address the scenarios outlined in the motivation, a clearly defined type description mechanism is required.
399
392
400
393
Workflows
401
394
^^^^^^^^^
@@ -408,10 +401,11 @@ Two potential workflows are considered for creating type descriptions:
408
401
409
402
Both workflows are valid, and the final decision is deferred pending further feasibility analysis.
410
403
411
-
Type Description Format
404
+
Type Structure Metadata
412
405
^^^^^^^^^^^^^^^^^^^^^^^
413
406
414
-
The format of the type description shall explicitly support versioning to allow schema evolution and backward compatibility. It must accommodate all data types described earlier in the ABI compatibility section. It should be simple, human-readable, and easily machine-parsable.
407
+
Precise information about the structure of the types is preserved for use during runtime, enabling a process without compile-time access to type definitions to correctly interpret a given memory location according to the previously established ABI rules.
408
+
The format of the type metadata shall explicitly support versioning to allow schema evolution and backward compatibility. It must accommodate all data types described earlier in the ABI compatibility section. It should be simple, human-readable, and easily machine-parsable.
415
409
416
410
The choice of serialization format is left open but may include RON, JSON5, or a custom DSL, based on readability, tooling support, and maintainability.
417
411
@@ -457,7 +451,7 @@ Reflection
457
451
Reflection, in this context, is the ability to inspect data at runtime even if its structure is not or not fully known at compile time.
458
452
Benefits of reflection include being able to translate recorded data into a human-readable format (e.g., JSON or CSV) without having to know the type definitions at compile time; this enables general-purpose data recording and transformation tools.
459
453
460
-
This ability requires some form of *type description* being available at runtime, so that a sequence of bytes can be interpreted as a data structure.
454
+
This ability requires some form of *type structure metadata* being available at runtime, so that a sequence of bytes can be interpreted as a data structure.
461
455
There are two primary approaches to achieve this goal:
462
456
463
457
* *inline type descriptions*, which precede each instance of every type, and
@@ -478,10 +472,10 @@ This approach, however, comes with significant downsides:
478
472
Alternative Approach
479
473
^^^^^^^^^^^^^^^^^^^^
480
474
481
-
Instead of inserting inline type descriptions into each instance of an ABI compatible type, the full type description can be made available to a consumer only once, either proactively or on request.
475
+
Instead of inserting inline type descriptions into each instance of an ABI compatible type, the full type structure metadata can be made available to a consumer only once, either proactively or on request.
482
476
The consumer decides if it uses or ignores this metadata.
483
477
484
-
This type description can be used to dynamically translate between the compact, non-reflective ABI compatible data structures on one side, and a reflective, inline-describing format on the other side.
478
+
This description of the type structure can be used to dynamically translate between the compact, non-reflective ABI compatible data structures on one side, and a reflective, inline-describing format on the other side.
485
479
Although this incurs a copy and some minor processing, the overhead should be negligible compared to other computational tasks involving the payload.
486
480
487
481
One method to efficiently translate a payload consisting of ABI compatible types to an inline-described reflective format is to convert the hierarchical type description to a flat list of *instructions* which can be executed by an interpreter.
@@ -523,7 +517,7 @@ Reflection will not be part of version 1.0 of this feature request.
523
517
3. The specification for SOME/IP types is incompatible with the requirement of ABI vectors that can grow dynamically during construction, i.e., vectors which contain fewer valid elements than they take up space in memory.
524
518
4. Inserting inline type descriptions on demand is expected to be a relatively cheap operation, which negates the main motivation for including them directly in ABI types in the first place.
525
519
526
-
* External type descriptions will probably be included in a later version of this feature request.
520
+
* External type structure descriptions will probably be included in a later version of this feature request.
527
521
For now, they're postponed until we have a better understanding of the relevant use cases.
0 commit comments