Skip to content

Commit a865ba7

Browse files
ABI CD: distinguish between type descriptions and structure metadata
1 parent f8d6fb8 commit a865ba7

File tree

1 file changed

+11
-17
lines changed
  • docs/features/communication/abi_compatible_data_types

1 file changed

+11
-17
lines changed

docs/features/communication/abi_compatible_data_types/index.rst

Lines changed: 11 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,9 @@ To activate this feature, use the following feature flag:
4141
Abstract
4242
========
4343

44-
This feature request defines a set of ABI-compatible data types and a runtime type description format to support zero-copy inter-process communication between C++17 and Rust 1.88 processes using the same endianness. It ensures consistent type layouts across languages by requiring fixed-size, statically allocated types without absolute pointers or language-specific metadata.
44+
This feature request defines a set of ABI-compatible data types and a runtime type metadata format to support zero-copy inter-process communication between C++17 and Rust 1.88 processes using the same endianness. It ensures consistent type layouts across languages by requiring fixed-size, statically allocated types without absolute pointers or language-specific metadata.
4545

46-
The specification covers primitive types, structs, enums, arrays, and introduces ABI-stable representations for vectors, strings, options, and results. An optional runtime-readable type description enables processes to interpret shared memory without compile-time access to type definitions.
46+
The specification covers primitive types, structs, enums, arrays, and introduces ABI-stable representations for vectors, strings, options, and results. An optional runtime-readable type structure metadata enables processes to interpret shared memory without compile-time access to type definitions.
4747

4848

4949
Motivation
@@ -53,7 +53,7 @@ This feature request addresses specific challenges in achieving type compatibili
5353

5454
1. **ABI Compatibility**: Processes implemented in different programming languages (C++17 and Rust 1.88) must interpret a shared memory location consistently as the same native type, provided both have compile-time access to the type definition. This scenario eliminates serialization overhead and allows direct memory access.
5555

56-
2. **Type Description**: It should be possible to record arbitrary data streams, and convert or analyze them at a later time and/or on a different system, without having to recompile the conversion or analysis tools for that particular data format. A machine-readable description of the format, including any user-defined data types, should be available on request during runtime. In addition, this description could potentially be used by gateway processes to perform relatively simple but generic transformations between different data representations.
56+
2. **Type Structure Metadata**: It should be possible to record arbitrary data streams, and convert or analyze them at a later time and/or on a different system, without having to recompile the conversion or analysis tools for that particular data format. A machine-readable description of the format, including any user-defined data types, should be available on request during runtime. In addition, this structure description could potentially be used by gateway processes to perform relatively simple but generic transformations between different data representations.
5757

5858

5959
ABI Compatibility
@@ -388,14 +388,7 @@ Language Conformance Summary
388388
Type Description
389389
----------------
390390

391-
To address the scenarios outlined in the motivation, a clearly defined type description mechanism is required. The type description provides sufficient information during runtime, enabling a process without compile-time access to type definitions to correctly interpret a given memory location according to the previously established ABI rules.
392-
393-
The goals are:
394-
395-
* Enable interpretation of shared memory content without compile-time access to type definitions.
396-
* Support all ABI-compatible data types previously defined.
397-
* Include versioning to manage schema evolution and compatibility.
398-
* Allow easy generation and parsing by tooling in both C++ and Rust.
391+
To address the scenarios outlined in the motivation, a clearly defined type description mechanism is required.
399392

400393
Workflows
401394
^^^^^^^^^
@@ -408,10 +401,11 @@ Two potential workflows are considered for creating type descriptions:
408401

409402
Both workflows are valid, and the final decision is deferred pending further feasibility analysis.
410403

411-
Type Description Format
404+
Type Structure Metadata
412405
^^^^^^^^^^^^^^^^^^^^^^^
413406

414-
The format of the type description shall explicitly support versioning to allow schema evolution and backward compatibility. It must accommodate all data types described earlier in the ABI compatibility section. It should be simple, human-readable, and easily machine-parsable.
407+
Precise information about the structure of the types is preserved for use during runtime, enabling a process without compile-time access to type definitions to correctly interpret a given memory location according to the previously established ABI rules.
408+
The format of the type metadata shall explicitly support versioning to allow schema evolution and backward compatibility. It must accommodate all data types described earlier in the ABI compatibility section. It should be simple, human-readable, and easily machine-parsable.
415409

416410
The choice of serialization format is left open but may include RON, JSON5, or a custom DSL, based on readability, tooling support, and maintainability.
417411

@@ -457,7 +451,7 @@ Reflection
457451
Reflection, in this context, is the ability to inspect data at runtime even if its structure is not or not fully known at compile time.
458452
Benefits of reflection include being able to translate recorded data into a human-readable format (e.g., JSON or CSV) without having to know the type definitions at compile time; this enables general-purpose data recording and transformation tools.
459453

460-
This ability requires some form of *type description* being available at runtime, so that a sequence of bytes can be interpreted as a data structure.
454+
This ability requires some form of *type structure metadata* being available at runtime, so that a sequence of bytes can be interpreted as a data structure.
461455
There are two primary approaches to achieve this goal:
462456

463457
* *inline type descriptions*, which precede each instance of every type, and
@@ -478,10 +472,10 @@ This approach, however, comes with significant downsides:
478472
Alternative Approach
479473
^^^^^^^^^^^^^^^^^^^^
480474

481-
Instead of inserting inline type descriptions into each instance of an ABI compatible type, the full type description can be made available to a consumer only once, either proactively or on request.
475+
Instead of inserting inline type descriptions into each instance of an ABI compatible type, the full type structure metadata can be made available to a consumer only once, either proactively or on request.
482476
The consumer decides if it uses or ignores this metadata.
483477

484-
This type description can be used to dynamically translate between the compact, non-reflective ABI compatible data structures on one side, and a reflective, inline-describing format on the other side.
478+
This description of the type structure can be used to dynamically translate between the compact, non-reflective ABI compatible data structures on one side, and a reflective, inline-describing format on the other side.
485479
Although this incurs a copy and some minor processing, the overhead should be negligible compared to other computational tasks involving the payload.
486480

487481
One method to efficiently translate a payload consisting of ABI compatible types to an inline-described reflective format is to convert the hierarchical type description to a flat list of *instructions* which can be executed by an interpreter.
@@ -523,7 +517,7 @@ Reflection will not be part of version 1.0 of this feature request.
523517
3. The specification for SOME/IP types is incompatible with the requirement of ABI vectors that can grow dynamically during construction, i.e., vectors which contain fewer valid elements than they take up space in memory.
524518
4. Inserting inline type descriptions on demand is expected to be a relatively cheap operation, which negates the main motivation for including them directly in ABI types in the first place.
525519

526-
* External type descriptions will probably be included in a later version of this feature request.
520+
* External type structure descriptions will probably be included in a later version of this feature request.
527521
For now, they're postponed until we have a better understanding of the relevant use cases.
528522

529523

0 commit comments

Comments
 (0)