Skip to content

Source Data Primitives v1#104

Open
beardedeagle wants to merge 5 commits into
mainfrom
source-data-primitives-v1
Open

Source Data Primitives v1#104
beardedeagle wants to merge 5 commits into
mainfrom
source-data-primitives-v1

Conversation

@beardedeagle

Copy link
Copy Markdown
Owner

Summary

Adds Source Data Primitives v1 as an end-to-end Strata-to-Mantle language/runtime slice.

This introduces source-authorable String and Bytes primitive values, carries them through Strata parsing, checking, lowering, Mantle target artifacts, runtime admission, runtime execution, observability, examples, fuzz seeds, assurance coverage, and documentation.

What changed

  • Adds canonical primitive shape and value support for String and Bytes in Mantle artifacts.
  • Extends Strata source parsing, display, checking, static validation, checked types, value resolution, equality handling, lowering, and target requirement declaration for primitive values.
  • Preserves the Strata/Mantle boundary by lowering checked typed primitive values into Mantle artifact IDs and typed value templates rather than executable source names.
  • Adds fail-closed validation for primitive collisions and malformed primitive data at the source, artifact, and runtime-loaded program boundaries.
  • Reserves primitive type names and primitive value-label forms so source declarations and payload-bearing variants cannot shadow String(...) or Bytes(...) artifact labels.
  • Adds runtime support for primitive equality and trace formatting while keeping runtime admission responsible for loaded artifact validation.
  • Adds a runnable examples/source_contract_data_primitives.str source-to-runtime gate proving checked source, built artifact, Mantle execution, and typed runtime trace behavior.
  • Adds fuzz seeds, performance smoke coverage, acceptance tests, negative tests, assurance inventory updates, and documentation for the new primitive surface.
  • Removes a redundant runtime test import so the Miri gate remains warning-free.

User impact

Strata programs can now use typed String and Bytes source literals in records, lists, maps, messages, process state, equality conditions, and runtime-observed data without treating those values as source-name dispatch or untyped strings. Mantle artifacts remain self-validating, and Mantle runtime admission rejects forged or malformed primitive-shaped values before execution.

Validation

  • cargo +stable fmt --all --check
  • cargo +stable test -p strata source_data_primitives --lib
  • cargo +stable test -p mantle-artifact target_requirements --lib
  • cargo +stable test -p mantle-runtime primitive --lib
  • cargo +stable test -p strata-mantle-acceptance source_contract_data_primitives_check_build_run_preserves_typed_values --test source_to_runtime_gates
  • cargo +stable test -p strata-mantle-acceptance --test language_surface_assurance
  • just run-example source_contract_data_primitives
  • just performance-smoke-profile source_contract_data_primitives.check_lower
  • just performance-smoke-profile source_contract_data_primitives.in_memory_runtime
  • just quality
  • just fuzz-ci
  • just miri-ci
  • pre-push native CI gate

String and Bytes now cross the Strata/Mantle boundary only as immutable typed data, preserving typed runtime dispatch and table-ID execution while allowing ordinary source values, collection entries, payloads, state, and exact primitive equality.

Constraint: Strata source text and byte literals must remain data, metadata, diagnostics, or provenance; executable meaning stays typed and table based.
Rejected: dynamic string operations | out of the v1 primitive-data slice and would blur immutable literal data with executable behavior.
Rejected: source text dispatch | runtime execution must continue through admitted typed IDs and value templates.
Confidence: high
Scope-risk: moderate
Directive: Keep primitive data immutable and never use primitive text or bytes for runtime dispatch, authority, or hidden control flow.
Tested: git diff --cached --check; staged content hygiene scans; file-by-file review zero findings; exhaustive cross-boundary review zero findings; just quality; just run-example source_contract_data_primitives; just fuzz-smoke; just miri-ci
Not-tested: local cross-OS PR-parity runner lanes
Complete the source data primitive review fixes across Strata checking,
Mantle artifact validation, runtime admission, diagnostics, tests, and
reference documentation.

The change reserves primitive type and value labels consistently so String and
Bytes cannot be shadowed by source declarations, decoded artifact values, or
runtime-loaded payload variants. It also keeps primitive label parsing defensive
for non-ASCII input, adds target requirement coverage for typed value templates,
and tightens runtime admission around primitive-shaped labels.

The source-level diagnostics and tests now describe the widened primitive
surface accurately, including ambiguous equality operands, malformed and
oversized primitive literals, primitive target requirements, and collision
rejections at the source, artifact, and runtime boundaries. Documentation now
spells out the reserved primitive forms and their boundary behavior.

Finally, remove a redundant runtime test import so the Miri gate stays warning
free while preserving the same test coverage.
Copilot AI review requested due to automatic review settings June 14, 2026 15:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the “Source Data Primitives v1” slice end-to-end across Strata (source syntax + checking + lowering) and Mantle (artifact schema + validation + runtime admission/execution/trace formatting), introducing first-class immutable String and Bytes primitive values and exact equality for those primitive types.

Changes:

  • Introduces String / Bytes literal syntax, AST nodes, canonical display, checker support, lowering support, and target requirement collection in Strata.
  • Extends Mantle artifact schema/codec/validation to include primitive value shapes and canonical lowercase-hex primitive value labels (e.g. String(7265...), Bytes(0102...)), plus admission/runtime validation.
  • Adds runnable example, fuzz seeds, acceptance/perf-smoke coverage, assurance inventory updates, and documentation updates for the new surface.

Reviewed changes

Copilot reviewed 95 out of 95 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Justfile Adds the new source-to-runtime success gate and updates failure-gate expectations to include primitives in diagnostics.
fuzz/seeds/strata_parse_check_lower/source_contract_data_primitives.str New fuzz seed covering parsing/checking/lowering of primitive literals and equality through collections/records.
fuzz/seeds/strata_parse_check_lower/source_contract_data_primitives_rejected.str New fuzz seed asserting malformed bytes escapes are rejected.
fuzz/seeds/mantle_runtime_from_source/source_contract_data_primitives.str New fuzz seed exercising full source→runtime flow with primitives.
fuzz/seeds/mantle_artifact_decode/source_contract_data_primitives.mta New artifact decode seed covering primitive shapes + primitive value labels in encoded artifacts.
examples/source_contract_data_primitives.str New runnable example demonstrating typed String/Bytes data and primitive equality at runtime.
docs/src/syntax-reference.md Documents new string/bytes literal grammar, escapes, equality surface updates, and reserved names.
docs/src/source-to-runtime-gates.md Adds a documented gate for the new primitives example.
docs/src/runtime-reference.md Updates runtime-supported predicate surface to include exact String/Bytes equality.
docs/src/language-surface-assurance.md Records the new primitives surface in the assurance inventory narrative.
docs/src/language-reference.md Updates language reference to include primitives + primitive equality and reserved-name rules.
docs/src/examples.md Inserts the new primitives example into the recommended reading order.
docs/src/diagnostics.md Updates diagnostics catalog for primitive-related errors and equality operand rules.
crates/strata/src/language/tests/source_value_equality.rs Updates equality diagnostics and adds positive coverage for primitive equality expressions.
crates/strata/src/language/tests/source_function_local_bindings.rs Updates expected diagnostics text to include primitives in the allowed type set.
crates/strata/src/language/tests/source_function_local_bindings_authority.rs Updates diagnostics expectations to include primitives.
crates/strata/src/language/tests/source_function_core.rs Updates function contract diagnostics expectations to include primitives.
crates/strata/src/language/tests/source_data_primitives.rs New unit tests covering primitive literal display, folding, lowering, rejection cases, and reserved-name/label collision behavior.
crates/strata/src/language/tests/process_return_match_arm_bounded_assurance/assertions.rs Extends assurance assertions to treat primitive artifact values as non-source-binding data.
crates/strata/src/language/tests/mod.rs Registers the new primitives test module.
crates/strata/src/language/tests/builtin_value_shapes.rs Extends enum-variant scanning to account for primitive artifact values.
crates/strata/src/language/target_requirements.rs Declares typed_value_templates runtime feature requirement when primitives are present.
crates/strata/src/language/parser/values.rs Adds parsing of string and bytes literals into new ValueExpr variants.
crates/strata/src/language/parser/tokens.rs Adds token APIs for bytes literals (peek/expect).
crates/strata/src/language/lowering/value_shapes.rs Lowers checked primitive shapes to artifact primitive shapes.
crates/strata/src/language/lexer.rs Adds bytes literal lexing, canonical string escapes (including \u{...}), and new error messages for malformed literals.
crates/strata/src/language/import_scope.rs Treats primitive literal expressions as valid value expressions for scope validation.
crates/strata/src/language/checker/types.rs Interns primitive types as checked value shapes.
crates/strata/src/language/checker/symbols/type_validation.rs Allows primitive type declarations in the “source value type” set validator.
crates/strata/src/language/checker/symbols/type_decls.rs Introduces TypeDecl::Primitive and corresponding kind string.
crates/strata/src/language/checker/symbols/build.rs Adds primitive type symbols, reserves primitive type names, and rejects payload-bearing enum variants named String/Bytes.
crates/strata/src/language/checker/symbols.rs Adds semantic lookup for primitive types and updates enum/record lookup exclusions.
crates/strata/src/language/checker/steps/transition/return_match_actions/substitution.rs Ensures primitive literal expressions are treated as terminal during substitution.
crates/strata/src/language/checker/steps/returns/matches/substitutions.rs Ensures primitive literal expressions are treated as terminal during substitution.
crates/strata/src/language/checker/steps/returns/matches.rs Adds artifact→source conversion for primitive values and updates enum-value handling to include primitive cases.
crates/strata/src/language/checker/static_validation/templates/equality.rs Extends equality operand validation and safe-pattern checks to include primitives.
crates/strata/src/language/checker/state_space/templates/scalars.rs Treats primitive literals as non-scalar in scalar-template typing paths.
crates/strata/src/language/checker/state_space/templates/equality.rs Extends template-side equality operand typing to recognize primitive literals and primitive-typed if/else operands.
crates/strata/src/language/checker/state_space/canonical/scalars.rs Excludes primitive literals from scalar canonical typing paths.
crates/strata/src/language/checker/state_space/canonical.rs Adds canonicalization of primitive values into artifact values for state-space enumeration and folding.
crates/strata/src/language/checker/state_space.rs Treats primitive literals as terminals in state-space evaluation paths.
crates/strata/src/language/checker/source_functions/values/type_check.rs Adds primitive-type checking behavior for resolved source values.
crates/strata/src/language/checker/source_functions/values/scalars.rs Excludes primitive literals from scalar expression typing.
crates/strata/src/language/checker/source_functions/values/resolution.rs Treats primitive literals as already-resolved values.
crates/strata/src/language/checker/source_functions/values/local_bindings.rs Updates local binding diagnostic text to include primitives.
crates/strata/src/language/checker/source_functions/values/equality.rs Extends source-side equality operand typing/validation to include primitives and updates ambiguity diagnostics.
crates/strata/src/language/checker/source_functions/values/dependencies.rs Treats primitive literals as non-resolution dependencies.
crates/strata/src/language/checker/source_functions/values.rs Routes primitive literal expressions through value-type checking paths.
crates/strata/src/language/checker/source_functions/value_resolution/substitution.rs Treats primitive literals as terminals for substitution.
crates/strata/src/language/checker/source_functions/value_resolution.rs Updates concrete enum/record handling to account for primitive literal cases.
crates/strata/src/language/checker/source_functions.rs Updates function contract diagnostics to include primitive types.
crates/strata/src/language/checked/types.rs Adds CheckedValueShape::Primitive.
crates/strata/src/language/ast/display.rs Adds canonical display for SourceStringLiteral / SourceBytesLiteral and prints them via ValueExpr display.
crates/strata/src/language/ast.rs Introduces SourceStringLiteral / SourceBytesLiteral AST types and length validation using MAX_PRIMITIVE_DATA_BYTES.
crates/strata/src/language.rs Re-exports the new literal AST types and reserves String / Bytes type names at the language level.
crates/strata-mantle-acceptance/tests/source_to_runtime_gates/source_functions/local_bindings.rs Updates acceptance expectations for diagnostics to include primitives.
crates/strata-mantle-acceptance/tests/source_to_runtime_gates/source_data_primitives.rs New source-to-runtime gate test for primitives end-to-end evidence in artifact encoding and runtime trace/output.
crates/strata-mantle-acceptance/tests/source_to_runtime_gates/runtime_branches/malformed_and_ordering.rs Updates runtime-gate diagnostics to include primitives in allowed equality operand kinds.
crates/strata-mantle-acceptance/tests/source_to_runtime_gates.rs Registers the new primitives source-to-runtime gate.
crates/strata-mantle-acceptance/tests/performance_smoke/source_data_primitives.rs Adds performance-smoke profiles for the new primitives example (check/lower + in-memory runtime).
crates/strata-mantle-acceptance/tests/performance_smoke/profiles.rs Registers the new profiles and updates profile selection key list.
crates/strata-mantle-acceptance/tests/performance_smoke.rs Executes the new primitives performance-smoke profiles.
crates/strata-mantle-acceptance/tests/language_surface_assurance/source.rs Updates assurance references to the updated equality diagnostics string.
crates/strata-mantle-acceptance/tests/language_surface_assurance/runtime.rs Adds a new assurance feature entry for primitives with evidence pointers (tests/docs/seeds/profiles).
crates/strata-mantle-acceptance/tests/language_surface_assurance/proof_domains.rs Adds a new proof domain covering the primitives surface.
crates/strata-mantle-acceptance/tests/language_surface_assurance/model.rs Registers the new assurance feature id in the expected model sets.
crates/mantle-runtime/src/run/tests/state_message_admission/predicate_templates.rs Updates admission rejection diagnostic text to include primitives in allowed operand kinds.
crates/mantle-runtime/src/run/tests/scalar_templates.rs Adds runtime test coverage for primitive equality branch selection and collision rejection.
crates/mantle-runtime/src/run/tests/effect_outcomes/boundary.rs Removes a redundant test import to keep warning-free builds.
crates/mantle-runtime/src/program/values.rs Updates embedded process-ref validation to treat primitive runtime values as non-authority carriers.
crates/mantle-runtime/src/program/type_validation.rs Validates primitive shapes and rejects payload-bearing enum variants colliding with primitive labels at load time.
crates/mantle-runtime/src/program/templates/admission/projections.rs Treats primitive shapes as non-process-ref-containing for projection admission checks.
crates/mantle-runtime/src/program/templates/admission/predicates.rs Extends operand type admission to allow primitive value types for equality predicates.
crates/mantle-runtime/src/program.rs Updates program loading imports/types to include primitive artifacts in runtime program model.
crates/mantle-runtime/src/event/jsonl/format.rs Adds JSONL formatting for primitive value labels using lowercase-hex encoding.
crates/mantle-artifact/src/validation/encoded_size.rs Accounts for primitive type shape encoded-size contributions.
crates/mantle-artifact/src/tests/target_requirements.rs Adds target-requirements validation test ensuring primitive shapes require typed_value_templates.
crates/mantle-artifact/src/tests/support.rs Adds test helper for appending primitive types.
crates/mantle-artifact/src/tests/identity_and_labels/artifact_value_shapes.rs Adds tests for primitive label canonicalization/parsing and collision/size validation.
crates/mantle-artifact/src/tests/codec/round_trip.rs Adds codec round-trip coverage for primitive shapes and primitive value templates.
crates/mantle-artifact/src/tests/codec/predicate_admission.rs Updates admission diagnostics to include primitives in allowed operand kinds.
crates/mantle-artifact/src/runtime_features.rs Declares runtime feature requirements for primitive shapes (typed_value_templates).
crates/mantle-artifact/src/lib.rs Re-exports ArtifactPrimitiveType from the artifact module.
crates/mantle-artifact/src/constants.rs Introduces MAX_PRIMITIVE_DATA_BYTES limit constant for primitive payload budgets.
crates/mantle-artifact/src/artifact/value_template/value.rs Adds primitive value validation, label encoding, label matching, and label-length computation.
crates/mantle-artifact/src/artifact/value_template/template/admission/projections.rs Treats primitive shapes as non-process-ref-containing in template projection admission.
crates/mantle-artifact/src/artifact/value_template/template/admission/predicates.rs Allows primitive operand types for equality predicates during template admission.
crates/mantle-artifact/src/artifact/value_template/parsing.rs Adds parsing of String(...) / Bytes(...) primitive value labels with lowercase-hex and size validation.
crates/mantle-artifact/src/artifact/value_template/model.rs Extends ArtifactValue with String and Bytes variants.
crates/mantle-artifact/src/artifact/validation.rs Validates primitive shapes, primitive values, and rejects payload-bearing enum variants colliding with primitive labels.
crates/mantle-artifact/src/artifact/primitives.rs Introduces ArtifactPrimitiveType enum with source/artifact naming and parsing helpers.
crates/mantle-artifact/src/artifact/codec/encode.rs Encodes primitive type shapes into the text artifact format.
crates/mantle-artifact/src/artifact/codec/decode.rs Decodes primitive type shapes from the text artifact format.
crates/mantle-artifact/src/artifact.rs Wires primitive type module, type shape variant, and constructor into the artifact model.
benchmarks/performance-smoke.baseline Adds baseline budgets for the new primitives performance-smoke profiles.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/src/diagnostics.md Outdated
Keep the diagnostics catalog aligned with the checker strings emitted for source
primitive redeclarations.

`String` and `Bytes` type redeclarations are rejected through the existing
reserved type-name diagnostic, so the documentation now points those primitive
names at `type name ... is reserved`. The core Bool conflict row is narrowed to
only the Bool-specific diagnostics that are actually emitted for `Bool`, `True`,
and `False` conflicts.

This preserves the Source Data Primitives v1 behavior while keeping the
user-facing diagnostics reference exact.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 95 out of 95 changed files in this pull request and generated 5 comments.

Comment thread crates/mantle-runtime/src/run/tests/scalar_templates.rs Outdated
Comment thread docs/src/syntax-reference.md Outdated
Comment thread crates/strata/src/language/tests/source_data_primitives.rs Outdated
Comment thread crates/strata/src/language/tests/source_data_primitives.rs Outdated
Comment thread crates/mantle-artifact/src/tests/codec/round_trip.rs Outdated
Derive primitive type and operand expectations from the assigned TypeIds instead of hard-coded table positions. This keeps the tests focused on the source-to-artifact contract while allowing harmless type table ordering changes.

Clarify the syntax reference so string and bytes literal bodies exclude unescaped terminators and escape introducers. Refresh the language-surface assurance marker to point at the new order-independent operand assertion.

Verification:

- cargo +stable fmt --all --check

- cargo +stable test -p mantle-runtime runtime_rejects_loaded_payload_enum_variants_that_collide_with_primitive_value_labels

- cargo +stable test -p mantle-artifact artifact_round_trips_primitive_shapes_and_values

- cargo +stable test -p strata checks_lowers_and_preserves_typed_string_and_bytes_values

- just docs

- cargo +stable test -p strata-mantle-acceptance --test language_surface_assurance evidence_files_and_markers_exist

- just quality

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 95 out of 95 changed files in this pull request and generated 1 comment.

Comment thread crates/mantle-runtime/src/program.rs Outdated
Move the primitive type import into the runtime type-validation module that actually uses it instead of carrying it through the parent program module. This keeps the loaded-program boundary imports explicit while preserving the existing primitive validation behavior.

Verification:

- cargo +stable fmt --all --check

- cargo +stable check -p mantle-runtime --all-targets

- git diff --check origin/main

- just quality
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants