Skip to content

Remove DataField accessor#377

Merged
elephantum merged 1 commit into
masterfrom
remove-datafield-accessor
Jun 5, 2026
Merged

Remove DataField accessor#377
elephantum merged 1 commit into
masterfrom
remove-datafield-accessor

Conversation

@elephantum
Copy link
Copy Markdown
Contributor

No description provided.

@elephantum elephantum force-pushed the remove-datafield-accessor branch from 7f14da1 to 1f77740 Compare May 21, 2026 07:48
@elephantum elephantum force-pushed the remove-datafield-accessor branch 3 times, most recently from 90c9a77 to 851d298 Compare June 5, 2026 19:46
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes the DataField key accessor and standardizes key mapping to only reference meta-table (primary key) columns, simplifying transform key handling across the meta/compute layers.

Changes:

  • Removed DataField / FieldAccessor types and updated InputSpec.keys / ComputeInput.keys to dict[str, str].
  • Simplified SQL meta aggregation logic to only use meta-table primary keys (no data-table joins for key extraction).
  • Updated tests and the key-mapping design doc to reflect the new “keys must be PK/meta columns” model.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/test_meta_transform_keys.py Updates schema + key mapping usage to no longer rely on DataField.
design-docs/2025-12-key-mapping.md Updates design doc examples/constraints to remove DataField and require PK-based keys.
datapipe/types.py Removes DataField and narrows InputSpec.keys typing to dict[str, str].
datapipe/meta/sql_meta.py Removes DataField handling and data-table join logic from get_agg_cte.
datapipe/meta/base.py Updates transform_idx_to_table_idx typing and mapping semantics to dict[str, str].
datapipe/compute.py Updates ComputeInput.keys typing and removes DataField-based schema logic.
CHANGELOG.md Removes mention of adding DataField (needs an explicit “removed/breaking change” note).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread CHANGELOG.md
@elephantum elephantum merged commit d73b87a into master Jun 5, 2026
29 checks passed
@elephantum elephantum deleted the remove-datafield-accessor branch June 5, 2026 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants