Skip to content

Add explicit output key mapping for batch transforms#383

Open
bobokvsky wants to merge 8 commits into
masterfrom
fix-meta-transform-kets
Open

Add explicit output key mapping for batch transforms#383
bobokvsky wants to merge 8 commits into
masterfrom
fix-meta-transform-kets

Conversation

@bobokvsky
Copy link
Copy Markdown
Collaborator

Summary

This PR extends transform key mapping to outputs and fixes cleanup behavior for aliased keys.

  • Add OutputSpec / ComputeOutput.keys so transform keys can be explicitly mapped to output table primary keys.
  • Make BatchTransformStep use explicit ComputeInput and ComputeOutput specs instead of accepting raw DataTable objects in compute-level APIs.
  • Fix incomplete transform keys generated by SQL metadata aggregation and output cleanup when output PK names differ from transform key names.
  • Add regression coverage for aliased input/output keys, composite aliases, duplicated physical id columns, multiple outputs, updates, and deletes.
  • Fix macOS local path expectations in filedir tests and update tqdm-loggable to remove Python 3.12 deprecation warnings.

@bobokvsky bobokvsky changed the title Fix meta transform kets Add explicit output key mapping for batch transforms May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant