Skip to content

[DwC export]: Implement export cache table creation mechanism #7737

@grantfitzsimmons

Description

@grantfitzsimmons

Goal

Implement the mechanism to create a flattened cache table in the database when an export is run, named after the sanitized mapping name (e.g., dwc_fish).

Background

The cache table enables post-processing pipelines and mirrors the Specify 6 approach. The table must be created or replaced and include a primary key column.

Acceptance Criteria

  • When an export is run, a table named {MappingName} (sanitized for MariaDB identifiers) is created in the database.
  • If the table already exists, it is replaced (CREATE OR REPLACE TABLE or equivalent).
  • The table has a primary key column (<table>Id).
  • Column types are inferred from the mapping's field types.
  • Bulk-load mechanisms are used for performance (e.g., INSERT … SELECT or LOAD DATA INFILE).
  • Table creation uses a transactional approach where possible; orphaned tables are cleaned up on failure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions