-
Notifications
You must be signed in to change notification settings - Fork 40
Open
Description
Goal
Implement the mechanism to create a flattened cache table in the database when an export is run, named after the sanitized mapping name (e.g., dwc_fish).
Background
The cache table enables post-processing pipelines and mirrors the Specify 6 approach. The table must be created or replaced and include a primary key column.
Acceptance Criteria
- When an export is run, a table named
{MappingName}(sanitized for MariaDB identifiers) is created in the database. - If the table already exists, it is replaced (CREATE OR REPLACE TABLE or equivalent).
- The table has a primary key column (
<table>Id). - Column types are inferred from the mapping's field types.
- Bulk-load mechanisms are used for performance (e.g.,
INSERT … SELECTorLOAD DATA INFILE). - Table creation uses a transactional approach where possible; orphaned tables are cleaned up on failure.
Reactions are currently unavailable