Enable nested to-many in Workbench by sharadsw · Pull Request #6216 · specify/specify7

sharadsw · 2025-02-10T18:43:02Z

Warning

This PR is based on #5417. Use either a db that was used to test #5417 or create a new db

This PR enables nested to many in the data mapper and fixes some automated tests. The underlying backend support for nested to many is in #5417.

For reference:
Nested to-manys in the workbench are mapping lines that involve multiple to-many relationships. For example: Collection Object -> Determination (to many) -> DeterminationCitation (to-many) -> ...

Checklist

Self-review the PR after opening it to make sure the changes look good and
self-explanatory (or properly documented)
Add automated tests
Add relevant issue to release milestone
Add relevant documentation (Tester - Dev)
Add a reverse migration if a migration is present in the PR

Testing instructions

Create a Workbench dataset
Map columns that include nested to-manys
- Examples:
  - CO -> Determination -> DeterminationCitation (see above)
  - CO -> Preparation -> PreparationProperty
  - CO -> Preparation -> PreparationAttribute
Enter data
Test upload
Verify data was uploaded correctly (you can verify through queries)
Test rollback

Triggered by 82033cd on branch refs/heads/issue-2331

lexiclevenger

Test upload
Verify data was uploaded correctly (you can verify through queries)
Test rollback

Looks good! I tested CO -> Determination -> DeterminationCitation and CO -> Preparation -> PreparationAttribute

emenslin

Test upload
Verify data was uploaded correctly (you can verify through queries)
Test rollback

Looks good, it seems like the RS owner issue was reintroduced in this PR though.

Issue branch:

03-05_14.14.mp4

Production:

03-05_14.20.mp4

sharadsw · 2025-03-06T21:32:57Z

@emenslin Should be fixed now. The branch didn't have the latest production changes

emenslin

Test upload
Verify data was uploaded correctly (you can verify through queries)
Test rollback

Looks good now!

grantfitzsimmons · 2025-03-28T19:48:42Z

KU Mammals and Herpetology needs this for tracking preservation chain

Triggered by f0822bf on branch refs/heads/issue-2331

alesan99

Testing instructions

Test upload
Verify data was uploaded correctly (you can verify through queries)
Test rollback

I also tried CO -> DNA Sequence -> DNA Sequencing Run. Looks good! 👍👍👍

Triggered by f27581b on branch refs/heads/issue-2331

* Enable trees in queries * Use query construct code from #4929 * Update unit test * Remove param_count * Display tree name in query error * Update TreeRankQuery to fix implicit ORs - see: #6196 (comment) * Allow removing last row in Batch Edit * Make a missing rank info dialog which proceeds to dataset creation * Augment tree queries with missing ranks for batch edit * Lint code with ESLint and Prettier Triggered by 65f7d21 on branch refs/heads/issue-6127 * Adjust apply_batch_edit_pack for multiple trees * Add a discipline type in tests * Un-enforce TreeRankRecord in upload plan * Add loading action to missing ranks dialog * Move table name to same line in missing fields dialog * Handle case when rank name has spaces * Add a close button to missing ranks dialog * Fix frontend missing field calculation * Fix frontend to many tree error * Lint code with ESLint and Prettier Triggered by 9407376 on branch refs/heads/issue-6127 * Restrict to manys only for tree fields * Fix multiple rank in row error * Fix navigator * Fix tests * Group missing ranks by tree * Lint code with ESLint and Prettier Triggered by 7f2149d on branch refs/heads/issue-6127 * Pass filtered treedef ids to the backend - Adds checkboxes to tree names in missing ranks dialog - Splits the main batch edit file into 4 smaller files * Lint code with ESLint and Prettier Triggered by 1c29ec9 on branch refs/heads/issue-6127 * Filter trees used when rewriting batch edit dataset * Fix tests * Use TreeRankRecord in upload plan * Remove unused string * Fix visual order - For multiple trees, columns will be grouped by tree first * Revert "Fix visual order" This reverts commit a8a2ad6. * Flag to-many in tree only queries * Lint code with ESLint and Prettier Triggered by f36521c on branch refs/heads/issue-6127 * Enable nested to-many in Workbench (#6216) * Enable nested to-many in Workbench * Update test * Add upload plan changes * Update tests * Lint code with ESLint and Prettier Triggered by 82033cd on branch refs/heads/issue-2331 * Fix tests * Lint code with ESLint and Prettier Triggered by f0822bf on branch refs/heads/issue-2331 * Lint code with ESLint and Prettier Triggered by f27581b on branch refs/heads/issue-2331 * Lint code with ESLint and Prettier Triggered by cc1f85b on branch refs/heads/issue-6127 * Check for lowercase tree table names when rewriting tree rank row plan * Handle None rank * Fix tree column order * Fix tests * Revert back to sort columns

* Enable trees in queries * Use query construct code from #4929 * Update unit test * Enable nested to-many in Workbench * Update test * Remove param_count * Display tree name in query error * Add upload plan changes * Update tests * Lint code with ESLint and Prettier Triggered by 82033cd on branch refs/heads/issue-2331 * Update TreeRankQuery to fix implicit ORs - see: #6196 (comment) * Allow removing last row in Batch Edit * Make a missing rank info dialog which proceeds to dataset creation * Augment tree queries with missing ranks for batch edit * Lint code with ESLint and Prettier Triggered by 65f7d21 on branch refs/heads/issue-6127 * Adjust apply_batch_edit_pack for multiple trees * Add a discipline type in tests * Enable relationships * Lint code with ESLint and Prettier Triggered by 350ee9c on branch refs/heads/issue-6126 * Enable data mapper and batch edit preferences * Fix localizations * Consider remote to ones as to many in upload plan * Add remote to ones method * Un-enforce TreeRankRecord in upload plan * Add loading action to missing ranks dialog * Move table name to same line in missing fields dialog * Handle case when rank name has spaces * Add a close button to missing ranks dialog * Fix frontend missing field calculation * Fix frontend to many tree error * Lint code with ESLint and Prettier Triggered by 9407376 on branch refs/heads/issue-6127 * Restrict to manys only for tree fields * Avoid cloning to-ones when committing - This was caused because we treat remote to-ones as to-many in the upload plan (affects COGs) * Fix to many for tree in relationships * Change revert to rollback in pref localization * Use TreeRankRecord in upload plan * Fix multiple rank in row error * Fix multiple rank in row error * Fix navigator * Fix tests * Group missing ranks by tree * Lint code with ESLint and Prettier Triggered by 7f2149d on branch refs/heads/issue-6127 * Pass filtered treedef ids to the backend - Adds checkboxes to tree names in missing ranks dialog - Splits the main batch edit file into 4 smaller files * Lint code with ESLint and Prettier Triggered by 1c29ec9 on branch refs/heads/issue-6127 * Filter trees used when rewriting batch edit dataset * Fix tests * Use TreeRankRecord in upload plan * Remove unused string * Fix visual order - For multiple trees, columns will be grouped by tree first * Revert "Fix visual order" This reverts commit a8a2ad6. * Fix tests * Lint code with ESLint and Prettier Triggered by f0822bf on branch refs/heads/issue-2331 * Handle (any rank) mapping for Batch Edit upload plans * Lint code with ESLint and Prettier Triggered by 3945527 on branch refs/heads/issue-6126 * Disable spauditlog for BE * Add title when batch edit is disabled * Fix deleted cells for many-to-one dependents * Use variant permissions for creating record sets * Fix undefined name error - Using tables doesn't work when the data hasn't loaded correctly * Disable changing batch edit prefs after upload * Add validation error for scope change * Add localization for other WB errors * Lint code with ESLint and Prettier Triggered by 34a3459 on branch refs/heads/issue-6126 * Remove description for null record - Removed for UX reasons. Users do not need to manually remove null record strings * Ensure at least 1 to-many column gets added to batch edit datasets * Flag to-many in tree only queries * Lint code with ESLint and Prettier Triggered by f36521c on branch refs/heads/issue-6127 * Lint code with ESLint and Prettier Triggered by 0647fa6 on branch refs/heads/issue-6126 * Enable nested to-many in Workbench (#6216) * Enable nested to-many in Workbench * Update test * Add upload plan changes * Update tests * Lint code with ESLint and Prettier Triggered by 82033cd on branch refs/heads/issue-2331 * Fix tests * Lint code with ESLint and Prettier Triggered by f0822bf on branch refs/heads/issue-2331 * Lint code with ESLint and Prettier Triggered by f27581b on branch refs/heads/issue-2331 * Lint code with ESLint and Prettier Triggered by cc1f85b on branch refs/heads/issue-6127 * Disable editing any rank tree relationships * Check for lowercase tree table names when rewriting tree rank row plan * Lint code with ESLint and Prettier Triggered by 633a7da on branch refs/heads/issue-6126 * Handle None rank * Batch edit: Disable editing dataset after rollback (#6428) * Add rolledback to SpDataset * Lint code with ESLint and Prettier Triggered by 7d2a86d on branch refs/heads/issue-6390 * Add text to indicate dataset cannot be edited * Make hot columns readonly based on context * Lint code with ESLint and Prettier Triggered by dcbb593 on branch refs/heads/issue-6390 * Reorder migration * Fix tree column order * Fix tests * Fix tests * Upgrade celery version (#6437) * Add rolledback to SpDataset * Lint code with ESLint and Prettier Triggered by 7d2a86d on branch refs/heads/issue-6390 * Add text to indicate dataset cannot be edited * Make hot columns readonly based on context * Lint code with ESLint and Prettier Triggered by dcbb593 on branch refs/heads/issue-6390 * Reorder migration * Upgrade celery and its dependencies * Revert back to sort columns * Enable matched and changed when readonly * Add missing import * Re-add lost code

* Enable trees in queries * Use query construct code from #4929 * Update unit test * Enable nested to-many in Workbench * Update test * Remove param_count * Display tree name in query error * Add upload plan changes * Update tests * Lint code with ESLint and Prettier Triggered by 82033cd on branch refs/heads/issue-2331 * Update TreeRankQuery to fix implicit ORs - see: #6196 (comment) * Allow removing last row in Batch Edit * Make a missing rank info dialog which proceeds to dataset creation * Augment tree queries with missing ranks for batch edit * Lint code with ESLint and Prettier Triggered by 65f7d21 on branch refs/heads/issue-6127 * Adjust apply_batch_edit_pack for multiple trees * Add a discipline type in tests * Enable relationships * Lint code with ESLint and Prettier Triggered by 350ee9c on branch refs/heads/issue-6126 * Enable data mapper and batch edit preferences * Fix localizations * Consider remote to ones as to many in upload plan * Add remote to ones method * Un-enforce TreeRankRecord in upload plan * Add loading action to missing ranks dialog * Move table name to same line in missing fields dialog * Handle case when rank name has spaces * Add a close button to missing ranks dialog * Fix frontend missing field calculation * Fix frontend to many tree error * Lint code with ESLint and Prettier Triggered by 9407376 on branch refs/heads/issue-6127 * Restrict to manys only for tree fields * Avoid cloning to-ones when committing - This was caused because we treat remote to-ones as to-many in the upload plan (affects COGs) * Fix to many for tree in relationships * Change revert to rollback in pref localization * Use TreeRankRecord in upload plan * Fix multiple rank in row error * Fix multiple rank in row error * Fix navigator * Fix tests * Group missing ranks by tree * Lint code with ESLint and Prettier Triggered by 7f2149d on branch refs/heads/issue-6127 * Pass filtered treedef ids to the backend - Adds checkboxes to tree names in missing ranks dialog - Splits the main batch edit file into 4 smaller files * Lint code with ESLint and Prettier Triggered by 1c29ec9 on branch refs/heads/issue-6127 * Filter trees used when rewriting batch edit dataset * Fix tests * Use TreeRankRecord in upload plan * Remove unused string * Fix visual order - For multiple trees, columns will be grouped by tree first * Revert "Fix visual order" This reverts commit a8a2ad6. * Fix tests * Lint code with ESLint and Prettier Triggered by f0822bf on branch refs/heads/issue-2331 * Handle (any rank) mapping for Batch Edit upload plans * Lint code with ESLint and Prettier Triggered by 3945527 on branch refs/heads/issue-6126 * Disable spauditlog for BE * Add title when batch edit is disabled * Fix deleted cells for many-to-one dependents * Use variant permissions for creating record sets * Fix undefined name error - Using tables doesn't work when the data hasn't loaded correctly * Disable changing batch edit prefs after upload * Add validation error for scope change * Add localization for other WB errors * Lint code with ESLint and Prettier Triggered by 34a3459 on branch refs/heads/issue-6126 * Remove description for null record - Removed for UX reasons. Users do not need to manually remove null record strings * Ensure at least 1 to-many column gets added to batch edit datasets * Flag to-many in tree only queries * Lint code with ESLint and Prettier Triggered by f36521c on branch refs/heads/issue-6127 * Lint code with ESLint and Prettier Triggered by 0647fa6 on branch refs/heads/issue-6126 * Enable nested to-many in Workbench (#6216) * Enable nested to-many in Workbench * Update test * Add upload plan changes * Update tests * Lint code with ESLint and Prettier Triggered by 82033cd on branch refs/heads/issue-2331 * Fix tests * Lint code with ESLint and Prettier Triggered by f0822bf on branch refs/heads/issue-2331 * Lint code with ESLint and Prettier Triggered by f27581b on branch refs/heads/issue-2331 * Lint code with ESLint and Prettier Triggered by cc1f85b on branch refs/heads/issue-6127 * Disable editing any rank tree relationships * Add rolledback to SpDataset * Lint code with ESLint and Prettier Triggered by 7d2a86d on branch refs/heads/issue-6390 * Add text to indicate dataset cannot be edited * Make hot columns readonly based on context * Lint code with ESLint and Prettier Triggered by dcbb593 on branch refs/heads/issue-6390 * Check for lowercase tree table names when rewriting tree rank row plan * Lint code with ESLint and Prettier Triggered by 633a7da on branch refs/heads/issue-6126 * Reorder migration * Handle None rank * Fix result cell link in Batch Edit and Workbench * Lint code with ESLint and Prettier Triggered by 4b40eec on branch refs/heads/issue-6164 * Remove auto height * Remove duplicate imports * Lint code with ESLint and Prettier Triggered by 19e6c3e on branch refs/heads/issue-6164

Enable nested to-many in Workbench

f7c30e1

sharadsw mentioned this pull request Feb 10, 2025

Support nested -to-many relationships in WorkBench #2331

Closed

sharadsw added this to the 7.10.1 milestone Feb 12, 2025

sharadsw added 4 commits February 12, 2025 15:41

Update test

f6b6562

Add upload plan changes

0fa6fc8

Update tests

82033cd

Lint code with ESLint and Prettier

a56c65b

Triggered by 82033cd on branch refs/heads/issue-2331

sharadsw mentioned this pull request Feb 14, 2025

Batch edit for multiple trees #6196

Merged

19 tasks

sharadsw changed the base branch from issue-5413 to issue-6127 February 26, 2025 18:42

sharadsw and others added 3 commits February 26, 2025 14:01

Merge remote-tracking branch 'origin/issue-6127' into issue-2331

e811142

Merge branch 'issue-6127' into issue-2331

06a4cc6

Merge branch 'issue-6127' into issue-2331

f1fa36e

sharadsw requested review from a team March 3, 2025 20:30

sharadsw marked this pull request as ready for review March 3, 2025 21:09

CarolineDenis approved these changes Mar 4, 2025

View reviewed changes

lexiclevenger approved these changes Mar 5, 2025

View reviewed changes

lexiclevenger requested a review from a team March 5, 2025 18:42

emenslin requested changes Mar 5, 2025

View reviewed changes

github-project-automation bot moved this from 📋Back Log to Dev Attention Needed in General Tester Board Mar 5, 2025

Merge branch 'issue-6127' into issue-2331

8574c50

sharadsw requested a review from emenslin March 6, 2025 21:32

emenslin approved these changes Mar 10, 2025

View reviewed changes

emenslin requested a review from a team March 10, 2025 17:47

sharadsw modified the milestones: 7.10.2, 7.10.3 Mar 11, 2025

sharadsw added 2 commits March 28, 2025 16:11

Merge remote-tracking branch 'origin/issue-6127' into issue-2331

f713380

Fix tests

f0822bf

Lint code with ESLint and Prettier

8f3891a

Triggered by f0822bf on branch refs/heads/issue-2331

CarolineDenis requested a review from acwhite211 April 1, 2025 13:49

alesan99 approved these changes Apr 1, 2025

View reviewed changes

acwhite211 approved these changes Apr 1, 2025

View reviewed changes

sharadsw added 3 commits April 15, 2025 13:16

Merge branch 'issue-6127' into issue-2331

2367a75

Merge branch 'issue-6127' into issue-2331

f27581b

Lint code with ESLint and Prettier

072b0dc

Triggered by f27581b on branch refs/heads/issue-2331

sharadsw merged commit cc1f85b into issue-6127 Apr 15, 2025
12 checks passed

github-project-automation bot moved this from Dev Attention Needed to ✅Done in General Tester Board Apr 15, 2025

sharadsw deleted the issue-2331 branch April 15, 2025 17:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable nested to-many in Workbench#6216

Enable nested to-many in Workbench#6216
sharadsw merged 15 commits intoissue-6127from
issue-2331

sharadsw commented Feb 10, 2025 •

edited

Loading

Uh oh!

lexiclevenger left a comment

Uh oh!

emenslin left a comment

Uh oh!

sharadsw commented Mar 6, 2025

Uh oh!

emenslin left a comment

Uh oh!

grantfitzsimmons commented Mar 28, 2025

Uh oh!

alesan99 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

sharadsw commented Feb 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Testing instructions

Uh oh!

lexiclevenger left a comment

Choose a reason for hiding this comment

Uh oh!

emenslin left a comment

Choose a reason for hiding this comment

Uh oh!

sharadsw commented Mar 6, 2025

Uh oh!

emenslin left a comment

Choose a reason for hiding this comment

Uh oh!

grantfitzsimmons commented Mar 28, 2025

Uh oh!

alesan99 left a comment

Choose a reason for hiding this comment

Testing instructions

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

sharadsw commented Feb 10, 2025 •

edited

Loading