Skip to content

Refactor/replace uuids with keys#16

Draft
mdondrup wants to merge 41 commits into
ds-wizard:mainfrom
DHP-stottepakke:refactor/replace-uuids-with-keys
Draft

Refactor/replace uuids with keys#16
mdondrup wants to merge 41 commits into
ds-wizard:mainfrom
DHP-stottepakke:refactor/replace-uuids-with-keys

Conversation

@mdondrup

Copy link
Copy Markdown

Added the keys from the Norwegian model and a key lookup layer.

MarekSuchanek and others added 30 commits July 1, 2020 10:55
* Added ORCID integration
* Added compatible KM to README
Fix ethical issues related to reuse of non-reference datasets
Updated minimum required version of compatible knowledge models to 2.6.11
Updated README with the latest link to the DCS Ontology
MarekSuchanek and others added 8 commits September 3, 2025 10:40
* Updated reused dataset name to use reply string value instead FAIRsharing integration
* Updated dependency on KM to 2.7.0
* Update metamodel version to 17.1
- Create src/_keys.j2: builds nested {parentUuid: {keyName: childUuid}} dicts
  from json.key annotations in the KM. Includes fail-loud collision detection.

- Add annotation(entity, key) helper macro to src/macros.j2 for generic
  annotation extraction.

- Update template.json:
  - Bump metamodelVersion from 18.0 to 20
  - Add research.data.no:norway-generic 1.2.0 to allowedPackages

With the updated KM workflow, all in-scope collisions are resolved (0 remaining).
Consumers (_mapping.j2, etc) continue unchanged; migration deferred to follow-up.
@mdondrup
mdondrup marked this pull request as ready for review May 19, 2026 09:33
mdondrup added 2 commits May 22, 2026 15:33
…der toolchain

- rewrote _mapping.j2 to use annotation-based extract_replies navigation instead of UUID-based reply_path lookups
- removed legacy UUID/key mapping files no longer needed: src/_uuids.j2 and src/_keys.j2
- fixed ethical-issues mapping for current KM answer shapes:
 -> handle Yes/No as booleans
 -> map uncertain personal/sensitive-data branches
 -> include legal/ethical free-text details in ethical_issues_description
- added local testing utilities under tools:
 - render.py with strict mode, dump-replies, list-chapters
 - walk_km.py for KM inspection
- installed minimal dsw-document-worker subset in tools/dsw_local

- setup/requirements and usage docs in tools/setup.sh, tools/requirements.txt, README.md
- update render CLI to use --input as primary export flag with --ctx as backward-compatible alias
@mdondrup
mdondrup marked this pull request as draft May 22, 2026 13:56
@korbinib
korbinib force-pushed the refactor/replace-uuids-with-keys branch from 77ecb2f to 03d4059 Compare May 23, 2026 14:10
@korbinib
korbinib force-pushed the refactor/replace-uuids-with-keys branch from af76a70 to f2a393b Compare June 12, 2026 08:33
@MarekSuchanek

Copy link
Copy Markdown
Member

Thanks for submitting this PR. There are a few things that blocks us from merging this such as bad git history clashing with what we have in this repo and including tools. We also have plans to rework our templates more consistently (this one, Horizon Europe, Science Europe, etc.).

Is it OK that we close this PR but use it for futher inspiration? Also, I will check the "tools" and we can think how to incorporate it in our existing tools like TDK... what were the aims there?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants