ENH: When importing ODIM files, extract the global attributes ‘wmo__i…#385
Open
chfer wants to merge 1 commit into
Open
ENH: When importing ODIM files, extract the global attributes ‘wmo__i…#385chfer wants to merge 1 commit into
chfer wants to merge 1 commit into
Conversation
…d’, ‘node’, ‘wmo__originating_centre’ and ‘wmo__wsi’ from the ODIM ‘/what/source’ attribute. * ADD: Added to xradar/model.py the FM301 global attributes ‘wmo__id’, ‘wmo__originating_centre’ and ‘wmo__wsi’ as required global attributes. Also one proposed OPERA FM301 extension, ‘node’ ,was added. * ADD: In the ODIM backend (‘xradar/io/backends/odim.py’), enhanced the method ‘OdimStore.get_attrs’ for extracting the global attributes ‘wmo__id’, ‘node’, ‘wmo__originating_centre’ and ‘wmo__wsi’ from the ODIM /what/source attribute. Also adding the global attribute ‘source’, which is set to ‘radar’, assuming that ODIM will only carry radar files and no LIDAR files. * TST: Added pytest fixtures in ‘conftest.py’ for accessing extra ODIM files. * TST: In ‘tests/io/test_odim.py’ added tests ‘test_odim_source_global_attributes’, ‘test_parse_odim_source_extensive’, ‘test_parse_odim_source_handles_non_string_input’ and test_parse_odim_source_surrogate_repair_unicodeerror_fallback’. * TST: Adapted test ‘tests/io/test_odim.py::test_open_odim_datatree’ to take into account one extra global attribute (site_name) * DOC: Adapted history.md.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #385 +/- ##
==========================================
+ Coverage 94.20% 94.23% +0.02%
==========================================
Files 29 29
Lines 6417 6449 +32
==========================================
+ Hits 6045 6077 +32
Misses 372 372
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Collaborator
|
Thanks @chfer, that looks like a missing part. Are you up to adding/updating also the writing part? If not we can defer that to a follow-up PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR extends the ODIM backend to extract additional global attributes from the
/what/sourcemetadata field when importing ODIM files.The extracted values are mapped onto the following FM301 global attributes in the xradar memory model:
wmo__idsite_namenodewmo__originating_centrewmo__wsiDetails
The ODIM
/what/sourcestring is parsed as key-value metadata and used to populate dataset global attributes during import.This change also handles source strings containing non-ASCII station names encoded through surrogate-escaped UTF-8 byte sequences, so values such as for example
Sürgavereare preserved correctly.Tests
Added coverage to verify:
/what/sourceinto global attributes