Add notes from SMDX files to JSON models

[bikeshedder]

The JSON models were missing the notes from the SMDX files. I do consider those very useful as they explain some of the rather cryptic abbreviations.

I quickly hacked together a Python script to copy the missing notes from the SMDX files to the JSON files:

---

#!/bin/python3
import json
from os import listdir
from xml.etree import ElementTree as ET

for fn in listdir('smdx'):
    if not (fn.startswith('smdx_') and fn.endswith('.xml')):
        continue
    model_id = int(fn[5:-4], base=10)
    dom = ET.parse(f'smdx/{fn}').getroot()
    strings = dom.find('strings')
    # model notes
    model_string = strings.find('model')
    model_notes = model_string.findtext('notes')
    if model_notes:
        model_notes = model_notes.strip()
    # point notes
    points_notes = {}
    symbols_notes = {}
    for point in strings.findall('point'):
        point_id = point.get('id')
        point_notes = point.findtext('notes')
        if point_notes:
            point_notes = point_notes.strip()
        if point_notes:
            points_notes[point_id] = point_notes
        for symbol in point.findall('symbol'):
            symbol_id = symbol.get('id')
            symbol_notes = symbol.findtext('notes')
            if symbol_notes:
                symbol_notes = symbol_notes.strip()
            if symbol_notes:
                symbols_notes[(point_id, symbol_id)] = symbol_notes
    if not (model_notes or points_notes):
        continue
    with open(f'json/model_{model_id}.json') as fh:
        jsondoc = json.load(fh)
    if not jsondoc['group'].get('notes', '') and model_notes:
        jsondoc['group']['notes'] = model_notes
    for point in jsondoc['group']['points']:
        point_name = point['name']
        point_notes = points_notes.get(point_name, '')
        if not point.get('notes', '') and point_notes:
            point['notes'] = point_notes
        for symbol in point.get('symbols', []):
            symbol_name = symbol['name']
            symbol_notes = symbols_notes.get((point_name, symbol_name), '')
            if not symbol.get('notes', '') and symbol_notes:
                symbol['notes'] = symbol_notes
    with open(f'json/model_{model_id}.json', 'w') as fh:
        json.dump(jsondoc, fh, indent=4)

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

[cla-bot]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @bikeshedder on file. In order for us to review and merge your code, please contact the project maintainers to get yourself added.

[bikeshedder]

I just realized that the symbol notes are missing as well. I haven't touched those yet so I didn't immediately notice them missing.

FYI. I use the models to generate code for the Rust sunspec crate:

• 📦 Crates.io: https://crates.io/crates/sunspec
• 🐙 GitHub repository: https://github.com/bikeshedder/sunspec
• 📖 Documentation: https://docs.rs/sunspec

[cla-bot]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @bikeshedder on file. In order for us to review and merge your code, please contact the project maintainers to get yourself added.

[bikeshedder]

I just updated the PR and the script in the first message to include the symbol notes.

[Kudrat9]

@bikeshedder Thanks for your work. We've had a lot of discussion about using the 'detail' field in the json models for such notes in the past. But there was no conclusion on how long the detail could be. So, it isn't being used. For now, we think it might be better to add any additional information or notes in our SunSpec DER Model Information Specification: https://sunspec.org/wp-content/uploads/2021/04/SunSpec-DER-Information-Model-Specification-V1-0.pdf

We'll go over the notes from the SMDX models and update the documentation.

[bikeshedder]

@Kudrat9 I find the notes quite useful in generated code. They contain some information that is missing otherwise:
e.g. https://docs.rs/sunspec/latest/sunspec/models/model3/struct.Model3.html

[bikeshedder]

I did sign the CLA quite a while ago, but the bot didn't pick that up.

@Kudrat9 If you don't consider the notes to be useful for the JSON files feel free to close this PR.

[dersecure]

Moved to #261