Edit YAML and JSON using "wilfred config"

[vilhelmprytz]

Is your feature request related to a problem? Please describe.
YAML and JSON configs are not editable by Wilfred.

Describe the solution you'd like
Ability to edit them using wilfred config <name>.

Describe alternatives you've considered
None.

Additional context
Related to #27.

[vilhelmprytz]

This is some background information.

Example of .yml file to read and write to (config.yml for BungeeCord).

connection_throttle: 4000
connection_throttle_limit: 3
timeout: 30000
stats: 948c0654-4a7a-473c-abfb-b07b37cfb3a2
permissions:
  default:
  - bungeecord.command.server
  - bungeecord.command.list
  admin:
  - bungeecord.command.alert
  - bungeecord.command.end
  - bungeecord.command.ip
  - bungeecord.command.reload
network_compression_threshold: 256
servers:
  lobby:
    motd: '&1Just another BungeeCord - Forced Host'
    address: localhost:25565
    restricted: false
player_limit: -1
prevent_proxy_connections: false
server_connect_timeout: 5000
remote_ping_timeout: 5000
forge_support: false
remote_ping_cache: -1
log_commands: false
log_pings: true
ip_forward: false
disabled_commands:
- disabledcommandhere
groups:
  md_5:
  - admin
listeners:
- query_port: 25568
  bind_local_address: true
  host: 0.0.0.0:25568
  motd: '&1Another Bungee server'
  max_players: 1
  force_default_server: false
  tab_size: 60
  forced_hosts:
    pvp.md-5.net: pvp
  tab_list: GLOBAL_PING
  ping_passthrough: false
  query_enabled: false
  proxy_protocol: false
  priorities:
  - lobby
online_mode: true

This is how Wilfred represents these values within the CLI.

╒═══════════════╤═══════════════════════════════════════╤═════════════════════════════════════════╕
│ Config File   │ Setting                               │ Value                                   │
╞═══════════════╪═══════════════════════════════════════╪═════════════════════════════════════════╡
│ config.yml    │ connection_throttle                   │ 4000                                    │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ connection_throttle_limit             │ 3                                       │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ timeout                               │ 30000                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ stats                                 │ 948c0654-4a7a-473c-abfb-b07b37cfb3a2    │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ permissions/default/0                 │ bungeecord.command.server               │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ permissions/default/1                 │ bungeecord.command.list                 │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ permissions/admin/0                   │ bungeecord.command.alert                │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ permissions/admin/1                   │ bungeecord.command.end                  │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ permissions/admin/2                   │ bungeecord.command.ip                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ permissions/admin/3                   │ bungeecord.command.reload               │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ network_compression_threshold         │ 256                                     │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ servers/lobby/motd                    │ &1Just another BungeeCord - Forced Host │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ servers/lobby/address                 │ localhost:25565                         │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ servers/lobby/restricted              │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ player_limit                          │ -1                                      │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ prevent_proxy_connections             │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ server_connect_timeout                │ 5000                                    │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ remote_ping_timeout                   │ 5000                                    │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ forge_support                         │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ remote_ping_cache                     │ -1                                      │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ log_commands                          │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ log_pings                             │ True                                    │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ ip_forward                            │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ disabled_commands/0                   │ disabledcommandhere                     │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ groups/md_5/0                         │ admin                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/query_port                │ 25568                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/bind_local_address        │ True                                    │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/host                      │ 0.0.0.0:25568                           │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/motd                      │ &1Another Bungee server                 │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/max_players               │ 1                                       │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/force_default_server      │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/tab_size                  │ 60                                      │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/forced_hosts/pvp.md-5.net │ pvp                                     │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/tab_list                  │ GLOBAL_PING                             │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/ping_passthrough          │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/query_enabled             │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/proxy_protocol            │ False                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ listeners/0/priorities/0              │ lobby                                   │
├───────────────┼───────────────────────────────────────┼─────────────────────────────────────────┤
│ config.yml    │ online_mode                           │ True                                    │
╘═══════════════╧═══════════════════════════════════════╧═════════════════════════════════════════╛

And this is how the parser looks like (that translates the .yml to a dictionary which is then displayed as a table like above).

def yaml_read(path):  # this function should be refactored later on!!!
    with open(path) as f:
        _raw = yaml.load(f.read(), Loader=yaml.FullLoader)

    _reformatted = {}

    def _iterate_dict(data, _name):
        """
        iterates a dictionary and continues to iterate the value of the dict if the
        type is dictionary, list or tuple (recursion). Adds the dictionary key and value
        to the _reformatted dict if it's a string, integer or boolean
        """

        _def = _name
        for k, v in data.items():
            _name = f"{_def}/{k}"
            if type(v) in [dict]:
                _iterate_dict(v, _name)
            if type(v) in [list, tuple]:
                _iterate_list(v, _name)

            if type(v) in [str, int, bool]:
                _reformatted[f"{_def}/{k}"] = v

    def _iterate_list(data, _name):
        """
        iterates a list and continues to iterate the values of the list if the
        type is dictionary, list or tuple (recursion). Adds the list value
        to the _reformatted dict if it's a string, integer or boolean
        """

        _def = _name

        i = 0
        for x in data:
            _name = f"{_def}/{i}"
            if type(x) in [dict]:
                _iterate_dict(x, _name)
            if type(x) in [list, tuple]:
                _iterate_list(x, _name)

            if type(x) in [str, int, bool]:
                _reformatted[f"{_def}/{i}"] = x

            i = i + 1

    for k, v in _raw.items():
        _name = k

        if type(v) in [dict]:
            _iterate_dict(v, _name)
        if type(v) in [list, tuple]:
            _iterate_list(v, _name)

        if type(v) in [str, int, bool]:
            _reformatted[k] = v

    return _reformatted

This is how the dictionary/JSON is represented when parsed by the Python YAML module (before my parser translates the settings into a CLI-friendly format), i.e. the _raw variable in the function above.

{
  "connection_throttle": 4000,
  "connection_throttle_limit": 3,
  "timeout": 30000,
  "stats": "948c0654-4a7a-473c-abfb-b07b37cfb3a2",
  "permissions": {
    "default": [
      "bungeecord.command.server",
      "bungeecord.command.list"
    ],
    "admin": [
      "bungeecord.command.alert",
      "bungeecord.command.end",
      "bungeecord.command.ip",
      "bungeecord.command.reload"
    ]
  },
  "network_compression_threshold": 256,
  "servers": {
    "lobby": {
      "motd": "&1Just another BungeeCord - Forced Host",
      "address": "localhost:25565",
      "restricted": false
    }
  },
  "player_limit": -1,
  "prevent_proxy_connections": false,
  "server_connect_timeout": 5000,
  "remote_ping_timeout": 5000,
  "forge_support": false,
  "remote_ping_cache": -1,
  "log_commands": false,
  "log_pings": true,
  "ip_forward": false,
  "disabled_commands": [
    "disabledcommandhere"
  ],
  "groups": {
    "md_5": [
      "admin"
    ]
  },
  "listeners": [
    {
      "query_port": 25568,
      "bind_local_address": true,
      "host": "0.0.0.0:25568",
      "motd": "&1Another Bungee server",
      "max_players": 1,
      "force_default_server": false,
      "tab_size": 60,
      "forced_hosts": {
        "pvp.md-5.net": "pvp"
      },
      "tab_list": "GLOBAL_PING",
      "ping_passthrough": false,
      "query_enabled": false,
      "proxy_protocol": false,
      "priorities": [
        "lobby"
      ]
    }
  ],
  "online_mode": true
}

And here comes the problem: the user specifies which variable to edit (they use the format settings are displayed as in the CLI), but Wilfred has to match this "search" variable to a nested data structure within the basic YAML dictionary.

The write function looks like this:

def yaml_write(path, key, value):

The path is the path of the config file, the key is the setting-name as it is shown in the CLI (e.g. listeners/0/host) and the value is the new value. The key can be split into a list so that it is easier to query for:

["listeners", 0, "host"]

[vilhelmprytz]

This is the file where the YAML parsing occurs wilfred/parser/yaml.py.

[vilhelmprytz]

Will skip this to next release.