feat: Onboarding (#740)

Co-authored-by: Koen Kanters <koenkanters94@gmail.com>

Author: Nerivec

.gitignore

@@ -109,4 +109,9 @@ venv.bak/
 secrets.json
 
 zigbee2mqtt/rootfs
-zigbee2mqtt-edge/rootfs
+zigbee2mqtt/Dockerfile
+zigbee2mqtt/build.yaml
+zigbee2mqtt-edge/rootfs
+zigbee2mqtt-edge/Dockerfile
+zigbee2mqtt-edge/build.yaml
+zigbee2mqtt-local/*

README.md

@@ -17,9 +17,6 @@
         <a href="https://discord.gg/dadfWYE">
             <img src="https://img.shields.io/discord/556563650429583360.svg">
         </a>
-        <a href="http://zigbee2mqtt.discourse.group/">
-            <img src="https://img.shields.io/discourse/https/zigbee2mqtt.discourse.group/status.svg">
-        </a>
     </div>
     <h1>Official Zigbee2MQTT Home Assistant add-on</h1>
 </div>
@@ -36,23 +33,15 @@
    - **Zigbee2MQTT** is the stable release that tracks the released versions of Zigbee2MQTT. (**recommended for most users**)
    - **Zigbee2MQTT Edge** tracks the `dev` branch of Zigbee2MQTT such that you can install the edge version if there are features or fixes in the Zigbee2MQTT dev branch that are not yet released.
 1. Click on the add-on and press **Install** and wait till the add-on is installed.
-1. Click on **Configuration**
-   - If you are **not** using the Mosquitto broker add-on fill in your MQTT details (leave empty when using the Mosquitto broker add-on) under the `mqtt` section. Format can be found [here](https://www.zigbee2mqtt.io/guide/configuration/mqtt.html#server-connection), but skip the initial `mqtt:` indent. e.g.: <br>
-     ```yaml
-     server: mqtt://localhost:1883
-     user: my_user
-     password: "my_password"
-     ```
-     Note: If the `password` includes certain special characters (reserved by yaml specification), the enclosing quotes are required. So it is recommended to always quote it when in doubt.
-   - Since Zigbee2MQTT automatically attempts to detect the adapter, you can leave the `serial` section empty for now; we may need it later in step 7.
-   - Click **Save**
-   - **Tip:** it is possible to refer to variables in the Home Assistant `secrets.yaml` file (not the Zigbee2MQTT one!) by using e.g. `password: '!secret mqtt_pass'`
-   - **CAUTION:** settings configured through the add-on configuration page will take precedence over settings in the `configuration.yaml` page (e.g. you set `rtscts: false` in add-on configuration page and `rtscts: true` in `configuration.yaml`, `rtscts: false` will be used).
 1. Start the add-on by going to **Info** and click **Start**
-1. Wait till Zigbee2MQTT starts and press **OPEN WEB UI** to verify Zigbee2MQTT started correctly.
+1. Wait a few seconds and press **OPEN WEB UI**, you will now see the onboarding page. More information about the onboarding can be found [here](https://www.zigbee2mqtt.io/guide/getting-started/#onboarding).
+1. Fill in the desired settings, for most setups changing the following is enough:
+   - Select your adapter under _Found Devices_, this will configure the _Coordinator/Adapter Port/Path_ and _Coordinator/Adapter Type/Stack/Driver_.
+   - Fill in the _Closests WiFi Channel_ to select the most optimal Zigbee channel.
+1. Press **Submit**, Zigbee2MQTT will now start, wait a few seconds and refresh the page. You should now see the Zigbee2MQTT frontend.
    - If it shows `502: Bad Gateway` wait a bit more and refresh the page.
    - If this takes too long (e.g. 2 minutes +) check the **Log** tab to see what went wrong.
-   - In case the add-on fails to start with the following error: `USB adapter discovery error (No valid USB adapter found). Specify valid 'adapter' and 'port' in your configuration.`, we need to fill in the `serial` section (which we skipped in step 5). Format can be found [here](https://www.zigbee2mqtt.io/guide/configuration/adapter-settings.html#adapter-settings), but skip the initial `serial:` indent. e.g.: <br>
+   - In case the add-on fails to start with the following error: `USB adapter discovery error (No valid USB adapter found). Specify valid 'adapter' and 'port' in your configuration.`, we need to fill in the `serial` section. Format can be found [here](https://www.zigbee2mqtt.io/guide/configuration/adapter-settings.html#adapter-settings), but skip the initial `serial:` indent. e.g.: <br>
      ```yaml
      adapter: zstack
      port: /dev/serial/by-id/usb-Texas_Instruments_TI_CC2531_USB_CDC___0X00124B0018ED3DDF-if00

common/rootfs/docker-entrypoint.sh

@@ -47,19 +47,7 @@ else
 fi
 
 export ZIGBEE2MQTT_DATA="$(bashio::config 'data_path')"
-if ! bashio::fs.file_exists "$ZIGBEE2MQTT_DATA/configuration.yaml"; then
-    mkdir -p "$ZIGBEE2MQTT_DATA" || bashio::exit.nok "Could not create $ZIGBEE2MQTT_DATA"
-
-    cat <<EOF > "$ZIGBEE2MQTT_DATA/configuration.yaml"
-version: 4
-homeassistant:
-  enabled: true
-advanced:
-  network_key: GENERATE
-  pan_id: GENERATE
-  ext_pan_id: GENERATE
-EOF
-fi
+mkdir -p "$ZIGBEE2MQTT_DATA" || bashio::exit.nok "Could not create $ZIGBEE2MQTT_DATA"
 
 if bashio::config.has_value 'watchdog'; then
     export Z2M_WATCHDOG="$(bashio::config 'watchdog')"
@@ -68,7 +56,13 @@ fi
 
 export NODE_PATH=/app/node_modules
 export ZIGBEE2MQTT_CONFIG_FRONTEND='{"enabled":true,"port": 8099}'
-export Z2M_ONBOARD_NO_SERVER="1"
+export ZIGBEE2MQTT_CONFIG_HOMEASSISTANT_ENABLED='true'
+export Z2M_ONBOARD_URL='http://0.0.0.0:8099'
+
+if bashio::config.true 'force_onboarding'; then
+    export Z2M_ONBOARD_FORCE_RUN="1"
+    bashio::log.info "Forcing onboard to run"
+fi
 
 if bashio::config.true 'disable_tuya_default_response'; then
     bashio::log.info "Disabling TuYa default responses"

zigbee2mqtt-edge/config.json

@@ -82,7 +82,8 @@
       "baudrate": "int?",
       "rtscts": "bool?"
     },
-    "watchdog": "str?"
+    "watchdog": "str?",
+    "force_onboarding": "bool?"
   },
   "image": "ghcr.io/zigbee2mqtt/zigbee2mqtt-edge-{arch}"
 }

zigbee2mqtt/DOCS.md

@@ -1,19 +1,59 @@
 # Pairing
+
 By default the add-on has `permit_join` set to `false`. To allow devices to join you need to activate this after the add-on has started. You can now use the [built-in frontend](https://www.zigbee2mqtt.io/information/frontend.html) to achieve this. For details on how to enable the built-in frontent see the next section.
 
 # Enabling the built-in frontend
+
 Enable `ingress` to have the frontend available in your UI: **Settings → Add-ons → Zigbee2MQTT → Show in sidebar**. You can find more details about the feature on the [Zigbee2MQTT documentation](https://www.zigbee2mqtt.io/information/frontend.html).
 
 # Configuration
+
+## Onboarding
+
+[Onboarding](https://www.zigbee2mqtt.io/guide/getting-started/#onboarding) allows you to setup Zigbee2MQTT without having to manually enter the details in the add-on configuration page. When starting the add-on with a brand new install (no configuration present), the frontend will show a quick setup page, allowing you to select various settings for Zigbee2MQTT to be able to start.
+
+> [!NOTE]
+> Successful detection of adapters, to select from, may vary based on your setup/network. You may have to enter these [details manually](https://www.zigbee2mqtt.io/guide/configuration/adapter-settings.html#basic-configuration) on the page instead.
+
+> [!TIP]
+> You can force the onboarding to re-run (e.g. changing adapter) using the toggle available in the add-on configuration page (visible after checking `Show unused optional configuration options`). This will force onboarding to run even after you have successfully configured it for the first time. Make sure to disable it once done.
+
+## Manual
+
 Configuration required to startup Zigbee2MQTT is available from the add-on configuration. The rest of the options can be configured via the Zigbee2MQTT frontend.
 
-**CAUTION:** settings configured through the add-on configuration page will take precedence over settings in the `configuration.yaml` page (e.g. you set `rtscts: false` in add-on configuration page and `rtscts: true` in `configuration.yaml`, `rtscts: false` will be used). _If you want to control the entire configuration through YAML, remove them from the add-on configuration page._
+> [!CAUTION]
+> Settings configured through the add-on configuration page will take precedence over settings in the `configuration.yaml` page (e.g. you set `rtscts: false` in add-on configuration page and `rtscts: true` in `configuration.yaml`, `rtscts: false` will be used). _If you want to control the entire configuration through YAML, remove them from the add-on configuration page._
+
+#### Examples for each configuration section
+
+- socat
+  ```yaml
+  enabled: false
+  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
+  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
+  options: "-d -d"
+  log: false
+  ```
+- mqtt
+  ```yaml
+  server: mqtt://localhost:1883
+  user: my_user
+  password: "my_password"
+  ```
+- serial
+  ```yaml
+  adapter: zstack
+  port: /dev/serial/by-id/usb-Texas_Instruments_TI_CC2531_USB_CDC___0X00124B0018ED3DDF-if00
+  ```
 
 # Configuration backup
-The add-on will create a backup of your configuration.yml within your data path: `$DATA_PATH/configuration.yaml.bk`. When upgrading, you should use this to fill in the relevant values into your new config, particularly the network key, to avoid breaking your network and having to repair all of your devices.
-The backup of your configuration is created on add-on startup if no previous backup was found. 
+
+The add-on will create a backup of your configuration.yml within your data path: `$DATA_PATH/configuration.yaml.bk`. When upgrading, you should use this to fill in the relevant values into your new config, particularly the network key, to avoid breaking your network and having to re-pair all of your devices.
+The backup of your configuration is created on add-on startup if no previous backup was found.
 
 # Enabling the watchdog
+
 To automatically restart Zigbee2MQTT in case of a soft failure (like "adapter disconnected"), the watchdog can be used. It can be enabled by adding the following to the add-on configuration:
 
 ```yaml
@@ -23,14 +63,17 @@ watchdog: default
 This will use the default watchdog retry delays of 1min, 5min, 15min, 30min, 60min. Custom delays are also supported, e.g. `watchdog: 5,10,30` will start Zigbee2MQTT with the watchdog's retry delays of 5min, 10min, 30min. For more information about the watchdog, read the [docs](https://www.zigbee2mqtt.io/guide/installation/15_watchdog.html).
 
 # Adding Support for New Devices
+
 If you are interested in adding support for new devices to Zigbee2MQTT see [How to support new devices](https://www.zigbee2mqtt.io/how_tos/how_to_support_new_devices.html).
 
 # Notes
+
 - Depending on your configuration, the MQTT server config may need to include the port, typically `1883` or `8883` for SSL communications. For example, `mqtt://core-mosquitto:1883` for Home Assistant's Mosquitto add-on.
 - To find out which serial ports you have exposed go to **Supervisor → System → Host system → ⋮ → Hardware**
 
 # Socat
-In some cases it is not possible to forward a serial device to the container that zigbee2mqtt runs in. This could be because the device is not physically connected to the machine at all. 
+
+In some cases it is not possible to forward a serial device to the container that zigbee2mqtt runs in. This could be because the device is not physically connected to the machine at all.
 
 Socat can be used to forward a serial device over TCP to zigbee2mqtt. See the [socat man pages](https://linux.die.net/man/1/socat) for more info.
 

zigbee2mqtt/config.json

@@ -85,7 +85,8 @@
       "baudrate": "int?",
       "rtscts": "bool?"
     },
-    "watchdog": "str?"
+    "watchdog": "str?",
+    "force_onboarding": "bool?"
   },
   "image": "ghcr.io/zigbee2mqtt/zigbee2mqtt-{arch}"
 }