CI: Add script to run all tests locally

Author: fhessel

extras/README.md

@@ -2,6 +2,10 @@
 
 This folder contains files that are not directly part of the library.
 
+## CI – Continuous Integration
+
+The [ci](ci) folder contains scripts and data used for automated testing.
+
 ## Documentation
 
 The [docs](docs/) folder contains documentation about the internal structure
@@ -49,4 +53,4 @@ unsigned char example_key_DER[] = {
 };
 unsigned int example_key_DER_len = 608;
 
-```
+```

extras/ci/README.md

@@ -0,0 +1,35 @@
+# Testing and Continuous Integration
+
+This folder contains the scripts and data that is used to run the tests that are specified in the GitHub Action workflows.
+
+The structure of this folder is as follows:
+
+- **apps** contains applications that are used during CI testing
+- **scripts** contains scripts and tools that are part of the workflows
+- **templates** contains project skeletons etc. used for testing
+- **tests** contains test suites written in Python which run against real hardware
+
+## Run Tests Locally
+
+You can (and should) run tests on your local machine before submitting an PR.
+
+System Requirements
+
+- A recent, Linux-based OS (it might work on macOS, too)
+- Python 3 installed (`sudo apt-get install python3` on Debian or Ubuntu)
+- Platform IO on your `PATH` (`source ~/.platformio/penv/bin/activate` if you're using an IDE like VSCode)
+- For the hardware checks: An ESP32 development board connected to your machine and a WiFi access point shared between you and the ESP32
+
+### Build Examples
+
+You can use `extras/ci/scripts/build-example.sh <"wroom"|"wrover"> <example-name>` to build a specific example.
+The main purpose of this is to check that everything still compiles as intended.
+The script will only try to build the example, but it will not flash it.
+The project folder is created in `tmp/<example-name>-<board-name>` in the root of the repositry.
+If you want, you can run `pio run -t upload -e <"wroom"|"wrover">` in these project directories to upload the example to a board.
+
+To build all examples (and to check that everything works), there is a script for manual testing: `extras/ci/scripts/build-example.sh`
+
+### Run Tests on Hardware
+
+(tbd)

extras/ci/scripts/build-all-examples.sh

@@ -0,0 +1,43 @@
+#!/bin/bash
+
+# Find the script and repository location based on the current script location
+SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+REPODIR=$(cd "$(dirname $SCRIPTDIR/../../../..)" && pwd)
+
+TOTAL_TESTS=0
+FAILED_TESTS=0
+OK_TESTS=0
+
+EXAMPLES_SUCCESS=()
+EXAMPLES_FAILURE=()
+for BOARD in wrover wroom; do
+  for EXAMPLENAME in $(ls "$REPODIR/examples"); do
+    if [ -d "$REPODIR/examples/$EXAMPLENAME" ] && [ -f "$REPODIR/examples/$EXAMPLENAME/$EXAMPLENAME.ino" ]; then
+      $SCRIPTDIR/build-example.sh "$BOARD" "$EXAMPLENAME"
+      RC=$?
+      TOTAL_TESTS=$[ TOTAL_TESTS + 1 ]
+      if [[ "$RC" == "0" ]]; then
+        OK_TESTS=$[ OK_TESTS + 1 ]
+        EXAMPLES_SUCCESS+=("$EXAMPLENAME ($BOARD)")
+      else
+        FAILED_TESTS=$[ FAILED_TESTS + 1 ]
+        EXAMPLES_FAILURE+=("$EXAMPLENAME ($BOARD)")
+      fi
+    fi
+  done # example loop
+done # board loop
+
+echo "Summary: $OK_TESTS/$TOTAL_TESTS succeeded"
+echo "-----------------------------------------"
+for exmpl in "${EXAMPLES_SUCCESS[@]}"; do
+  printf "  \u2714 $exmpl\n"
+done
+for exmpl in "${EXAMPLES_FAILURE[@]}"; do
+  printf "  \u274c $exmpl\n"
+done
+if [[ "$FAILED_TESTS" != "0" ]]; then
+  echo "$FAILED_TESTS Tests failed."
+  exit 1
+else
+  echo "Success."
+fi