PIOT-CDA-08-001-A: Create / edit module AsyncCoapServerAdapter and connect it into DeviceDataManager

[labbenchstudios]

Description

• Create a new, or edit the existing, Python module named AsyncCoapServerAdapter and within the module a class with the same name. This will provide your CoAP server functionality and host your local resource implementations.
  • NOTE: These instructions make use of the following CoAP library:
    • The aiocoap open source CoAP library, located at: aiocoap. Reference: Amsüss, Christian and Wasilak, Maciej. aiocoap: Python CoAP Library. Energy Harvesting Solutions, 2013–. http://github.com/chrysn/aiocoap/.
  • This card also references the coapthon3 library:
    • The CoAPthon3 open source CoAP library, located at: CoAPthon3. Reference: G.Tanganelli, C. Vallati, E.Mingozzi, "CoAPthon: Easy Development of CoAP-based IoT Applications with Python", IEEE World Forum on Internet of Things (WF-IoT 2015).
• Connect AsyncCoapServerAdapter into DeviceDataManager.

Review the README

• Please see README.md for further information on, and use of, this content.
• License for embedded documentation and source codes: PIOT-DOC-LIC

Estimated effort may vary greatly

• The estimated level of effort for this exercise shown in the 'Estimate' section below is a very rough approximation. The actual level of effort may vary greatly depending on your development and test environment, experience with the requisite technologies, and many other factors.

IMPORTANT NOTE

• The aiocoap specific code samples provided below within this exercise have been tested in a limited environment - non-Linux environments may or may not yield the correct results. You'll also need to use pip to install LinkHeader to retrieve resource discovery results.
  • One way to install LinkHeader using pip is as follows (your environment may require a slightly different command):
  • NOTE: If using virtualenv or venv, be sure to change to activate the virtual environment before executing the command below.

pip install LinkHeader

Actions

NOTE 1: As a reminder, and as mentioned in Chapter 1 of Programming the IoT within the Application configuration section, you may want to consider setting the DEFAULT_CONFIG_FILE_NAME property in ConfigConst.py to the absolute path for PiotConfig.props. While this shouldn't be necessary, as the ConfigUtil.py module (and ConfigUtil class) will attempt to locate it automatically, this will enable both the test classes and the application to find the configuration file, as the relative path will be different for each due to the path delta between the main source branch programmingtheiot and the test branch tests.

NOTE 2: There are two CoAP library dependencies included within your project's requirements.txt file:

• An asynchronous server library using asyncio (and the aiocoap library)
• A synchronous and asynchronous library using threads (and the coapthon3 library)
• For this card (and this lab module's -A named cards, we'll focus on the asynchronous approach using asyncio. This relies upon the aiocoap library.
  • Regardless of which library you choose, the calls into the CoAP server will essentially be the same - just the name of the module will be different. For this exercise, the module name will be AsyncCoapServerAdapter.py, with the class name AsyncCoapServerAdapter.

NOTE 3: Regarding updates to the PiotConfig.props config file: You may need to configure your server to bind to a specific IP address. There are many reasons for this, including some that are security-related. If using aiocoap, please be sure to review the FAQ here: https://aiocoap.readthedocs.io/en/latest/faq.html

Step 1: Create the module, class, and import statements

• Within the programmingtheiot.cda.connection package, create a new (or edit the existing) Python module named AsyncCoapServerAdapter.py.
• Create the class AsyncCoapServerAdapter within the module.
• At the beginning of the module - before the class declaration - you'll need the following import statements (if you're using the existing template, these should already be in place for you):

import logging

import asyncio
import time
import traceback
import threading

import aiocoap
import aiocoap.resource as resource

from aiocoap.resource import Resource

from typing import Optional, Dict, Any
from contextlib import suppress

import programmingtheiot.common.ConfigConst as ConfigConst

from programmingtheiot.common.ConfigUtil import ConfigUtil
from programmingtheiot.common.ResourceNameEnum import ResourceNameEnum

from programmingtheiot.common.IDataMessageListener import IDataMessageListener
from programmingtheiot.cda.connection.handlers.AsyncGetTelemetryResourceHandler import AsyncGetTelemetryResourceHandler
from programmingtheiot.cda.connection.handlers.AsyncGetSystemPerformanceResourceHandler import AsyncGetSystemPerformanceResourceHandler
from programmingtheiot.cda.connection.handlers.AsyncUpdateActuatorResourceHandler import AsyncUpdateActuatorResourceHandler

Step 2: Create the constructor

• Add the __init__() constructor.
  • Include a parameter for dataMsgListener: IDataMessageListener = None, and set a class-scoped instance of self.dataMsgListener to this parameter's reference.
  • NOTE 1: You might also want to add a new method named def setDataMessageListener(self, listener: IDataMessageListener = None): to set the IDataMessageListener reference after initialization. However, if you do, you'll need to re-think how you'll create your internal resource handlers, as they'll need access to the IDataMessageListener to handle callbacks.
  • The constructor initialization should look similar to the following:

	def __init__(self, dataMsgListener = None):
		self.config = ConfigUtil()
		self.dataMsgListener = dataMsgListener
		self.enableConfirmedMsgs = False
		
		self.host = self.config.getProperty(ConfigConst.COAP_GATEWAY_SERVICE, ConfigConst.HOST_KEY, ConfigConst.DEFAULT_HOST)
		self.port = self.config.getInteger(ConfigConst.COAP_GATEWAY_SERVICE, ConfigConst.PORT_KEY, ConfigConst.DEFAULT_COAP_PORT)
		
		self.serverUri = f"coap://{self.host}:{self.port}"

		self.coapServer   = None
		self.rootResource = None
		
		self._serverTask: Optional[asyncio.Task] = None
		self._eventLoopThread: Optional[asyncio.AbstractEventLoop] = None
		self._executionThread: Optional[threading.Thread] = None
		self._shutdownEvent: Optional[asyncio.Event] = None
		self._shutdownFuture: Optional[asyncio.Future] = None

		self._initServer()
		
		logging.info(f"Async CoAP Server configured at {self.serverUri}")

Step 2: Add the requisite helper methods

• Public helper methods
  • Add a setter for registering the callback type that will receive incoming messages from the CoAP server's handlers:

	def setDataMessageListener(self, listener: IDataMessageListener = None) -> bool:
		if listener:
			self.dataMsgListener = listener
			return True
		
		return False

• Internal helper methods:

  • Add the following helper method - it can just implement pass for now:

    • def _initServer(self)

  • Add the following async helper methods - they can all just implement pass for now:

    • async def _runServer(self)
    • async def _runServerTask(self)
    • async def _keepServerRunning(self)
    • async def _shutdownServer(self)

Step 3: Add a public facing method to add resources to the server.

• Recall that CoAP resources look very much like URI paths, so there's some string parsing that will be needed
  • This functionality will take all resource names and add them - in the appropriate sequence - to self.rootResource.
    • NOTE: For the CoAP server to function, it needs resource handlers that will implement the GET, PUT, POST and DELETE request methods for a given resource name (e.g., PIOT/ConstrainedDevice/SystemPerfMsg). The server will need to create these instances and register them as resource handlers or permit an external caller to do so (such as DeviceDataManager. How you choose to implement this is up to you.
  • There are quite a few options for building this functionality. Here's one suggested implementation approach:

	def addResource(self, resourcePath: ResourceNameEnum = None, endName: str = None, resource: Resource = None):
		if resourcePath and resource:
			uriPath = resourcePath.value
			
			if endName:
				uriPath = uriPath + '/' + endName
				
			resourceList = uriPath.split('/')
			
			if not self.rootResource:
				self.rootResource = resource.Site()
				
			logging.info(f"Adding resource to server: {resourceList}")

			try:
				self.rootResource.add_resource(resourceList, resource)
				
			except Exception as e:
				logging.error(f"Failed to add resource to server: {resourceList}")
		else:
			logging.warning(f"No resource provided for path: {uriPath}")

Step 3: Add the public facing methods to start and stop the server

• Add the server START method.
  • This will be called by DeviceDataManager (eventually) from within its startManager() method, provided enableCoapServer is True within PiotConfig.props.
  • This method will make use of some of the class members declared within the constructor, as well as some of the internal methods previously defined (but not yet fully implemented).

	def startServer(self) -> bool:
		if self._serverTask and not self._serverTask.done():
			logging.warning("Server already running. Ignoring start request.")
			return False
		
		if not self.rootResource:
			logging.error("No resources configured. Nothing for server to do.")
			return False
		
		try:
			logging.info(f"Starting Async CoAP Server at {self.serverUri}")

			self._eventLoopThread = asyncio.new_event_loop()
			self._shutdownEvent = asyncio.Event()

			self._executionThread = threading.Thread(
				target = self._runServerTask,
				daemon = True,
				name = "Async-CoAP-Server-Thread")
			
			self._executionThread.start()

			# wait a fraction of a second so server can spin up
			time.sleep(0.5)

			logging.info(f"Async CoAP Server started at {self.serverUri}")

			return True
		
		except Exception as e:
			logging.error(f"Failed to start Async CoAP Server at {self.serverUri}")
			traceback.print_exception(type(e), e, e.__traceback__)
			return False

• Add the server STOP method.
  • This will be called by DeviceDataManager (eventually) from within its stopManager() method, provided enableCoapServer is True within PiotConfig.props and the Async CoAP Server instance is not None.
  • This method will make use of some of the class members declared within the constructor, as well as some of the internal methods previously defined (but not yet fully implemented).

	def stopServer(self) -> bool:
		if not self._eventLoopThread or not self._serverTask:
			logging.warning("Async CoAP Server not yet running. Ignoring stop request.")
			return False

		try:
			logging.info(f"Shutting down Async CoAP Server at {self.serverUri}")

			if self._eventLoopThread.is_running():
				asyncio.run_coroutine_threadsafe(
					self._shutdownServer(),
					self._eventLoopThread)

			for _ in range(5):
				if self._serverTask.done():
					break

				time.sleep(1.0)

			if self._eventLoopThread.is_running():
				self._eventLoopThread.call_soon_threadsafe(self._eventLoopThread.stop)

			logging.info(f"Async CoAP Server shutdown: {self.serverUri}")

			return True
		
		except Exception as e:
			logging.error(f"Failed to shutdown Async CoAP Server at {self.serverUri}")
			traceback.print_exception(type(e), e, e.__traceback__)
			return False

Step 4: Integrate with DeviceDataManager

NOTE: These activities will follow the same pattern described for integrating the MQTT client adapter within the DeviceDataManager, except they'll use the instance of AsyncCoapServerAdapter. Be sure to review PIOT-CDA-06-004 for details.

• Within the config path at the CDA_HOME top level director, edit PiotConfig.props to set enableCoapServer to True.
  • Add (or edit) the enableCoapServer key / value (boolean flag) in PiotConfig.props within the ConstrainedDevice section. Be sure to set it to True
  • Enable (or disable) the CoAP server within the DeviceDataManager constructor by storing a class-scoped boolean named enableCoapServer to determine if DeviceDataManager will host a CoAP server or not. This will be loaded from PiotConfig.props.
• Update DeviceDataManager with a reference to AsyncCoapServerAdapter. This will only need to be set (and subsequently activated as indicated below) if enableCoapServer is true. For this activity, it will need to be true of course.
  • Add the requisite import statement at the beginning of the DeviceDataManager.py module:

from programmingtheiot.cda.connection.AsyncCoapServerAdapter import AsyncCoapServerAdapter

• Create a class-scoped instance of AsyncCoapServerAdapter within the DeviceDataManager constructor called coapServer
  • Add the following code to DeviceDataManager - within the __init__() method (constructor):

self.enableCoapServer = \
	self.configUtil.getBoolean( \
		section = ConfigConst.CONSTRAINED_DEVICE, key = ConfigConst.ENABLE_COAP_SERVER_KEY)

if self.enableCoapServer:
	self.coapServer = AsyncCoapServerAdapter(dataMsgListener = self)

• Edit the startManager() method to include a call to self.coapServer.startServer(). Here's an example of what the implementation might look like:

if self.coapServer:
	self.coapServer.startServer()

• Edit the stopManager() method to include a call to self.coapServer.stopServer(). Here's an example of what the implementation might look like:

if self.coapServer:
	self.coapServer.stopServer()

Estimate (Small = < 2 hrs; Medium = 4 hrs; Large = 8 hrs)

• Medium

Tests

• You can use the Californium client described in PIOT-CFG-08-001 to test your CoAP server running within the CDA.

Configure the CDA to run with CoAP server enabled

• Make sure your CDA's configuration file is updated to enable the CoAP server
  • Update the config file and start the CDA in a separate terminal (or within your IDE)
  • NOTE: The config info below is JUST AN EXAMPLE

#
# CoAP client configuration information
#
[Coap.GatewayService]
credFile       = ./cred/PiotCoapCred.props
certFile       = ./cert/PiotCoapLocalCertFile.pem
host           = localhost
port           = 5683
securePort     = 5684
enableAuth     = False
enableCrypt    = False

#
# CDA specific configuration information
#
[ConstrainedDevice]
deviceLocationID = constraineddevice001
enableSimulator  = True
enableEmulator   = False
enableSenseHAT   = False
enableMqttClient = False
enableCoapServer = True
enableCoapClient = False
enableSystemPerformance = True
enableSensing    = True
enableLogging    = True
pollCycleSecs    = 5
testGdaDataPath  = /tmp/gda-data
testCdaDataPath  = /tmp/cda-data
testEmptyApp     = False
runForever       = True

Configure and run the Californium Client

• Open a separate terminal window on your system to run the Californium client.
  • Change directory to the Californium Tools path (INSTALL_PATH/californium.tools)
  • Run cf-client with a simple discover or get request

CoAP GET Example

cd cf-client/target
java -jar cf-client-3.10.0-SNAPSHOT.jar -m GET coap://localhost:5683

CoAP DISCOVER Example

cd cf-client/target
java -jar cf-client-3.10.0-SNAPSHOT.jar -m GET coap://localhost:5683/.well-known/core

Californium client help

• For a list of supported client parameters, use the following:

java -jar cf-client-3.10.0-SNAPSHOT.jar --help

Results (Initial)

• You should see something similar to the following, depending on how the server is configured:
  • NOTE: The following is an example response from a DISCOVER request

Results (Upon Full Lab Module Completion)

• Once you've completed all the exercises in this lab module, you should see an output that looks similar to the following, depending on how the server is configured:
  • NOTE: The following is an example response from a DISCOVER request when using the aiocoap library.