[FEATURE]: remove hard requirement of Circle+ during startup, or in case of migration

[degerrit]

Describe the solution you are looking for

Currently the python-plugwise-usb integration require the Circle+ to be reachable during initial installation AND during every Home Assistant restart.

I see a few reasons why this should not be necessary :-)

1. in case of an unreachable or failing Circle+, it will be hit or miss whether you can still get data from all other Circles upon the next planned or unplanned Home Assistant restart, upgrade, etc.. Many people have invested hundreds of bucks in Circles, which become essentially useless in Home Assistant when the Circle+ is unreachable or dies.
2. it's more robust. Both Plugwise Source and cyberjunky's deprecated DomotiGa software, prove that it isn't necessary: I can get data from all my other Circle's in both systems without the Circle+ (and have for several months already!)
3. in a more tricky one, as a bonus feature, it could allow people with existing Plugwise networks to migrate to Home Assistant, keeping their network (this one means that it would probably be necessary to input the Circle+/network name manually?). (Note: for me, my Circle+ lived long enough that I could start the initial setup, and was able to create my devices)

Is your feature request related to a problem? (optional)

Yes, my Circle+ is essentially more often broken than it is not (relay clicks endlessly on a bad day).

Describe alternatives you've considered

For me: replacing all my plugs with Ikea INSPELNING or HomeWizard plugs has come to mind, or looking out for a new secondhand Circle+ if I can get one cheap, but ugh: migration.

What type of installation are you running?

Home Assistant OS

How did you install python-plugwise-usb (or plugwise_usb) for short

HA using HACS

What firmware version is your Plugwise product at?

2011-06-27 firmware

Your setup

None

Additional context

Here is some logging from the docker container where it keeps looking for the Circle+, despite other Circles being available:

2024-12-31 15:00:09.737 DEBUG (MainThread) [custom_components.plugwise_usb] Connect to USB-Stick
2024-12-31 15:00:09.738 DEBUG (SyncWorker_2) [plugwise_usb.controller] Open USB serial connection to Plugwise USB stick
2024-12-31 15:00:09.738 DEBUG (SyncWorker_2) [plugwise_usb.connections.serial] Open serial port /dev/ttyUSB0
2024-12-31 15:00:09.747 DEBUG (SyncWorker_2) [plugwise_usb.connections.serial] Successfully connected to serial port /dev/ttyUSB0
2024-12-31 15:00:09.747 DEBUG (SyncWorker_2) [plugwise_usb.controller] Starting message controller threads...
2024-12-31 15:00:09.750 DEBUG (SyncWorker_2) [plugwise_usb.controller] All message controller threads started
2024-12-31 15:00:09.751 DEBUG (MainThread) [custom_components.plugwise_usb] Initialize USB-stick
2024-12-31 15:00:09.751 DEBUG (SyncWorker_6) [plugwise_usb] Send init request to Plugwise Zigbee stick
2024-12-31 15:00:09.752 DEBUG (SyncWorker_6) [plugwise_usb.controller] Queue StickInitRequest to be send to  with retry counter 0 and priority 2
2024-12-31 15:00:09.752 INFO (send_messages_thread) [plugwise_usb.controller] Send StickInitRequest to  using seq_id b'0000'
2024-12-31 15:00:09.753 DEBUG (serial_writer_thread) [plugwise_usb.connections] Sending StickInitRequest to plugwise stick (b'\x05\x05\x03\x03000AB43C\r\n')
2024-12-31 15:00:09.778 DEBUG (serial_reader_thread) [plugwise_usb.parser] Feed data: b'\x05\x05\x03\x030000003000C19A5C\r\n\x05\x05\x03\x0300110030000D6F0000B81C700101B90D6F0000998BED09B9FFCB5D\r\n\x83'
2024-12-31 15:00:09.778 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'\x05\x05\x03\x030000003000C19A5C\r\n\x05\x05\x03\x0300110030000D6F0000B81C700101B90D6F0000998BED09B9FFCB5D\r\n\x83'
2024-12-31 15:00:09.778 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'\x05\x05\x03\x030000003000C19A5C\r\n\x05\x05\x03\x0300110030000D6F0000B81C700101B90D6F0000998BED09B9FFCB5D\r\n\x83')
2024-12-31 15:00:09.778 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message header found at index 0
2024-12-31 15:00:09.778 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message footer (b'\r\n') in (b'\x05\x05\x03\x030000003000C19A5C\r\n\x05\x05\x03\x0300110030000D6F0000B81C700101B90D6F0000998BED09B9FFCB5D\r\n\x83')
2024-12-31 15:00:09.778 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message footer found at index 20
2024-12-31 15:00:09.779 DEBUG (serial_reader_thread) [plugwise_usb.controller] Received success NodeAckSmallResponse for request with seq_id b'0030'
2024-12-31 15:00:09.779 DEBUG (serial_reader_thread) [plugwise_usb.parser] Reset parser : b'\x05\x05\x03\x0300110030000D6F0000B81C700101B90D6F0000998BED09B9FFCB5D\r\n\x83'
2024-12-31 15:00:09.779 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'\x05\x05\x03\x0300110030000D6F0000B81C700101B90D6F0000998BED09B9FFCB5D\r\n\x83'
2024-12-31 15:00:09.779 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'\x05\x05\x03\x0300110030000D6F0000B81C700101B90D6F0000998BED09B9FFCB5D\r\n\x83')
2024-12-31 15:00:09.779 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message header found at index 0
2024-12-31 15:00:09.779 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message footer (b'\r\n') in (b'\x05\x05\x03\x0300110030000D6F0000B81C700101B90D6F0000998BED09B9FFCB5D\r\n\x83')
2024-12-31 15:00:09.779 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message footer found at index 58
2024-12-31 15:00:09.779 INFO (serial_reader_thread) [plugwise_usb.controller] Received StickInitResponse from 000D6F0000B81C70 with sequence id b'0030'
2024-12-31 15:00:09.781 DEBUG (serial_reader_thread) [plugwise_usb.parser] Reset parser : b'\x83'
2024-12-31 15:00:09.852 DEBUG (MainThread) [custom_components.plugwise_usb] Discover Circle+ node
2024-12-31 15:00:09.853 DEBUG (SyncWorker_0) [plugwise_usb.controller] Queue NodeInfoRequest to be send to b'000D6F0000998BED' with retry counter 0 and priority 2
2024-12-31 15:00:09.902 INFO (send_messages_thread) [plugwise_usb.controller] Send NodeInfoRequest to b'000D6F0000998BED' using seq_id b'0031'
2024-12-31 15:00:09.904 DEBUG (serial_writer_thread) [plugwise_usb.connections] Sending NodeInfoRequest to plugwise stick (b'\x05\x05\x03\x030023000D6F0000998BEDF9D4\r\n')
2024-12-31 15:00:09.944 DEBUG (serial_reader_thread) [plugwise_usb.parser] Feed data: b'\x05\x05\x03\x030000003100C1300D\r\n# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003100E19AAB\r\n\x83'
2024-12-31 15:00:09.944 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'\x05\x05\x03\x030000003100C1300D\r\n# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003100E19AAB\r\n\x83'
2024-12-31 15:00:09.944 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'\x05\x05\x03\x030000003100C1300D\r\n# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003100E19AAB\r\n\x83')
2024-12-31 15:00:09.944 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message header found at index 0
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message footer (b'\r\n') in (b'\x05\x05\x03\x030000003100C1300D\r\n# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003100E19AAB\r\n\x83')
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message footer found at index 20
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.controller] Received success NodeAckSmallResponse for request with seq_id b'0031'
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Reset parser : b'# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003100E19AAB\r\n\x83'
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003100E19AAB\r\n\x83'
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003100E19AAB\r\n\x83')
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message header found at index 109
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message footer (b'\r\n') in (b'\x05\x05\x03\x030000003100E19AAB\r\n\x83')
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message footer found at index 20
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.controller] Received timeout NodeAckSmallResponse for request with seq_id b'0031'
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.controller] Skip retry NodeInfoRequest to 000D6F0000998BED to speedup discover process
2024-12-31 15:00:09.945 DEBUG (serial_reader_thread) [plugwise_usb.parser] Reset parser : b'\x83'
2024-12-31 15:00:14.781 INFO (watchdog_thread) [plugwise_usb] Circle+ not yet discovered, resubmit discovery request
2024-12-31 15:00:14.781 DEBUG (watchdog_thread) [plugwise_usb.controller] Queue NodeInfoRequest to be send to b'000D6F0000998BED' with retry counter 0 and priority 3
2024-12-31 15:00:14.782 INFO (send_messages_thread) [plugwise_usb.controller] Send NodeInfoRequest to b'000D6F0000998BED' using seq_id b'0032'
2024-12-31 15:00:14.782 DEBUG (serial_writer_thread) [plugwise_usb.connections] Sending NodeInfoRequest to plugwise stick (b'\x05\x05\x03\x030023000D6F0000998BEDF9D4\r\n')
2024-12-31 15:00:14.813 DEBUG (serial_reader_thread) [plugwise_usb.parser] Feed data: b'\x05\x05\x03\x030000003200C1DEDF\r\n# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003200E17479\r\n\x83'
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'\x05\x05\x03\x030000003200C1DEDF\r\n# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003200E17479\r\n\x83'
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'\x05\x05\x03\x030000003200C1DEDF\r\n# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003200E17479\r\n\x83')
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message header found at index 0
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message footer (b'\r\n') in (b'\x05\x05\x03\x030000003200C1DEDF\r\n# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003200E17479\r\n\x83')
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message footer found at index 20
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.controller] Received success NodeAckSmallResponse for request with seq_id b'0032'
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Reset parser : b'# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003200E17479\r\n\x83'
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003200E17479\r\n\x83'
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003200E17479\r\n\x83')
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message header found at index 109
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message footer (b'\r\n') in (b'\x05\x05\x03\x030000003200E17479\r\n\x83')
2024-12-31 15:00:14.814 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message footer found at index 20
2024-12-31 15:00:14.815 DEBUG (serial_reader_thread) [plugwise_usb.controller] Received timeout NodeAckSmallResponse for request with seq_id b'0032'
2024-12-31 15:00:14.815 DEBUG (serial_reader_thread) [plugwise_usb.controller] Skip retry NodeInfoRequest to 000D6F0000998BED to speedup discover process
2024-12-31 15:00:14.815 DEBUG (serial_reader_thread) [plugwise_usb] Discover Circle+ at 000D6F0000998BED
2024-12-31 15:00:14.815 DEBUG (serial_reader_thread) [plugwise_usb.controller] Queue NodeInfoRequest to be send to b'000D6F0000998BED' with retry counter 0 and priority 3
2024-12-31 15:00:14.815 DEBUG (serial_reader_thread) [plugwise_usb.parser] Reset parser : b'\x83'
2024-12-31 15:00:14.932 INFO (send_messages_thread) [plugwise_usb.controller] Send NodeInfoRequest to b'000D6F0000998BED' using seq_id b'0033'
2024-12-31 15:00:14.933 DEBUG (serial_writer_thread) [plugwise_usb.connections] Sending NodeInfoRequest to plugwise stick (b'\x05\x05\x03\x030023000D6F0000998BEDF9D4\r\n')
2024-12-31 15:00:14.947 DEBUG (serial_reader_thread) [plugwise_usb.parser] Feed data: b'\x05\x05\x03\x030000003300C1748E\r\n# APSRequestNodeIn'
2024-12-31 15:00:14.947 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'\x05\x05\x03\x030000003300C1748E\r\n# APSRequestNodeIn'
2024-12-31 15:00:14.947 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'\x05\x05\x03\x030000003300C1748E\r\n# APSRequestNodeIn')
2024-12-31 15:00:14.947 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message header found at index 0
2024-12-31 15:00:14.948 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message footer (b'\r\n') in (b'\x05\x05\x03\x030000003300C1748E\r\n# APSRequestNodeIn')
2024-12-31 15:00:14.948 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message footer found at index 20
2024-12-31 15:00:14.948 DEBUG (serial_reader_thread) [plugwise_usb.controller] Received success NodeAckSmallResponse for request with seq_id b'0033'
2024-12-31 15:00:14.948 DEBUG (serial_reader_thread) [plugwise_usb.parser] Reset parser : b'# APSRequestNodeIn'
2024-12-31 15:00:14.948 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'# APSRequestNodeIn'
2024-12-31 15:00:14.948 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'# APSRequestNodeIn')
2024-12-31 15:00:14.948 DEBUG (serial_reader_thread) [plugwise_usb.parser] No valid message header found yet
2024-12-31 15:00:14.969 DEBUG (serial_reader_thread) [plugwise_usb.parser] Feed data: b'fo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003300E1DE28\r\n\x83'
2024-12-31 15:00:14.969 DEBUG (serial_reader_thread) [plugwise_usb.parser] Parse data: b'# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003300E1DE28\r\n\x83'
2024-12-31 15:00:14.969 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message header (b'\x05\x05\x03\x03') in (b'# APSRequestNodeInfo: Source MAC: 000D6F0000B81C70# APSRequestNodeInfo: Destination MAC: \r\n000D6F0000998BED\r\n\x05\x05\x03\x030000003300E1DE28\r\n\x83')
2024-12-31 15:00:14.969 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message header found at index 109
2024-12-31 15:00:14.969 DEBUG (serial_reader_thread) [plugwise_usb.parser] Lookup message footer (b'\r\n') in (b'\x05\x05\x03\x030000003300E1DE28\r\n\x83')
2024-12-31 15:00:14.969 DEBUG (serial_reader_thread) [plugwise_usb.parser] Valid message footer found at index 20
2024-12-31 15:00:14.969 DEBUG (serial_reader_thread) [plugwise_usb.controller] Received timeout NodeAckSmallResponse for request with seq_id b'0033'
2024-12-31 15:00:14.969 DEBUG (serial_reader_thread) [plugwise_usb.controller] Skip retry NodeInfoRequest to 000D6F0000998BED to speedup discover process
2024-12-31 15:00:14.969 ERROR (serial_reader_thread) [plugwise_usb] Circle+ is not discovered yet
2024-12-31 15:00:14.970 DEBUG (serial_reader_thread) [plugwise_usb.parser] Reset parser : b'\x83'
2024-12-31 15:00:25.032 WARNING (MainThread) [custom_components.plugwise_usb] Failed to connect to Circle+ node

Maybe related to #8 or plugwise/plugwise-beta#324

[brefra]

This is currently worked on at the async branch

[dirixmjm]

@degerrit, I did not do a lot of testing but this is basically working in 0.44.13 (and 0.55.10 of the home assistant (HA) integration).
I basically unplugged my Circle+ and started up HA which showed all circles and other nodes online without problems.

If non-existent, you need to manually generate the nodetype.cache cache file. which has the following contents (but containing the MACs of your nodes):

000D6F0002778E9B;NodeType.CIRCLE
000D6F00003FEE4F;NodeType.CIRCLE
000D6F00019DC76F;NodeType.SENSE
000D6F00019DC918;NodeType.SCAN
000D6F0001A45B39;NodeType.SWITCH

This cache file in HA is located in /config/.storage/plugwise-/nodetype.cache

I have no Idea if you can make it work for a bare (newly added) system. I am also pretty sure it will be impossible to add further circles/nodes.

[degerrit]

@degerrit, I did not do a lot of testing but this is basically working in 0.44.13 (and 0.55.10 of the home assistant (HA) integration). I basically unplugged my Circle+ and started up HA which showed all circles and other nodes online without problems.

If non-existent, you need to manually generate the nodetype.cache cache file. which has the following contents (but containing the MACs of your nodes):

000D6F0002778E9B;NodeType.CIRCLE
000D6F00003FEE4F;NodeType.CIRCLE
000D6F00019DC76F;NodeType.SENSE
000D6F00019DC918;NodeType.SCAN
000D6F0001A45B39;NodeType.SWITCH

This cache file in HA is located in /config/.storage/plugwise-/nodetype.cache

I have no Idea if you can make it work for a bare (newly added) system. I am also pretty sure it will be impossible to add further circles/nodes.

Thanks a lot @dirixmjm! I have been following the release of the async version and this thread and meaning to move over my Plugwise systems (& automations) to Home Assistant, but haven't yet found the time to do it. Will keep you posted when I do!