Support for Z-Stack 3 (Z-Stack 3.0.x and Z-Stack 3.x.x) - Texas Instruments Zigbee 3.0 stack on adapters like CC2652 and CC1352

[Hedda]

Is your feature request related to a problem? Please describe.

This is a feature request as there is today currently no support for Z-Stack 3 / Z-Stack 3.x.x (Texas Instruments Zigbee 3.x stack)?

Z-Stack 3 support is needed to be compliant with adapters like CC2652/CC2652P/CC2652R/CC2652RB and CC1352/CC1352P.

https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/README.md

As it looks there is currently only support for the older and obsolete/deprecated Z-Stack Home 1.2 from Texas Instruments?

https://www.ti.com/tool/Z-STACK-ARCHIVE

Texas Instruments USB adapters based on CC2652/CC2652P/CC2652R/CC2652RB are very popular as Zigbee 3.0 compliant Zigbee coordinators among other open source home automation projects/communities (inc. Zigbee2MQTT, IoBroker, and Home Assistant).

Personally, I can recommend Electrolama zzh (zig-a-zig-ah) CC2652R USB stick/dongle which is open-source hardware:

https://electrolama.com/projects/zig-a-zig-ah/

https://github.com/electrolama/zig-a-zig-ah

Describe the solution you'd like

Full support for Z-Stack 3 API for Texas Instruments adapters running Z-Stack 3 firmware (Z-Stack 3.0.x and Z-Stack 3.x.x).

The primary goal would be to use the newer and more powerful USB adapter based on Texas Instruments CC2652/CC2652P/CC2652R/CC2652RB and CC1352/CC1352P which today can be found from many suppliers:

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator

Hopefully, it will also lead to full Zigbee 3.0 compliance but that is only a secondary goal and probably not really a priority.

http://www.ti.com/lit/an/swra615a/swra615a.pdf

Describe alternatives you've considered

The primary alternative today, if you like to like a Zigbee 3.0 compliant Zigbee coordinator USB adapter for creating a Zigbee 3 network, is to go with a newer Silicon Labs EmberZNet based adapter like EFR32 (EFR32MG12 or EFR32MG21).

Additional context

While not recommended for a "production" Zigbee network it is possible for testing and development purposes to flash CC2538, CC2531 and CC2530 based USB adapters with Z-Stack 3.0.x firmware:

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_3.0.x/bin

If you today have a CC2531 USB adapter that is already flashed with Z-Stack Home 1.2 firmware then you can now actually upgrade to a Z-Stack 3.0.x firmware yourself quite easily via USB using zigpy-znp command-line tools (as a stand-alone tool):

https://github.com/zigpy/zigpy-znp/blob/dev/TOOLS.md

Note! Be sure to do a backup before you upgrade from Z-Stack Home 1.2 firmware to Z-Stack 3.0.x firmware as you will need to restore that backup afterwards as the upgrade procedure will perform an initialization of the adapter.

https://github.com/zigpy/zigpy-znp/blob/dev/TOOLS.md#backup-and-restore

By the way, zigpy-znp supports bidirectional migration between any coordinators via Open ZigBee Coordinator Backup Format

https://github.com/zigpy/open-coordinator-backup/

PS: Zigbee2MQTT have lists of TI adapters that are made for Z-Stack 3 (Z-Stack 3.x.x) as well as precompiled firmware images:

https://www.zigbee2mqtt.io/information/supported_adapters.html

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_3.x.0/bin

Adapter	TI Chip/Module Used	Firmware to Flash	BSL Trigger Pin (1)	Auto-BSL (2)	RF Switch Control Pins (3)	LED(s)
TI LAUNCHXL-CC26xR1	CC2652R	CC2652R_*.zip	DIO_13	No	N/A	DIO_6 (Red)DIO_7 (Green)
TI LAUNCHXL-CC1352P-2	CC1352P	CC1352P2_CC2652P_launchpad_*.zip	DIO_15	No	DIO_28: 2.4GhzDIO_29: 20dBm PADIO_30: Sub-1GHz	DIO_6 (Red)DIO_7 (Green)
Electrolama zzh	CC2652R	CC2652R_*.zip	DIO_13	No	N/A	DIO_7 (Pink)
Electrolama zzhp-lite	CC2652P(Ebyte E72)	CC1352P2_CC2652P_other_*.zip	DIO_15	Yes	DIO_5: 20dBm PA ??DIO_6: 2.4GHz ??	DIO_7 (Pink)
Electrolama zzhp	CC2652P	CC1352P2_CC2652P_other_*.zip	DIO_15	Yes	DIO_5: 20dBm PA ??DIO_6: 2.4GHz ??	DIO_7 (Pink)
Electrolama zoe2	CC1352P(Ebyte E79)	CC1352P2_CC2652P_other_*.zip	DIO_15	No	DIO_5: 20dBm PA ??DIO_6: 2.4GHz ??	DIO_7 (Pink)
Slaesh's CC2652RB stick	CC2652RB	CC2652RB_*.zip	DIO_13	Yes	N/A	DIO_7 (Blue)
ZigStar Stick v4	CC2652P(RFSTAR RF-BM-2652P2)	CC1352P2_CC2652P_launchpad_*.zip	DIO_15	Only for CH340C ver.	DIO_28: 2.4GhzDIO_29: 20dBm PA	DIO_6 (Green)DIO_7 (Red)
Tube's CC2652P2 USB Coordinator	CC2652P(RFSTAR RF-BM-2652P2)	CC1352P2_CC2652P_launchpad_*.zip	DIO_15	N/A	DIO_28: 2.4GhzDIO_29: 20dBm PA	N/A
Tube's Zigbee Gateways (CC2652P2 variant)	CC2652P(RFSTAR RF-BM-2652P2)	CC1352P2_CC2652P_launchpad_*.zip	DIO_15	N/A	DIO_28: 2.4GhzDIO_29: 20dBm PA	N/A
Tube's Zigbee PoE (Power Over Ethernet) Serial Coordinator (CC2652P2 variant)	CC2652P(RFSTAR RF-BM-2652P2)	CC1352P2_CC2652P_launchpad_*.zip	DIO_15	N/A	DIO_28: 2.4GhzDIO_29: 20dBm PA	N/A
Egony Stick V4 (Ebyte ver.)	CC2652P(Ebyte E72-2G4M20S1E)	CC1352P2_CC2652P_other_*.zip	DIO_15	Yes(from Rev.2.0)	DIO_5: 20dBm PADIO_6: 2.4GHz	DIO_8 (Green)DIO_7 (Red)
Gio-dot Z-Bee Duo with CC2652P	CC2652P(Ebyte E72-2G4M20S1E)	CC1352P2_CC2652P_other_*.zip	DIO_15	Yes(from Rev.2.0)	DIO_5: 20dBm PADIO_6: 2.4GHz	DIO_8 (Green)DIO_7 (Red)
Egony Stick V4 (RFSTAR ver.)	CC2652P(RFSTAR RF-BM-2652P2)	CC1352P2_CC2652P_launchpad_*.zip	DIO_15	Yes	DIO_28: 2.4GhzDIO_29: 20dBm PA	DIO_7 (Green)DIO_6 (Red)
cod.m Zigbee CC2652P RPi Module	CC2652P(RFSTAR RF-BM-2652P2)	CC1352P2_CC2652P_launchpad_*.zip	DIO_15	Yes	DIO_28: 2.4GhzDIO_29: 20dBm PA	DIO_7 (Green)DIO_6 (Red)
Gio-dot Z-Bee Duo with CC2652P	CC2652P (Ebyte E72-2G4M20S1E)	CC1352P2_CC2652P_other_*.zip	DIO_15	Yes (from Rev.2.0)	DIO_5: 20dBm PA	DIO_8 (Green) DIO_7 (Red)
cyijun OpenZ3Gateway	CC2652P (RFSTAR RF-BM-2652P2 SMA Ant.)	CC1352P2_CC2652P_launchpad_*.zip	DIO_15	No	DIO_28: 2.4Ghz DIO_29: 20dBm PA	DIO_7 (Green) DIO_6 (Red)
SONOFF Zigbee 3.0 USB Dongle Plus	CC2652P	CC1352P2_CC2652P_launchpad_*.zip	?	No	?	?

More information on those adapters including recommendations can be found here:

https://www.zigbee2mqtt.io/guide/adapters (previously https://www.zigbee2mqtt.io/information/supported_adapters.html )

[cdjackson]

I'm certainly happy for someone to provide a PR to add this support, but it is not something that I am personally likely to implement as all commercial users that I know of using this library are using Ember chipsets. So please feel free to provide a PR @Hedda

[Hedda]

I'm certainly happy for someone to provide a PR to add this support, but it is not something that I am personally likely to implement as all commercial users that I know of using this library are using Ember chipsets.

OK. Then posted to openhab/org.openhab.binding.zigbee#649 as it is a community request for the OpenHAB Zigbee Binding.

Texas Instruments USB dongles/sticks based on CC2652/CC2652P/CC2652R/CC2652RB are very popular among other many open source home automation projects/communities (inc. Zigbee2MQTT, IoBroker, and Home Assistant) as the CC26x2 series ais both relatively inexpensive, readily available, and very powerful Zigbee 3.0 compliant Zigbee coordinators.

PS: I don't have the coding skills to code this myself, (plus OpenHAB is not the primary application that I use myself today).

[cdjackson]

OK. Then posted to openhab/org.openhab.binding.zigbee#649

That's fine, but clearly it's not possible to implement the openHAB enhancement if the additions are not first made to this library.

Again though, unless someone comes along to implement this, it is unlikely to be implemented by myself as I'm just too busy with other things.

Texas Instruments USB dongles/sticks based on CC2652/CC2652P/CC2652R/CC2652RB are very popular

I appreciate that this is the case for the hobby market as they are cheap, however the Silabs chipset is more popular for commercial users. Of around 20 customers I've supported, only 1 has used the TI chip - the others have used Silabs.

[emilm]

Is the command set totally different between 1.2 and 3.0 ? Because adapter shows up with "Unknown" status.
I have a CC2652P2 based adapter with the 3.0 stack installed. I know Java, so how can I help?
What needs to be done, exactly to get this properly recognized and operational in OpenHAB?

[cdjackson]

Is the command set totally different between 1.2 and 3.0 ?

Personally I have no idea as I'm not familiar with the recent firmware. I know there were a number of changes a couple of years ago as I started to rewrite the TI driver for a customer who started using it, but they then moved to Silabs.

What needs to be done, exactly to get this properly recognized and operational in OpenHAB?

There would need to be a driver implemented to support the chipset - similar to the ones that are implemented for the other chips.

[emilm]

There would need to be a driver implemented to support the chipset - similar to the ones that are implemented for the other chips.

zigbee2mqtt supports both adapters in the same code, it seems like:
https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/adapter/nvItems.ts
https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/adapter/zStackAdapter.ts

Would this shed some light on how different they are?

[cdjackson]

The best place to look is the API documentation - reverse engineering other code is really a pain. However it is up to the implementor - if someone finds it easier to try to reverse other code, then it's a valid approach even if it's more difficult and error prone.

As I've said above, unfortunately I personally do not have the time to look at this - there just aren't enough hours in the day at the moment.

[emilm]

OK. Can't find the API documentation, and I do it faster reading the code.
I bought 2 zigbee devices thinking they would be compatible with 1.2 but seems like many new devices are strictly 3.0 . So my guess is that there will more of this.

To me, it seems like there's not much differences, but it fails to reset in my case:

2021-04-28 14:29:51.165 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - Starting ZigBeeDiscoveryService
2021-04-28 14:29:51.676 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Initializing ZigBee network [zigbee:coordinator_cc2531:baaa34bafc].
2021-04-28 14:29:51.680 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Channel 11
2021-04-28 14:29:51.682 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - PANID 31409
2021-04-28 14:29:51.684 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - EPANID AB8165E5491D8BDA
2021-04-28 14:29:51.688 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Network Key 3903473C0DB0CA4AA92A6B5FBACB919C
2021-04-28 14:29:51.689 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Link Key 5A6967426565416C6C69616E63653039
2021-04-28 14:29:51.692 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Config: zigbee_initialise found, initializeNetwork=false
2021-04-28 14:29:51.695 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Network Key String 3903473C0DB0CA4AA92A6B5FBACB919C
2021-04-28 14:29:51.697 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Network key final array 3903473C0DB0CA4AA92A6B5FBACB919C
2021-04-28 14:29:51.699 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Link Key String 5A6967426565416C6C69616E63653039
2021-04-28 14:29:51.703 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Link key final array 5A6967426565416C6C69616E63653039
2021-04-28 14:29:51.705 [DEBUG] [.zigbee.cc2531.handler.CC2531Handler] - Initializing ZigBee CC2531 serial bridge handler.
2021-04-28 14:29:51.827 [DEBUG] [.zigbee.cc2531.handler.CC2531Handler] - ZigBee CC2531 Coordinator opening Port:'/dev/ttyZIGBEE0' PAN:7ab1, EPAN:AB8165E5491D8BDA, Channel:11
2021-04-28 14:29:51.842 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Scheduling ZigBee start
2021-04-28 14:29:52.854 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - ZigBee network starting
2021-04-28 14:29:52.856 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Initialising ZigBee coordinator
2021-04-28 14:29:52.914 [DEBUG] [e.transaction.ZigBeeTransactionQueue] - Default: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=1, interTransactionDelay=50, maxRetries=2]
2021-04-28 14:29:52.917 [DEBUG] [e.transaction.ZigBeeTransactionQueue] - Broadcast: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=2, interTransactionDelay=4000, maxRetries=0]
2021-04-28 14:29:52.919 [DEBUG] [e.transaction.ZigBeeTransactionQueue] - Multicast: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=3, interTransactionDelay=1200, maxRetries=0]
2021-04-28 14:29:52.940 [DEBUG] [tsystems.zigbee.ZigBeeNetworkManager] - ZigBeeNetworkManager initialize: networkState=UNINITIALISED
2021-04-28 14:29:52.942 [DEBUG] [tsystems.zigbee.ZigBeeNetworkManager] - Network state is updated to INITIALISING
2021-04-28 14:29:52.949 [DEBUG] [e.dongle.cc2531.ZigBeeDongleTiCc2531] - CC2531 transport initialize
2021-04-28 14:29:52.953 [DEBUG] [nding.zigbee.serial.ZigBeeSerialPort] - Connecting to serial port [/dev/ttyZIGBEE0] at 38400 baud, flow control FLOWCONTROL_OUT_RTSCTS.
2021-04-28 14:29:52.958 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - null: networkStateUpdated called with state=INITIALISING
2021-04-28 14:29:53.041 [DEBUG] [nding.zigbee.serial.ZigBeeSerialPort] - Serial port [/dev/ttyZIGBEE0] is initialized.
2021-04-28 14:29:58.082 [WARN ] [.cc2531.network.ZigBeeNetworkManager] - Dongle reset failed. Assuming bootloader is running and sending magic byte 0xef.
2021-04-28 14:29:58.340 [INFO ] [zwave.handler.ZWaveControllerHandler] - Attempting to add listener when controller is null
2021-04-28 14:30:03.084 [WARN ] [.cc2531.network.ZigBeeNetworkManager] - Attempt to get out from bootloader failed.
2021-04-28 14:30:03.116 [DEBUG] [nding.zigbee.serial.ZigBeeSerialPort] - Serial port '/dev/ttyZIGBEE0' closed.

So in zigbee2mqtt it first requests version: https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/adapter/zStackAdapter.ts#L106

And from there it initializes : https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/adapter/startZnp.ts#L161

    await znp.request(Subsystem.SYS, 'resetReq', {type: Constants.SYS.resetType.SOFT});

is kinda similar to your dongleReset() .

The difference is that you specify the whole frame while the js code builds it from the different components of the frame as an object. https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/unpi/frame.ts#L35

Further on in that file, you can see how the startup sequence works with the special logic for version 3 of the stack.
I was hoping for you with the expertise on the protocol itself and your own library could quickly (relative term I know) spot the extra 3.x stuff. My hunch, it isn't all that much.

But if you don't even have time to look, it would be difficult for me to jump in and support it.

[cdjackson]

But if you don't even have time to look, it would be difficult for me to jump in and support it.

I'm happy to answer a few questions, but a question like "what is the difference between API 1 and API 2" is a broad ranging question that can take a lot of time to understand. If you have the knowledge and capability to implement and debug this, then I'm happy to answer targeted questions, but I don't have a lot of time to start digging into the system to work out how something works in MQTT and implement it in Java here.

2021-04-28 14:30:03.084 [WARN ] [.cc2531.network.ZigBeeNetworkManager] - Attempt to get out from bootloader failed.

Maybe the new firmware uses a different sequence to escape from the bootloader? Or maybe some response before this is invalid such that the driver thinks it is in the bootloader and then can't escape. Just run through a debug sequence to work this out.

[emilm]

Thanks a lot, maybe I can get it to work locally first.
Is there some way to do a quick setup and run a sequence of commands to verify that it works without running OpenHAB?
I run Windows and I would imagine this could be set up to work with a COM port or something.

[cdjackson]

Yes, you can use the console application that is part of this repository. This is a full console application that's used as a demo since most users of this repository do not use openHAB.

[emilm]

Thanks a lot! I totally missed that.

[Hedda]

zigbee2mqtt supports both adapters in the same code, it seems like:
https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/adapter/nvItems.ts
https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/adapter/zStackAdapter.ts

Would this shed some light on how different they are?

FYI, zigpy-znp and zigpy-cc also support both stack versions in same code and while not Java they might shed light on differences.

In those examples; zigpy-cc was written for Z-Stack Home 1.2 API/CLI and later @sanyatuning added support for Z-Stack 3 CLI/API.

https://github.com/zigpy/zigpy-cc

In the other example; zigpy-znp was written for Z-Stack 3.x API/CLI and later @puddly added support for Z-Stack Home 1.2 CLI/API.

https://github.com/zigpy/zigpy-znp

Both now support Z-Stack Home 1.2 API/CLI and Z-Stack 3 API/CLI, but zigpy-cc is now obsolete in favour of the newer zigpy-znp.

For reference; zigpy-znp was written from scratch/docs, while zigpy-cc was reverse-engineered from zigbee-herdsman by @Koenkk