zwave: Necessary package not maintained currently and fix not in sight

[bmxp]

The package python-openzwave from https://pypi.org/project/python-openzwave/ is needed for ZWave Plugin to work.
Unfortunately this package won't install with Python 3.9 e.g. on Debian bullseye.
The Package is currently unmaintaned and does not get any fixes. It is unlikely that existing problems will be fixed in nearer future.

There are other projects like Eventghost or Homeassistant that seem to face the same problems. They seem to have a solution, see at blacklight/platypush#128 @kdschlosser seems to have developed a solution.

I don't know if https://pypi.org/project/homeassistant-pyozw/ can provide a solution. However I do not have hardware an can not test or develop another integration. So if we do not find a solution I vote for removing the plugin from the plugin repo so that Travis CI tests won't fail any more.

[kdschlosser]

python-openzwave.zip

for a general installation
setup.py install --dev

to install as a service
setup.py install --dev --service

I was pretty crafty with the service portion of it. If you install it as a service it does not need to be on the same machine as the automation program accessing the ZWave network.

For example, if you have the computer that hosts the automation software on a PC at one end of a large house with the original design this will cause congestion problems on the network because only a small number of nodes would be in contact with the controller and everything would have to be forwarded through those nodes. With the new setup you have the ability to plug a ZStick into say a Raspberry Pi and install pyopenzwave as a service on that Pi and then install a "client" onto the PC. The Pi being as small as it is makes it easy to centrally locate in a home. You can use a wired network connection or a WiFi connection to allow the PC to communicate with the Pi. The data sent between the 2 is encrypted.

use --help against the service script to list the options.

to build the documentation
setup.py build_docs

There are actual command classes now. The command classes are only going to have functions available that are specific to that command class. All you have to do is create a subclass of a command class to be able to inject any GUI/operation code. As an example for a dimmer you want to be able to show a slider code for the dimming portion which you would do by sub classing the SwitchMultilevel command class. Because dimmer switches are also a simple on off switch you subclass the SwitchBinary command class. Each one of those sub classed command classes would have a method added that would build the GUI control that represents that command class. In the case of SwitchMultilevel you would make a slider control and for SwitchBinary you would create a simple on/off switch. So when you want to display the GUI controls for a device you access the device and then iterate over the list of command classes the device has and call that method you added to build the controls.

it has been a while since I have run it and I do not remember how much of it I have finished up. I believe that most of it is good to go and that the documentation may be lacking.

There is an important property when setting the options. ZWaveOption.single_notification_handler. In the past python openzwave was incredibly slow, not only in starting up but also in running. What this option does is it changes from using single threaded system to handle the notifications to having a thread for each node on the network. Fear not there is not going to be 200 threads always running if a network has 200 nodes. If a node thread is idle for a few seconds the thread will shut down so it doesn't hold up resources. When a notification comes in it will start up. I believe that timeout is 10 seconds, I would have to double check tho.

The first time a network is loaded takes the longest for it to startup. after that is takes only a few seconds for enough information to populate so a representation of available devices can be displayed. Being able to control a specific device may not be ready right away and the length of time it takes to be able to control it will vary based on when it get checked and also how many devices are on the network and how much congestion there is on the network. so a 200 node network and the device being queried last is going to take a few minutes until it is able to be controlled.

[bmxp]

@kdschlosser thanks for sharing your work. Wouldn't it make sense to put it on pypi as a package?
Doing so would possibly provide a way to just get it with pip install

[kdschlosser]

WARNING
REALLY LONG READ

It would cause a conflict with the one that is currently on pypi. There is a whole lot that would have to be changed just to change the name of the package. Can't do a simple replacement across the files and the code would have to be gone over line by line. The paths to config files would have to change an the c extension names would have to change as well. That's only a couple of things.

Then I would also want to get permission from the library author if it is OK that I release the package under a different name and also to change the licensing from GPL to MIT or the like due to the GPL needing to be the primary license on a project that may want to use the library. So if a library that wants to use python-openzwave is licensed using the MIT or some other license currently in order to use python-openzwave that library would have to change their license to GPL in order to use it. where is if python-openzwave is MIT and the library is GPL then it can be used without the need to change any licensing.

I have to look and see what the license is for openzwave and that could be the thing that is forcing python-openzwave to be GPL.

I had a falling out with the developer of openzwave because he had wanted me to make changes that specifically tailored python-openzwave to the needs of HomeAssistant. HomeAssistant should have made changes to make it compatible with python-openzwave but the author of HomeAssistant didn't want to make change to his software in order to make it work properly. HomeAssistant overrides quite a bit of the standard library and that was done with no regard to making those changes to keep the original functionality of the overridden component. This caused issues with python-openzwave and they wanted additional code to be added to Python-openzwave so it would specifically would work with HomeAssistant. Adding that code would cause issues for other projects wanting to use python-openzwave. I refused to do it and removed all traces of the version of python-openzwave I created so my code would not be used in a manner that I deemed not correct. That was 2 years ago and since then HomeAssistant had stated originally that they were going to write a new connector to openzwav and apparently jumped ship on that idea and is now using a javascript driver instead. Personally I feel that to be a bad idea because of how much of a resource hog javascript is. to each their own.

So I pulled the code that I had written and I walked away. I am not being paid to make the changes I made and I made the changes to suit my own needs and wanted to share it with other people that saw value in the changes I had made. But to have someone sit on the sidelines and tell me that I am doing things wrong and offering no help to do them the way they felt was right or even offer information as to why they felt it was wrong because an annoyance. Once I started getting aggravated over it then I am not going to offer my assistance anymore. That is pretty much how it went. Oh the other thing that rubbed me the wrong way is I am the person that got python-openzwave to work properly on Windows based PC's It would not compile. So anyone wanting to use python-openzwave on a windows computer couldn't do it. I spent 4 months writing a damned near bulletproof script to detect Visual C and Windows SDKs that are installed to set up an identical build environment that Visual Studio creates when compiling in order to compile openzwave and the extension modules for python-openzwave properly on Windows. I did not use the vcvars* scripts that are included with Visual C, Visual Studio or Visual Studio Build Tools because vcvars* doesn't always set up a correct build environment and there are bugs int it.

I never got 1 thank you from neither the author of Home Assistant or the author of openzwave for making it possible to run their software and have zwave support possible on some 1,500,000,000 (1.5 billion) devices or 73% of the computers in the world.

So that is the way it went and it sucks because folks like yourself and this project are ultimately the ones who got the crap end of the stick in the deal. Seeing that it is 2 years later and the chances of my code getting used are now pretty slim I am OK with posting it up again for people to use if they like. There really isn't that much left that is original and when I have the time to rewrite the last bit of code that is original I will be able to release it under a different name and with a different license without any possible backlash. I will make a repository for it and provide support for those that want to use it when i make those final changes.

[ThomasCr]

I would recommend to drop the plugin, because of the python version from open-zwave and run zwave-js btw. zwave-js-ui like I do and connect it with MQTT together.

Maybe, if needed, we can hook up to the Websocket Implementation for Home Assistant, which is included in the project.

[ThomasCr]

This is how I connect my Fibaro ZWave Plug to zwave-js using MQTT:

        zwSteckdose1:
            name: Esszimmer
            type: bool
            on_change: .mqtt_out = value if not sh..self.changed_by() == "On_Change:{}".format(sh..mqtt_in.property.path) else None
            mqtt_in:
                type: dict
                visu_acl: ro
                mqtt_topic_in: zwave/2/37/0/currentValue
                on_change: .. = value['value']
            mqtt_out:
                type: bool
                visu_acl: ro
                mqtt_topic_out: zwave/2/37/0/targetValue/set
            watt:
                type: num
                visu_acl: ro
            watt_dict:
                type: dict
                visu_acl: ro 
                mqtt_topic_in: zwave/2/49/0/Power
                on_change: ..watt = value['value']
            wh:
                type: num
                visu_acl: ro
                eval: sh...kwh() * 1000
                eval_trigger: ..kwh
            kwh:
                type: num
                visu_acl: ro
            kwh_dict:
                type: dict
                visu_acl: ro
                mqtt_topic_in: zwave/2/50/0/value/65537
                on_change: ..kwh = value['value']

It has Node ID 2, thats why it has topic zwave/2/....
But you need to take care on your settings. I recommend the following settings in zwave-js .

For better understanding / finding the right topic I can recommend MQTT Explorer

[kdschlosser]

I am working on releasing a new binder to openzwave.. it is almost finished. It supports multi instance nodes and also association groups. It is easy to integrate into a GUI because of how command class identification is handled.

[bmxp]

To be honest I like the idea of having a direct connection instead of a detour via mqtt.

@kdschlosser When you are ready with your binder to openzwave I would try to implement it. Unless it is proven that the current plugin is useless I would not drop it in favour of a mqtt solution.

[kdschlosser]

well here is the beauty of some of the things that I have done.

One of the most problematic things about ZWave and any of the controllers that are made is that the controller it's self is horribly under powered and they are usually slow as molasses. so when a person decides to go the route of using a ZWave USB stick and plugging it onto something that has a bit more giddy up the location of said device is usually not able to be placed centrally in a home.

So what I did is I made it so the library can be run as a service or a daemon and placed onto something like a raspberry pi and that raspberry would have the ZWave USB stick plugged into it and then elsewhere in the home the library gets would then get installed into the "controller" or the machine that you want to use to control the ZWave devices. you can then connect the 2 together and the API is identical as if it the ZWave stick was plugged into the controller. The communications between the 2 is encrypted and can optionally be encrypted with user made certificates,

The other nice thing about a design like this is the controller can be rebooted without having to reboot the zwave stack. restarting the zwave stack takes some time to do and unless it needs to be reset there is no reason to do it even if the controller needs to be restarted.

This same kind of a setup can be done all on a single machine as well to allow the controller to operate separately from the zwave stack.

if you tell me what OS you are using I can get you a copy of the compiled library.

There is a whole slew of features that make it way better than the original library. the library is incredibly fast compared to python-openzwave. and it is a whole lot easier to integrate into a GUI.

[Morg42]

For this to run, we'd need a plethora of compiled libraries, as shng is mostly platform-independent. Intel- and ARM-architectures on at least Windows (7, 10, 11, XP?), macOS (10.x-13/14?), several linuxes, possibly more.

If you can "only" supply compiled libraries, we might find a way if you can provide the libraries and the API documentation.