CTR-01 switches no longer come up in HA #205

Mikro007 · 2021-06-15T16:12:15Z

After I made an upgrade of HA to the latest version 2021.06.04 and followed the guide in this app, I did not succeed.

Has uninstalled both plejd and mosquitto apps. Restarted HA and done as it says in the guide. But now when I start running the used app again, I only get 16 units out of 33. And from what I have seen, it seems to be CTR-01 units that are missing.

Any idea what I can do to make it work again?

SweVictor · 2021-06-15T17:40:55Z

The main change in 0.8 is that we register outputs rather than physical devices. Plejd devices without outputs configured will not show up, which should be consistent with the plejd app.

If you get complete verbose or silly logs from the startup you will see everything that happens, what devices are configured/not and hopefully also why.

Mikro007 · 2021-06-15T17:59:44Z

SweVictor
I don´t understand? "devices without outputs configured"???

Everything is working fine in my plejd app.

Forget about the ctr-01!

I saw now that I have them also, but it´s always the same units that is found in HA???

2021-06-15 19:59:48 WRN [device-registry] Trying to set state for null which is not in the list of known outputs.
2021-06-15 19:59:48 WRN [plejd-mqtt] Unknown output id null - not handled by us.

This you mean about in the log?

Mikro007 · 2021-06-15T18:50:26Z

5 times I deleted and reinstall the plugins without success

SweVictor · 2021-06-16T07:48:56Z

I don´t understand? "devices without outputs configured"???

Typical example would be REL-02 which is a Plejd device with two outputs. From 0.8 that physical device will show op as TWO devices in HA, each with one controllable switch entity. Plejd devices configured as "no load" will not show up at all.

2021-06-15 19:59:48 WRN [device-registry] Trying to set state for null which is not in the list of known outputs.
2021-06-15 19:59:48 WRN [plejd-mqtt] Unknown output id null - not handled by us.

If you turn on verbose logging you will see what led up to this, but typically this is an update sent by a device not known by the addon. I get this a bit for example for devices without loads. It could probably be improved code-wise and log-wise but it's not a real problem. The only thing we could do with these messages is log that "we got this message which does not affect any state in the system, discarding".

5 times I deleted and reinstall the plugins without success

Sorry to hear that. It seems that some things are up and running so hopefully it is solvable. If you can, please export the full logs in verbose or silly mode, scrub sensitive information and paste a link to the file and I'll have a look at it. Please also include what you are missing specifically and what is included on the HA side.

If you want to look into this yourself the first thing that happens is API request and parsing of all devices which is logged. You can also look into what the addon registers to mqtt using for example the tool Mqtt Explorer.

Mikro007 · 2021-06-16T09:14:10Z

Now I even tried to reinstall whole HA with the same result. These 16 units come in as they should, but 19 units are still missing and not showing up in HA. I´m running on RPI4 with SSD disk.

I´m not able to look at api problems, just a regular user and it feels like this is something wrong. I used this plugin from when it was released, and I have never got any problems with the units.

I upload two logging files. One from start of the addon, and one when it have running a little.
loggning.txt
logging-start.txt

SweVictor · 2021-06-16T11:51:46Z

Hmm... From the logging.txt it seems the below devices are registered, logs cut off over the first device ("taklampa kök") though, so probably there are more registered.

Taklampa kök (C1729049EB4B_0). online
Soffbord (DA4ABBBDD7B3_0). online
Hall Groventré (D1F037ABCF90_0). online
Taklampa sovrum (F88E5274E2B3_0). online
Vitrinskåp (EFCEB3FAC37B_0). online
Taklampa barnrum (C46014B1A54D_0). online
Hall entré (FC4D45E40C3A_0). online
Taklampa entre (CBA4569C068E_0). online
Wc (EF5CB366811C_0). online
Taklampa badrum (C681D1584836_0). online

Do you have any issues with the above devices?

Mikro007 · 2021-06-16T15:57:04Z

No, thats 10 of the 16 that is working in HA.

Mikro007 · 2021-06-16T15:59:32Z

And I can not find a common thread because there are various used devices that do not come in. Always the same 16 units that enter HA even after a reinstallation of HA.

SweVictor · 2021-06-17T06:51:19Z

Well, I understand you have issues and no, I don't think reinstalling will help. I do however need more complete logs to be able to help.

As an example, from restarting my own plejd addon and getting full logs I get this:

Starting Plejd addon and reading configuration...
2021-06-17 08:42:22 VRB [plejd-main] Log level set to verbose
2021-06-17 08:42:22 INF [plejd-main] Plejd add-on, version 0.8.0-beta
...
2021-06-17 08:42:24 INF [plejd-api] Getting devices from site details response...
2021-06-17 08:42:24 INF [plejd-api] Plejd gateway 'Mediaskåpet' found on site
2021-06-17 08:42:24 VRB [plejd-api] No outputSettings found for Lampetter badrum (F9AFCBF4E4DC), assuming output 0
2021-06-17 08:42:24 VRB [device-registry] Added/updated output device: {"bleOutputAddress":37,"deviceId":"F9AFCBF4E4DC","dimmable":true,"hiddenFromRoomList":false,"name":"Lampetter badrum","output":0,"roomId":"69b2a2d5-476a-425f-8753-11cce233570a","roomName":"Badrum uppe","type":"light","typeName":"DIM-01","version":"2.0","uniqueId":"F9AFCBF4E4DC_0"}. 1 output devices in total.
2021-06-17 08:42:24 VRB [device-registry] Added device to room 69b2a2d5-476a-425f-8753-11cce233570a: ["F9AFCBF4E4DC_0"]
2021-06-17 08:42:24 VRB [device-registry] Added/updated output device: {"bleOutputAddress":81,"deviceId":"FCDA8C56CBDD","dimmable":true,"hiddenFromRoomList":false,"name":"Matbord 1","output":0,"roomId":"151601f8-cf85-4689-a949-0bc59d88688a","roomName":"Kök / matrum","type":"light","typeName":"DIM-01","version":"2.0","uniqueId":"FCDA8C56CBDD_0"}. 2 output devices in total.
...
2021-06-17 08:42:24 VRB [device-registry] Device Dimmer 1 balkongdörr is hidden and will not be included. 
          Hidden from room list: true
          Hidden from integrations: false
...
2021-06-17 08:42:24 VRB [plejd-api] No outputSettings found for Matbord 2 (E8153A3CBF90), assuming output 0
2021-06-17 08:42:24 VRB [device-registry] Added/updated output device: {"bleOutputAddress":85,"deviceId":"E8153A3CBF90","dimmable":true,"hiddenFromRoomList":false,"name":"Matbord 2","output":0,"roomId":"151601f8-cf85-4689-a949-0bc59d88688a","roomName":"Kök / matrum","type":"light","typeName":"DIM-01","version":"2.0","uniqueId":"E8153A3CBF90_0"}. 18 output devices in total.
...
2021-06-17 08:42:24 VRB [plejd-main] connected to mqtt.
2021-06-17 08:42:24 INF [plejd-mqtt] Sending discovery for 32 Plejd output devices
...
2021-06-17 08:42:24 INF [plejd-mqtt] Discovered DIM-01 (light) named Matbord 1 (81 : FCDA8C56CBDD_0).
2021-06-17 08:42:24 DBG [plejd-mqtt] Sending discovery for Kök fönster
..

So, if you could follow the readme and extract full logs from the startup as well as identify what devices you are missing I can hopefully help out. Currently I don't see enough in the logs unfortunately.

Mikro007 · 2021-06-17T07:02:56Z

How will I get a complete logfile when the start disappears after click on update?

SweVictor · 2021-06-17T07:04:46Z

Please refer to the https://github.com/icanos/hassio-plejd/tree/master/plejd#troubleshooting section for general troubleshooting (you could for example look at the mqtt messages coming to HA from Plejd) and the https://github.com/icanos/hassio-plejd/tree/master/plejd#logs section for extracting full logs.

Mikro007 · 2021-06-17T07:39:25Z

Here you get a new logfile[
plejd.log
](url)

SweVictor · 2021-06-17T07:51:15Z

Nice, thanks, that for sure IS a very complete log :-) I agree it's quite cumbersome to get logs, but I don't know how to fix that since Home Assistant sandboxes all addons in their own docker containers and only exposes a very limited log.

Either way - the logs shows quite a lot of devices hidden, could these be the ones you're missing? If not, please specify a few devices you are missing.

Relevant part of log

2021-06-17 09:18:23 VRB [device-registry] Device Entré is hidden and will not be included. 
          Hidden from room list: false
          Hidden from integrations: true
2021-06-17 09:18:23 VRB [device-registry] Device Köksfönster is hidden and will not be included. 
          Hidden from room list: false
          Hidden from integrations: true
2021-06-17 09:18:23 VRB [device-registry] Device Carport is hidden and will not be included. 
          Hidden from room list: false
          Hidden from integrations: true
2021-06-17 09:18:23 VRB [device-registry] Device Baksida garage is hidden and will not be included. 
          Hidden from room list: false
          Hidden from integrations: true
2021-06-17 09:18:23 VRB [device-registry] Device Baksida dörr is hidden and will not be included. 
          Hidden from room list: false
          Hidden from integrations: true
2021-06-17 09:18:23 VRB [device-registry] Device Baksida sovrum is hidden and will not be included. 
          Hidden from room list: false
          Hidden from integrations: true
2021-06-17 09:18:23 VRB [device-registry] Device Allrumfönster is hidden and will not be included. 
          Hidden from room list: false
          Hidden from integrations: true

Mikro007 · 2021-06-17T08:12:31Z

All the devies you named is not in my HA. But what you mean with hidden? Of 35 devices I have just one hidden in my plejd app, because it´s a stair coupling.

Mikro007 · 2021-06-17T08:15:50Z

This ones showes up in my HA

SweVictor · 2021-06-17T08:30:46Z

Alright, so we have the reason then. This was discussed in #179 and in the end we (probably I) decided to hide both devices hiddenFromRooms=true´ (the UI setting in the Plejd app) and hiddenFromIntegrations` (a setting I don't know how to set as a user).

In MY installation hiddenFromIntegrations is always false.

@faanskit @vBrolin - since you were part of the discussing in #179 - any ideas on what hiddenFromIntegrations does and what it's intended for? If no-one objects I suggest we ignore that value if we can't find how to set it.

Mikro007 · 2021-06-17T17:35:09Z

@SweVictor
When do you think there is a fix ready for this?

faanskit · 2021-06-17T17:59:58Z

Alright, so we have the reason then. This was discussed in #179 and in the end we (probably I) decided to hide both devices hiddenFromRooms=true´ (the UI setting in the Plejd app) and hiddenFromIntegrations` (a setting I don't know how to set as a user).

I'd speculate that it stems from

devices[
  {
      "deviceId": "FD9F50DBAD2B",
      "hiddenFromRoomList": true,
  }
]

That is in the Plejd app, if you select "Hide from room view" for a device - parameter will be true.
The rationale in the code would be; if you don't want it in the app - why do you want it in the Home Assistant?
These are more of utilitarian devices, like in one above referenced example - slave for "trapp".

I can find good arguments for both variants. And with that comes my sad conclusion:

We need another configuration:
overrideHideFromRoomList - which default should be FALSE with current behaviour.

BR, Marcus

Mikro007 · 2021-06-17T18:12:09Z

@faanskit
That plejd unit should be hidden and it's the only one that is so in the plejd app.

faanskit · 2021-06-17T18:26:16Z

@Mikro007 , ok. To quick to conclude then. :-)

You have 17 devices that have hiddenFromIntegrations set to true.

One example is:

      "deviceId": "C620C580B8E6",
      "title": "Baksida garage",
      "hiddenFromRoomList": false,
      "hiddenFromIntegrations": true,

Can you tell if there are any special settings with that device since it is registered with Plejd as hiddenFromIntegrations.
I do not really grasp the meaning of this property in the app. I seek why the app registers this differently. User behaviour or app logic.

I think the resolution will almost be the almost the same; a new configuration:
overrideHideFromIntegrations - which default should be FALSE with current behavior.

I'll poke around in the app and my testsystem, but if you can already point of what is different in the app it will be faster to reach a conclusion.

Mikro007 · 2021-06-17T18:45:32Z

Nothing special with that unit, just a regular ctr-01. And it's not hidden in my app, like the other 17 that's are missing.

faanskit · 2021-06-17T18:57:50Z

Ok, I know what it means now.

hiddenFromIntegrations=true means it does not appear in Google Home Assistant.

Configure->Gateway->Integrations->Settings for integrations
Select a device and press "..."

Here you can decide how it shall be integrated:

Excluded
Light
Other

The implementation in HA makes sense. This is a "kind of" integration and if you as a user has requested in the app to exclude it from integrations, it should not be integrated in HA either.

Please confirm by enabling one of the problematic devices for integrations.

If my hypothesis is correct; this is a feature not a bug ;-)

axlthorell · 2021-06-17T21:19:07Z

Just tested with changing the exclude/light option in the plejd app and as expected the device shows up in HA when I set it to light.
Please make the HA integration load all lights, I hide lights from Google that I only want HA to control using automations. This is not possible now.

faanskit · 2021-06-18T04:59:41Z

Just tested with changing the exclude/light option in the plejd app and as expected the device shows up in HA when I set it to light.

Thanks for confirming my hypothesis. We can thus conclude that this is a INABIAF.

Please make the HA integration load all lights, I hide lights from Google that I only want HA to control using automations. This is not possible now.

I still consider the current behavior is the correct and expected one. However, I still honor the though of an alternative reality. My humble proposal to facilitate this feature request is to add two configuration options:

overrideHideFromRoomList
overrideHideFromIntegrations

/Marcus

axlthorell · 2021-06-18T05:15:47Z

I still consider the current behavior is the correct and expected one. However, I still honor the though of an alternative reality. My humble proposal to facilitate this feature request is to add two configuration options:

overrideHideFromRoomList

overrideHideFromIntegrations

/Marcus

This would be much appreciated.
I use the gateway to communicate with Google and don't want all devices to be visible for voice command.

SweVictor · 2021-06-18T05:25:12Z

Interesting, and very nice finds eveyone. At least we know WHY this happens, good job everyone!

Possibly the setting is a good way to solve this, a bit annoying to have too many settings though.

Btw - after connecting Google Home to Home Assistant I suddenly got double all Plejd devices. So that will also be a thing for may people...

Mikro007 · 2021-06-18T05:36:58Z

I vote for @axlthorell solution.

In my situation I will now need to by a new gateway because my old is broken. I can't reach this option in the plejd app.

faanskit · 2021-06-18T05:57:43Z

Possibly the setting is a good way to solve this, a bit annoying to have too many settings though.

Yea, configuration options are bad. Both for usability and for maintainability. In this case dual behavior are reasonable and can motivate. I do have an idea of a variant, but not sure it is fully thought through. I'll collect my thoughts on that and revert back.

Btw - after connecting Google Home to Home Assistant I suddenly got double all Plejd devices. So that will also be a thing for may people...

Yea, that is a drag and I honestly think the only way is to only have one integration to Google. But the option to exclude things from Google is available in Plejd as well as in Home Assistants Google Integration via the expose entity parameter.
With that you as a user can decide which integration that shall control a particular device. You don't want them duplicated.

I think that the Plejd code will change in a not so distant future. As soon as their integration to HomeKit is released, with an assumption that they will allow parallel integrations to both Google and Apple; a single parameter is not enough.
I predict that overrideHideFromIntegrations will become overrideHideFromGoogleIntegrations and overrideHideFromHomeKitIntegrations. This will further complicate the setting I proposed.
Obviously we can ask Plejd to add a an integration option for Hassio Plejd, but I doubt they will comply :-)

In my situation I will now need to by a new gateway because my old is broken. I can't reach this option in the plejd app

@Mikro007, there is one more option for you if you are in a hurry. I hate to tell it to you with the amount of devices you have; but you could remove the problematic ones and add them again.

axlthorell · 2021-06-18T06:51:39Z

Would it be better to have HA include all plejd devices as it did before? It is possible to hide devices in HA.

faanskit · 2021-06-18T06:53:47Z

I'll collect my thoughts on that and revert back.

Nope, did not work. I was thinking that we could set !hiddenFromIntegrations to the entity parameter used in the Google Home Assistant integration. I do not see that as an option. And it would be hard to explain such behaviour.

I have second thoughts about the proposed override configuration, especially with the roadmap towards HomeKit. The parameter is there to hide the integration from towards Google - not Home Assistant.

The current implementation only limits the use of HA, and is against the "spirit" of HA. The user can hide those devices himself in HA if he does not want them there. It's not up to this integration to judge. And the user can hide it from Google both in Plejd and HA.

Therefore: Remove the check. Get all devices in. Let them free.

///

About hideFromRoom list. I do not know the current behavior. But; similar to above - ha-plejd should not be the judge. Let the devices in and let the user remove them from the Ui instead.

SweVictor · 2021-06-18T07:32:56Z

In my situation I will now need to by a new gateway because my old is broken. I can't reach this option in the plejd app.

If you mean Plejd gateway you should be able to get very similar functionality with HA + this addon without the Plejd Gateway if you want to save some money.

I have second thoughts about the proposed override configuration, especially with the roadmap towards HomeKit. The parameter is there to hide the integration towards Google - not Home Assistant.

Well, conceptually I'd say it's a setting for all integrations (Google, Amazon, custom-hacked Plejd, ...), but sure, I see your point.

Would it be better to have HA include all plejd devices as it did before? It is possible to hide devices in HA.

The current implementation only limits the use of HA, and is against the "spirit" of HA. The user can hide those devices himself in HA if he does not want them there. It's not up to this integration to judge. And the user can hide it from Google both in Plejd and HA.

Therefore: Remove the check. Get all devices in. Let them free.

About hideFromRoom list. I do not know the current behavior. But; similar to above - ha-plejd should not be the judge. Let the devices in and let the user remove them from the Ui instead.

This is an interesting option I think. No-config adding of all things, up to user to hide stuff or include in custom dashboards. While I personally use hideFromRooms to remove things I don't want to/can't control (at all) and don't "need" them in HA I wouldn't have problem if they were shown in HA.

What do you think guys, anyone against just adding everything? @icanos wanna veto such a change? 😄

faanskit · 2021-06-18T07:49:57Z

If you mean Plejd gateway you should be able to get very similar functionality with HA + this addon without the Plejd Gateway if you want to save some money.

I think he meant that he cannot use Plejd app to change the Hide From Integrations since he no longer has a Gateway. Without a gateway, you cannot even get to that part of the Ui.

Well, conceptually I'd say it's a setting for all integrations (Google, Amazon, custom-hacked Plejd, ...), but sure, I see your point.

I think that was the original intent, but current implementation differs. Check the app. You need to go all the way into the Google Integration to turn them on/off. So; Ux wise you do it per integration. Codewise, it's just one parameter.
When they add HomeKit, likely it will be the same. Therefore i predict the change. But I have noted Plejd often take the easy way out. Ref. DAL-01 implementation, so it might as well end up as you say.

What do you think guys, anyone against just adding everything?

You have my vote. Revert back to original behavior, unless that config option is desired.

SweVictor · 2021-06-18T08:12:31Z

You have my vote. Revert back to original behavior, unless that config option is desired.

Alright, good. For the record I think we used the "hide from rooms" settings "originally".

axlthorell · 2021-06-21T07:19:13Z

You have my vote. Revert back to original behavior, unless that config option is desired.

Alright, good. For the record I think we used the "hide from rooms" settings "originally".

Can you create a PR and have the addon include all devices?

Thanks!

SweVictor · 2021-06-21T10:09:48Z

So - PR is up, finalized what I've been doing here #207

You can download feature branch files and manually install to test the changed functionality and report back if you have a possibility.

axlthorell · 2021-06-22T08:21:33Z

So - PR is up, finalized what I've been doing here #207

You can download feature branch files and manually install to test the changed functionality and report back if you have a possibility.

Works perfectly!
Thanks!

SweVictor · 2021-06-22T08:43:27Z

Great to hear, I'll close this and merge the PR to develop at least quite soon. Hold off with full release a while pending feedback on v0.8

Mikro007 closed this as completed Jun 15, 2021

Mikro007 reopened this Jun 15, 2021

SweVictor closed this as completed Jun 22, 2021

CTR-01 switches no longer come up in HA #205

CTR-01 switches no longer come up in HA #205

Comments

Mikro007 commented Jun 15, 2021

SweVictor commented Jun 15, 2021

Mikro007 commented Jun 15, 2021 • edited Loading

Mikro007 commented Jun 15, 2021

SweVictor commented Jun 16, 2021

Mikro007 commented Jun 16, 2021 • edited Loading

SweVictor commented Jun 16, 2021

Mikro007 commented Jun 16, 2021

Mikro007 commented Jun 16, 2021

SweVictor commented Jun 17, 2021

Mikro007 commented Jun 17, 2021

SweVictor commented Jun 17, 2021

Mikro007 commented Jun 17, 2021

SweVictor commented Jun 17, 2021

Mikro007 commented Jun 17, 2021

Mikro007 commented Jun 17, 2021

SweVictor commented Jun 17, 2021

Mikro007 commented Jun 17, 2021

faanskit commented Jun 17, 2021

Mikro007 commented Jun 17, 2021 • edited Loading

faanskit commented Jun 17, 2021

Mikro007 commented Jun 17, 2021

faanskit commented Jun 17, 2021

axlthorell commented Jun 17, 2021

faanskit commented Jun 18, 2021

axlthorell commented Jun 18, 2021

SweVictor commented Jun 18, 2021

Mikro007 commented Jun 18, 2021

faanskit commented Jun 18, 2021

axlthorell commented Jun 18, 2021

faanskit commented Jun 18, 2021

SweVictor commented Jun 18, 2021 • edited Loading

faanskit commented Jun 18, 2021 • edited Loading

SweVictor commented Jun 18, 2021

axlthorell commented Jun 21, 2021

SweVictor commented Jun 21, 2021

axlthorell commented Jun 22, 2021

SweVictor commented Jun 22, 2021

Mikro007 commented Jun 15, 2021 •

edited

Loading

Mikro007 commented Jun 16, 2021 •

edited

Loading

Mikro007 commented Jun 17, 2021 •

edited

Loading

SweVictor commented Jun 18, 2021 •

edited

Loading

faanskit commented Jun 18, 2021 •

edited

Loading