Add mac address as connection for matter device

[MrEbbinghaus]

Proposed change

This PR adds a mac address connection for matter devices that have a mac address, so that they can be matched with existing devices in the registry.

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Deprecation (breaking change to happen in the future)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• I have followed the perfect PR recommendations
• The code has been formatted using Ruff (ruff format homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[home-assistant]

Hi @MrEbbinghaus

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

[home-assistant]

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

[home-assistant]

Hey there @home-assistant/matter, mind taking a look at this pull request as it has been labeled with an integration (matter) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of matter can trigger bot actions by commenting:

• @home-assistant close Closes the pull request.
• @home-assistant rename Awesome new title Renames the pull request.
• @home-assistant reopen Reopen the pull request.
• @home-assistant unassign matter Removes the current integration label and assignees on the pull request, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

Stale

[frenck]

I've marked this PR, as changes are requested that need to be processed.
Please un-draft it once it is ready for review again by clicking the "Ready for review" button.

Thanks! 👍

../Frenck

Learn more about our pull request process.

[marcelveldt]

Nice work!

[agners]

So this PR seems to cause a bit of headache here. I see devices with many MAC addresses, and things merging weirdly on dev

[MrEbbinghaus]

@agners Oh no, that doesn't look right.
Can you send the matter diagnostics?

I suspect the devices all share the invalid 00:00:00:00:00:00 address.
I knew this could happen (see #121257 (comment)), but this is a HA core level issue, so I removed the check for invalid mac addresses.

[agners]

I suspect the devices all share the invalid 00:00:00:00:00:00 address.

Yeah I have at least two devices which return 00:00:00:00:00:00.

I guess technically, this is a valid MAC address. But these devices don't use this as physical address (according to my AP). Technically, the Matter standard says NetworkInterface hardwareAddress field is mandatory and should contain "a" MAC address. I guess very strictly speaking, this is "a" MAC address, just not the valid/used one for that device 🙈

Anyhow, given that Matter devices return such addresses, this is more a Matter problem. I'd suggest to filter them out on Matter integration side.

[agners]

Btw, what is the exact use case/motivation behind this PR? Do you have a device where this connects up with other integrations or something?

I am a bit worried since MAC address privacy extensions become more and more commonplace. The Matter standard also mentions that when it comes to host name generation. It seems we have to expect that devices rotate MAC addresses:

For DNS‐SD a target host name is required, in addition to the instance name. The target host name
SHALL be constructed using one of the available link-layer addresses, such as a 48-bit device MAC
address (for Ethernet and Wi‐Fi) or a 64-bit MAC Extended Address (for Thread) expressed as a
fixed-length twelve-character (or sixteen-character) hexadecimal string, encoded as ASCII (UTF-8)
text using capital letters, e.g., B75AFB458ECD.. In the event that a device performs MAC
address randomization for privacy, then the target host name SHALL use the privacy-preserving
randomized version and the hostname SHALL be updated in the record every time the underlying
link-layer address rotates. Note that it is legal to reuse the same hostname on more than one inter­
face, even if the underlying link-layer address does not match the hostname on that interface, since
the goal of using a link-layer address is to ensure local uniqueness of the generated hostname. If
future link layers are supported by Matter that do not use 48-bit MAC addresses or 64-bit MAC
Extended Address identifiers, then a similar rule will be defined for those technologies.

[MrEbbinghaus]

Unfortunately, I can't work on this today.
@marcelveldt Should this PR be reverted and put on hold until there is a patch for the core code to validate mac addresses?
Or should I add the checks for now, and maybe it gets fixed later in the core?

---

@agners 00:00:00:xx:xx:xx (and a bunch of other addresses) are not valid identifiers.
'Values based on a zero-valued OUI [...] shall not be used as identifiers.'
See: https://standards.ieee.org/wp-content/uploads/import/documents/tutorials/eui.pdf

---

I still have the checks that I removed here:

https://github.com/MrEbbinghaus/core/blob/e3f84fd007d08b060b4a5c478427df94fa82dcb5/homeassistant/components/matter/adapter.py#L50-L67

    def is_NULL(self) -> bool:
        """Check if the address is NULL value/should not be used.

        See: https://standards.ieee.org/wp-content/uploads/import/documents/tutorials/eui.pdf
        'Values based on a zero-valued OUI [...] shall not be used as identifiers.'
        """
        return self.address[:3] == b"\x00\x00\x00"

    def is_individual(self) -> bool:
        """Check if the address is an individual address, not a group address."""
        return self.address[0] & 0x01 == 0

    def _valid_connection(self) -> bool:
        return self.is_individual() and not self.is_NULL()

[agners]

@agners 00:00:00:xx:xx:xx (and a bunch of other addresses) are not valid identifiers.
'Values based on a zero-valued OUI [...] shall not be used as identifiers.'
See: https://standards.ieee.org/wp-content/uploads/import/documents/tutorials/eui.pdf

Oh I see. I just looked up OUI, and with 00:00:00 Xerox came up, so assumed it is valid.

[MrEbbinghaus]

Btw, what is the exact use case/motivation behind this PR? Do you have a device where this connects up with other integrations or something?

@agners I have a matter device that wants to connect to the manufacturer to make auto-updates, which I don't want.

My router integration adds a switch for every connected device to block its internet access.

And it annoys me that I regularly have duplicate devices in my list.

[MrEbbinghaus]

@agners You are right to point out that there will be more devices with address rotation in the future, which could be an issue for mac addresses to be used as unique device identifiers. However, this should not cause any severe problems in HA.
In the worst case, two integrations would have two different Mac addresses for the same device and the devices can't be merged automatically. But this is the status quo, so no UX degradation.
It is unlikely that 'randomly' generated Mac addresses for privacy will collide, and it would not only be a problem for HA alone but possibly for the entire network.

Also, as I understand it, the addresses are only stored until the integration that provides the entity reloads. So even when the addresses rotate, it wouldn't spam the device registry with addresses.

(Also, I guess that most Matter devices won't move from network to network regularly, and therefore manufacturers won't bother with implementing privacy enhancing rotating addresses.)

[MrEbbinghaus]

@marcelveldt Another issue. This time it's an issue with the frontend.
Once again, the matter frontend should be fixed

The Fritz!Box integration sets via_device. (Technically, the matter device is connected via the Wi-Fi Router…)

This breaks this frontend code:
getMatterDeviceActions and HaDeviceInfoMatter

export const getMatterDeviceActions = async (
  el: HTMLElement,
  hass: HomeAssistant,
  device: DeviceRegistryEntry
): Promise<DeviceAction[]> => {
  if (device.via_device_id !== null) {
    // only show device actions for top level nodes (so not bridged)
    return [];
  }
  
  // ...

---

The semantic of via_device isn't clear from the docs:

https://developers.home-assistant.io/docs/device_registry_index/#what-is-a-device
states:

A device that offers multiple endpoints, where parts of the device sense or output in different areas, should be split into separate devices and refer back to parent device with the via_device attribute.

This sounds like the Fritz!Box integration shouldn't set via_device since that matter device isn't part of it.

But https://developers.home-assistant.io/docs/device_registry_index#device-properties states:

Attribute	Description
via_device	Identifier of a device that routes messages between this device and Home Assistant. Examples of such devices are hubs, or parent devices of a sub-device. This is used to show device topology in Home Assistant.

And this WIFI-Router is definitely "a device that routes messages between this device and Home Assistant".

[agners]

Hm, in the Matter integration we use via_device for bridges. This is a single Matter node, which generates multiple devices: one root device (the bridge), and multiple bridged devices. The latter all have via_device set. We definitely make assumption that those are all are part of one Matter node, e.g. when deleting the bridge device. But I guess that still should be fine as the device which makes trouble here is from another integration? 🤔

[MrEbbinghaus]

That assumption that a device is a bridged matter device if it has via_device set is wrong.
There is at least the "AVM Fritz!Box Tools" integration that breaks that.

That might be against the original idea of via_device, but it's not wrong. The Philips Hue integration also declares that all its devices are via itself, but they don't have to.
Nothing would stop them releasing a light fixture with multiple independent lights that has a central controller which communicates with the bridge… Now what?

This is a hard problem to solve, when you think about that lower network layers are transparent to the upper levels. Neither my SmartPlug nor my Home Assistant can tell how the two are connected.

The only chance you have is that if every device involved shouts: "I know that one, it's connected through me!" That's what the Fritz and the Philips Hue integration are doing.

---

Thinking about meshed networks like in this graphic:

What would be the via_device for the green node?
The next hop? The thread border router? The matter controller?

[agners]

The next hop? The thread border router? The matter controller?

IMHO, via_device only make sense for really static things. However, mesh network are dynamic, they reconfigure all the time. Also there can be multiple BR, and they can be selected at random. And finally there could be multiple Matter controllers controlling the same device. So not sure if via_device makes sense at all for Matter devices, TBH.

[marcelveldt]

We are going to revert this PR because mac address is not a stable identifier and it has proven to cause side effects.

[marcelveldt]

And about via_device: that is designed in HA for bridges/hubs and nothing else. So the current implementation in Matter, Hue etc. is correct.

[marcelveldt]

If you think the architecture needs to be changed, you can file an architecture proposal in our architecture repository.
Thanks!

[MrEbbinghaus]

We are going to revert this PR because mac address is not a stable identifier and it has proven to cause side effects.

I agree, that this PR should be reverted (for the moment), but I don't agree that Mac addresses not being stable identifiers. (I mean they are called "Extended Unique Identifiers" and are standardized by the IEEE since forever.)

The issue in #121257 (comment) comes from insufficient filtering by the HA core of addresses that are specified not to be used as identifiers.

And my issue in #121257 (comment) comes from the Fritz!Box integration incorrectly setting the via_device attribute (based on the attributes intention) and that the frontend assumes that a matter device with via_device must be a bridged device. (see home-assistant/frontend#21415)

I think that being able to merge devices in HA is an important organizational feature, and unfortunately there is no general solution to do this manually yet. (Although there is some progress. See the Blog Post for version 2024.7)

But you are right, this is not the place to discuss this.

---

And about via_device: that is designed in HA for bridges/hubs and nothing else. So the current implementation in Matter, Hue etc. is correct.

I guess it could be better documented. Currently, the docs say:

Identifier of a device that routes messages between this device and Home Assistant.

Which would include routers, switches, "mesh extenders", ...

One could say:
via_device should be set only if it acts on the application layer.

The Hue Bridge translates HueBridge API messages into their ZigBee protocol.
A Matter parent node translates Matter into whatever it uses to "communicate" with the sub node. So their integrations should set via_device.

But a (Wi-Fi) router, thread border router or a mesh extender only acts on the network layer or below.

[marcelveldt]

For now, this PR has been reverted. First the core side needs to be fixed to filter out invalid mac addresses (such as the 00:00:00:00:00) case and then we could try it again but I will keep an eye on the various (cheap) devices how well they set the mac address within the network diagnostics cluster. Somehow I have a bad feeling about this as we already ran into multiple cases where the mac address was just filled with a bogus mac.

So step 1 is adjust core to at least filter out the all-zeros mac, maybe we can even find a better test to ensure the mac is 100% valid.

Step 2 would be inspecting various matter devices for the mac address and if we feel comfortable, add back the change that was now reverted in Matter.

As for "via_device" your conclusion is correct and this should maybe be better defined in our docs.

[MrEbbinghaus]

@marcelveldt Regarding the validation of addresses suitable for identification, I opened a discussion in the architecture repo: home-assistant/architecture#1113