Skip to content

Bluetooth: Host: Reassemble extended advertising reports #39352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

hermabe
Copy link
Member

@hermabe hermabe commented Oct 12, 2021

The host reassembles fragmented advertising reports from the controller.

Fragmented advertising reports from different advertisers may be
interleaved. If fragmented advertising reports from multiple advertisers
are received, only the first one is kept and reassembled. The rest are
discarded until the first one is complete.

If a complete advertising report is not received from an advertiser
within the timeout, the data is discarded and another advertiser is
allowed to be reassembled.

This PR is based on the work by @rugeGerritsen in #37377.

Fixes: #37368


There are a couple issues that need to be discussed:

  • A single complete "short" advertising report and the last fragment in a series of fragmented advertising reports are equal. There is no way to disambiguate them by only looking at the advertising report. To be able to reassemble the data correctly, one thus need to know if an advertiser has sent incomplete data before. A list of a possibly unbounded number of advertisers must be kept. An advertiser can only be removed from the list if one has received an advertising report with Data status "complete".
  • There is no way to know if two advertising reports from anonymous advertisers are from the same device/advertiser set and should be combined or if they are from different devices.

Most of the complexity in this PR arises because the controller may interleave fragmented advertising reports from different advertisers. For controllers that do no interleave advertising reports from different advertisers (like the Softdevice Controller), a much simpler implementation like the one in #37377 can be used. One solution to this is to hide the complex implementation that handles interleaving behind a Kconfig and let controllers that do not need it use the simpler implementation.

A solution in the spec is to add another bit to the Event_Type/Data status field which tells whether the advertising report is part of a chain of fragmented advertising reports or if it is complete by its own.

Tagging some people from previous discussions: @rugeGerritsen @Thalley @cvinayak

The host reassembles fragmented advertising reports from the controller.

Fragmented advertising reports from different advertisers may be
interleaved. If fragmented advertising reports from multiple advertisers
are received, only the first one is kept and reassembled. The rest are
discarded until the first one is complete.

If a complete advertising report is not reveived from an advertiser
within the timeout, the data is discarded and another advertiser is
allowed to be reassembled.

Signed-off-by: Herman Berget <herman.berget@nordicsemi.no>
@github-actions github-actions bot added area: API Changes to public APIs area: Bluetooth area: Bluetooth Host Bluetooth Host (excluding BR/EDR) area: Tests Issues related to a particular existing or missing test platform: STM32 ST Micro STM32 labels Oct 12, 2021
@rugeGerritsen
Copy link
Collaborator

@sjanc , FYI

@sjanc
Copy link
Collaborator

sjanc commented Oct 12, 2021

"A solution in the spec is to add another bit to the Event_Type/Data status field which tells whether the advertising report is part of a chain of fragmented advertising reports or if it is complete by its own."

Data status:
0b00 = Complete
0b01 = Incomplete, more data to come
0b10 = Incomplete, data truncated, no more to come

Why is this not enough?

@hermabe
Copy link
Member Author

hermabe commented Oct 12, 2021

Why is this not enough?

If we get an adv report with data_status=complete, there is no way to know if this is a non-fragmented adv report and this is all the data. If this is all the data, we can create an event. If we previously have discarded data from this advertiser we can not create an event because the data would be incomplete.

@sjanc
Copy link
Collaborator

sjanc commented Oct 12, 2021

I'd assume that if you discard event with "0b01 = Incomplete, more data to come" then you discard all until you get either complete or truncated status

@hermabe
Copy link
Member Author

hermabe commented Oct 12, 2021

I think you would still have the same problem. If you discard until you receive complete or truncated reports, you may have discarded the start of advertising data from other advertisers.

I am thinking of a scenario like this, where fragmented adv reports from multiple advertisers (A and B) are interleaved:

  • A1, More to come, added to reassembly buffer
  • B1, More to come, discarded
  • A2, Complete, added to reassembly buffer and event created
  • B2, Complete, Must also be discarded because B1 was discarded. If event is created from this, data would be missing.

@sjanc
Copy link
Collaborator

sjanc commented Oct 12, 2021

You have advertiser address and SID in report to handle matching, haven't you?

@hermabe
Copy link
Member Author

hermabe commented Oct 12, 2021

Or even:

  • A1, More to come, added to reassembly buffer
  • B1, More to come, discarded
  • A2, Complete, added to reassembly buffer and event created
  • B2, Complete, Must also be discarded because B1 was discarded. If event is created from this, data would be missing.
  • C1, Complete, Should not be discarded as this is the only data from this advertiser

I see no way to disambiguate between C1 and B2 without knowing that B1 has been discarded.

@hermabe
Copy link
Member Author

hermabe commented Oct 12, 2021

You have advertiser address and SID in report to handle matching, haven't you?

Yes

@Thalley
Copy link
Collaborator

Thalley commented Oct 12, 2021

@sjanc @hermabe I think it is also relevant to refer to #37336 where the conclusion that assembling the reports in host can be done, and should be done in the host to provide a better user experience.

@hermabe Is this PR ready for review, or is it still a WIP?

@hermabe
Copy link
Member Author

hermabe commented Oct 12, 2021

Is this PR ready for review, or is it still a WIP?

The issue with anonymous advertisers is not resolved. That needs to be discussed. As mentioned above I would also like to keep a simpler solution for use with controllers that do no interleave advertising reports from different advertisers. A Kconfig for that is not implemented yet.

But please do comment on any issues in the current implementation you see.

@@ -27,6 +31,105 @@ static bt_le_scan_cb_t *scan_dev_found_cb;
static sys_slist_t scan_cbs = SYS_SLIST_STATIC_INIT(&scan_cbs);

#if defined(CONFIG_BT_EXT_ADV)
/* A buffer pool used to reassemble advertisement data from the controller. */
NET_BUF_POOL_FIXED_DEFINE(ext_scan_buf_pool, 1, CONFIG_BT_EXT_SCAN_BUF_SIZE, NULL);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only one chain is expected to be reassembled at time?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's OK to start with I reckon, but we should support multiple.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the relationsship between this and the fragmented_advertiser_heap?

@Thalley
Copy link
Collaborator

Thalley commented Oct 12, 2021

There are a couple issues that need to be discussed:

* A single complete "short" advertising report and the last fragment in a series of fragmented advertising reports are equal. There is no way to disambiguate them by only looking at the advertising report. To be able to reassemble the data correctly, one thus need to know if an advertiser has sent incomplete data before. A list of a possibly unbounded number of advertisers must be kept. An advertiser can only be removed from the list if one has received an advertising report with `Data status` "complete".

So you are saying that if a device has 2 advertisement sets, S1 and S2, that sends data D1 and D2, where S1 sends D1 in multiple segments D1_1, D1_2, etc., we can't know to tell them apart? I think @sjanc already mentioned this, but the advertising set ID would tell them apart, so I don't think this would be an issue.
What could potentially be an issue, is if when we start scanning, we miss e.g. D1_1 and D1_2, and we get the 3rd and final segment D1_3 which would be "complete", but we wouldn't actually know this. In that case, I guess it's up to the upper layers to discard it as it would likely be invalid for whatever purpose it has.

* There is no way to know if two advertising reports from anonymous advertisers are from the same device/advertiser set and should be combined or if they are from different devices.

That is true, and it looks like it isn't something that is intended we can do. From the score spec:

For this purpose, all anonymous advertising is treated as being from a single device different to all non-anonymous devices.

Which I read as we should treat all adv reports from anonymous devices as a single device. If two anonymous devices advertise segmented data at once, it will very often be gibberish unfortunately. I guess this is a known limitation in BLE (we could add an erratum asking for a recommendation not to use segmented adv data with the anonymous address).

@@ -27,6 +31,105 @@ static bt_le_scan_cb_t *scan_dev_found_cb;
static sys_slist_t scan_cbs = SYS_SLIST_STATIC_INIT(&scan_cbs);

#if defined(CONFIG_BT_EXT_ADV)
/* A buffer pool used to reassemble advertisement data from the controller. */
NET_BUF_POOL_FIXED_DEFINE(ext_scan_buf_pool, 1, CONFIG_BT_EXT_SCAN_BUF_SIZE, NULL);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's OK to start with I reckon, but we should support multiple.

config BT_EXT_SCAN_BUF_SIZE
int "Maximum advertisement report size"
range 1 1650
default 255
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest a default of 229 which is the maximum data length of a single report.

@hermabe
Copy link
Member Author

hermabe commented Oct 13, 2021

So you are saying that if a device has 2 advertisement sets, S1 and S2, that sends data D1 and D2, where S1 sends D1 in multiple segments D1_1, D1_2, etc., we can't know to tell them apart?

We have the sid, so we can tell them apart. The problem is with interleaving fragmented reports from multiple advertiser sets (either from the same device or different devices) as in the example in my comment above.

What could potentially be an issue, is if when we start scanning, we miss e.g. D1_1 and D1_2, and we get the 3rd and final segment D1_3 which would be "complete", but we wouldn't actually know this. In that case, I guess it's up to the upper layers to discard it as it would likely be invalid for whatever purpose it has.

Yes, I guess the application should discard the first report from each advertising set/device as it may have never received the first fragment(s) and the data could be incomplete.

For this purpose, all anonymous advertising is treated as being from a single device different to all non-anonymous devices.

Which I read as we should treat all adv reports from anonymous devices as a single device. If two anonymous devices advertise segmented data at once, it will very often be gibberish unfortunately. I guess this is a known limitation in BLE (we could add an erratum asking for a recommendation not to use segmented adv data with the anonymous address).

This text is from the LL spec, so I am not sure it applies to the host. Although it would make sense. In addition the spec says right above that:

The Host may request that duplicate advertising reports are filtered and so not sent.

Which implies that there may also be a situation where the controller does not do this deduplication.

@Thalley
Copy link
Collaborator

Thalley commented Oct 13, 2021

So you are saying that if a device has 2 advertisement sets, S1 and S2, that sends data D1 and D2, where S1 sends D1 in multiple segments D1_1, D1_2, etc., we can't know to tell them apart?

We have the sid, so we can tell them apart. The problem is with interleaving fragmented reports from multiple advertiser sets (either from the same device or different devices) as in the example in my comment above.

I feel like I'm missing some crucial amount information. In your example, I assume A, B and C are advertising sets
Each of those would have a different SID, and we can tell them apart, and so I don't see the issue of reassembling then, as C1 clearly wouldn't be related to B1 nor B2.

For the sake of reassembly, the pair of {address, sid} is unique (except for anonymous advertising), so it should be possibly to perform reassembly, even with multiple sets interleaved (assuming that the reassembly buffers supports reassembly of multiple advertising data).

@hermabe
Copy link
Member Author

hermabe commented Oct 13, 2021

For the sake of reassembly, the pair of {address, sid} is unique (except for anonymous advertising)

Yes, agree. However, we need to know if we previously have discarded any reports from this addr-sid pair. If we have, we must discard all further reports from this pair until a report with data_status=complete is received. If we reassemble reports for an addr-sid pair for which we have previously discarded reports from, the data would be incomplete.

Imagine two different series of adv reports received from the controller. Here A, B and C are different addr-sir pairs.

First example:

  • A1, More to come, added to the reassembly buffer
  • B1, More to come, have no more reassembly buffers, discarded
  • A2, Complete, added to the reassembly buffer and event created. Reassembly buffer cleared.
  • B2, Complete, Must be discarded because B1 was discarded. If event is created from this, data would be missing.

Second example:

  • A1, More to come, added to the reassembly buffer
  • B1, More to come, have no more reassembly buffers, discarded
  • A2, Complete, added to the reassembly buffer and event created. Reassembly buffer cleared.
  • C1, Complete, Should not be discarded as this is the only data from this advertiser.

In these examples B2 and C1 may have the exact same data. The only difference is the addr-sid pair. In the first example we have discarded data from B, so we cannot use the rest of the data until it sends a report with data_status=complete. In the second example, we have not discarded any data from C, so we should use the data and create an event.

@Thalley
Copy link
Collaborator

Thalley commented Oct 13, 2021

For the sake of reassembly, the pair of {address, sid} is unique (except for anonymous advertising)

Yes, agree. However, we need to know if we previously have discarded any reports from this addr-sid pair. If we have, we must discard all further reports from this pair until a report with data_status=complete is received. If we reassemble reports for an addr-sid pair for which we have previously discarded reports from, the data would be incomplete.

Imagine two different series of adv reports received from the controller. Here A, B and C are different addr-sir pairs.

First example:

* A1, More to come, added to the reassembly buffer

* B1, More to come, have no more reassembly buffers, discarded

* A2, Complete, added to the reassembly buffer and event created. Reassembly buffer cleared.

* B2, Complete, Must be discarded because B1 was discarded. If event is created from this, data would be missing.

Second example:

* A1, More to come, added to the reassembly buffer

* B1, More to come, have no more reassembly buffers, discarded

* A2, Complete, added to the reassembly buffer and event created. Reassembly buffer cleared.

* C1, Complete, Should not be discarded as this is the only data from this advertiser.

In these examples B2 and C1 may have the exact same data. The only difference is the addr-sid pair. In the first example we have discarded data from B, so we cannot use the rest of the data until it sends a report with data_status=complete. In the second example, we have not discarded any data from C, so we should use the data and create an event.

Thanks for the clarification - I think I understand your concerns now.
Regarding

However, we need to know if we previously have discarded any reports from this addr-sid pair.

Given the lack of a "start" event type, can we ever guarantee that the reassembled data is correctly assembled?

Is there any scenario where we would not receive e.g. the first segment from the LL, but only the subsequent ones? e.g. if a device sends data in 3 segments D1, D2 and D3; can we in the host ever just receive D2 and D3?

In any case, in order to handle the previously discarded issue, I guess we need to have a discarded list as well, right? Alternatively we simply discard any events, even complete ones, if it is not in the reassembly buffer. This is perhaps not the best solution (especially if we only support a single reassembly buffer), but adv reports are considered discardable in Zephyr, and any host discarding would only be in the case that we are reassembling another PDU.

This feature is definitely going to be memory demanding, but that's not surprising.

@hermabe
Copy link
Member Author

hermabe commented Oct 14, 2021

Given the lack of a "start" event type, can we ever guarantee that the reassembled data is correctly assembled?

Is there any scenario where we would not receive e.g. the first segment from the LL, but only the subsequent ones? e.g. if a device sends data in 3 segments D1, D2 and D3; can we in the host ever just receive D2 and D3?

I think the controller knows whether a PDU is the first in the chain or not, so it should never send adv reports starting from the middle of the chain. I don't know how likely it is to lose reports over HCI.

I guess we need to have a discarded list as well, right?

I think the current list does that job, unless i misunderstand something. If an advertiser is in the list and we should not keep its reports, we have previously received a report from that advertiser and discarded it.

@Thalley
Copy link
Collaborator

Thalley commented Oct 14, 2021

I don't know how likely it is to lose reports over HCI.

That should never happen

I guess we need to have a discarded list as well, right?

I think the current list does that job, unless i misunderstand something. If an advertiser is in the list and we should not keep its reports, we have previously received a report from that advertiser and discarded it.

Does the current implementation support the scenario you created in #39352 (comment)?

adv_info->interval = sys_le16_to_cpu(evt->interval);
adv_info->adv_type = get_adv_type(evt->evt_type);
/* Convert "Legacy" property to Extended property. */
adv_info->adv_props = evt->evt_type ^ BT_HCI_LE_ADV_PROP_LEGACY;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recognize that this is moved code. But can someone check this line for me? This converts BT_HCI_LE_ADV_PROP_ to BT_GAP_ADV_PROP_. But there is a mismatch in meaning between these bitfields on BIT(3), BIT(5) and BIT(6). Is this safe?

We should consider making the conversion explicit:

Suggested change
adv_info->adv_props = evt->evt_type ^ BT_HCI_LE_ADV_PROP_LEGACY;
adv_info->adv_props = (((evt->evt_type & BT_HCI_LE_ADV_PROP_CONN) ?
BT_GAP_ADV_PROP_CONNECTABLE : 0)
& ((evt->evt_type & BT_HCI_LE_ADV_PROP_SCAN) ?
BT_GAP_ADV_PROP_SCANNABLE : 0)
& (!(evt->evt_type & BT_HCI_LE_ADV_PROP_LEGACY) ?
BT_GAP_ADV_PROP_EXT_ADV : 0));

@alwa-nordic
Copy link
Collaborator

alwa-nordic commented Oct 14, 2021

Does the current implementation support the scenario you created in #39352 (comment)?

Looks to me that it is handled correctly so long as add_frag_advertiser does not fail to allocate. If allocation fails when discarding B1 then B2 will be delivered to the application as a complete advertisement.

It might be fine to pass on some corrupted advertisements. So long as they are rare. A real world application should anyway be resistant to adversarial advertisers that try to disrupt the application by sending bad advertisements.

@carlescufi carlescufi removed the platform: STM32 ST Micro STM32 label Oct 14, 2021
Copy link
Collaborator

@cvinayak cvinayak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we

*/
static sys_dnode_t *add_frag_advertiser(const bt_addr_le_t *addr, uint8_t sid)
{
struct fragmented_advertiser *frag_adv = k_malloc(sizeof(struct fragmented_advertiser));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no way to know how many advertisers we will see

Advertisers seen should be limited using a Kconfig, old advertising reports that have been reassembled before can be replaced with newly seen advertisers. Use of k_malloc will have non-deterministic application behavior based on use of allocations across the application (outside Bluetooth subsystem).

/* Get a pointer to the fragmented advertiser the dnode pointer points to */
#define FRAG_ADVERTISER_FROM_DNODE(dnode_ptr) \
((struct fragmented_advertiser *)((dnode_ptr)-offsetof(struct fragmented_advertiser, node)))
#define FRAGMENTED_ADVERTISER_TIMEOUT_MS 10000
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a Kconfig and the range should be as per the maximum auxiliary offset possible. I am not very familiar with the current implementation in the PR, but I would expect, that a fragment that does not received the next report within the timeout will be given to the application as truncated data, right?

Comment on lines 776 to 779
/* We have truncated the data and the controller will provide more data.
* Therefore we must discard the remaining part of the advertisement.
*/
FRAG_ADVERTISER_FROM_DNODE(current_advertiser)->should_keep_reports = false;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Life of current_advertiser in case of timeout and truncated, to be able to correctly handle reports continue to come from the controller for the host timed out and host truncated, is current_advertiser valid until controller returns truncated or incomplete?

Misc. style changes from PR comments.

Signed-off-by: Berget, Herman <herman.berget@nordicsemi.no>
Make the fragmented_advertiser functions take and return
`struct fragmented_advertiser *` instead of `sys_dnode_t *`.

Signed-off-by: Berget, Herman <herman.berget@nordicsemi.no>
Make `struct fragmented_advertiser` have a pointer to the reassembly
buffer. Currently only pointing to the global `ext_scan_buf` if it has a
buffer. Makes it easier to add support for multiple reassembly buffers.

Signed-off-by: Herman Berget <herman.berget@nordicsemi.no>
Use a local heap instead of the global one. The size is configured by
a Kconfig.

Signed-off-by: Herman Berget <herman.berget@nordicsemi.no>
Remove debug logging.

Signed-off-by: Herman Berget <herman.berget@nordicsemi.no>
@github-actions github-actions bot added the platform: STM32 ST Micro STM32 label Oct 22, 2021
@@ -125,6 +125,12 @@ enum {
BT_GAP_ADV_PROP_SCAN_RESPONSE = BIT(3),
/** Extended advertising. */
BT_GAP_ADV_PROP_EXT_ADV = BIT(4),
/** The reported data is a truncated.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/** The reported data is a truncated.
/** @brief The reported data is a truncated.

range 1 1650
default 255
help
Maximum size of an advertisement report. If the advertisement
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Maximum size of an advertisement report. If the advertisement
Maximum size of an advertisement report in octets. If the advertisement

provided by the controller is larger than this buffer size,
the remaining data will be discarded.

config BT_EXT_SCAN_MAX_FRAGMENTED_ADVERTISERS
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
config BT_EXT_SCAN_MAX_FRAGMENTED_ADVERTISERS
config BT_EXT_SCAN_MAX_REASSEMBLY_BUFFERS

The term "fragmented advertiser" isn't particularly clear, and may indicate that this is related to the advertiser role.

@@ -27,6 +31,105 @@ static bt_le_scan_cb_t *scan_dev_found_cb;
static sys_slist_t scan_cbs = SYS_SLIST_STATIC_INIT(&scan_cbs);

#if defined(CONFIG_BT_EXT_ADV)
/* A buffer pool used to reassemble advertisement data from the controller. */
NET_BUF_POOL_FIXED_DEFINE(ext_scan_buf_pool, 1, CONFIG_BT_EXT_SCAN_BUF_SIZE, NULL);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the relationsship between this and the fragmented_advertiser_heap?

@alwa-nordic
Copy link
Collaborator

It seems that discussion has stagnated a bit. Maybe we could have a meeting to recap the discussion and kickoff a new iteration?

I'm thinking, to get it going again, we could at first merge a simplistic implementation supporting only non-interleaved adv reports. This is sufficient for the SoftDeviceController at least. (Does anyone know if the Zephyr Controller interleaves adv reports?)

We can add detection of interleaving so we can log it as an error. Then we can extend the implementation to support a config-bounded adv report concurrency.

@cvinayak
Copy link
Collaborator

cvinayak commented Nov 1, 2021

Does anyone know if the Zephyr Controller interleaves adv reports?

Zephyr Controller does not interleave Extended Advertising Reports, but it appears Periodic Advertising Report implementation is heading towards a possibility of being interleaved as HCI LE Periodic Advertising Reports are generated without buffering of complete AUX_SYNC_IND+AUX_CHAIN_IND PDU chains.

@hermabe
Copy link
Member Author

hermabe commented Nov 1, 2021

It seems that discussion has stagnated a bit. Maybe we could have a meeting to recap the discussion and kickoff a new iteration?
I'm thinking, to get it going again, we could at first merge a simplistic implementation supporting only non-interleaved adv reports. This is sufficient for the SoftDeviceController at least.

Agree. The simple implementation seems sufficient for Softdevice controller and the Zephyr Link Layer for now.

@hermabe
Copy link
Member Author

hermabe commented Dec 20, 2021

Closed in favor of #41337

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: API Changes to public APIs area: Bluetooth Host Bluetooth Host (excluding BR/EDR) area: Bluetooth area: Tests Issues related to a particular existing or missing test platform: STM32 ST Micro STM32
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bluetooth: Host: Support receiving long advertising data
7 participants