Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyValueStoreMgr().Get() fails on Darwin #12174

Closed
rcasallas-silabs opened this issue Nov 23, 2021 · 2 comments · Fixed by #14342
Closed

KeyValueStoreMgr().Get() fails on Darwin #12174

rcasallas-silabs opened this issue Nov 23, 2021 · 2 comments · Fixed by #14342
Assignees
Labels

Comments

@rcasallas-silabs
Copy link
Contributor

Problem

This behaviour has been observed on the Darwing implementation of KeyValueStoreMgr:

  • If more than one new key is store with KeyValueStoreMgr().Put(), the following KeyValueStoreMgr().Get() call for the second key returns CHIP_NO_ERROR, but fails to retrieve the stored value (the output buffer remains unchanged).
  • However, if KeyValueStoreMgr().Get() is called immediatelly after the KeyValueStoreMgr().Put() call, the value is later returned correctly.
  • In any case, the KeyValueStoreMgr().Get() call does not modify the value of the in/out size parameter.
    The following code illustrate the problem:

static constexpr size_t kBufferSize = 512;
static constexpr size_t kEntriesCount = 4;
static constexpr size_t kEntrySizeMax = 128;
static constexpr size_t kKeySizeMax = 16;

void FormatKey(char *key, size_t max_size, size_t index)
{
    snprintf(key, max_size, "e%zu", index);
}

void TestKeyValueStoreMgr(nlTestSuite * apSuite, void * apContext)
{
    static uint8_t buffer[kBufferSize] = { 0 };
    ByteSpan entries[kEntriesCount];
    char key[kKeySizeMax];
    uint8_t out[kEntrySizeMax];
    size_t out_size;

    // Initialize

    for (size_t i = 0; i < sizeof(buffer); i++)
    {
        buffer[i] = static_cast<uint8_t>(i % 0xff);
    }
    
    // Delete

    for (size_t i = 0; i < kEntriesCount; i++)
    {
        FormatKey(key, sizeof(key), i);
        NL_TEST_ASSERT(apSuite, CHIP_NO_ERROR == chip::DeviceLayer::PersistedStorage::KeyValueStoreMgr().Delete(key));
    }

    // Save

    for (size_t i = 0; i < kEntriesCount; i++)
    {
        FormatKey(key, sizeof(key), i);
        entries[i] = ByteSpan(&buffer[16 + i], (1 + (i % kEntrySizeMax)));
        NL_TEST_ASSERT(apSuite, CHIP_NO_ERROR == chip::DeviceLayer::PersistedStorage::KeyValueStoreMgr().Put(key, entries[i].data(), entries[i].size()));
        out_size = sizeof(out);
        // FIXME: Loading will fail if the following line is commented
        // NL_TEST_ASSERT(apSuite, CHIP_NO_ERROR == chip::DeviceLayer::PersistedStorage::KeyValueStoreMgr().Get(key, out, out_size));
    }

    // Load

    for (size_t i = 0; i < kEntriesCount; i++)
    {
        FormatKey(key, sizeof(key), i);
        out_size = sizeof(out);
        memset(out, 0x00, out_size);
        NL_TEST_ASSERT(apSuite, CHIP_NO_ERROR == chip::DeviceLayer::PersistedStorage::KeyValueStoreMgr().Get(key, out, out_size));
        NL_TEST_ASSERT(apSuite, !memcmp(out, entries[i].data(), entries[i].size()));
    }
}

Proposed Solution

None so far.

@rcasallas-silabs rcasallas-silabs changed the title KeyValueStoreMgr().Get() fails on Darwing KeyValueStoreMgr().Get() fails on Darwin Nov 30, 2021
@bzbarsky-apple
Copy link
Contributor

@sagar-apple Is this code you know about? Or does @pan-apple ?

@bzbarsky-apple
Copy link
Contributor

It's possible that #14233 will fix this. At least it fixes the above-pasted test for me....

bzbarsky-apple added a commit to bzbarsky-apple/connectedhomeip that referenced this issue Jan 26, 2022
There were two remaining issues in the Darwin KVS impl that needed
to be fixed to get this to pass:

1) The errors returned did not match the API documentation, which
   caused some consumers to fail.  The most important problem here was
   returning CHIP_ERROR_KEY_NOT_FOUND instead of
   CHIP_ERROR_PERSISTED_STORAGE_VALUE_NOT_FOUND from _Get, but I went
   ahead and aligned the other error cases (not initialized, buffer
   passed to _Get too small, deleting unknown key, failures to save to
   persistent storage) with the API documentation.

2) In _Put, if the item already existed the change of "value" was not
   being picked up by the NSManagedObject machinery, so updating the
   value of an already-existing key did not work.  The fix for this is
   to use @dynamic instead of @synthesize for the properties of
   KeyValueItem (shamelessly copy/pasting from the documentation at
   https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreData/LifeofaManagedObject.html),
   which I _think_ causes the getter/setter to be provided by the
   NSMnagedObject bits, which then know about the set we perform and
   know to update the data store.

Fixes project-chip#12174
bzbarsky-apple added a commit to bzbarsky-apple/connectedhomeip that referenced this issue Jan 26, 2022
There were two remaining issues in the Darwin KVS impl that needed
to be fixed to get this to pass:

1) The errors returned did not match the API documentation, which
   caused some consumers to fail.  The most important problem here was
   returning CHIP_ERROR_KEY_NOT_FOUND instead of
   CHIP_ERROR_PERSISTED_STORAGE_VALUE_NOT_FOUND from _Get, but I went
   ahead and aligned the other error cases (not initialized, buffer
   passed to _Get too small, deleting unknown key, failures to save to
   persistent storage) with the API documentation.

2) In _Put, if the item already existed the change of "value" was not
   being picked up by the NSManagedObject machinery, so updating the
   value of an already-existing key did not work.  The fix for this is
   to use @dynamic instead of @synthesize for the properties of
   KeyValueItem (shamelessly copy/pasting from the documentation at
   https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreData/LifeofaManagedObject.html),
   which I _think_ causes the getter/setter to be provided by the
   NSMnagedObject bits, which then know about the set we perform and
   know to update the data store.

Fixes project-chip#12174
bzbarsky-apple added a commit to bzbarsky-apple/connectedhomeip that referenced this issue Jan 26, 2022
There were two remaining issues in the Darwin KVS impl that needed
to be fixed to get this to pass:

1) The errors returned did not match the API documentation, which
   caused some consumers to fail.  The most important problem here was
   returning CHIP_ERROR_KEY_NOT_FOUND instead of
   CHIP_ERROR_PERSISTED_STORAGE_VALUE_NOT_FOUND from _Get, but I went
   ahead and aligned the other error cases (not initialized, buffer
   passed to _Get too small, deleting unknown key, failures to save to
   persistent storage) with the API documentation.

2) In _Put, if the item already existed the change of "value" was not
   being picked up by the NSManagedObject machinery, so updating the
   value of an already-existing key did not work.  The fix for this is
   to use @dynamic instead of @synthesize for the properties of
   KeyValueItem (shamelessly copy/pasting from the documentation at
   https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreData/LifeofaManagedObject.html),
   which I _think_ causes the getter/setter to be provided by the
   NSMnagedObject bits, which then know about the set we perform and
   know to update the data store.

Fixes project-chip#12174
bzbarsky-apple added a commit to bzbarsky-apple/connectedhomeip that referenced this issue Jan 27, 2022
There were two remaining issues in the Darwin KVS impl that needed
to be fixed to get this to pass:

1) The errors returned did not match the API documentation, which
   caused some consumers to fail.  The most important problem here was
   returning CHIP_ERROR_KEY_NOT_FOUND instead of
   CHIP_ERROR_PERSISTED_STORAGE_VALUE_NOT_FOUND from _Get, but I went
   ahead and aligned the other error cases (not initialized, buffer
   passed to _Get too small, deleting unknown key, failures to save to
   persistent storage) with the API documentation.

2) In _Put, if the item already existed the change of "value" was not
   being picked up by the NSManagedObject machinery, so updating the
   value of an already-existing key did not work.  The fix for this is
   to use @dynamic instead of @synthesize for the properties of
   KeyValueItem (shamelessly copy/pasting from the documentation at
   https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreData/LifeofaManagedObject.html),
   which I _think_ causes the getter/setter to be provided by the
   NSMnagedObject bits, which then know about the set we perform and
   know to update the data store.

Fixes project-chip#12174
bzbarsky-apple added a commit to bzbarsky-apple/connectedhomeip that referenced this issue Jan 27, 2022
There were two remaining issues in the Darwin KVS impl that needed
to be fixed to get this to pass:

1) The errors returned did not match the API documentation, which
   caused some consumers to fail.  The most important problem here was
   returning CHIP_ERROR_KEY_NOT_FOUND instead of
   CHIP_ERROR_PERSISTED_STORAGE_VALUE_NOT_FOUND from _Get, but I went
   ahead and aligned the other error cases (not initialized, buffer
   passed to _Get too small, deleting unknown key, failures to save to
   persistent storage) with the API documentation.

2) In _Put, if the item already existed the change of "value" was not
   being picked up by the NSManagedObject machinery, so updating the
   value of an already-existing key did not work.  The fix for this is
   to use @dynamic instead of @synthesize for the properties of
   KeyValueItem (shamelessly copy/pasting from the documentation at
   https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreData/LifeofaManagedObject.html),
   which I _think_ causes the getter/setter to be provided by the
   NSMnagedObject bits, which then know about the set we perform and
   know to update the data store.  But even with this change things
   were failing with ASAN errors, so for now we just remove the old
   item and create a new one on every Put() call.

Fixes project-chip#12174
bzbarsky-apple added a commit to bzbarsky-apple/connectedhomeip that referenced this issue Jan 27, 2022
There were two remaining issues in the Darwin KVS impl that needed
to be fixed to get this to pass:

1) The errors returned did not match the API documentation, which
   caused some consumers to fail.  The most important problem here was
   returning CHIP_ERROR_KEY_NOT_FOUND instead of
   CHIP_ERROR_PERSISTED_STORAGE_VALUE_NOT_FOUND from _Get, but I went
   ahead and aligned the other error cases (not initialized, buffer
   passed to _Get too small, deleting unknown key, failures to save to
   persistent storage) with the API documentation.

2) In _Put, if the item already existed the change of "value" was not
   being picked up by the NSManagedObject machinery, so updating the
   value of an already-existing key did not work.  The fix for this is
   to use @dynamic instead of @synthesize for the properties of
   KeyValueItem (shamelessly copy/pasting from the documentation at
   https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreData/LifeofaManagedObject.html),
   which I _think_ causes the getter/setter to be provided by the
   NSMnagedObject bits, which then know about the set we perform and
   know to update the data store.

Fixes project-chip#12174
bzbarsky-apple added a commit that referenced this issue Jan 27, 2022
There were two remaining issues in the Darwin KVS impl that needed
to be fixed to get this to pass:

1) The errors returned did not match the API documentation, which
   caused some consumers to fail.  The most important problem here was
   returning CHIP_ERROR_KEY_NOT_FOUND instead of
   CHIP_ERROR_PERSISTED_STORAGE_VALUE_NOT_FOUND from _Get, but I went
   ahead and aligned the other error cases (not initialized, buffer
   passed to _Get too small, deleting unknown key, failures to save to
   persistent storage) with the API documentation.

2) In _Put, if the item already existed the change of "value" was not
   being picked up by the NSManagedObject machinery, so updating the
   value of an already-existing key did not work.  The fix for this is
   to use @dynamic instead of @synthesize for the properties of
   KeyValueItem (shamelessly copy/pasting from the documentation at
   https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreData/LifeofaManagedObject.html),
   which I _think_ causes the getter/setter to be provided by the
   NSMnagedObject bits, which then know about the set we perform and
   know to update the data store.

Fixes #12174
selissia pushed a commit to selissia/connectedhomeip that referenced this issue Jan 28, 2022
There were two remaining issues in the Darwin KVS impl that needed
to be fixed to get this to pass:

1) The errors returned did not match the API documentation, which
   caused some consumers to fail.  The most important problem here was
   returning CHIP_ERROR_KEY_NOT_FOUND instead of
   CHIP_ERROR_PERSISTED_STORAGE_VALUE_NOT_FOUND from _Get, but I went
   ahead and aligned the other error cases (not initialized, buffer
   passed to _Get too small, deleting unknown key, failures to save to
   persistent storage) with the API documentation.

2) In _Put, if the item already existed the change of "value" was not
   being picked up by the NSManagedObject machinery, so updating the
   value of an already-existing key did not work.  The fix for this is
   to use @dynamic instead of @synthesize for the properties of
   KeyValueItem (shamelessly copy/pasting from the documentation at
   https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreData/LifeofaManagedObject.html),
   which I _think_ causes the getter/setter to be provided by the
   NSMnagedObject bits, which then know about the set we perform and
   know to update the data store.

Fixes project-chip#12174
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants