Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query cache and use setParameters not working properly #927

Closed
ffyliu opened this issue Nov 13, 2020 · 11 comments
Closed

Query cache and use setParameters not working properly #927

ffyliu opened this issue Nov 13, 2020 · 11 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@ffyliu
Copy link

ffyliu commented Nov 13, 2020

🚨 First, please check:

Describe the bug
When I query data, I cache the query object and do the query, the result is normal. Then the next same method query uses setParameters method to modify parameters, and the result is not right.

Basic info (please complete the following information):

  • ObjectBox version 2.8.0
  • Reproducibility: always
  • Device: huawei mate 9
  • OS: Android 9

attachments are corresponding demo
1.compile and install apk
2.start the app and click on the database test in the upper right corner
3.click the bottom button and check the log.

demo explanation:
Entity:Message
method getMsgListWithType use setParameters(Property<?> property, int[] values) not work
method getMsgListWithType2 use setParameters(String alias, int[] values) not work
method getMsgList use setParameter work right.

TestAll.zip

@ffyliu ffyliu added the bug Something isn't working label Nov 13, 2020
@greenrobot
Copy link
Member

It might speedup response time if you could show the relevant code parts directly in the ticket.

@ffyliu
Copy link
Author

ffyliu commented Nov 16, 2020

Thank you very much for your reply. The main code is as follows:

@Entity
public class Message {

    private static Map<String, Query<Message>> queryMap = new ConcurrentHashMap<>(5);

    public static List<Message> getMsgList(String sid, boolean isDesc) {
        try {
            String keyName = "getMsgList".concat(isDesc ? "1" : "2");
            Query<Message> query = queryMap.get(keyName);
            if (query == null) {
                QueryBuilder<Message> queryBuilder = 
                        StoreManager.getBoxStore().boxFor(Message.class).query().equal(Message_.sid, sid);
                query = (isDesc ? queryBuilder.orderDesc(Message_.serverId) : queryBuilder.order(Message_.serverId)).build();
                queryMap.put(keyName, query);
            } else {
                query.setParameter(Message_.sid, sid);
            }
            return query.find();
        } catch (Exception e) {
            return new LinkedList<>();
        }
    }

    public static List<Message> getMsgListWithType(String sid, int[] types, boolean isDesc) {
        try {
            String keyName = "getMsgListWithType".concat(isDesc ? "1" : "2");
            Query<Message> query = queryMap.get(keyName);
            if (query == null) {
                QueryBuilder<Message> queryBuilder = StoreManager.getBoxStore().boxFor(Message.class).query()
                        .equal(Message_.sid, sid)
                        .in(Message_.tp, types);
                query = (isDesc ? queryBuilder.orderDesc(Message_.serverId) : queryBuilder.order(Message_.serverId)).build();
                queryMap.put(keyName, query);
            } else {
                query.setParameter(Message_.sid, sid).setParameters(Message_.tp, types);
            }
            return query.find();
        } catch (Exception e) {
            return new LinkedList<>();
        }
    }

    public static List<Message> getMsgListWithType2(String sid, int[] types, boolean isDesc) {
        try {
            String keyName = "getMsgListWithType2".concat(isDesc ? "1" : "2");
            Query<Message> query = queryMap.get(keyName);
            if (query == null) {
                QueryBuilder<Message> queryBuilder = StoreManager.getBoxStore().boxFor(Message.class).query()
                        .equal(Message_.sid, sid)
                        .in(Message_.tp, types).parameterAlias("1");
                query = (isDesc ? queryBuilder.orderDesc(Message_.serverId) : queryBuilder.order(Message_.serverId)).build();
                queryMap.put(keyName, query);
            } else {
                query.setParameter(Message_.sid, sid).setParameters("1", types);
            }
            return query.find();
        } catch (Exception e) {
            return new LinkedList<>();
        }
    }
    @Id
    public long id;
    @Index
    public long serverId; // 消息id
    @Index
    public String fr; // 发送者用户名
    public int tp; // 消息类型
    public int s; // 发送平台
    public String m;
    public String fn; // 发送者名字
    public String nn; // 发送者昵称,在昵称发生变化时,需要进行对应消息的更新.
    public long ts; // 发送时间
    public String oz; // 扩展字段
    public String at; // 是否有@别人
    @Index
    public String sid; // topicId
    public Message(int tp, String sid, String m) {
        this.tp = tp;
        this.sid = sid;
        this.m = m;
    }
}

method getMsgListWithType use setParameters(Property<?> property, int[] values) not work
method getMsgListWithType2 use setParameters(String alias, int[] values) not work
method getMsgList use setParameter work right.

@greenrobot
Copy link
Member

greenrobot commented Nov 17, 2020

I didn't see anything obvious with your code. 🤔

Maybe this helps: this unit test ensures that setParamters() with int[] works.

PS.: is queryMap the same static instance? That would explain it. You need to use different keys for different queries.

@ffyliu
Copy link
Author

ffyliu commented Nov 18, 2020

sorry,the testCase code is below:

public static final String[] SIDS = {"adfadfasdfqweqweadfasd", "vczvcvzcdfgafgwaasdasd", "qfhgsdfgsasdasdasdfas", "khjkkjhdafhadjfhasd", "qwyeqwyudnadhfahfadf"};

@DebugLog
@OnClick(R.id.tv_start_test)
public void startQueryTest() {
    Log.d(TAG, "startQueryTest start");
    checkAndAddData();
    int[] types = new int[]{1, 3};
    for (String sid : SIDS) {
        Log.d(TAG, "startQueryTest sid=" + sid + " start");
        List<Message> messages = Message.getMsgListWithType(sid, types, false);
        Log.d(TAG, "startQueryTest sid=" + sid + ",method1MsgSize=" + messages.size());
        messages = Message.getMsgListWithType2(sid, types, false);
        Log.d(TAG, "startQueryTest sid=" + sid + ",method2MsgSize=" + messages.size());
        messages = Message.getMsgList(sid, false);
        Log.d(TAG, "startQueryTest sid=" + sid + ",method3MsgSize=" + messages.size());
    }
    Log.d(TAG, "startQueryTest end");
}

@DebugLog
public void checkAndAddData() {
    long count = Message.getMsgNum();
    Log.d(TAG, "checkAndAddData count=" + count);
    if (count >= 2000) {
        Log.d(TAG, "checkAndAddData data is OK");
        return;
    }
    count = Message.clearMessages();
    Log.d(TAG, "checkAndAddData clearCount=" + count);
    List<Message> messageList = new LinkedList<>();
    int num = 300;
    for (String sid : SIDS) {
        for (int i = 0; i < num; i++) {
            messageList.add(new Message(i / 100 + 1, sid, "随机测试大数据发哈加快速度华发商都后方可哈师大" + i));
        }
        num += 100;
    }
    Message.addOrUpdateMessage(messageList);
    Log.d(TAG, "checkAndAddData success and nowCount=" + Message.getMsgNum());
}

the testcase log is below:

2020-11-18 10:14:05.009 14438-14438/com.sogou.test V/QueryCacheFragment: ⇢ startQueryTest()
2020-11-18 10:14:05.009 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest start
2020-11-18 10:14:05.010 14438-14438/com.sogou.test V/QueryCacheFragment: ⇢ checkAndAddData()
2020-11-18 10:14:05.142 14438-14438/com.sogou.test D/QueryCacheFragment: checkAndAddData count=2500
2020-11-18 10:14:05.142 14438-14438/com.sogou.test D/QueryCacheFragment: checkAndAddData data is OK
2020-11-18 10:14:05.142 14438-14438/com.sogou.test V/QueryCacheFragment: ⇠ checkAndAddData [132ms]
2020-11-18 10:14:05.142 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd start
2020-11-18 10:14:05.149 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method1MsgSize=200
2020-11-18 10:14:05.154 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method2MsgSize=200
2020-11-18 10:14:05.160 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method3MsgSize=300
2020-11-18 10:14:05.160 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=vczvcvzcdfgafgwaasdasd start
2020-11-18 10:14:05.161 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=vczvcvzcdfgafgwaasdasd,method1MsgSize=0
2020-11-18 10:14:05.162 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=vczvcvzcdfgafgwaasdasd,method2MsgSize=0
2020-11-18 10:14:05.169 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=vczvcvzcdfgafgwaasdasd,method3MsgSize=400
2020-11-18 10:14:05.169 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qfhgsdfgsasdasdasdfas start
2020-11-18 10:14:05.169 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qfhgsdfgsasdasdasdfas,method1MsgSize=0
2020-11-18 10:14:05.170 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qfhgsdfgsasdasdasdfas,method2MsgSize=0
2020-11-18 10:14:05.180 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qfhgsdfgsasdasdasdfas,method3MsgSize=500
2020-11-18 10:14:05.180 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=khjkkjhdafhadjfhasd start
2020-11-18 10:14:05.181 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=khjkkjhdafhadjfhasd,method1MsgSize=0
2020-11-18 10:14:05.182 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=khjkkjhdafhadjfhasd,method2MsgSize=0
2020-11-18 10:14:05.192 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=khjkkjhdafhadjfhasd,method3MsgSize=600
2020-11-18 10:14:05.192 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qwyeqwyudnadhfahfadf start
2020-11-18 10:14:05.193 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qwyeqwyudnadhfahfadf,method1MsgSize=0
2020-11-18 10:14:05.194 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qwyeqwyudnadhfahfadf,method2MsgSize=0
2020-11-18 10:14:05.204 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qwyeqwyudnadhfahfadf,method3MsgSize=700
2020-11-18 10:14:05.204 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest end
2020-11-18 10:14:05.205 14438-14438/com.sogou.test V/QueryCacheFragment: ⇠ startQueryTest [195ms]

the question is:
1. method getMsgList use only setParameter to modify the sid param,the result is always right
2.method getMsgListWithType and getMsgListWithType2 use setParameter and setParameters to modify the sid and tp param,the result is first with create right,next with cache is always wrong.

image

@ffyliu
Copy link
Author

ffyliu commented Nov 18, 2020

I didn't see anything obvious with your code. 🤔

Maybe this helps: this unit test ensures that setParamters() with int[] works.

PS.: is queryMap the same static instance? That would explain it. You need to use different keys for different queries.

the queryMap is static for the three methods and the query cache keys of the three methods are all different。

@greenrobot-team
Copy link
Member

greenrobot-team commented Nov 23, 2020

Played with the given example and indeed it looks like setParameter is broken. E.g. replacing with:

Query<Message> query = queryMap.get(keyName);
if (query == null) {
    QueryBuilder<Message> queryBuilder = StoreManager.getBoxStore().boxFor(Message.class).query()
            .equal(Message_.sid, "")
            .in(Message_.tp, new int[]{});
    query = (isDesc ? queryBuilder.orderDesc(Message_.serverId) : queryBuilder.order(Message_.serverId)).build();
    queryMap.put(keyName, query);
}
query.setParameter(Message_.sid, sid).setParameters(Message_.tp, types);
return query.find();

so setParameter is always used: on the first try 0 results are returned as well.

However, I can't reproduce this (it's not about multiple conditions, not about String and Integer, not about count, not about desktop vs Android). It works when switching the in to an equal condition, so not an issue with the example data either.

I'll try to create a minimal example that copies what the given example does, and maybe I'll find something.

Edit: query parameters also look fine:

2020-11-23 16:06:53.998 10131-10131/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd start
2020-11-23 16:06:54.001 10131-10131/com.sogou.test I/Box: Parameters for query #2:
    (sid ==(i) "adfadfasdfqweqweadfasd"
     AND tp in [1|3])
2020-11-23 16:06:54.001 10131-10131/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method1MsgSize=0
2020-11-23 16:06:54.002 10131-10131/com.sogou.test I/Box: Parameters for query #3:
    (sid ==(i) "adfadfasdfqweqweadfasd"
     AND tp in [1|3])
2020-11-23 16:06:54.004 10131-10131/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method2MsgSize=100
2020-11-23 16:06:54.004 10131-10131/com.sogou.test I/Box: Parameters for query #4:
    sid ==(i) "adfadfasdfqweqweadfasd"

@ffyliu
Copy link
Author

ffyliu commented Nov 24, 2020

Thank you very much for your reply.
Looking forward to good news.😊

@greenrobot-team
Copy link
Member

greenrobot-team commented Nov 24, 2020

@ffyliu You found a bug, thank you!

It appears this only affects the Android 32-bit libraries (so x86 and armeabi-v7) running on 64-bit devices. The 64-bit libraries (x86_x64, arm64-v8a), which most devices probably use are not affected.

Update: running the Android 32-bit libraries on 32-bit devices also do not have this issue.

Your app/build.gradle excludes the 64-bit libraries, hence the issue always occurs in your example app:

ndk { abiFilters "armeabi", "armeabi-v7a", "x86", "mips" }

As a workaround you can use a combination of equal + or/and to achieve the same result as an in condition.

You should update (or even remove) your ABI filters to include the 64-bit libraries. ObjectBox only supports x86, x86_x64, armeabi-v7, arm64-v8a.

@greenrobot-team greenrobot-team modified the milestone: 3.0 Nov 24, 2020
@greenrobot-team greenrobot-team self-assigned this Nov 24, 2020
@greenrobot-team greenrobot-team added this to the 3.0 milestone Nov 24, 2020
@ffyliu
Copy link
Author

ffyliu commented Nov 25, 2020

OK,I will remove my app ndk filters and retry。
Thank you very much。

@greenrobot-team I change the ndk filters to below
ndk { abiFilters "armeabi", "armeabi-v7a", "arm64-v8a", "x86", "x86_64", "mips" }
or remove it,the demo run right,Thanks.👍

@greenrobot-team
Copy link
Member

greenrobot-team commented Nov 30, 2020

Update: this issue only occurs when running the Android 32-bit libraries (x86 and armeabi-v7) on 64-bit capable hardware. In the above case the use of ABI filters excluded the 64-bit libraries.

Removing/updating the ABI filters so the 64-bit libraries are included resolves this issue.

Projects that do not exclude the 64-bit libraries are not affected. Put differently: the 32-bit Android libraries (x86 and armeabi-v7) work fine on 32-bit hardware.

Regardless, a fix to support running the 32-bit lib on 64-bit hardware exists internally and will be released in an upcoming release.

Updated my message above in that regard.

Side note: since August 2019 Play Store requires new apps and app updates to ship 64-bit versions of native libraries.

@greenrobot-team greenrobot-team modified the milestones: 3.0, 2.9.0 Feb 16, 2021
@greenrobot-team
Copy link
Member

Running the x86 Android library on x64 hardware should be possible again with the upcoming 2.9.0 release, though we do not recommend it (check your ABI filters as mentioned above!).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants