Find behaves differently from MongoDB

[joshq00]

Document:

{ _id: 1,
  author: { name: "John", email: "abc@xyz.com" },
  comment: ".....",
  .....
}

Searching for author.name: "John" should return record 1, but does not.

One has to search author: { name: "John" } which returns record 1.

This differs from mongo in that the first example would return record 1 and the second would not. Nested objects must fully match. Searching for nested properties uses "author.name" syntax.

[Irrelon]

Hmm, this is true. I don't think we can change this though. There are a couple of reasons why doing it the mongo way will be inefficient too (for Forerunner). The main one is that each property would need to be checked for a period character and then split into parts, then traversed.

Apart from that, there are tons of projects using this library that already query data that way... would be horrible to change the convention now.

That said, we could support both ways because we could pre-process the query, detect the properties that are accessing child properties via the path syntax (with a period) and then convert them into the object syntax that ForerunnerDB uses before executing the query. Something for version 2 perhaps.

I'll update the docs to reflect the difference between ForerunnerDB and MongoDB in this regard.

[joshq00]

That's a reasonable solution.

Rather than just purely changing the way it works, what if the user is given a choice of the search method?

I know personally I was looking for a kind of browser ified mongo (so I may reuse most code...), But with differing behaviors, and search being the key functionality I'm looking for, forerunner unfortunately can't be used.

It looks like an awesome project. If mongo style queries can't be used, maybe I can try in a fork, but I believe if you move closer and closer to matching mongo API it will attract a wide audience

[kevteljeur]

For what it's worth, I set about implementing Forerunner into a project (to replace my own, horrendous in-memory query system) but then I discovered this difference with the query language through trial and error. It's true that other libraries do this and of course a precedent has been set by now for existing users, but: The MongoDB implementation is very specific, and this implementation shouldn't be called 'Mongo-like', because the expectation is that it will behave the same way (in fact, with regard to nesting and dot-notation, Forerunner behaves exactly the opposite to MongoDB).

I appreciate that readme.md has been updated since I first arrived at it, but it isn't compatible with MongoDB queries, and that should be explicit. I do look forward to the point when it is, it does look like a promising library to use.

[Irrelon]

@kevteljeur Thanks for the feedback and for giving Forerunner a go!

I think you are correct that an explicit point should be made about the differences. I will update the readme.md again to try to be more explicit.

Hopefully other than this, ForerunnerDB is working well for your use case! :)

[kevteljeur]

Unfortunately not (because the query behaviour is a deal-breaker), but I am tracking this because I like the integration with LocalForage (another wheel which I unfortunately reinvented) and you're obviously working on this on an ongoing basis so I'll be back. I did check through your code (you've got a tidy tree traverser in there) so I know it's design decision but hopefully you'll revisit it. And yes, tree traversal is costly but there are some checks that you can make (such as first level checking for a fast match) that might mitigate it a bit. As was suggested above, perhaps it could be optional, so a developer could make a conscious choice to take the performance hit to get compatibility.

If I can squeeze a few minutes out sometime to have a closer look then maybe I'll submit some proposals to you but I doubt I'll be giving you better code than you could write yourself. Keep up the good work!

[Irrelon]

Dev branch now has mongoEmulation() method on fdb, db and collection instances. Find, update and remove all tested and working with dot notation-based queries. Can you guys give it a try and let me know your results?

Can you respond further on #46 issue card please? Closing this one down now.

[joshq00]

I quickly looked at convert to fdb function. I may be wrong but, off first glance, is it just changing
{"a.b": true} to {a:{b:true}} ? If so, there will still be different results in matching

[Irrelon]

@joshq00 That is essentially correct, I thought that would solve the issue in the least costly fashion... would it still break something?

[joshq00]

Not "break," just won't fully fix. Nested object criteria in mongo tests for a full match and dot criteria searches partial

[kevteljeur]

@Irrelon Keep in mind that in MongoDB queries, {"a.b": true} and {a:{b:true}} have explicitly different meanings; They might return the same results, but wouldn't be expected to. {"a.b": true} would return any document with a b child that had a value of true, while {a:{b:true}} would only return documents that had a b child document that was exactly {b:true} (including key order).

Sorry, I'm not sure if you were aware of that and apologies if you are (i'm getting pinged on this thread too).

[Irrelon]

Ahh ok I see. Are there any other gotchas in the spec or does that cover it? I can make the two behave differently in the emulation mode but just want to make sure I'm not missing anything else! :)

Thanks for the clarification by the way, very helpful 👍

[joshq00]

I'm not fully aware of how the FDB search works for arrays (I used another lib that has Mongo API).

It would be beneficial to read the Query Documents section from Mongo's documentation.

From their page,
Inventory Collection

{ _id: 5, type: "food", item: "aaa", ratings: [ 5, 8, 9 ] }
{ _id: 6, type: "food", item: "bbb", ratings: [ 5, 9 ] }
{ _id: 7, type: "food", item: "ccc", ratings: [ 9, 5, 8 ] }

Exact match

db.inventory.find( { ratings: [ 5, 8, 9 ] } )
=> { _id: 5, type: "food", item: "aaa", ratings: [ 5, 8, 9 ] }

Match element

db.inventory.find( { ratings: 5 } )
=> { "_id" : 5, "type" : "food", "item" : "aaa", "ratings" : [ 5, 8, 9 ] }
=> { "_id" : 6, "type" : "food", "item" : "bbb", "ratings" : [ 5, 9 ] }
=> { "_id" : 7, "type" : "food", "item" : "ccc", "ratings" : [ 9, 5, 8 ] }

Match specific element

db.inventory.find( { 'ratings.0': 5 } )
=> { "_id" : 5, "type" : "food", "item" : "aaa", "ratings" : [ 5, 8, 9 ] }
=> { "_id" : 6, "type" : "food", "item" : "bbb", "ratings" : [ 5, 9 ] }

There's a lot more on embedded documents