throwing error when the something equals _

[luxdo]

I am writing to report a potential bug I encountered while using the RediSearch om library.

Issue:

When attempting to search for terms containing an underscore character (_), we receive a SearchError indicating a syntax error in the query. The specific error message states:

Syntax error at offset 10 near token "_".
This is often the result of using a stop word in the query.

However, the provided link to the list of default stop words in RediSearch (https://github.com/stopwords-iso/stopwords-en) does not include the underscore character. (also the stopwords link in the description of redis-om-node is not available-showing 404 error)

Steps to Reproduce:
const blockedIp = await limitedsRepository
.search()
.where('ip')
.equals(ip)
.and('token')
.equals(token) // throws an error when token = '_'
.return.first()
Execute the search query containing an underscore (e.g., "search_term").
Observe the SearchError with the aforementioned message.
Expected Behavior:

The search query should be processed successfully, even with terms containing underscores. This is because the underscore is a commonly used character in various contexts and shouldn't be treated as a stop word.

Impact:

This issue prevents us from searching for data containing underscores in our application, potentially limiting the flexibility of our search functionality.

Request:

We kindly request you to investigate this potential bug and consider the following:

Is this a documented limitation of RediSearch queries? If so, I would appreciate clarification on how to handle such cases.
Could there be a configuration option to exclude the underscore character from the default stop word list?
I believe this issue would be beneficial to address for broader usability of RediSearch.

---

Dear Guy and Redis OM Development Team,

I am writing to request the addition of the following features to the Redis OM library:

Relationship-based querying: The ability to query data based on relationships between entities would significantly enhance the flexibility and power of the library. This would allow developers to perform complex queries that involve multiple entities and their connections.

Database transactions: Implementing support for database transactions would ensure data consistency and integrity, especially in scenarios where multiple operations need to be performed atomically.

Simplified configuration for Redis persistence: Providing an easy-to-use API function to configure Redis persistence data permanently (either AOF or RDB) would streamline the setup process and make it more accessible to developers.

These features would greatly improve the usability and capabilities of the Redis OM library, making it a more compelling choice for developers working with Redis.

Once these features have been added, I would be interested in hearing your perspective on the advantages of using Redis OM over other competing libraries. What sets Redis OM apart and makes it the preferred choice for developers?

Thank you for your time and consideration.

[guyroyse]

Could you provide the code that you used to create your schema? This impacts how search works in Redis itself. I'm guessing these fields are either text or string but which makes a difference.

Also—and this isn't essential but it does make everyone's life easier—you can escape code with back-ticks to add a fixed-width font and syntax highlighting. For example the very nice looking:

let hello = () => { return 'Hello World' };

Can be accomplished by typing:

```javascript
let hello = () => { return 'Hello World' };
```

This works inline as well with a single back-tick. For example:

This is fixed-width text.

Can be accomplished by typing:

This is `fixed-width` text.

Regarding the other feature requests, those would be better placed in another issue so this one can be focused on the particular error you are seeing.

Thanks!

[luxdo]

Sure, here is the schema

const limitedsSchema = new Schema('limiteds', {
	ip: { type: 'string' },
	token: { type: 'string' },
	createdAt: { type: 'date' }
});

[guyroyse]

Successfully reproduced this with a simplified schema:

import { createClient } from "redis";
import { Repository, Schema } from "redis-om";

const redis = await createClient()
  .on("error", (err) => console.error("Redis Client Error", err))
  .connect();

// WARNING: wipes entire database. For test use only.
await redis.flushAll();

const schema = new Schema("underscore", {
  field: { type: "string" },
});

const repository = new Repository(schema, redis);
await repository.createIndex();

await repository.save({ field: "alfa" });
await repository.save({ field: "bravo" });
await repository.save({ field: "charlie_delta" });
await repository.save({ field: "_charlie_delta_" });
await repository.save({ field: "_" });
await repository.save({ field: "_charlie" });
await repository.save({ field: "delta_" });
await repository.save({ field: "_echo_" });

// works fine, of course
let entities = await repository
  .search()
  .where("field")
  .equals("alfa")
  .return.all();

console.log(entities);

// this works too
entities = await repository
  .search()
  .where("field")
  .equals("charlie_delta")
  .return.all();

console.log(entities);

// this fails
entities = await repository
  .search()
  .where("field")
  .equals("_")
  .return.all();

console.log(entities);

redis.quit();

[guyroyse]

I know what the problem is.

The query parser for Redis search doesn't know how to handle a query that looks like this: (@field:{_}).
It will, however, handle it if you escape the underscore: (@field:{\_}).

It's also happy to deal with strings that contain underscores. Just not ones that are exclusively underscores.

I can patch Redis OM to handle this scenario and escape the underscore automatically. In the meantime, you can work around this by executing a raw query instead and escaping the underscore for Redis' benefit:

entities = await repository.searchRaw("@field:{\\_}").return.all();

Hope this helps in the short term and thanks for finding this! I'll get to it as soon as I can.