-
Notifications
You must be signed in to change notification settings - Fork 65
Feat/sql redis query #467
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Feat/sql redis query #467
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -30,6 +30,7 @@ | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| from redisvl.query.hybrid import HybridQuery | ||||||||||||||||||||||||||||||||||||||||||
| from redisvl.query.query import VectorQuery | ||||||||||||||||||||||||||||||||||||||||||
| from redisvl.query.sql import SQLQuery | ||||||||||||||||||||||||||||||||||||||||||
| from redisvl.redis.utils import ( | ||||||||||||||||||||||||||||||||||||||||||
| _keys_share_hash_tag, | ||||||||||||||||||||||||||||||||||||||||||
| async_cluster_create_index, | ||||||||||||||||||||||||||||||||||||||||||
|
|
@@ -894,6 +895,49 @@ def _aggregate(self, aggregation_query: AggregationQuery) -> List[Dict[str, Any] | |||||||||||||||||||||||||||||||||||||||||
| storage_type=self.schema.index.storage_type, | ||||||||||||||||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| def _sql_query(self, sql_query: SQLQuery) -> List[Dict[str, Any]]: | ||||||||||||||||||||||||||||||||||||||||||
| """Execute a SQL query and return results. | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| Args: | ||||||||||||||||||||||||||||||||||||||||||
| sql_query: The SQLQuery object containing the SQL statement. | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| Returns: | ||||||||||||||||||||||||||||||||||||||||||
| List of dictionaries containing the query results. | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| Raises: | ||||||||||||||||||||||||||||||||||||||||||
| ImportError: If sql-redis package is not installed. | ||||||||||||||||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||||||||||||||||
| try: | ||||||||||||||||||||||||||||||||||||||||||
| from sql_redis.executor import Executor | ||||||||||||||||||||||||||||||||||||||||||
| from sql_redis.schema import SchemaRegistry | ||||||||||||||||||||||||||||||||||||||||||
| except ImportError: | ||||||||||||||||||||||||||||||||||||||||||
| raise ImportError( | ||||||||||||||||||||||||||||||||||||||||||
| "sql-redis is required for SQL query support. " | ||||||||||||||||||||||||||||||||||||||||||
| "Install it with: pip install redisvl[sql]" | ||||||||||||||||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| registry = SchemaRegistry(self._redis_client) | ||||||||||||||||||||||||||||||||||||||||||
| registry.load_all() # Loads index schemas from Redis | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| executor = Executor(self._redis_client, registry) | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| # Execute the query with any params | ||||||||||||||||||||||||||||||||||||||||||
| result = executor.execute(sql_query.sql, params=sql_query.params) | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| # Decode bytes to strings in the results (Redis may return bytes) | ||||||||||||||||||||||||||||||||||||||||||
| decoded_rows = [] | ||||||||||||||||||||||||||||||||||||||||||
| for row in result.rows: | ||||||||||||||||||||||||||||||||||||||||||
| decoded_row = {} | ||||||||||||||||||||||||||||||||||||||||||
| for key, value in row.items(): | ||||||||||||||||||||||||||||||||||||||||||
| # Decode key if bytes | ||||||||||||||||||||||||||||||||||||||||||
| str_key = key.decode("utf-8") if isinstance(key, bytes) else key | ||||||||||||||||||||||||||||||||||||||||||
| # Decode value if bytes | ||||||||||||||||||||||||||||||||||||||||||
| str_value = value.decode("utf-8") if isinstance(value, bytes) else value | ||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+932
to
+935
|
||||||||||||||||||||||||||||||||||||||||||
| # Decode key if bytes | |
| str_key = key.decode("utf-8") if isinstance(key, bytes) else key | |
| # Decode value if bytes | |
| str_value = value.decode("utf-8") if isinstance(value, bytes) else value | |
| # Decode key if bytes, but preserve non-UTF-8 bytes | |
| if isinstance(key, bytes): | |
| try: | |
| str_key = key.decode("utf-8") | |
| except UnicodeDecodeError: | |
| str_key = key | |
| else: | |
| str_key = key | |
| # Decode value if bytes, but preserve non-UTF-8 bytes | |
| if isinstance(value, bytes): | |
| try: | |
| str_value = value.decode("utf-8") | |
| except UnicodeDecodeError: | |
| str_value = value | |
| else: | |
| str_value = value |
Copilot
AI
Jan 29, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docstring for the query method states "This method takes a BaseQuery, AggregationQuery, or HybridQuery object directly" but should also mention SQLQuery since it's now a supported query type. Additionally, the Args section references the old type hint without SQLQuery.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| """SQL Query class for executing SQL-like queries against Redis.""" | ||
|
|
||
| from typing import Any, Dict, Optional | ||
|
|
||
|
|
||
| class SQLQuery: | ||
| """A query class that translates SQL-like syntax into Redis queries. | ||
|
|
||
| This class allows users to write SQL SELECT statements that are | ||
| automatically translated into Redis FT.SEARCH or FT.AGGREGATE commands. | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| from redisvl.query import SQLQuery | ||
| from redisvl.index import SearchIndex | ||
|
|
||
| index = SearchIndex.from_existing("products", redis_url="redis://localhost:6379") | ||
|
|
||
| sql_query = SQLQuery(''' | ||
| SELECT title, price, category | ||
| FROM products | ||
| WHERE category = 'electronics' AND price < 100 | ||
| ''') | ||
|
|
||
| results = index.query(sql_query) | ||
|
|
||
| Note: | ||
| Requires the optional `sql-redis` package. Install with: | ||
| ``pip install redisvl[sql]`` | ||
| """ | ||
|
|
||
| def __init__(self, sql: str, params: Optional[Dict[str, Any]] = None): | ||
| """Initialize a SQLQuery. | ||
|
|
||
| Args: | ||
| sql: The SQL SELECT statement to execute. | ||
| params: Optional dictionary of parameters for parameterized queries. | ||
| Useful for passing vector data for similarity searches. | ||
| """ | ||
| self.sql = sql | ||
| self.params = params or {} | ||
|
|
||
| def redis_query_string( | ||
| self, | ||
| redis_client: Optional[Any] = None, | ||
| redis_url: str = "redis://localhost:6379", | ||
| ) -> str: | ||
| """Translate the SQL query to a Redis command string. | ||
|
|
||
| This method uses the sql-redis translator to convert the SQL statement | ||
| into the equivalent Redis FT.SEARCH or FT.AGGREGATE command. | ||
|
|
||
| Args: | ||
| redis_client: A Redis client connection used to load index schemas. | ||
| If not provided, a connection will be created using redis_url. | ||
| redis_url: The Redis URL to connect to if redis_client is not provided. | ||
| Defaults to "redis://localhost:6379". | ||
|
|
||
| Returns: | ||
| The Redis command string (e.g., 'FT.SEARCH products "@category:{electronics}"'). | ||
|
|
||
| Raises: | ||
| ImportError: If sql-redis package is not installed. | ||
|
|
||
| Example: | ||
| .. code-block:: python | ||
|
|
||
| from redisvl.query import SQLQuery | ||
|
|
||
| sql_query = SQLQuery("SELECT * FROM products WHERE category = 'electronics'") | ||
|
|
||
| # Using redis_url | ||
| redis_cmd = sql_query.redis_query_string(redis_url="redis://localhost:6379") | ||
|
|
||
| # Or using an existing client | ||
| from redis import Redis | ||
| client = Redis() | ||
| redis_cmd = sql_query.redis_query_string(redis_client=client) | ||
|
|
||
| print(redis_cmd) | ||
| # Output: FT.SEARCH products "@category:{electronics}" | ||
| """ | ||
| try: | ||
| from sql_redis.schema import SchemaRegistry | ||
| from sql_redis.translator import Translator | ||
| except ImportError: | ||
| raise ImportError( | ||
| "sql-redis is required for SQL query support. " | ||
| "Install it with: pip install redisvl[sql]" | ||
| ) | ||
|
|
||
| # Get or create Redis client | ||
| if redis_client is None: | ||
| from redis import Redis | ||
|
|
||
| redis_client = Redis.from_url(redis_url) | ||
|
|
||
| # Load schemas from Redis | ||
| registry = SchemaRegistry(redis_client) | ||
| registry.load_all() | ||
|
|
||
| # Translate SQL to Redis command | ||
| translator = Translator(registry) | ||
|
|
||
| # Substitute non-bytes params in SQL before translation | ||
| sql = self.sql | ||
| for key, value in self.params.items(): | ||
| placeholder = f":{key}" | ||
| if isinstance(value, (int, float)): | ||
| sql = sql.replace(placeholder, str(value)) | ||
| elif isinstance(value, str): | ||
| sql = sql.replace(placeholder, f"'{value}'") | ||
| # bytes (vectors) are handled separately | ||
|
Comment on lines
+105
to
+113
|
||
|
|
||
| translated = translator.translate(sql) | ||
| return translated.to_command_string() | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The sql-redis dependency is using an absolute local file path specific to a developer's machine (/Users/robert.shelton/Documents/sql-redis/...). This will break on other systems and in CI/CD environments. According to the PR description, this is blocked on a release/update of sql-redis package. Before merging, this should be changed to reference a published version on PyPI (e.g., "sql-redis>=0.1.0").