superset.exceptions.SupersetErrorsException: [SupersetError(message='questdb error: QuestDbEngineSpec.execute() takes 3 positional arguments but 4 were given', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'QuestDB', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error

[mehrdad2000]

Bug description

Hi
try this integrate superset docker version with QuestDB as explain here: https://questdb.io/docs/third-party-tools/superset/

after start docker and add database when i run simple select query got this error:

Here is the error that i've got in superset webui:

QuestDB Error
questdb error: QuestDbEngineSpec.execute() takes 3 positional arguments but 4 were given
This may be triggered by:
Issue 1002 - The database returned an unexpected error. 

Here is the log:

superset_app          | 2024-08-21 06:15:19,877:WARNING:superset.views.error_handling:SupersetErrorsException
superset_app          | Traceback (most recent call last):
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request
superset_app          |     rv = self.dispatch_request()
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request
superset_app          |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
superset_app          |     return f(self, *args, **kwargs)
superset_app          |   File "/app/superset/views/base_api.py", line 119, in wraps
superset_app          |     duration, response = time_function(f, self, *args, **kwargs)
superset_app          |   File "/app/superset/utils/core.py", line 1364, in time_function
superset_app          |     response = func(*args, **kwargs)
superset_app          |   File "/app/superset/views/base_api.py", line 91, in wraps
superset_app          |     return f(self, *args, **kwargs)
superset_app          |   File "/app/superset/utils/log.py", line 303, in wrapper
superset_app          |     value = f(*args, **kwargs)
superset_app          |   File "/app/superset/sqllab/api.py", line 409, in execute_sql_query
superset_app          |     command_result: CommandResult = command.run()
superset_app          |   File "/app/superset/utils/decorators.py", line 266, in wrapped
superset_app          |     return on_error(ex)
superset_app          |   File "/app/superset/utils/decorators.py", line 236, in on_error
superset_app          |     raise ex
superset_app          |   File "/app/superset/utils/decorators.py", line 259, in wrapped
superset_app          |     result = func(*args, **kwargs)
superset_app          |   File "/app/superset/commands/sql_lab/execute.py", line 106, in run
superset_app          |     status = self._run_sql_json_exec_from_scratch()
superset_app          |   File "/app/superset/commands/sql_lab/execute.py", line 161, in _run_sql_json_exec_from_scratch
superset_app          |     return self._sql_json_executor.execute(
superset_app          |   File "/app/superset/sqllab/sql_json_executer.py", line 113, in execute
superset_app          |     raise SupersetErrorsException(
superset_app          | superset.exceptions.SupersetErrorsException: [SupersetError(message='questdb error: QuestDbEngineSpec.execute() takes 3 positional arguments but 4 were given', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'QuestDB', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]



gunicorn 22.0.0
questdb-connect==1.0.12
Redis version=7.4.0
PostgreSQL 15.8 (Debian 15.8-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
celery@d41d4bb1f20b v5.4.0 (opalescent)
Linux-5.14.0-70.13.1.el9_0.x86_64-x86_64-with-glibc2.36 (edited) 

How to reproduce the bug

..

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

[dosubot]

Hey @mehrdad2000! I'm here to help you with any issues you're facing. Let's work together to solve this problem!

To resolve the error "QuestDbEngineSpec.execute() takes 3 positional arguments but 4 were given" when integrating Superset with QuestDB using Docker, you need to ensure that the execute method in the QuestDbEngineSpec class is defined to accept the correct number of arguments.

The error message indicates that the execute method is currently defined to accept 3 positional arguments, but it is being called with 4 arguments. You need to modify the method definition to accept 4 arguments.

Here is an example of how you can modify the execute method to accept 4 arguments:

class QuestDbEngineSpec(BaseEngineSpec):
    # Other methods and attributes

    @classmethod
    def execute(cls, cursor, query, database, **kwargs):
        # Method implementation
        pass

In this example, the execute method is defined to accept cursor, query, database, and **kwargs as arguments. The **kwargs allows for any additional keyword arguments to be passed without causing an error.

Make sure to update the execute method in the QuestDbEngineSpec class accordingly to resolve the error ^[1][2].

To continue talking to Dosu, mention @dosu.

[mehrdad2000]

where is the QuestDbEngineSpec file?

[dosubot]

The QuestDbEngineSpec class should be located in the file that defines the database engine specifications for QuestDB. Typically, this would be in a file within the superset/db_engine_specs directory. Look for a file named something like questdb.py or similar within that directory.

If you don't find it there, you can search the repository for the QuestDbEngineSpec class definition to locate the exact file. Once you find the file, you can modify the execute method as needed.

To continue talking to Dosu, mention @dosu.

[mehrdad2000]

questdb.py not exist in that path!

[dosubot]

You can find the QuestDbEngineSpec file in the superset/db_engine_specs/questdb.py module ^[1]. If the file does not exist, it might be that the QuestDB integration is not included in your current version of Superset. You may need to create the file yourself or check if there are any updates or plugins that need to be installed to support QuestDB.

To continue talking to Dosu, mention @dosu.

[mehrdad2000]

solution wrong and there is no file like that.

[uthay-logicsoft]

So we are also experiencing the same issue,

We are experimenting the Superset for our use cases on this process, We are trying to connect to our SQL Server which is connected via LAN, I am able to connect to the Postgresql database which is available along side of the same SQL Server on the destination machine,

From : Superset application, (Docker Engine on Linux Superset deployed on via docker compose)
Destination: Windows machine SQLServer 2019
Running the Container Via: Docker Compose File.
Error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 127, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 121, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1470, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/utils/log.py", line 255, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 93, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/databases/api.py", line 920, in test_connection
    TestConnectionDatabaseCommand(item).run()
  File "/app/superset/commands/database/test_connection.py", line 200, in run
    raise SupersetErrorsException(errors) from ex
superset.exceptions.SupersetErrorsException: [SupersetError(message="(builtins.NoneType) None\n[SQL: (pymssql.exceptions.OperationalError) (20009, b'DB-Lib error message 20009, severity 9:\\nUnable to connect: Adaptive Server is unavailable or does not exist (#@<ip-masked>)\\n')\n(Background on this error at: https://sqlalche.me/e/14/e3q8)]\n(Background on this error at: https://sqlalche.me/e/14/dbapi)", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Microsoft SQL Server', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]

[rusackas]

Did you install the driver?
https://superset.apache.org/docs/configuration/databases/#installing-database-drivers

[uthay-logicsoft]

Did you install the driver? https://superset.apache.org/docs/configuration/databases/#installing-database-drivers

Yeah we did that step also using docker exec via pip

[kalai-logicsoft]

In our case we have not passed the password properly, the thing is we have to pass the password / all values as URL encoded if at all it has special characters like ` @#$%^&*()!~.,/', ect., Most cases or in our case we have special characters in password.

Read More about this Here in the - SQLAlchemy 2.0 Documentation

Sample connection string looks like mssql+pymssql://test:test%40123@localhost:1433/sampledb
Here the password is test@123 which is turned into test%40123 by doing URL encoding.

Sample code to encode a URL using python is,

import urllib.parse

print(urllib.parse.quote_plus('test@123'))

# which will result
# test%40123
# %40 -means @

Actually the error code 20009 means more, see here. By This any error on the database side like database not found, login failed, timeout, or any error from MSSqlServer will be informed with 20009.

Do a sample python program to validate is that connection able to be made to the destination machine.
I tried to find the error codes page, couldn't be able to find it, anyone has that information which will help please post here.

[uthay-logicsoft]

Now its resolved, thank you to @kalai-logicsoft

[rusackas]

@mehrdad2000 if you agree, maybe we can close this issue?

Would anyone on the thread be able/willing to make a documentation update to help others running into this in the future? I'm happy to help anyone with that dev flow and navigating the PR cycle if needed.

[kalai-logicsoft]

@rusackas Hi, I will do the required documentation update and related things, please guide me through the steps and what to do.

[rusackas]

@rusackas Hi, I will do the required documentation update and related things, please guide me through the steps and what to do.

I wasn't sure as I neither encountered nor fixt the issue. If it is indeed just URL encoding, you can see that this page has a note for Athena.

If this fix applies to other DBs, we should add it in those entries. QuestDB doesn't even have an entry, so that may be useful. If ALL databases are better off with URL encoding, we should look for a more generalized place to add that note.

[kalai-logicsoft]

@rusackas I have added required documentation changes on this PR.

If this fix applies to other DBs, we should add it in those entries. QuestDB doesn't even have an entry, so that may be useful. If ALL databases are better off with URL encoding, we should look for a more generalized place to add that note.

I understand your point, but I'm also unsure where to add this information. Let's discuss it further within the PR itself to gain more clarity.