fix: ODPS data source table preview failed #32301

[zhutong6688]

SUMMARY

I fixed an error when using sqllab to preview data after configuring the odps data source in superset, and added partition restrictions for previewing SQL

fixes: #32301

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

This is a preview of the odps table data before the repair：

This is a preview of the repaired ODPS table data：



In addition, when using other data sources, there is no problem with testing and it does not affect other data sources

TESTING INSTRUCTIONS

1. First, configure the ODPS data source
2. Create a new SQLlab page in the "SQL" column, select the database Schema
3. Select partitioned tables, non partitioned tables, and multi partitioned tables in sequence to test the correctness of data preview
4. Choose another data source, such as an example data source like SQLite, and follow the same steps as above to verify the validity of the data preview

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[github-actions]

Congrats on making your first PR and thank you for contributing to Superset! 🎉 ❤️

We hope to see you in our Slack community too! Not signed up? Use our Slack App to self-register.

[korbit-ai]

Review by Korbit AI

Korbit automatically attempts to detect when you fix issues in new commits.

Category	Issue	Fix Detected
	Undefined partition variable in ODPS preview logic ▹ view	✅

Files scanned

​

File Path	Reviewed
superset/databases/utils.py	✅
superset/daos/database.py	✅
superset/sql/parse.py	✅
superset/models/core.py	✅
superset/databases/api.py	✅
superset/db_engine_specs/base.py	✅

Explore our documentation to understand the languages and file types we support and the files we ignore.

Need a new review? Comment /korbit-review on this PR and I'll review your latest changes.

Korbit Guide: Usage and Customization

Interacting with Korbit

• You can manually ask Korbit to review your PR using the /korbit-review command in a comment at the root of your PR.
• You can ask Korbit to generate a new PR description using the /korbit-generate-pr-description command in any comment on your PR.
• Too many Korbit comments? I can resolve all my comment threads if you use the /korbit-resolve command in any comment on your PR.
• Chat with Korbit on issues we post by tagging @korbit-ai in your reply.
• Help train Korbit to improve your reviews by giving a 👍 or 👎 on the comments Korbit posts.

Customizing Korbit

• Check out our docs on how you can make Korbit work best for you and your team.
• Customize Korbit for your organization through the Korbit Console.

Feedback and Support

• Tell us what you think of Korbit
• Schedule a call with our team
• Email us @ support@korbit.ai

[hainenber]

Thank you for the PR! I left a few comments and also you'll need to run added changes against established pre-commit hooks here.

[hainenber]

Re-assign DB's password to a 3rd variable seems odd here.

[zhutong6688]

Access_key=passwd, I assigned passwd to Access_key, so that when using ODPS to build connections later, I have a visual experience of AK and Secret

[hainenber]

Are there any benefits aside of readability? I'm not holding strong opinion on this but feels like this assignment is a bit excessive.

[zhutong6688]

Could you please help me run a CI again so that I can see if there are any other check errors，Thanks again

[zhutong6688]

please help me run a CI again，thanks a lot

[zhutong6688]

@hainenber If you have time now，please help me run a CI again，thanks a lot

[zhutong6688]

@hainenber May I ask if this error is caused by not installing and using pre commit?

[hainenber]

I'd recommend to have precommit installed and used to catch early CI issues.

[zhutong6688]

I'd recommend to have precommit installed and used to catch early CI issues.

Okay, got it

[zhutong6688]

@hainenber please help me run a CI again，thanks a lot

[zhutong6688]

Could you please help me agree and check the code online? Thank you

[hainenber]

CI has already run. I’ll be reviewing your code.

[zhutong6688]

I found that the code for table preview has been greatly adjusted, which is exactly where I fixed it. The interface calls are different from what I modified before, and the only thing fixed is the improvement. It is suggested to slow down the improvement of this section of code, as there are indeed many bugs and bug fixes cannot keep up with the speed of improving the code. This is not reasonable, and it is more appropriate to prioritize bug fixes in the code.

[zhutong6688]

@hainenber I have adjusted the code. Thank you for agreeing to run the constantly blocked reviews。

[zhutong6688]

@hainenber @rusackas @sadpandajoe @bgreenlee Let's take a look at this PR (#32556) later. The previous PR (#32310) modified a lot of underlying basic classes. Although other data sources were not affected according to my testing, the testing unit classes of superset need to be adjusted too much. Therefore, I submitted a new PR, which tries not to modify the underlying basic classes as much as possible, but modifies the logical collection to the odps.by file, without causing too much impact on a large number of unit testing classes.

[zhutong6688]

Let's pay attention to this PR（https://github.com/apache/superset/pull/32556） in the future. This PR will be closed for now. It has modified many underlying classes, which requires too many modifications to the test classes of the superset project itself。