feat(SIP-95): permissions for catalogs

[betodealmeida]

SUMMARY

Continuing the work on #22862. This PR:

• Adds catalog support to the Postgres DB engine spec (in Postgres, a catalog is called a "database").
• Adds a new endpoint /catalogs/ under the DB API, to list catalogs.
• Adds an optional parameter catalog to the /schemas/ DB API, to specify a non-default catalog.
• Updates the security manager and model to manage catalog-level permissions (automatically creating, renaming, and deleting them, similar to how we do for for schemas).

These are all backend changes that shouldn't affect the current behavior of Superset, other than introducing the new permissions. Note that as of today users can query different catalogs in SQL Lab, and even create (virtual) datasets in different catalogs, and our SQL parser is able to handle catalogs correctly; because of that, introducing these new permissions can be considered a security improvement.

In the next PR I'll start working on the UI to expose the catalog functionality, and enabling support for multi-catalog in a given database will be optional and default to false.

One thing that I'm not sure about is when we should rename existing schema permissions from [db].[schema] to [db].[default-catalog].[schema]. In this PR I'm doing it when the database gets updated. I don't think doing it in a migration is a good solution, because we'd have to add a new migration every time a DB engine spec is updated to support catalogs. Another option would be to do it during superset init — @dpgaspar do you have any thoughts here?

Edit: added a migration for the schema permissions, done by a helper function, and a comment in the base DB engine spec informing that when supports_catalog is set to True a migration is necessary.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

I've updated existing tests, and I'm adding new tests for all the changes. The PR is draft so it's easier to track which changes have been covered with tests.

ADDITIONAL INFORMATION

• Has associated issue: [SIP-95] Proposal for Catalog Support in SQL Lab #22862
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[betodealmeida]

Changing to false here and in other DB engine specs, since the specs should only have it set to true if they implement their own get_default_catalog. I'll update the DB engine specs in the near future.

[betodealmeida]

The current way memoized_func works is that the decorated function needs to take these parameters, even though they are only used by the decorator. This makes the function tightly coupled with the decorator, which is bad design.

I've fixed the decorator to pop these attributes, and removed them from the decorated functions.

[dpgaspar]

LGTM, I vote for db migrations instead of superset init

[john-bodley]

@betodealmeida @dpgaspar et al. the reason I believe this was a sorted list (as opposed to a set) was to ensure the results were always consistently rendered in SQL Lab et al. I don't disagree that a set is a more appropriate container, but it's contents aren't ordered. Maybe we should being using the ordered-set package for improved transparency.

CC @michael-s-molina

[betodealmeida]

Ah, thanks! I'll update the logic in SQL Lab to show it sorted there then.