[SIP-127] User Preferences

[villebro]

[SIP] Proposal for User Preferences

Motivation

Currently it's possible to customize various aspects of Superset:

• Color palettes
• Enable/disable thumbnails
• Default viz type
• Number formats and currencies
• etc

However, these are configured per deployment, meaning users can't override the globally defined defaults (note: Thumbnails can be enabled/disabled via a toggle switch in the chart/dashboard page when the FF is enabled, which isn't really the right place for it). In addition, the lack of user-specific configurability has made it difficult to introduce typically available customizability, like

• Timezone support (local vs fixed timezone)
• Light/dark mode
• Replacing colors in charts with decals/patterns for visually impaired

Proposed Change

To make Superset slightly less one-size-fits-all, we propose introducing a settings page where new user-specific settings can be easily introduced and changed per user. This would make it possible to change the functionality and appearance of Superset per user, without having to impose the same for all.

New or Changed Public Interfaces

A new config would be introduced which defines the default user settings. These could then be overridden per user, via a User Preferences page. The preferences would be stored in the metastore, and would be exposed via the API for normal CRUD operations. The API would be hosted on the /api/v1/user_preferences/ resource with typical CRUD verbs, and could be queried to retrieve a user's full set of preferences during page initialization, and then stored in Redux. This would be a move away from bootstrap data to make the initial html payload leaner. This also makes it easier to update the preference state during runtime without having to refresh the page.

Due to the simple data structure and limited need for querying individual preference keys, the full preference data would be stored in the key_value table, with a version id:

{
  "version": 1,
  "revision": 10,
  "preferences": {
    "timeZone": "PST",
    "somePreferenceState": {
      "key1": "value1",
      "key2": 123,
      "booleanKey": true,
    },
  }
}

The payload structure would be a TypedDict in the backend, making the key-value pairs strongly typed:

class SomePreferenceState(TypedDict):
    key1: str,
    key2: int
    booleanKey: bool

class UserPreferences(TypedDict):
    version: int
    revision: int
    timeZone: str
    somePreferenceState: SomePreferenceState

The version key would leave open the option to migrate keys in the future, if breaking changes need to be made. The revision key would indicate the revision of the preference state (this would be incremented on each update). The other keys (timeZone and somePreferenceState in the example) would be reserved for individual preferences. To protect against race conditions on updates, the preference object would be locked with KeyValueDistributedLock. This would ensure that only a single thread is updating it at a time. Note, that collisions in this state are highly unexpected, but are annoying to debug as they can cause weird side-effects.

Draft of User Preferences page

We should add the settings that we already know we'll need, even if we don't implement them in the POC. This will help us plan and organize the interface to be extended with additional settings later.

Migration Plan and Compatibility

The feature would be hidden behind a new feature flag until the user preferences page becomes stable. After that, the original feature flag would be removed, with new experimental preferences being introduced via an experimental flag to ensure they're not exposed to users before they're stable.

Rejected Alternatives

Adding light/dark/auto mode, timezone etc to the config, which takes effect for all users.

Pilot

As a pilot for this, I propose implementing one of the following:

1. Accessibility through decals

As discussed in #25379, ECharts has good support for adding decals to chart series to improve readability for users for whom the current color coding scheme doesn't work well (e.g. color blindness). By adding an option to the User Preferences page, we could render at least all ECharts charts with decals, making them more widely accessible to users.

2. Timezone support

a first version of the UI, and add user customizable timezone support, as Superset currently only works with naive or single timezone timestamps. As many databases work in UTC, it's often painful for users to convert UTC to whichever timezone they're in. While it's possible to force a certain timezone for a dataset/chart, enforcing a single timezone for all users can be difficult, especially if the users are distributed across multiple timezones.

I did some hacking on adding timezone support to the chart data endpoint a while ago, but decided against implementing it until it can be made generally available via a per-user customizable setting. See here: #23511

[rusackas]

Another needed example would be enabling accessibility features (e.g. ECharts decals/patterns) since few would want to turn this on deployment-wide, but any org would want their users to opt into such capabilities.

[villebro]

Another needed example would be enabling accessibility features (e.g. ECharts decals/patterns) since few would want to turn this on deployment-wide, but any org would want their users to opt into such capabilities.

Oh, super good example - I'll add it to the summary 👍

[michael-s-molina]

Thank you for the SIP @villebro. This is a much needed feature.

I'm always trying to dive deeper during analysis phase, even if our first version will be just a subset of the whole. I think it's important to have a vision of where we want to go and avoid structural decisions that block that vision because they were made considering only the subset. For that reason, I think it would be good if the SIP includes:

• A draft of the User Preferences page. We could add the settings that we already know we'll need, even if we don't implement them in the POC. This will help us plan and organize the interface to be extended with additional settings later. @kasiazjc
• The proposed API changes. We generally don't do this upfront and that's why many of our APIs are poorly designed. This is simply specifying the new or changed endpoints, params, payload, response, etc. Something like this.
• How the frontend should access this information. I believe we're going to use Redux here, so it would be good to see where this information will be placed and how it will be modeled in the Redux state.

I'm happy to collaborate with you and @kasiazjc on this effort as I think it's a very valuable feature for users.

[villebro]

thanks @michael-s-molina - I added an initial proposal for the API, state management and frontend handling of the preference object.

[rusackas]

Think this is ready for a DISCUSS thread?