Merge remote-tracking branch 'upstream/master' into feat/add_all_dash…

…boards_permissions * upstream/master: (32 commits) docs: Add a note to contributing.md on reporting security vulnerabilities (apache#10796) Fix: Include RLS filters for cache keys (apache#10805) feat: filters for database list view (apache#10772) fix: MVC show saved query (apache#10781) added creator column and adjusted order columns (apache#10789) security: disallow uuid package on jinja2 (apache#10794) feat: CRUD REST API for saved queries (apache#10777) fix: disable domain sharding on explore view (apache#10787) fix: can not type `0.05` in `TextControl` (apache#10778) fix: pivot table timestamp grouping (apache#10774) fix: add validator information to email/slack alerts (apache#10762) More Label touchups (margins) (apache#10722) fix: dashboard extra filters (apache#10692) fix: re-installing local superset in cache image (apache#10766) feat: SIP-34 table list view for databases (apache#10705) refactor: convert DatasetList schema filter to use new distinct api (apache#10746) chore: removing fsevents dependency (apache#10751) Fix precommit hook for docs/installation.rst (apache#10759) feat(database): POST, PUT, DELETE API endpoints (apache#10741) docs: Update OAuth configuration in installation.rst (apache#10748) ...
ofekisr · Sep 7, 2020 · d709758 · d709758
2 parents 0dc2d4b + 1d76c59
commit d709758
Show file tree

Hide file tree

Showing 97 changed files with 5,207 additions and 3,373 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -42,6 +42,7 @@ little bit helps, and credit will always be given.
       - [Merging](#merging)
       - [Post-merge Responsibility](#post-merge-responsibility)
   - [Managing Issues and PRs](#managing-issues-and-prs)
+  - [Reporting a Security Vulnerability](#reporting-a-security-vulnerability)
   - [Revert Guidelines](#revert-guidelines)
   - [Setup Local Environment for Development](#setup-local-environment-for-development)
     - [Documentation](#documentation)
@@ -264,6 +265,12 @@ If the PR passes CI tests and does not have any `need:` labels, it is ready for
 
 If an issue/PR has been inactive for >=30 days, it will be closed. If it does not have any status label, add `inactive`.
 
+## Reporting a Security Vulnerability
+
+Please report security vulnerabilities to private@superset.apache.org.
+
+In the event a community member discovers a security flaw in Superset, it is important to follow the [Apache Security Guidelines](https://www.apache.org/security/committers.html) and release a fix as quickly as possible before public disclosure. Reporting security vulnerabilities through the usual GitHub Issues channel is not ideal as it will publicize the flaw before a fix can be applied.
+
 ## Revert Guidelines
 
 Reverting changes that are causing issues in the master branch is a normal and expected part of the development process. In an open source community, the ramifications of a change cannot always be fully understood. With that in mind, here are some considerations to keep in mind when considering a revert:

diff --git a/Dockerfile-dev b/Dockerfile-dev
@@ -20,11 +20,13 @@
 ######################################################################
 FROM preset/superset:dev
 
-COPY ./requirements/*.txt ./docker/requirements-*.txt/ /app/requirements/
+COPY ./requirements/*.txt ./docker/requirements-*.txt /app/requirements/
+COPY ./setup.py ./MANIFEST.in /app/
 
 USER root
 # Cache everything for dev purposes...
 RUN cd /app \
-    && pip install --no-cache -r requirements/docker.txt \
-    && pip install --no-cache -r requirements/requirements-local.txt || true
+    && pip install -e . \
+    && pip install -r requirements/docker.txt \
+    && pip install -r requirements/requirements-local.txt || true
 USER superset
diff --git a/UPDATING.md b/UPDATING.md
@@ -23,6 +23,8 @@ assists people when migrating to a new version.
 
 ## Next
 
+* [10794](https://github.com/apache/incubator-superset/pull/10794): Breaking change: `uuid` python package is not supported on Jinja2 anymore, only uuid functions are exposed eg: `uuid1`, `uuid3`, `uuid4`, `uuid5`.
+
 * [10674](https://github.com/apache/incubator-superset/pull/10674): Breaking change: PUBLIC_ROLE_LIKE_GAMMA was removed is favour of the new PUBLIC_ROLE_LIKE so it can be set it whatever role you want.
 
 * [10590](https://github.com/apache/incubator-superset/pull/10590): Breaking change: this PR will convert iframe chart into dashboard markdown component, and remove all `iframe`, `separator`, and `markup` slices (and support) from Superset. If you have important data in those slices, please backup manually.

diff --git a/docker-compose.yml b/docker-compose.yml
@@ -71,9 +71,9 @@ services:
     volumes: *superset-volumes
 
   superset-node:
-    image: node:10-jessie
+    image: node:12
     container_name: superset_node
-    command: ["bash", "-c", "cd /app/superset-frontend && npm install --global webpack webpack-cli && npm install && npm run dev"]
+    command: ["bash", "-c", "cd /app/superset-frontend && npm install -f --no-optional --global webpack webpack-cli && npm install -f --no-optional && npm run dev"]
     env_file: docker/.env
     depends_on: *superset-depends-on
     volumes: *superset-volumes

diff --git a/docs/installation.rst b/docs/installation.rst
@@ -54,43 +54,89 @@ The Superset web server and the Superset Celery workers (optional)
 are stateless, so you can scale out by running on as many servers
 as needed.
 
-Start with Docker
------------------
+Install and Deploy Superset Locally with Docker
+-----------------------------------------------
 
-.. note ::
-    The Docker-related files and documentation are actively maintained and
-    managed by the core committers working on the project. Help and contributions
-    around Docker are welcomed!
+To try Superset locally, the
+best-supported currently method is via Docker, using ``docker-compose``. Superset
+does not have official support for Windows, so we have provided a VM
+workaround below. (We will update this documentation once Windows is
+supported.)
+
+**Step 0 - Install a Docker Engine and Docker Compose**
+
+*Mac OSX:*
+
+    `Install Docker for Mac <https://docs.docker.com/docker-for-mac/install/>`__, which includes the Docker engine and a recent version of `docker-compose` out of the box.
+
+    Once you have Docker for Mac installed, open up the preferences pane for Docker, go to the "Resources" section and increase the allocated memory to 6GB. With only the 2GB of RAM allocated by default, Superset will fail to start.
+
+
+*Linux:*
+
+    `Install Docker on Linux <https://docs.docker.com/engine/install/>`__ by following Docker’s instructions for whichever flavor of Linux suits you.
+
+    Because ``docker-compose`` is not installed as part of the base Docker installation on Linux, once you have a working engine, follow the `docker-compose installation instructions <https://docs.docker.com/compose/install/>`__ for Linux.
+
+
+*Windows:*
+
+    NOTE: Windows is currently not a supported environment for Superset installation.
+
+    For Windows users, the best option may be to install an Ubuntu Desktop VM via `VirtualBox <https://www.virtualbox.org/>`__ and proceed with the Docker on Linux instructions inside of that VM. It is recommended to assign at least 8GB of RAM to the virtual machine as well as provisioning a hard drive of at least 40GB, so that there will be enough space for both the OS and all of the required dependencies.
 
-If you know docker, then you're lucky, we have shortcut road for you to
-initialize development environment: ::
+**Step 1 - Clone Superset's Github repository**
 
-    git clone https://github.com/apache/incubator-superset/
-    cd incubator-superset
-    # you can run this command everytime you need to start superset now:
-    docker-compose up
+`Clone Superset's repo <https://github.com/apache/incubator-superset>`__
+in your terminal with the following command:
 
-After several minutes for superset initialization to finish, you can open
-a browser and view `http://localhost:8088` to start your journey. By default
-the system configures an admin user with the username of `admin` and a password
-of `admin` - if you are in a non-local environment it is highly recommended to
-change this username and password at your earliest convenience.
+.. code:: bash
 
-From there, the container server will reload on modification of the superset python
-and javascript source code.
-Don't forget to reload the page to take the new frontend into account though.
+    $ git clone https://github.com/apache/incubator-superset.git
 
-See also `CONTRIBUTING.md#building <https://github.com/apache/incubator-superset/blob/master/CONTRIBUTING.md#building>`_,
-for alternative way of serving the frontend.
+Once that command completes successfully, you should see a new
+``incubator-superset`` folder in your current directory.
 
-It is currently not recommended to run docker-compose in production.
+**Step 2 - Launch Superset via `docker-compose up`**
 
-If you are attempting to build on a Mac and it exits with 137 you need to increase your docker resources.
-OSX instructions: https://docs.docker.com/docker-for-mac/#advanced (Search for memory)
+Next, `cd` into the folder you created in Step 1:
 
-Or if you're curious and want to install superset from bottom up, then go ahead.
+.. code:: bash
 
-See also `docker/README.md <https://github.com/apache/incubator-superset/blob/master/docker/README.md>`_
+    $ cd incubator-superset
+
+Once you're in the directory, run the following command:
+
+.. code:: bash
+
+    $ docker-compose up
+
+You should see a wall of logging output from the containers being
+launched on your machine. Once this output slows to a crawl, you should
+have a running instance of Superset on your local machine!
+
+**Step 3 - Log In to Superset**
+
+Your Superset local instance also includes a Postgres server to store
+your data and is already pre-loaded with some example datasets that ship
+with Superset. You can access Superset now via your web browser by
+visiting ``http://localhost:8088``. Note that many browsers now default
+to ``https`` - if yours is one of them, please make sure it uses
+``http``.
+
+Log in with the default username and password:
+
+::
+
+    username: admin
+    password: admin
+
+Congrats! You have successfully installed Superset!
+
+.. note ::
+    The Docker-related files and documentation are actively maintained and
+    managed by the core committers working on the project. Help and contributions
+    around Docker are welcomed! See also `docker/README.md <https://github.com/apache/incubator-superset/blob/master/docker/README.md>`_ for additional information.
 
 OS dependencies
 ---------------
@@ -1449,19 +1495,15 @@ The first step: Configure authorization in Superset ``superset_config.py``.
             'token_key':'access_token', # Name of the token in the response of access_token_url
             'icon':'fa-address-card',   # Icon for the provider
             'remote_app': {
-                'consumer_key':'myClientId',  # Client Id (Identify Superset application)
-                'consumer_secret':'MySecret', # Secret for this Client Id (Identify Superset application)
-                'request_token_params':{
+                'client_id':'myClientId',  # Client Id (Identify Superset application)
+                'client_secret':'MySecret', # Secret for this Client Id (Identify Superset application)
+                'client_kwargs':{
                     'scope': 'read'               # Scope for the Authorization
                 },
-                'access_token_method':'POST',    # HTTP Method to call access_token_url
                 'access_token_params':{        # Additional parameters for calls to access_token_url
                     'client_id':'myClientId'
                 },
-                'access_token_headers':{    # Additional headers for calls to access_token_url
-                    'Authorization': 'Basic Base64EncodedClientIdAndSecret'
-                },
-                'base_url':'https://myAuthorizationServer/oauth2AuthorizationServer/',
+                'api_base_url':'https://myAuthorizationServer/oauth2AuthorizationServer/',
                 'access_token_url':'https://myAuthorizationServer/oauth2AuthorizationServer/token',
                 'authorize_url':'https://myAuthorizationServer/oauth2AuthorizationServer/authorize'
             }

diff --git a/docs/sqllab.rst b/docs/sqllab.rst
@@ -73,7 +73,10 @@ Superset's Jinja context:
 
 - ``time``: ``time``
 - ``datetime``: ``datetime.datetime``
-- ``uuid``: ``uuid``
+- ``uuid1``: ``uuid1``
+- ``uuid3``: ``uuid3``
+- ``uuid4``: ``uuid4``
+- ``uuid5``: ``uuid5``
 - ``random``: ``random``
 - ``relativedelta``: ``dateutil.relativedelta.relativedelta``
 

diff --git a/requirements/base.txt b/requirements/base.txt
@@ -9,16 +9,16 @@
 aiohttp==3.6.2            # via slackclient
 alembic==1.4.2            # via flask-migrate
 amqp==2.6.1               # via kombu
-apispec[yaml]==3.3.1      # via flask-appbuilder
+apispec[yaml]==3.3.2      # via flask-appbuilder
 async-timeout==3.0.1      # via aiohttp
 attrs==20.1.0             # via aiohttp, jsonschema
 babel==2.8.0              # via flask-babel
 backoff==1.10.0           # via apache-superset
 billiard==3.6.3.0         # via celery
 bleach==3.1.5             # via apache-superset
-boto3==1.14.48            # via tabulator
-botocore==1.17.48         # via boto3, s3transfer
-brotli==1.0.7             # via flask-compress
+boto3==1.14.51            # via tabulator
+botocore==1.17.51         # via boto3, s3transfer
+brotli==1.0.9             # via flask-compress
 cached-property==1.5.1    # via tableschema
 cachelib==0.1.1           # via apache-superset
 celery==4.4.7             # via apache-superset
@@ -29,7 +29,7 @@ click==7.1.2              # via apache-superset, flask, flask-appbuilder, tables
 colorama==0.4.3           # via apache-superset, flask-appbuilder
 contextlib2==0.6.0.post1  # via apache-superset
 croniter==0.3.34          # via apache-superset
-cryptography==3.0         # via apache-superset
+cryptography==3.1         # via apache-superset
 dataclasses==0.6          # via apache-superset
 decorator==4.4.2          # via retry
 defusedxml==0.6.0         # via python3-openid
@@ -41,7 +41,7 @@ flask-appbuilder==3.0.1   # via apache-superset
 flask-babel==1.0.0        # via flask-appbuilder
 flask-caching==1.9.0      # via apache-superset
 flask-compress==1.5.0     # via apache-superset
-flask-cors==3.0.8         # via apache-superset
+flask-cors==3.0.9         # via apache-superset
 flask-jwt-extended==3.24.1  # via flask-appbuilder
 flask-login==0.4.1        # via flask-appbuilder
 flask-migrate==2.5.3      # via apache-superset
@@ -88,7 +88,7 @@ polyline==1.4.0           # via apache-superset
 prison==0.1.3             # via flask-appbuilder
 psycopg2-binary==2.8.5    # via apache-superset
 py==1.9.0                 # via retry
-pyarrow==0.17.1           # via apache-superset
+pyarrow==1.0.1            # via apache-superset
 pycparser==2.20           # via cffi
 pydruid==0.6.1            # via apache-superset
 pyhive[hive]==0.6.3       # via apache-superset

diff --git a/requirements/testing.txt b/requirements/testing.txt
@@ -22,13 +22,13 @@ isort==5.4.2              # via pylint
 jedi==0.17.2              # via ipython
 lazy-object-proxy==1.4.3  # via astroid
 mccabe==0.6.1             # via pylint
-more-itertools==8.4.0     # via pytest
+more-itertools==8.5.0     # via pytest
 openapi-spec-validator==0.2.9  # via -r requirements/testing.in
 parameterized==0.7.4      # via -r requirements/testing.in
 parso==0.7.1              # via jedi
 pexpect==4.8.0            # via ipython
 pickleshare==0.7.5        # via ipython
-prompt-toolkit==3.0.6     # via ipython
+prompt-toolkit==3.0.7     # via ipython
 ptyprocess==0.6.0         # via pexpect
 pygments==2.6.1           # via ipython
 pyhive[hive,presto]==0.6.3  # via -r requirements/testing.in, apache-superset

diff --git a/setup.py b/setup.py
@@ -90,14 +90,14 @@ def get_git_sha():
         "isodate",
         "markdown>=3.0",
         "msgpack>=1.0.0, <1.1",
-        "pandas>=1.0.3, <1.1",
+        "pandas>=1.0.5, <1.1",
         "parsedatetime",
         "pathlib2",
         "polyline",
         "python-dateutil",
         "python-dotenv",
         "python-geohash",
-        "pyarrow>=0.17.0, <0.18",
+        "pyarrow>=1.0.1, <1.1",
         "pyyaml>=5.1",
         "retry>=0.9.2",
         "selenium>=3.141.0",

diff --git a/superset-frontend/.storybook/main.js b/superset-frontend/.storybook/main.js
@@ -22,7 +22,7 @@ const path = require('path');
 const customConfig = require('../webpack.config.js');
 
 module.exports = {
-  stories: ['../src/components/**/*.stories.@(t|j)sx'],
+stories: ['../src/@(components|common)/**/*.stories.@(t|j)sx'],
   addons: [
     '@storybook/addon-essentials',
     '@storybook/addon-links',