SIP-32: Moving frontend code to the base of the repo (apache#9098)

* move assets out, get webpack dev working

* update docs to reference superset-frontend

* draw the rest of the owl

* fix docs

* fix webpack script

* rats

* correct docs

* fix tox dox

.dockerignore

@@ -34,7 +34,7 @@
 tests/
 docs/
 install/
-superset/assets/node_modules/
-superset/assets/cypress/
-superset/assets/coverage/
+superset-frontend/node_modules/
+superset-frontend/cypress/
+superset-frontend/coverage/
 venv

.fossa.yml

@@ -26,8 +26,8 @@ analyze:
   modules:
   - name: assets
     type: npm
-    target: superset/assets
-    path: superset/assets
+    target: superset-frontend
+    path: superset-frontend
   - name: docs
     type: pip
     target: docs

.gitignore

@@ -51,16 +51,18 @@ local_config.py
 superset.egg-info/
 superset/bin/supersetc
 tmp
+rat-results.txt
 
 # Node.js, webpack artifacts
 *.entry.js
 *.js.map
 node_modules
 npm-debug.log*
-superset/assets/coverage/*
-superset/assets/cypress/screenshots
-superset/assets/cypress/videos
-superset/assets/version_info.json
+superset-frontend/coverage/*
+superset-frontend/cypress/screenshots
+superset-frontend/cypress/videos
+superset-frontend/version_info.json
+superset/static
 yarn-error.log
 
 # IntelliJ

.rat-excludes

@@ -18,6 +18,7 @@ node_modules/*
 rat-results.txt
 babel-node
 dist
+superset/static/*
 build
 superset.egg-info
 apache_superset.egg-info
@@ -32,6 +33,7 @@ apache_superset.egg-info
 env/*
 docs/_build/*
 docs/_modules/*
+docs/_static/*
 _build/*
 _static/*
 .buildinfo

.travis.yml

@@ -110,7 +110,7 @@ cache:
     - ~/.npm
     - ~/.cache
     - ~/.travis_cache/
-    - superset/assets/.terser-plugin-cache/
+    - superset-frontend/.terser-plugin-cache/
 addons:
   apt:
     packages:

CONTRIBUTING.md

@@ -65,7 +65,7 @@ Here's a list of repositories that contain Superset-related packages:
   distributed on
   [pypi](https://pypi.org/project/apache-superset/). This repository
   also includes Superset's main Javascript bundles and react apps under
-  the [superset/assets](https://github.com/apache/incubator-superset/tree/master/superset/assets)
+  the [superset-frontend](https://github.com/apache/incubator-superset/tree/master/superset-frontend)
   folder.
 - [apache-superset/superset-ui](https://github.com/apache-superset/superset-ui)
   contains core Superset's
@@ -322,15 +322,15 @@ Then, [open a pull request](https://help.github.com/articles/about-pull-requests
 If you're adding new images to the documentation, you'll notice that the images
 referenced in the rst, e.g.
 
-    .. image:: _static/img/tutorial/tutorial_01_sources_database.png
+    .. image:: _static/images/tutorial/tutorial_01_sources_database.png
 
 aren't actually stored in that directory. Instead, you should add and commit
-images (and any other static assets) to the `superset/assets/images` directory.
+images (and any other static assets) to the `superset-frontend/images` directory.
 When the docs are deployed to https://superset.incubator.apache.org/, images
-are copied from there to the `_static/img` directory, just like they're referenced
+are copied from there to the `_static/images` directory, just like they're referenced
 in the docs.
 
-For example, the image referenced above actually lives in `superset/assets/images/tutorial`. Since the image is moved during the documentation build process, the docs reference the image in `_static/img/tutorial` instead.
+For example, the image referenced above actually lives in `superset-frontend/images/tutorial`. Since the image is moved during the documentation build process, the docs reference the image in `_static/images/tutorial` instead.
 
 #### API documentation
 
@@ -418,7 +418,7 @@ app.logger.info(form_data)
 
 ### Frontend Assets
 
-Frontend assets (JavaScript, CSS, and images) must be compiled in order to properly display the web UI. The `superset/assets` directory contains all NPM-managed front end assets. Note that there are additional frontend assets bundled with Flask-Appbuilder (e.g. jQuery and bootstrap); these are not managed by NPM, and may be phased out in the future.
+Frontend assets (JavaScript, CSS, and images) must be compiled in order to properly display the web UI. The `superset-frontend` directory contains all NPM-managed front end assets. Note that there are additional frontend assets bundled with Flask-Appbuilder (e.g. jQuery and bootstrap); these are not managed by NPM, and may be phased out in the future.
 
 #### nvm and node
 
@@ -437,7 +437,7 @@ Install third-party dependencies listed in `package.json`:
 
 ```bash
 # From the root of the repository
-cd superset/assets
+cd superset-frontend
 
 # Install dependencies from `package-lock.json`
 npm ci
@@ -481,7 +481,7 @@ See docs [here](docker/README.md)
 #### Updating NPM packages
 
 Use npm in the prescribed way, making sure that
-`superset/assets/package-lock.json` is updated according to `npm`-prescribed
+`superset-frontend/package-lock.json` is updated according to `npm`-prescribed
 best practices.
 
 #### Feature flags
@@ -494,7 +494,7 @@ FEATURE_FLAGS = {
 }
 ```
 
-If you want to use the same flag in the client code, also add it to the FeatureFlag TypeScript enum in `superset/assets/src/featureFlags.ts`. For example,
+If you want to use the same flag in the client code, also add it to the FeatureFlag TypeScript enum in `superset-frontend/src/featureFlags.ts`. For example,
 
 ```
 export enum FeatureFlag {
@@ -524,7 +524,7 @@ Lint the project with:
 tox -e flake8
 
 # for javascript
-cd superset/assets
+cd superset-frontend
 npm ci
 npm run lint
 ```
@@ -616,7 +616,7 @@ def sqrt(x: Union[float, int]) -> Union[float, int]:
 We use [Jest](https://jestjs.io/) and [Enzyme](https://airbnb.io/enzyme/) to test Javascript. Tests can be run with:
 
 ```bash
-cd superset/assets
+cd superset-frontend
 npm run test
 ```
 
@@ -636,7 +636,7 @@ superset run --port 8081
 Run Cypress tests:
 
 ```bash
-cd superset/assets
+cd superset-frontend
 npm run build
 
 cd cypress-base
@@ -653,7 +653,7 @@ npm run cypress run -- --spec cypress/integration/dashboard/index.test.js --conf
 npm run cypress open
 ```
 
-See [`superset/assets/cypress_build.sh`](https://github.com/apache/incubator-superset/blob/master/superset/assets/cypress_build.sh).
+See [`superset-frontend/cypress_build.sh`](https://github.com/apache/incubator-superset/blob/master/superset-frontend/cypress_build.sh).
 
 ## Translating
 

Dockerfile

@@ -42,15 +42,15 @@ RUN cd /app \
 FROM node:10-jessie AS superset-node
 
 # NPM ci first, as to NOT invalidate previous steps except for when package.json changes
-RUN mkdir -p /app/superset/assets
-COPY ./superset/assets/package* /app/superset/assets/
-RUN cd /app/superset/assets \
+RUN mkdir -p /app/superset-frontend
+COPY ./superset-frontend/package* /app/superset-frontend/
+RUN cd /app/superset-frontend \
         && npm ci
 
 # Next, copy in the rest and let webpack do its thing
-COPY ./superset/assets /app/superset/assets
+COPY ./superset-frontend /app/superset-frontend
 # This is BY FAR the most expensive step (thanks Terser!)
-RUN cd /app/superset/assets \
+RUN cd /app/superset-frontend \
         && npm run build \
         && rm -rf node_modules
 
@@ -81,7 +81,7 @@ RUN useradd --user-group --no-create-home --no-log-init --shell /bin/bash supers
 COPY --from=superset-py /usr/local/lib/python3.6/site-packages/ /usr/local/lib/python3.6/site-packages/
 # Copying site-packages doesn't move the CLIs, so let's copy them one by one
 COPY --from=superset-py /usr/local/bin/gunicorn /usr/local/bin/celery /usr/local/bin/flask /usr/bin/
-COPY --from=superset-node /app/superset/assets /app/superset/assets
+COPY --from=superset-node /app/superset-frontend /app/superset-frontend
 
 ## Lastly, let's install superset itself
 COPY superset /app/superset
@@ -112,4 +112,4 @@ COPY ./requirements-dev.txt ./docker/requirements-extra.txt /app/
 USER root
 RUN cd /app \
     && pip install --no-cache -r requirements-dev.txt -r requirements-extra.txt
-USER superset
+USER superset

MANIFEST.in

@@ -20,18 +20,7 @@ graft licenses/
 include README.md
 recursive-include superset/examples *
 recursive-include superset/migrations *
-
-include superset/assets/package.json
-
-# Whitelist anything that needs to be added to the static bundle
-recursive-exclude superset/static *
-recursive-include superset/static/assets/branding *
-recursive-include superset/static/assets/dist *
-recursive-include superset/static/assets/images *
-recursive-exclude superset/static/images/viz_thumbnails_large *
-recursive-include superset/static/assets/stylesheets *
-include superset/static/assets/version_info.json
-
 recursive-include superset/templates *
 recursive-include superset/translations *
+recursive-include superset/static *
 recursive-exclude tests *

README.md

@@ -25,7 +25,7 @@ Superset
 [![PyPI](https://img.shields.io/pypi/pyversions/apache-superset.svg?maxAge=2592000)](https://pypi.python.org/pypi/apache-superset)
 [![Get on Slack](https://img.shields.io/badge/slack-join-orange.svg)](https://join.slack.com/t/apache-superset/shared_invite/enQtNDMxMDY5NjM4MDU0LWJmOTcxYjlhZTRhYmEyYTMzOWYxOWEwMjcwZDZiNWRiNDY2NDUwNzcwMDFhNzE1ZmMxZTZlZWY0ZTQ2MzMyNTU)
 [![Documentation](https://img.shields.io/badge/docs-apache.org-blue.svg)](https://superset.incubator.apache.org)
-[![dependencies Status](https://david-dm.org/apache/incubator-superset/status.svg?path=superset/assets)](https://david-dm.org/apache/incubator-superset?path=superset/assets)
+[![dependencies Status](https://david-dm.org/apache/incubator-superset/status.svg?path=superset-frontend)](https://david-dm.org/apache/incubator-superset?path=superset-frontend)
 
 <img
   src="https://cloud.githubusercontent.com/assets/130878/20946612/49a8a25c-bbc0-11e6-8314-10bef902af51.png"
@@ -42,23 +42,23 @@ Screenshots & Gifs
 
 **View Dashboards**
 
-<kbd><img title="View Dashboards" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset/assets/images/screenshots/bank_dash.png"></kbd><br/>
+<kbd><img title="View Dashboards" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset-frontend/images/screenshots/bank_dash.png"></kbd><br/>
 
 **Slice & dice your data**
 
-<kbd><img title="Slice & dice your data" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset/assets/images/screenshots/explore.png"></kbd><br/>
+<kbd><img title="Slice & dice your data" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset-frontend/images/screenshots/explore.png"></kbd><br/>
 
 **Query and visualize your data with SQL Lab**
 
-<kbd><img title="SQL Lab" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset/assets/images/screenshots/sqllab.png"></kbd><br/>
+<kbd><img title="SQL Lab" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset-frontend/images/screenshots/sqllab.png"></kbd><br/>
 
 **Visualize geospatial data with deck.gl**
 
-<kbd><img title="Geospatial" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset/assets/images/screenshots/deckgl_dash.png"></kbd><br/>
+<kbd><img title="Geospatial" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset-frontend/images/screenshots/deckgl_dash.png"></kbd><br/>
 
 **Choose from a wide array of visualizations**
 
-<kbd><img title="Visualizations" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset/assets/images/screenshots/visualizations.png"></kbd><br/>
+<kbd><img title="Visualizations" src="https://raw.githubusercontent.com/apache/incubator-superset/master/superset-frontend/images/screenshots/visualizations.png"></kbd><br/>
 
 Apache Superset
 ---------------

RELEASING/Dockerfile.from_local_tarball

@@ -48,7 +48,7 @@ ARG SUPERSET_RELEASE_RC_TARBALL
 # Can fetch source from svn or copy tarball from local mounted directory
 COPY $SUPERSET_RELEASE_RC_TARBALL ./
 RUN tar -xvf *.tar.gz
-WORKDIR /home/superset/apache-superset-incubating-$VERSION/superset/assets
+WORKDIR /home/superset/apache-superset-incubating-$VERSION/superset-frontend
 
 RUN npm ci \
     && npm run build \

RELEASING/Dockerfile.from_svn_tarball

@@ -49,7 +49,7 @@ RUN svn co https://dist.apache.org/repos/dist/dev/incubator/superset/$VERSION ./
 RUN tar -xvf *.tar.gz
 WORKDIR apache-superset-incubating-$VERSION
 
-RUN cd superset/assets \
+RUN cd superset-frontend \
     && npm ci \
     && npm run build \
     && rm -rf node_modules

RELEASING/README.md

@@ -56,9 +56,9 @@ need to be done at every release.
 
 ## Crafting a source release
 
-When crafting a new minor or major release we create 
+When crafting a new minor or major release we create
 a branch named with the release MAJOR.MINOR version (on this example 0.34).
-This new branch will hold all PATCH and release candidates 
+This new branch will hold all PATCH and release candidates
 that belong to the MAJOR.MINOR version.
 
 The MAJOR.MINOR branch is normally a "cut" from a specific point in time from the master branch.
@@ -77,7 +77,7 @@ deprecations and upgrading-related topics,
 make sure to move the content now under the `Next Version` section under a new
 section for the new release.
 
-Finally bump the version number on `superset/static/assets/package.json`:
+Finally bump the version number on `superset-frontend/package.json`:
 
 ```json
     "version": "0.34.1"
@@ -120,7 +120,7 @@ The script will output the exported variables. Here's example for 0.34.1 RC1:
 
 The first step of preparing an Apache Release is packaging a release candidate
 to be voted on. Make sure you have correctly prepared and tagged the ready to ship
-release on Superset's repo (MAJOR.MINOR branch), the following script will clone 
+release on Superset's repo (MAJOR.MINOR branch), the following script will clone
 the tag and create a signed source tarball from it:
 
 ```bash
@@ -131,14 +131,14 @@ the tag and create a signed source tarball from it:
 
 Note that `make_tarball.sh`:
 
-- By default assumes you have already executed an SVN checkout to `$HOME/svn/superset_dev`. 
-This can be overriden by setting `SUPERSET_SVN_DEV_PATH` environment var to a different svn dev directory 
+- By default assumes you have already executed an SVN checkout to `$HOME/svn/superset_dev`.
+This can be overriden by setting `SUPERSET_SVN_DEV_PATH` environment var to a different svn dev directory
 - Will refuse to craft a new release candidate if a release already exists on your local svn dev directory
 - Will check `package.json` version number and fails if it's not correctly set
 
 ### Build and test the created source tarball
 
-To build and run the just created tarball 
+To build and run the just created tarball
 ```bash
     # Build and run a release candidate tarball
     ./test_run_tarball.sh local
@@ -174,12 +174,12 @@ https://lists.apache.org/thread.html/e60f080ebdda26896214f7d3d5be1ccadfab95d48fb
 To easily send a voting request to Superset community, still on the `superset/RELEASING` directory:
 
 ```bash
-    # Note: use Superset's virtualenv 
+    # Note: use Superset's virtualenv
     (venv)$ python send_email.py vote_pmc
 ```
 
 The script will interactively ask for extra information so it can authenticate on the Apache Email Relay.
-The release version and release candidate number are fetched from the previously set environment variables. 
+The release version and release candidate number are fetched from the previously set environment variables.
 
 ```bash
     Sender email (ex: user@apache.org): your_apache_email@apache.org
@@ -194,12 +194,12 @@ https://lists.apache.org/thread.html/50a6b134d66b86b237d5d7bc89df1b567246d125a71
 To easily send the result email, still on the `superset/RELEASING` directory:
 
 ```bash
-    # Note: use Superset's virtualenv 
+    # Note: use Superset's virtualenv
     (venv)$ python send_email.py result_pmc
 ```
 
 The script will interactively ask for extra information needed to fill out the email template. Based on the
-voting description, it will generate a passing, non passing or non conclusive email. 
+voting description, it will generate a passing, non passing or non conclusive email.
 here's an example:
 
 ```bash
@@ -217,7 +217,7 @@ started at general@incubator.apache.org.
 To easily send the voting request to Apache community, still on the `superset/RELEASING` directory:
 
 ```bash
-    # Note: use Superset's virtualenv 
+    # Note: use Superset's virtualenv
     (venv)$ python send_email.py vote_ipmc
 ```
 
@@ -227,12 +227,12 @@ least 72 hours have past, you can post a [RESULT] thread
 To easily send the result email, still on the `superset/RELEASING` directory:
 
 ```bash
-    # Note: use Superset's virtualenv 
+    # Note: use Superset's virtualenv
     (venv)$ python send_email.py result_ipmc
 ```
 
 Again, the script will interactively ask for extra information needed to fill out the email template. Based on the
-voting description, it will generate a passing, non passing or non conclusive email. 
+voting description, it will generate a passing, non passing or non conclusive email.
 here's an example:
 
 ```bash
@@ -291,7 +291,7 @@ while requesting access to push packages.
 Once it's all done, an [ANNOUNCE] thread announcing the release to the dev@ mailing list is the final step.
 
 ```bash
-    # Note use Superset's virtualenv 
+    # Note use Superset's virtualenv
     (venv)$ python send_email.py announce
 ```