[AIRFLOW-2691] Manage JS dependencies via npm

Closes #3572 from verdan/AIRFLOW-2691-npm-webpack

Author: verdan

.gitignore

@@ -138,3 +138,9 @@ rat-results.txt
 *.generated
 *.tar.gz
 scripts/ci/kubernetes/kube/.generated/airflow.yaml
+
+# Node & Webpack Stuff
+*.entry.js
+node_modules
+npm-debug.log*
+static/dist

.rat-excludes

@@ -37,3 +37,5 @@ bootstrap-toggle.min.js
 bootstrap-toggle.min.css
 d3.v3.min.js
 ace.js
+airflow/www_rbac/node_modules
+.*json

CONTRIBUTING.md

@@ -242,3 +242,60 @@ $ alembic revision -m "add new field to db"
   Generating
 ~/airflow/airflow/migrations/versions/12341123_add_new_field_to_db.py
 ```
+
+## Setting up the node / npm javascript environment (ONLY FOR www_rbac)
+
+`airflow/www_rbac/` contains all npm-managed, front end assets.
+Flask-Appbuilder itself comes bundled with jQuery and bootstrap.
+While these may be phased out over time, these packages are currently not
+managed with npm.
+
+### Node/npm versions
+Make sure you are using recent versions of node and npm. No problems have been found with node>=8.11.3 and npm>=6.1.3
+
+### Using npm to generate bundled files
+
+#### npm
+First, npm must be available in your environment. If it is not you can run the following commands
+(taken from [this source](https://gist.github.com/DanHerbert/9520689))
+```
+brew install node --without-npm
+echo prefix=~/.npm-packages >> ~/.npmrc
+curl -L https://www.npmjs.com/install.sh | sh
+```
+
+The final step is to add `~/.npm-packages/bin` to your `PATH` so commands you install globally are usable.
+Add something like this to your `.bashrc` file, then `source ~/.bashrc` to reflect the change.
+```
+export PATH="$HOME/.npm-packages/bin:$PATH"
+```
+
+#### npm packages
+To install third party libraries defined in `package.json`, run the
+following within the `airflow/www_rbac/` directory which will install them in a
+new `node_modules/` folder within `www_rbac/`.
+
+```bash
+# from the root of the repository, move to where our JS package.json lives
+cd airflow/www_rbac/
+# run npm install to fetch all the dependencies
+npm install
+```
+
+To parse and generate bundled files for airflow, run either of the
+following commands. The `dev` flag will keep the npm script running and
+re-run it upon any changes within the assets directory.
+
+```
+# Compiles the production / optimized js & css
+npm run prod
+
+# Start a web server that manages and updates your assets as you modify them
+npm run dev
+```
+
+#### Upgrading npm packages
+
+Should you add or upgrade a npm package, which involves changing `package.json`, you'll need to re-run `npm install` 
+and push the newly generated `package-lock.json` file so we get the reproducible build.
+

MANIFEST.in

@@ -6,9 +6,9 @@
 # to you under the Apache License, Version 2.0 (the
 # "License"); you may not use this file except in compliance
 # with the License.  You may obtain a copy of the License at
-# 
+#
 #   http://www.apache.org/licenses/LICENSE-2.0
-# 
+#
 # Unless required by applicable law or agreed to in writing,
 # software distributed under the License is distributed on an
 # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -24,10 +24,12 @@ include README.md
 graft licenses/
 graft airflow/www/templates
 graft airflow/www/static
+graft airflow/www_rbac
 graft airflow/www_rbac/static
 graft airflow/www_rbac/templates
 graft airflow/www_rbac/translations
 include airflow/alembic.ini
 graft scripts/systemd
 graft scripts/upstart
 graft airflow/config_templates
+recursive-exclude airflow/www_rbac/node_modules *

airflow/www_rbac/app.py

@@ -31,7 +31,7 @@
 from airflow import settings
 from airflow import configuration as conf
 from airflow.logging_config import configure_logging
-
+from airflow.www_rbac.static_config import configure_manifest_files
 
 app = None
 appbuilder = None
@@ -64,6 +64,7 @@ def create_app(config=None, session=None, testing=False, app_name="Airflow"):
     app.register_blueprint(routes)
 
     configure_logging()
+    configure_manifest_files(app)
 
     with app.app_context():
 

airflow/www_rbac/compile_assets.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# first bump up package.json manually, commit and tag
+rm airflow/www_rbac/static/dist/*
+cd airflow/www_rbac/
+npm install
+npm run build
+cd ../..