Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New monorepo deploys #366

Merged
merged 17 commits into from
Dec 16, 2019
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 18 additions & 34 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,33 +41,19 @@ jobs:
exit 0
fi

git config --global user.email '<>' # Leave email blank
git config --global user.name "Circle MonoRepo Automatic Deployer"

for APP in $APPS_MODIFIED
do
if [ "$APP" == "$CIRCLE_BRANCH" ]
if [ "$APP" != "$CIRCLE_BRANCH" ]
then
git config --global user.email '<>' # Leave email blank
git config --global user.name "Circle MonoRepo Automatic Deployer"
if [[ $APP =~ "dashr" ]]
then
cp ~/dash-sample-apps/apps/"$APP"/Aptfile ~/dash-sample-apps/
cp ~/dash-sample-apps/apps/"$APP"/Procfile ~/dash-sample-apps/
cp ~/dash-sample-apps/apps/"$APP"/.buildpacks ~/dash-sample-apps/
cp ~/dash-sample-apps/apps/"$APP"/init.R ~/dash-sample-apps/
# app.json tries to call python predeploy, which is not needed for R apps:
rm ~/dash-sample-apps/app.json
git add . && git commit -a -m "Deployed commit: $CIRCLE_SHA1"
fi
if [ ! -d ~/dash-sample-apps/apps/"$APP"/assets/ ]
then
echo "Deploy failed because there is no assets/ folder. Exiting..."
exit 1
fi
git commit -m "Deployed commit: $CIRCLE_SHA1"
git push playground $CIRCLE_BRANCH:master --force
exit 0
else
echo "appname: $APP is not same as the branchname: $CIRCLE_BRANCH. Not deploying..."
continue
fi

CREATE_APP=true ./deploy "$APP"
exit 0
done
echo "Deploy failed because of a branchname/appname mismatch. Exiting..."
exit 1
Expand All @@ -90,20 +76,18 @@ jobs:
name: Deploy
command: |
APPS_MODIFIED=$(ls apps | grep dash- | sort -u)
if [ -z "$APPS_MODIFIED" ]
then
echo "No app change detected. Skipping the deploy.."
exit 0
fi

git config --global user.email '<>' # Leave email blank
git config --global user.name "Circle MonoRepo Automatic Deployer"

for APP in $APPS_MODIFIED
do
if [[ $APP =~ "dashr" ]]
then
continue
fi
git config --global user.email '<>' # Leave email blank
git config --global user.name "Circle MonoRepo Automatic Deployer"
if [ -d ~/dash-sample-apps/apps/"$APP"/ ]
then
git commit -m "Deployed commit: $CIRCLE_SHA1"
git remote add $APP https://dash-playground.plotly.host/GIT/$APP
git push $APP master --force || echo "Deploy failed because there is no such app on Dash Gallery: $APP. Please use the web interface to create an app with this name. Continuing deployments..."
fi
./deploy "$APP"
done
- run:
name: Push to production
Expand Down
7 changes: 7 additions & 0 deletions .deployignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# these apps are ignored for deployment
dash-image-processing
dash-mapd-demo
dash-salesforce-crm
dash-windstreaming
dashr-alignment-viewer
dashr-salesforce-crm
12 changes: 3 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Contributing an app written with Dash for R is very similar to the steps outline
3. The `Procfile` should contain

```
web: R -f /app/apps/"$DASH_APP_NAME"/app.R
web: R -f /app/app.R
```

4. Routing and request pathname prefixes should be set. One approach might be to include
Expand All @@ -88,12 +88,6 @@ at the head of your `app.R` file.
app$run_server(host = "0.0.0.0", port = Sys.getenv('PORT', 8050))
``

6. For convenience, it is probably easiest to set the working directory in `app.R` as well:

``
setwd(sprintf("/app/apps/%s", appName))
``

### Making changes to an existing app

Switch to the branch that has the same name as the DDS app (the "app
Expand All @@ -117,7 +111,7 @@ branch. Once you have passed your code review, you can merge your PR.

- **`Procfile`** gets run at root level for deployment
- Make sure python working directory is at the app level
- Ex. `web: gunicorn --pythonpath apps/{DASH_APP_NAME} app:server`
- Ex. `web: gunicorn app:server`
- **`requirements.txt`**
- Install project dependecies in a virtual environment
- **`runtime.txt`**
Expand All @@ -139,7 +133,7 @@ branch. Once you have passed your code review, you can merge your PR.

#### Handle relative path

Since deployment happens at the root level `/` and not at the app level (`/apps/{DASH_APP_NAME}`), we need to make sure our application is able to run at both levels for flexibility.
Assets should never use a relative path, as this will fail when deployed to Dash Enterprise due to use of subdirectories for serving apps.

Reading from assets and data folder
```Python
Expand Down
2 changes: 1 addition & 1 deletion apps/dash-brain-viewer/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-brain-viewer app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-clinical-analytics/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-clinical-analytics app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-cytoscape-phylogeny/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-cytoscape-phylogeny app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-cytoscape/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-cytoscape app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-daq-iv-tracer/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-daq-iv-tracer demo:server
web: gunicorn demo:server
2 changes: 1 addition & 1 deletion apps/dash-daq-satellite-dashboard/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-daq-satellite-dashboard app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-datashader/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-datashader app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-drug-discovery/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-drug-discovery app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-financial-report/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-financial-report app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-image-processing/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-image-processing app:server
web: gunicorn app:server
5 changes: 2 additions & 3 deletions apps/dash-image-processing/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
import csv
import sys
import pathlib
import importlib

import boto3
import dash
Expand All @@ -16,8 +15,8 @@
from dash.dependencies import Input, Output, State
from flask_caching import Cache

drc = importlib.import_module("apps.dash-image-processing.dash_reusable_components")
utils = importlib.import_module("apps.dash-image-processing.utils")
import dash_reusable_components as drc
import utils

DEBUG = True
LOCAL = False
Expand Down
2 changes: 0 additions & 2 deletions apps/dash-image-processing/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@

from PIL import Image, ImageFilter, ImageDraw, ImageEnhance

# drc = importlib.import_module("apps.dash-iamge-processing.dash_reusable_components")

#
APP_PATH = str(pathlib.Path(__file__).parent.resolve())

Expand Down
2 changes: 1 addition & 1 deletion apps/dash-lastodash/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web:gunicorn --pythonpath apps/dash-lastodash lastodash:server
web: gunicorn lastodash:server
2 changes: 1 addition & 1 deletion apps/dash-live-model-training/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-live-model-training app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-manufacture-spc-dashboard/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-manufacture-spc-dashboard app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-mapd-demo/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-mapd-demo app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-medical-provider-charges/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-medical-provider-charges app:server
web: gunicorn app:server
2 changes: 0 additions & 2 deletions apps/dash-medical-provider-charges/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,6 @@
for state in state_list:
p = os.getcwd().split(os.path.sep)
csv_path = "data/processed/df_{}_lat_lon.csv".format(state)
if p[-1] != "dash-medical-provider-charges":
csv_path = "apps/dash-medical-provider-charges/" + csv_path
state_data = pd.read_csv(csv_path)
data_dict[state] = state_data

Expand Down
2 changes: 1 addition & 1 deletion apps/dash-multipage-report/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-multipage-report app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-nlp/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-nlp app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-object-detection/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-object-detection app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-oil-and-gas/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-oil-and-gas app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-oil-gas-ternary/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-oil-gas-ternary app:server
web: gunicorn app:server
2 changes: 0 additions & 2 deletions apps/dash-oil-gas-ternary/app.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import importlib
import pathlib
import os

Expand All @@ -11,7 +10,6 @@
import plotly.graph_objs as go
from dash.dependencies import Input, Output, State

# constants = importlib.import_module("apps.dash-oil-gas-ternary.constants")
import constants

# app initialize
Expand Down
2 changes: 1 addition & 1 deletion apps/dash-opioid-epidemic/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-opioid-epidemic app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-pk-calc/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-pk-calc app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-salesforce-crm/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-salesforce-crm index:server
web: gunicorn index:server
2 changes: 1 addition & 1 deletion apps/dash-stitching/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-stitching app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-study-browser/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-study-browser app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-svm/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-svm app:server
web: gunicorn app:server
4 changes: 2 additions & 2 deletions apps/dash-svm/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
from sklearn import datasets
from sklearn.svm import SVC

drc = importlib.import_module("apps.dash-svm.utils.dash_reusable_components")
figs = importlib.import_module("apps.dash-svm.utils.figures")
import utils.dash_reusable_components as drc
import utils.figures as figs

app = dash.Dash(
__name__,
Expand Down
2 changes: 1 addition & 1 deletion apps/dash-tsne/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-tsne app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-uber-rides-demo/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-uber-rides-demo app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-web-trader/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-web-trader app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-wind-streaming/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-wind-streaming app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dash-yield-curve/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --pythonpath apps/dash-yield-curve app:server
web: gunicorn app:server
2 changes: 1 addition & 1 deletion apps/dashr-alignment-viewer/.buildpacks
Original file line number Diff line number Diff line change
@@ -1 +1 @@
https://github.com/virtualstaticvoid/heroku-buildpack-r#heroku-16
https://github.com/plotly/heroku-buildpack-r#heroku-18
2 changes: 1 addition & 1 deletion apps/dashr-alignment-viewer/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: R -f /app/apps/"$DASH_APP_NAME"/app.R
web: R -f /app/app.R
1 change: 0 additions & 1 deletion apps/dashr-alignment-viewer/app.R
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ if (appName != "") {
pathPrefix <- sprintf("/%s/", appName)
Sys.setenv(DASH_ROUTES_PATHNAME_PREFIX = pathPrefix,
DASH_REQUESTS_PATHNAME_PREFIX = pathPrefix)
setwd(sprintf("/app/apps/%s", appName))
}

library(dashBio)
Expand Down
2 changes: 1 addition & 1 deletion apps/dashr-brain-viewer/.buildpacks
Original file line number Diff line number Diff line change
@@ -1 +1 @@
https://github.com/virtualstaticvoid/heroku-buildpack-r#heroku-16
https://github.com/plotly/heroku-buildpack-r#heroku-18
2 changes: 1 addition & 1 deletion apps/dashr-brain-viewer/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: R -f /app/apps/"$DASH_APP_NAME"/app.R
web: R -f /app/app.R
2 changes: 0 additions & 2 deletions apps/dashr-brain-viewer/app.R
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ if (appName != ""){

Sys.setenv(DASH_ROUTES_PATHNAME_PREFIX = pathPrefix,
DASH_REQUESTS_PATHNAME_PREFIX = pathPrefix)

setwd(sprintf("/app/apps/%s", appName))
}

read_mniobj <- function(file){
Expand Down
2 changes: 1 addition & 1 deletion apps/dashr-clustergram/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: R -f /app/apps/"$DASH_APP_NAME"/app.R
web: R -f /app/app.R
2 changes: 0 additions & 2 deletions apps/dashr-clustergram/app.R
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,6 @@ if (appName != ""){

Sys.setenv(DASH_ROUTES_PATHNAME_PREFIX = pathPrefix,
DASH_REQUESTS_PATHNAME_PREFIX = pathPrefix)

setwd(sprintf("/app/apps/%s", appName))
}

source("utils/dash_bio_utils.R")
Expand Down
1 change: 1 addition & 0 deletions apps/dashr-daq-iv-tracer/.buildpacks
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://github.com/plotly/heroku-buildpack-r#heroku-18
2 changes: 1 addition & 1 deletion apps/dashr-daq-iv-tracer/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: R -f /app/apps/"$DASH_APP_NAME"/app.R
web: R -f /app/app.R
2 changes: 0 additions & 2 deletions apps/dashr-daq-iv-tracer/app.R
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ if (!appName == "") {

Sys.setenv(DASH_ROUTES_PATHNAME_PREFIX = pathPrefix,
DASH_REQUESTS_PATHNAME_PREFIX = pathPrefix)

setwd(sprintf("/app/apps/%s", appName))
}

source("utils/helperfuns.R")
Expand Down
1 change: 1 addition & 0 deletions apps/dashr-drug-discovery/.buildpacks
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://github.com/plotly/heroku-buildpack-r#heroku-18
2 changes: 1 addition & 1 deletion apps/dashr-drug-discovery/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: R -f /app/apps/"$DASH_APP_NAME"/app.R
web: R -f /app/app.R
4 changes: 0 additions & 4 deletions apps/dashr-drug-discovery/app.R
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,10 @@ if (appName != ""){

Sys.setenv(DASH_ROUTES_PATHNAME_PREFIX = pathPrefix,
DASH_REQUESTS_PATHNAME_PREFIX = pathPrefix)

setwd(sprintf("./app/apps/%s", appName))
}

app <- Dash$new()

setwd(sprintf("./app/apps/%s", appName))

df <- read.csv('./data/small_molecule_drugbank.csv', header = TRUE, sep = ",")


Expand Down
1 change: 0 additions & 1 deletion apps/dashr-drug-discovery/buildpacks.txt

This file was deleted.

2 changes: 1 addition & 1 deletion apps/dashr-financial-report/.buildpacks
Original file line number Diff line number Diff line change
@@ -1 +1 @@
https://github.com/virtualstaticvoid/heroku-buildpack-r#heroku-16
https://github.com/plotly/heroku-buildpack-r#heroku-18
2 changes: 1 addition & 1 deletion apps/dashr-financial-report/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: R -f /app/apps/"$DASH_APP_NAME"/app.R
web: R -f /app/app.R
9 changes: 0 additions & 9 deletions apps/dashr-financial-report/app.R
Original file line number Diff line number Diff line change
@@ -1,19 +1,10 @@
appName <- Sys.getenv("DASH_APP_NAME")

if (appName != ""){

pathPrefix <- sprintf("/%s/", appName)



Sys.setenv(DASH_ROUTES_PATHNAME_PREFIX = pathPrefix,

DASH_REQUESTS_PATHNAME_PREFIX = pathPrefix)



setwd(sprintf("/app/apps/%s", appName))

}


Expand Down
2 changes: 1 addition & 1 deletion apps/dashr-ideogram/.buildpacks
Original file line number Diff line number Diff line change
@@ -1 +1 @@
https://github.com/virtualstaticvoid/heroku-buildpack-r#heroku-16
https://github.com/plotly/heroku-buildpack-r#heroku-18
2 changes: 1 addition & 1 deletion apps/dashr-ideogram/Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: R -f /app/apps/"$DASH_APP_NAME"/app.R
web: R -f /app/app.R
Loading