Ensure that the token list is stored on the server so that the same image works for both studies and programs

[shankari]

concretely, how to configure a list of tokens on the production server and to potentially have other program vs. study differences.

Dynamic configs are stored at https://github.com/e-mission/nrel-openpath-deploy-configs

Please ask @zackAemmer or @sebastianbarry if you have any questions about dynamic configs since they have worked with them recently.

[shankari]

e-mission-server supports multiple different authentication methods.
You can find the documentation for it here:

e-mission-docs/docs/install/configuring_authentication.md

Line 48 in 4fad0c4

### `token_list`/`prompted-auth` ###

One of them is token_list and if the chose auth method is token_list, the server expects to find a file of strings, and whenever the user tries to register with an op_code it must be in that list.

This is implemented at emission/net/auth/token_list.py

[shankari]

Options are:

1. We spin up a new container
   • Container image has the files baked in, so we would need to rebuild it every time there is a new deployment
   • Container would presumably expose an API to get token list given program name
     • This would need auth because otherwise anybody can read the list of tokens which defeats the entire purpose
     • The container could accessible only inside the NREL intranet and only from the webapp-tagged containers
       • (SDN should support something like this)
     • Question: whether cloud services supports dynamically configured networking?
2. Enterprise GitHub repo
   • similar to public repo with config but only internal
   • we would commit the files to the repo
   • webapp containers would be connected to the repo and pull
   • Question: Can containers on a medium cluster be connected to the NREL github instance? And is the security of the NREL github repo sufficient that we can store keys in there?
3. DocumentDB
   • host the token list as a "table" and just change token_list.py to read from the database instead of a file
   • how do we populate the token list in the database?
     • Do we assume that one of us has write access to the DB and will run a script to load the table?
     • Question: Will there be a little web service/jenkins job that will allow us to do this?

[shankari]

Cleanest solution appears to be the database.
Answer to the question for the database:

• Long-term: This should be in the admin dashboard. We can then give access to the admins according to the standard admin dashboard procedures. We can let them generate their own tokens and QR codes and download them, so this can all be self-service and will not need me to generate tokens and send them via onedrive. But the admin dashboard will take a long time to implement (at least a month or two).
• Short-term: I will load the token list into the database using a script and my current write access.

The actual task for @aGuttman is:

• write a script to load a token list into a table (aka collection in mongodb) into the database. make sure to access it via wrapper code in emission/storage so that if we do want to support a different database (e.g. couchdb), it should be easy to find all locations.
• change the emission/net/auth/token_list.py to read from the database
• change the auth mode selection (skip or token_list or...), currently defined in conf/net/api/webserver.conf and read around auth_method = config_data["server"]["auth"] to read from the dynamic config on the public github instead.
  • those configs should already have study vs. program and if it is study we want it to be skip and if it is program we want it to be token_list

[shankari]

• All database collections are defined in emission/core/get_database.py
• you can just create a new collection in there and it would be good to add some kind of access/wrapper method for it in emission/storage/decorators so we can swap out the DB later if needed
• then just read and write using mongodb queries.

[shankari]

Wed: write a script to load a token list into a table (aka collection in mongodb) into the database. make sure to access it via wrapper code in emission/storage so that if we do want to support a different database (e.g. couchdb), it should be easy to find all locations.

[aGuttman]

This issue worked on in PR e-mission/e-mission-server#885

[shankari]

wrt "read from the dynamic config on the public github instead", the server does not currently read from the public github, which is why it is not dynamically configured per the title of this issue.
#790 (comment)

However, the viz_scripts do read from the dynamic config in python to determine which graphs to generate and you should be able to reuse that code.

https://github.com/e-mission/em-public-dashboard/blob/main/viz_scripts/bin/generate_plots.py#L26

[shankari]

At least initially, it would be good if we would be able to support non-dynamic config as well, for those community members that do not want to have dynamically configurable instances. So I would suggest creating a new auth method called "dynamic", read the config in there, and then choose skip or token_list from there depending on the config.

Another aspect to consider during the design is that you don't want to read the config for every auth because that would be too much of an overhead; we should read the config once at startup or once a day or something and then use it forever.

[aGuttman]

My approach is to resolve auth_method in cdc_webapp.py between dynamic and anything else.

i.e. if webapp.conf has skip or token_list or similar, we leave it alone. If the resolution sees dynamic it will attempt to read the dynamic conf from a url as the viz_script example does. On success, it will change auth_method based on what is specified at dynamic_conf[intro][program_or_study] to either token_list or skip.

This avoids the problem of reading from the dynamic conf every time we want to verify a token, as adding a dynamic case to the elif block in auth.py would do. In fact, by resolving in cdc_webapp.py into cases we already know, auth.py can remain untouched as should function normally.

[shankari]

server code is checked in
docker-compose changes in e-mission-docker and the internal NREL OpenPATH repo are pending.
e-mission/e-mission-server#886 (review)

[shankari]

docker-compose changes are complete and solution has been deployed.
e-mission/e-mission-docker#26