If webtask dump secret store

[andrewkrug]

Webtask has a secret store... turns out you can just ask nicely.

https://webtask.io/docs/editor/secrets

[andrewkrug]

To be clear... the task is get secrets to dict and sanitize. Then mash them into ES.

[danielhartnell]

I decided to take a step back and look at the overall execution context. It was interesting to review. For now, I'll continue to focus on checking for secrets and gathering them. Happy to include any other information if you want.

{
  "meta": {},
  "storage": {
    "timeout": 10000,
    "token": "my-super-cool-token"
  },
  "data": {
    "NAME": "daniel"
  },
  "params": {},
  "query": {},
  "secrets": {
    "NAME": "daniel"
  },
  "headers": {
    "host": "wt-0ad02e9d22fed48c956fa4d6b11a93d3-0.run.webtask.io",
    "user-agent": "curl/7.51.0",
    "accept": "*/*",
    "accept-version": "2.0.0",
    "x-wt-params": "Scroll down for this base64 decoded content",
    "x-forwarded-for": "::ffff:172.31.25.153",
    "x-forwarded-proto": "https",
    "x-forwarded-port": "8721",
    "connection": "close"
  },
  "token": "my-super-cool-token",
  "id": "1494114432064.206135"
}

And the decoded value for the x-wt-params above:

{
  "req_id": "1494114432064.206135",
  "container": "wt-0ad02e9d22fed48c956fa4d6b11a93d3-0",
  "resolved_modules": null,
  "url_format": 3,
  "ectx": {
    "NAME": "daniel"
  },
  "url": "webtask://localhost/api/data/code/wt-0ad02e9d22fed48c956fa4d6b11a93d3-0%2Fsecrets",
  "pb": 2,
  "mb": 1,
  "jtn": "secrets",
  "token": "my-super-cool-token"
}

[danielhartnell]

I've learned a few new things this evening. If you review the following excerpt, you'll see the following:

• The wt CLI provides a bundle parameter out of the box so we don't have to use wt-bundle. This is described in Bundling Exceeds 100kb Limit auth0/wt-cli#121.
• Webtask has a maximum payload size of 500KB (including Node modules) and we appear to be exceeding that with our current profiler.

I'll think about how to address these findings.

λ MacBook-Pro node-lambda-inspector → λ git master* → wt create webtask-profiler.js --bundle

* Hint: A package.json file has been detected adjacent to your webtask. Ensuring that all dependencies from that file are avialable on the platform. This may take a few minutes for new versions of modules so please be patient.
* Hint: If you would like to opt-out from this behaviour, pass in the --ignore-package-json flag.
Resolving 1 module...
Provisioning 1 module...
uuid4@1.0.0 is available
[BABEL] Note: The code generator has deoptimised the styling of "/Users/dhartnell/node-lambda-inspector/webtask-profiler.js" as it exceeds the max of "500KB".
onWebtaskError Error: Payload Too Large
    at Request.callback (/usr/local/lib/node_modules/wt-cli/node_modules/superagent/lib/node/index.js:675:11)
    at /usr/local/lib/node_modules/wt-cli/node_modules/superagent/lib/node/index.js:883:18
    at IncomingMessage.<anonymous> (/usr/local/lib/node_modules/wt-cli/node_modules/superagent/lib/node/parsers/json.js:16:7)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:188:7)
    at endReadableNT (_stream_readable.js:975:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)
Uncaught error:  Request body size exceeds 122880
Error: Payload Too Large
    at Request.callback (/usr/local/lib/node_modules/wt-cli/node_modules/superagent/lib/node/index.js:675:11)
    at /usr/local/lib/node_modules/wt-cli/node_modules/superagent/lib/node/index.js:883:18
    at IncomingMessage.<anonymous> (/usr/local/lib/node_modules/wt-cli/node_modules/superagent/lib/node/parsers/json.js:16:7)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:188:7)
    at endReadableNT (_stream_readable.js:975:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)
Please report this at: https://github.com/auth0/wt-cli/issues

[danielhartnell]

With my current Node knowledge, I'm having some trouble understanding how I can make context available to the profiler itself. Creating a new Webtask with the following code is simple enough:

module.exports = function(context, callback) {
	console.log(context.secrets.mySecret);
} 

This will yield what we are looking for. But I have been unable to get access to the same information from the profiler. I started by trying to create a new lookup called getSecrets which would handle the results and pass it back to make_lookup_callback().

I keep seeing: ReferenceError: context is not defined. I think this makes sense but I'm not sure how I can solve it.

[danielhartnell]

I've been thinking about this more and I think I've identified how I will solve this. I was a little grumpy yesterday and I probably got to a point where I was overthinking this issue. From webtask.js, when I call the profiler, I can pass in this context and then do what I need from within the profiler. I'll experiment with that and see if I can wrap this up.