PostgreSQL storage backend configurable with environment variables

[KristapsT]

Is your feature request related to a problem? Please describe.
When configuring Vault to run in Kubernetes and use PostgreSQL storage backend, I ran in to an issue where my only option to configure PostgreSQL connection string is to provide it in configurations file - this is a problem to use since we store configurations for deployments in Git and in this case the configuration would contain the database password.

Describe the solution you'd like
libpq already has a set of environment variables pq should be able to pick up - https://www.postgresql.org/docs/current/libpq-envars.html these can be used, or specific set of environment variables can be created to cover these configuration options - this is already the case for other storage backends for Vault - DynamoDB, Manta, S3 and Swift.

Explain any additional use-cases
Not storing secrets in configuration files greatly helps with implementing GitOps type of deployments where the state of application is stored in git - secrets instead can be passed as environment variables to the running application during deployment.

This especially has become the case now when Vault has Helm Chart deployment option and configuration of Vault is part of valuesfile.

[mccurdyc]

I'd love to pick this one up.

Is there a reason why we wouldn't want to do it similar to how it is done in DynamoDB, Manta or S3?

I did notice that S3 doesn't support environment variables for all inputs (e.g., AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY)

Are there particular PostgreSQL envs that we would want to force the user to submit the entire connection url for?

[catsby]

Hello -

We would certainly welcome and review a pull request that allows the Postgres storage backend to read the connection url from the environment. I would probably start by requiring the full connection string as opposed to accepting the individual parts (username, password, url, port, so on...). I admit off-hand I'm not sure if other storage backends accept both (full vs piecemeal) so maybe check those out for reference.

Thanks!

[mccurdyc]

Hi @catsby @KristapsT

TL;DR - After thinking about this a bit, I don't know that "hiding" these configuration options in environment variables is a good idea. Terraform with the vault provider --- here is an example --- and terraform variables (a good post on secret managment with terraform) is a potential solution to this problem.

---

The way it currently exists, the configuration is explicit --- see the PostgreSQL Storage Backend configuration docs. If the connection_url specifically, were sourced through env (i.e., removing the required field for configuring the storage backend), then it would be less clear to other developers who would, by default, be looking in the vault configuration file. Also, sourcing from envs enables configuration to be done possibly in many places rather than in a centralized, vault configuration, file.

With the existing configuration --- while it may require more tooling and additional steps --- a user still has the ability to source parts of the connection_url (e.g., username, password, host, etc.) from the env, while remaining explicit and indicating to a future contributor that certain pieces are sourced from the env.

I'm totally open to discuss this further or help in any way possible. Also, please correct me if I have misstated anything!

Kind Regards!

[KristapsT]

@mccurdyc my need for this came when I was starting to look at running Vault in Kubernetes using relatively recently released Helm chart for Vault. Since providing secret values to applications running in Kubernetes through environment variables is quite common, I was looking to do the same when configuring PostgreSQL backed of Vault.
I am not sure how Terraform would help in that particular case.

[mccurdyc]

@KristapsT you are correct. I apologize for the confusion. I am working on automated tests and manually testing my change with the Vault helm chart now. Thank you for your patience!

[marcaurele]

I would think this issue can be closed since the PR #7937 fixed it, right?