Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VAULT-32657 prevent duplicate keys in HCL files #29862

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

bosouza
Copy link
Contributor

@bosouza bosouza commented Mar 7, 2025

Description

As discussed in hashicorp/hcl#35, HCL has allowed duplicate keys where the last one "wins", which has been flagged as a security risk. A fix preventing duplicate keys in HCL was added in hashicorp/hcl#704, and additional methods for preserving the old behavior came in hashicorp/hcl#707.

This PR upgrades all HCL dependencies to the fixed version, which effectively introduces a breaking change where all HCL files accepted by Vault that might have previously worked fine containing duplicate keys will now fail to be parsed, erroring out the operation. This is a comprehensive list of affected features:

  • HCL configs accepted by vault CLI commands:
    • $HOME/.vault HCL file with token_helper field
    • -config HCL file passed to vault operator migrate command
    • -config HCL file passed to vault agent command
    • -config HCL file passed to vault proxy command
    • -config HCL file passed to vault operator diagnose command
    • -config HCL file passed to vault server command
      • users trying to upgrade their Vault clusters with duplicate keys in their config will fail to upgrade, this should be mentioned in the upgrade guide.
  • HCL policy definitions:

TODO only if you're a HashiCorp employee

  • Backport Labels: If this fix needs to be backported, use the appropriate backport/ label that matches the desired release branch. Note that in the CE repo, the latest release branch will look like backport/x.x.x, but older release branches will be backport/ent/x.x.x+ent.
    • LTS: If this fixes a critical security vulnerability or severity 1 bug, it will also need to be backported to the current LTS versions of Vault. To ensure this, use all available enterprise labels.
  • ENT Breakage: If this PR either 1) removes a public function OR 2) changes the signature
    of a public function, even if that change is in a CE file, double check that
    applying the patch for this PR to the ENT repo and running tests doesn't
    break any tests. Sometimes ENT only tests rely on public functions in CE
    files.
  • Jira: If this change has an associated Jira, it's referenced either
    in the PR description, commit message, or branch name.
  • RFC: If this change has an associated RFC, please link it in the description.
  • ENT PR: If this change has an associated ENT PR, please link it in the
    description. Also, make sure the changelog is in this PR, not in your ENT PR.

@github-actions github-actions bot added the hashicorp-contributed-pr If the PR is HashiCorp (i.e. not-community) contributed label Mar 7, 2025
Copy link

github-actions bot commented Mar 7, 2025

CI Results: failed ❌
Failures:

Test Type Package Test Logs
standard api/cliconfig TestParseConfig_HclDuplicateKey view test results
standard command TestAgent_Config_HclDuplicateKeyInvalid view test results
standard command TestMigration view test results
standard command TestMigration/Config_parsing view test results
standard command TestOperatorDiagnoseCommand_Run view test results
standard command TestOperatorDiagnoseCommand_Run/validations view test results
standard command TestOperatorDiagnoseCommand_Run/validations/diagnose_invalid_config_duplicate_keys view test results
standard command TestPolicyFmtCommand_Run view test results
standard command TestPolicyFmtCommand_Run/hcl_duplicate_key view test results
standard command TestProxy_Config_HclDuplicateKeyInvalid view test results

and 11 - 10 other tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hashicorp-contributed-pr If the PR is HashiCorp (i.e. not-community) contributed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant