Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust account-specific terraform to manually-applied changes. #532

Merged
merged 4 commits into from
Nov 13, 2020

Conversation

adunkman
Copy link
Contributor

These tweaks have been made to update our logging infrastructure; apply them to terraform to bring things back in line. This change goes hand-in-hand with an update to our Court-specific environment documentation to document these values.

These tweaks have been made to update our logging infrastrucutre; apply
them to terraform to bring things back in line.
Copy link
Collaborator

@mmarcotte mmarcotte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super

Thanks for this Andrew!

AWS Elasticsearch requires that it sets up its own identity provider and
user pool client.
@adunkman
Copy link
Contributor Author

Added one more commit to fix #496:

  • When deploying a domain, AWS Elasticsearch creates a Cognito user pool client and its mapped identity provider, configured to point at the domain endpoint.
  • This creates an issue when Terraform runs against the Cognito identity pool, since it sees no such mapped identity provider and removes it. (Reported at BUG: log_viewers cognito pool recreated on account-specific terraform run #496). This breaks authentication for Kibana when the run happens.
  • Any time the domain is deployed — on changes to its volumes, instances, configuration, etc — it re-creates the user pool client and mapped identity provider if it doesn’t exist. This fixes authentication.

FIGHT!

  • I initially attempted to bring these resources under Terraform control, which would allow us to manage them ourselves as we do all other AWS resources. This creates a cycle:
    • To configure the user pool client, you need the endpoint for Elasticsearch.
    • To determine the endpoint for Elasticsearch, you need the user pool client.
  • My attempted fix was to use a known domain for the Elasticsearch endpoint (logs.ef-cms.ustaxcourt.gov) to break the cycle. This worked great! Except…

There is no permanent fix without a change in AWS behavior (hashicorp/terraform-provider-aws#5557) — c853d8b implements a workaround (to have Terraform ignore changes to cognito_identity_providers, sidestepping the fight).

This prevented adding log subscriptions to all of the court’s
environments.
@adunkman
Copy link
Contributor Author

Since this PR is still open, tacked on one more commit to fix a bash scripting bug which we discovered right before our load test yesterday — the IF-statement conditional is reversed for determining if LOG_GROUP_ENVIRONMENTS is set, which ensured the default value was always used. Fixed in de5aa71.

@mmarcotte mmarcotte merged commit 163653f into staging Nov 13, 2020
@adunkman adunkman deleted the update-elasticsearch-config branch November 13, 2020 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants