Skip to content

Commit 2b69482

Browse files
bryantbiggsbryantbiggs
committed
fix: Updates from testing with Outposts, still need final remote validation hence the git push
1 parent 2596ea3 commit 2b69482

File tree

10 files changed

+373
-160
lines changed

10 files changed

+373
-160
lines changed

‎examples/outposts/README.md

+24-16
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,25 @@ Note: This example requires an an AWS Outpost to provision.
1010

1111
To run this example you need to:
1212

13-
1. Copy the `terraform.tfvars.example` to `terraform.tfvars` and fill in the required variables
14-
2. Execute:
13+
1. Deploy the remote host where the cluster will be provisioned from. The remote host is required since only private access is permitted to clusters created on Outposts. If you have access to the network where Outposts are provisioned (VPN, etc.), you can skip this step:
1514

1615
```bash
16+
$ cd prerequisites
17+
$ terraform init
18+
$ terraform plan
19+
$ terraform apply
20+
```
21+
22+
2. If provisioning using the remote host deployed in step 1, connect to the remote host using SSM. Note, you will need to have the [SSM plugin for the AWS CLI installed](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html). You can use the output generated by step 1 to connect:
23+
24+
```bash
25+
$ aws ssm start-session --region <REGION> --target <INSTANCE_ID>
26+
```
27+
28+
3. Once connected to the remote host, navigate to the cloned project example directory and deploy the example:
29+
30+
```bash
31+
$ cd $HOME/terraform-aws-eks/examples/outposts
1732
$ terraform init
1833
$ terraform plan
1934
$ terraform apply
@@ -35,27 +50,31 @@ Note that this example may create resources which cost money. Run `terraform des
3550
| Name | Version |
3651
|------|---------|
3752
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.34 |
53+
| <a name="provider_kubernetes"></a> [kubernetes](#provider\_kubernetes) | >= 2.10 |
3854

3955
## Modules
4056

4157
| Name | Source | Version |
4258
|------|--------|---------|
4359
| <a name="module_eks"></a> [eks](#module\_eks) | ../.. | n/a |
44-
| <a name="module_vpc_cni_irsa"></a> [vpc\_cni\_irsa](#module\_vpc\_cni\_irsa) | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | ~> 5.0 |
4560

4661
## Resources
4762

4863
| Name | Type |
4964
|------|------|
65+
| [kubernetes_storage_class_v1.this](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/storage_class_v1) | resource |
66+
| [aws_outposts_outpost_instance_types.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/outposts_outpost_instance_types) | data source |
5067
| [aws_outposts_outposts.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/outposts_outposts) | data source |
68+
| [aws_subnet.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source |
69+
| [aws_subnets.lookup](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) | data source |
5170
| [aws_subnets.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) | data source |
71+
| [aws_vpc.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) | data source |
5272

5373
## Inputs
5474

5575
| Name | Description | Type | Default | Required |
5676
|------|-------------|------|---------|:--------:|
57-
| <a name="input_outpost_instance_type"></a> [outpost\_instance\_type](#input\_outpost\_instance\_type) | Instance type supported by the Outposts instance | `string` | `"m5.large"` | no |
58-
| <a name="input_region"></a> [region](#input\_region) | The AWS region to deploy into (e.g. us-east-1) | `string` | n/a | yes |
77+
| <a name="input_region"></a> [region](#input\_region) | The AWS region to deploy into (e.g. us-east-1) | `string` | `"us-west-2"` | no |
5978

6079
## Outputs
6180

@@ -64,32 +83,21 @@ Note that this example may create resources which cost money. Run `terraform des
6483
| <a name="output_aws_auth_configmap_yaml"></a> [aws\_auth\_configmap\_yaml](#output\_aws\_auth\_configmap\_yaml) | Formatted yaml output for base aws-auth configmap containing roles used in cluster node groups/fargate profiles |
6584
| <a name="output_cloudwatch_log_group_arn"></a> [cloudwatch\_log\_group\_arn](#output\_cloudwatch\_log\_group\_arn) | Arn of cloudwatch log group created |
6685
| <a name="output_cloudwatch_log_group_name"></a> [cloudwatch\_log\_group\_name](#output\_cloudwatch\_log\_group\_name) | Name of cloudwatch log group created |
67-
| <a name="output_cluster_addons"></a> [cluster\_addons](#output\_cluster\_addons) | Map of attribute maps for all EKS cluster addons enabled |
6886
| <a name="output_cluster_arn"></a> [cluster\_arn](#output\_cluster\_arn) | The Amazon Resource Name (ARN) of the cluster |
6987
| <a name="output_cluster_certificate_authority_data"></a> [cluster\_certificate\_authority\_data](#output\_cluster\_certificate\_authority\_data) | Base64 encoded certificate data required to communicate with the cluster |
7088
| <a name="output_cluster_endpoint"></a> [cluster\_endpoint](#output\_cluster\_endpoint) | Endpoint for your Kubernetes API server |
7189
| <a name="output_cluster_iam_role_arn"></a> [cluster\_iam\_role\_arn](#output\_cluster\_iam\_role\_arn) | IAM role ARN of the EKS cluster |
7290
| <a name="output_cluster_iam_role_name"></a> [cluster\_iam\_role\_name](#output\_cluster\_iam\_role\_name) | IAM role name of the EKS cluster |
7391
| <a name="output_cluster_iam_role_unique_id"></a> [cluster\_iam\_role\_unique\_id](#output\_cluster\_iam\_role\_unique\_id) | Stable and unique string identifying the IAM role |
7492
| <a name="output_cluster_id"></a> [cluster\_id](#output\_cluster\_id) | The name/id of the EKS cluster. Will block on cluster creation until the cluster is really ready |
75-
| <a name="output_cluster_identity_providers"></a> [cluster\_identity\_providers](#output\_cluster\_identity\_providers) | Map of attribute maps for all EKS identity providers enabled |
7693
| <a name="output_cluster_oidc_issuer_url"></a> [cluster\_oidc\_issuer\_url](#output\_cluster\_oidc\_issuer\_url) | The URL on the EKS cluster for the OpenID Connect identity provider |
7794
| <a name="output_cluster_platform_version"></a> [cluster\_platform\_version](#output\_cluster\_platform\_version) | Platform version for the cluster |
7895
| <a name="output_cluster_primary_security_group_id"></a> [cluster\_primary\_security\_group\_id](#output\_cluster\_primary\_security\_group\_id) | Cluster security group that was created by Amazon EKS for the cluster. Managed node groups use this security group for control-plane-to-data-plane communication. Referred to as 'Cluster security group' in the EKS console |
7996
| <a name="output_cluster_security_group_arn"></a> [cluster\_security\_group\_arn](#output\_cluster\_security\_group\_arn) | Amazon Resource Name (ARN) of the cluster security group |
8097
| <a name="output_cluster_security_group_id"></a> [cluster\_security\_group\_id](#output\_cluster\_security\_group\_id) | ID of the cluster security group |
8198
| <a name="output_cluster_status"></a> [cluster\_status](#output\_cluster\_status) | Status of the EKS cluster. One of `CREATING`, `ACTIVE`, `DELETING`, `FAILED` |
82-
| <a name="output_cluster_tls_certificate_sha1_fingerprint"></a> [cluster\_tls\_certificate\_sha1\_fingerprint](#output\_cluster\_tls\_certificate\_sha1\_fingerprint) | The SHA1 fingerprint of the public key of the cluster's certificate |
83-
| <a name="output_eks_managed_node_groups"></a> [eks\_managed\_node\_groups](#output\_eks\_managed\_node\_groups) | Map of attribute maps for all EKS managed node groups created |
84-
| <a name="output_eks_managed_node_groups_autoscaling_group_names"></a> [eks\_managed\_node\_groups\_autoscaling\_group\_names](#output\_eks\_managed\_node\_groups\_autoscaling\_group\_names) | List of the autoscaling group names created by EKS managed node groups |
85-
| <a name="output_fargate_profiles"></a> [fargate\_profiles](#output\_fargate\_profiles) | Map of attribute maps for all EKS Fargate Profiles created |
86-
| <a name="output_kms_key_arn"></a> [kms\_key\_arn](#output\_kms\_key\_arn) | The Amazon Resource Name (ARN) of the key |
87-
| <a name="output_kms_key_id"></a> [kms\_key\_id](#output\_kms\_key\_id) | The globally unique identifier for the key |
88-
| <a name="output_kms_key_policy"></a> [kms\_key\_policy](#output\_kms\_key\_policy) | The IAM resource policy set on the key |
8999
| <a name="output_node_security_group_arn"></a> [node\_security\_group\_arn](#output\_node\_security\_group\_arn) | Amazon Resource Name (ARN) of the node shared security group |
90100
| <a name="output_node_security_group_id"></a> [node\_security\_group\_id](#output\_node\_security\_group\_id) | ID of the node shared security group |
91-
| <a name="output_oidc_provider"></a> [oidc\_provider](#output\_oidc\_provider) | The OpenID Connect identity provider (issuer URL without leading `https://`) |
92-
| <a name="output_oidc_provider_arn"></a> [oidc\_provider\_arn](#output\_oidc\_provider\_arn) | The ARN of the OIDC Provider if `enable_irsa = true` |
93101
| <a name="output_self_managed_node_groups"></a> [self\_managed\_node\_groups](#output\_self\_managed\_node\_groups) | Map of attribute maps for all self managed node groups created |
94102
| <a name="output_self_managed_node_groups_autoscaling_group_names"></a> [self\_managed\_node\_groups\_autoscaling\_group\_names](#output\_self\_managed\_node\_groups\_autoscaling\_group\_names) | List of the autoscaling group names created by self-managed node groups |
95103
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

‎examples/outposts/main.tf

+121-45
Original file line numberDiff line numberDiff line change
@@ -9,15 +9,17 @@ provider "kubernetes" {
99
exec {
1010
api_version = "client.authentication.k8s.io/v1beta1"
1111
command = "aws"
12-
# This requires the awscli to be installed locally where Terraform is executed
13-
args = ["eks", "get-token", "--cluster-name", module.eks.cluster_id]
12+
args = ["eks", "get-token", "--cluster-id", module.eks.cluster_id, "--region", var.region]
1413
}
1514
}
1615

1716
locals {
1817
name = "ex-${basename(path.cwd)}"
1918
cluster_version = "1.21" # Required by EKS on Outposts
2019

20+
outpost_arn = element(tolist(data.aws_outposts_outposts.this.arns), 0)
21+
instance_type = element(tolist(data.aws_outposts_outpost_instance_types.this.instance_types), 0)
22+
2123
tags = {
2224
Example = local.name
2325
GithubRepo = "terraform-aws-eks"
@@ -32,84 +34,158 @@ locals {
3234
module "eks" {
3335
source = "../.."
3436

35-
cluster_name = local.name
36-
cluster_version = local.cluster_version
37-
cluster_endpoint_public_access = true
37+
cluster_name = local.name
38+
cluster_version = local.cluster_version
39+
40+
cluster_endpoint_public_access = false # Not available on Outpost
41+
cluster_endpoint_private_access = true
42+
43+
vpc_id = data.aws_vpc.this.id
44+
subnet_ids = data.aws_subnets.this.ids
3845

3946
outpost_config = {
40-
control_plane_instance_type = var.outpost_instance_type
41-
outpost_arns = [tolist(data.aws_outposts_outposts.this.arns)[0]]
47+
control_plane_instance_type = local.instance_type
48+
outpost_arns = [local.outpost_arn]
4249
}
4350

44-
cluster_addons = {
45-
coredns = {
46-
most_recent = true
51+
# Extend cluster security group rules
52+
cluster_security_group_additional_rules = {
53+
ingress_vpc_https = {
54+
description = "Remote host to control plane"
55+
protocol = "tcp"
56+
from_port = 443
57+
to_port = 443
58+
type = "ingress"
59+
cidr_blocks = [data.aws_vpc.this.cidr_block]
4760
}
48-
kube-proxy = {
49-
most_recent = true
61+
inress_nodes_ephemeral_ports_tcp = {
62+
description = "To node 1025-65535"
63+
protocol = "tcp"
64+
from_port = 1025
65+
to_port = 65535
66+
type = "ingress"
67+
source_node_security_group = true
5068
}
51-
vpc-cni = {
52-
most_recent = true
53-
service_account_role_arn = module.vpc_cni_irsa.iam_role_arn
69+
egress_nodes_ephemeral_ports_tcp = {
70+
description = "To node 1025-65535"
71+
protocol = "tcp"
72+
from_port = 1025
73+
to_port = 65535
74+
type = "egress"
75+
source_node_security_group = true
5476
}
5577
}
5678

57-
# Encryption key
58-
create_kms_key = true
59-
cluster_encryption_config = {
60-
resources = ["secrets"]
79+
# Extend node-to-node security group rules
80+
node_security_group_additional_rules = {
81+
ingress_vpc_all = {
82+
description = "VPC"
83+
protocol = "-1"
84+
from_port = 0
85+
to_port = 0
86+
type = "ingress"
87+
cidr_blocks = [data.aws_vpc.this.cidr_block]
88+
}
89+
ingress_self_all = {
90+
description = "Node to node all ports/protocols"
91+
protocol = "-1"
92+
from_port = 0
93+
to_port = 0
94+
type = "ingress"
95+
self = true
96+
}
97+
egress_all = {
98+
description = "Node all egress"
99+
protocol = "-1"
100+
from_port = 0
101+
to_port = 0
102+
type = "egress"
103+
cidr_blocks = ["0.0.0.0/0"]
104+
ipv6_cidr_blocks = ["::/0"]
105+
}
61106
}
62-
kms_key_deletion_window_in_days = 7
63-
enable_kms_key_rotation = true
64-
65-
create_cluster_security_group = false
66-
create_node_security_group = false
67-
subnet_ids = [tolist(data.aws_subnets.this.ids)[0]]
68-
69-
manage_aws_auth_configmap = true
70-
71-
eks_managed_node_group_defaults = {
72-
instance_types = [var.outpost_instance_type]
73107

108+
self_managed_node_group_defaults = {
74109
attach_cluster_primary_security_group = true
110+
111+
iam_role_additional_policies = {
112+
AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
113+
}
75114
}
76115

77-
eks_managed_node_groups = {
116+
self_managed_node_groups = {
78117
outpost = {
79118
name = local.name
119+
120+
min_size = 2
121+
max_size = 5
122+
desired_size = 3
123+
instance_type = local.instance_type
80124
}
81125
}
82126

127+
# We need to add the node group IAM role to the aws-auth configmap
128+
create_aws_auth_configmap = true
129+
83130
tags = local.tags
84131
}
85132

133+
resource "kubernetes_storage_class_v1" "this" {
134+
metadata {
135+
name = "ebs-sc"
136+
annotations = {
137+
"storageclass.kubernetes.io/is-default-class" = "true"
138+
}
139+
}
140+
141+
storage_provisioner = "ebs.csi.aws.com"
142+
volume_binding_mode = "WaitForFirstConsumer"
143+
allow_volume_expansion = true
144+
145+
parameters = {
146+
type = "gp2"
147+
encrypted = "true"
148+
}
149+
}
150+
86151
################################################################################
87152
# Supporting Resources
88153
################################################################################
89154

90155
data "aws_outposts_outposts" "this" {}
91156

92-
data "aws_subnets" "this" {
157+
data "aws_outposts_outpost_instance_types" "this" {
158+
arn = local.outpost_arn
159+
}
160+
161+
# This just grabs the first Outpost and returns its subnets
162+
data "aws_subnets" "lookup" {
93163
filter {
94164
name = "outpost-arn"
95-
values = [tolist(data.aws_outposts_outposts.this.arns)[0]]
165+
values = [local.outpost_arn]
96166
}
97167
}
98168

99-
module "vpc_cni_irsa" {
100-
source = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
101-
version = "~> 5.0"
169+
# This grabs a single subnet to reverse lookup those that belong to same VPC
170+
# This is whats used for the cluster
171+
data "aws_subnet" "this" {
172+
id = element(tolist(data.aws_subnets.lookup.ids), 0)
173+
}
102174

103-
role_name_prefix = "VPC-CNI-IRSA"
104-
attach_vpc_cni_policy = true
105-
vpc_cni_enable_ipv4 = true
175+
# These are subnets for the Outpost and restricted to the same VPC
176+
# This is whats used for the cluster
177+
data "aws_subnets" "this" {
178+
filter {
179+
name = "outpost-arn"
180+
values = [local.outpost_arn]
181+
}
106182

107-
oidc_providers = {
108-
main = {
109-
provider_arn = module.eks.oidc_provider_arn
110-
namespace_service_accounts = ["kube-system:aws-node"]
111-
}
183+
filter {
184+
name = "vpc-id"
185+
values = [data.aws_subnet.this.vpc_id]
112186
}
187+
}
113188

114-
tags = local.tags
189+
data "aws_vpc" "this" {
190+
id = data.aws_subnet.this.vpc_id
115191
}

0 commit comments

Comments
 (0)