Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate azure to ignition #71

Merged
merged 21 commits into from
Apr 19, 2018
Merged

Migrate azure to ignition #71

merged 21 commits into from
Apr 19, 2018

Conversation

calvix
Copy link
Contributor

@calvix calvix commented Apr 12, 2018

migrate azure to ignition - start with bastion,vault and worker

Update added also master.

Few notes:

  • we use same ignition for bastion and vault for aws/azure
  • all users are now in one file only for both aws and azure
  • master ignition is too big to be passed as user data so we pass only super simple ignition which has info where it should fetch the real big ignition
  • the big ignition is uploaded to Azure blob storage
  • added boot diagnostic to master instance

@calvix calvix self-assigned this Apr 12, 2018
@calvix calvix requested review from corest and r7vme April 18, 2018 10:59
@calvix
Copy link
Contributor Author

calvix commented Apr 18, 2018

Should be final, this now works nicely, I tried recreating from freash few times and all seem to be good.

Copy link
Contributor

@r7vme r7vme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work.

btw there are few things to address and one critical thing with public storage access.


variable "boot_diagnostics_storage_uri" {
type = "string"
description = "storage account uri fro boot diagnostics"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick. Please start from upper case and there is typo in fro

filename = "${path.cwd}/generated/master-ignition.yaml"
}

resource "azurerm_storage_account" "storage_acc" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please move all storage related part to separate module. Like it's done for S3 buckets

https://github.com/giantswarm/giantnetes-terraform/tree/master/modules/aws/s3

container_access_type = "container"
}

resource "azurerm_storage_blob" "ignition_blob" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please move ignition_blob and loader to master module. Like done for aws

https://github.com/giantswarm/giantnetes-terraform/blob/master/modules/aws/master/master.tf#L142-L165

type = "string"
type = "string"

#default = "Standard_D2s_v3"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's switch to v3 in separate PR

}

# Convert ignition config to raw json and merge users part.
data "ct_config" "master_big" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be remove _big suffix as you switched to ignition_config already.

pretty_print = false
}

resource "local_file" "master_ignition_big" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's also move this to master module

name = "${var.cluster_name}machine"
resource_group_name = "${module.resource_group.name}"
location = "westeurope"
account_tier = "Standard"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also account_kind=BlobStorage and access_tier=Cold , because we need only objects and not going to host VM disks there.

name = "ignition"
resource_group_name = "${module.resource_group.name}"
storage_account_name = "${azurerm_storage_account.storage_acc.name}"
container_access_type = "container"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This actually gives full anonymous access to container

@@ -93,6 +93,11 @@ resource "azurerm_virtual_machine" "master" {
}
}

boot_diagnostics {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need boot diagnostics? we are experts already :trollface:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it gives us logs for instance if something fuck ups. If nothing breaks we don't need it.
I am fine with removing it.

Copy link
Contributor Author

@calvix calvix Apr 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed for now

@calvix
Copy link
Contributor Author

calvix commented Apr 18, 2018

moved storage to separate module

Copy link
Contributor

@r7vme r7vme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, let's proceed with that and think how we can restrict container access.

for n in bastion master vault; do
ed --quiet ${WORKDIR}/cloud-config/${n}.yaml.tmpl << EOF
3i
cat >> ${WORKDIR}/ignition/users.yaml << EOF
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, can you please also remove ed installation above?

@calvix
Copy link
Contributor Author

calvix commented Apr 19, 2018

@r7vme I took a look at container restriction and it seems not to be implemented in terraform.

this is the functionality: https://docs.microsoft.com/en-us/azure/storage/common/storage-dotnet-shared-access-signature-part-1

Here is the issue in terraform: hashicorp/terraform-provider-azurerm#90

And we can see that only resource quota is available in terraform: https://www.terraform.io/docs/providers/azurerm/r/storage_share.html

@r7vme
Copy link
Contributor

r7vme commented Apr 19, 2018

For storage access we can use following:

  • private access mode for container
  • MSI for VM with proper role (docs)

The only concern that MSI can be not available on really early stage of ignition retrieval. I think MSI smth to do with waagent, that runs later.

@r7vme
Copy link
Contributor

r7vme commented Apr 19, 2018

The easiest solution would be to restrict access to storage account only from virtual network, but it's not implemented yet.

hashicorp/terraform-provider-azurerm#416

@calvix
Copy link
Contributor Author

calvix commented Apr 19, 2018

Here we go, azure is green.

@calvix calvix merged commit d003b14 into master Apr 19, 2018
@calvix calvix deleted the migrate-azure-to-ignition branch April 19, 2018 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants