Skip to content

Commit 4e1a300

Browse files
wenwu449wenwu449
committed
Squashed commit of the following:
commit d0ebb9e Author: chshou <shou3301@outlook.com> Date: Mon Mar 26 18:06:57 2018 -0700 Fix merge errors (#22) * move sed to kubelet.sh, remove unnecessary exit 0 (Azure#2520) * move sed to kubelet.sh, remove unnecessary exit 0 * circleci bump * circleci bump * fix merge errors and deployment succeeded * added example * sanitize example commit 5639dce Merge: 1294a58 89f4b2e Author: Wenjun Wu <wenjun.wu@live.com> Date: Mon Mar 26 17:51:08 2018 -0700 Merge pull request #21 from yolo3301/mig-merge Merge from upstream commit 89f4b2e Merge: 375d0c0 1294a58 Author: chshou <shou3301@outlook.com> Date: Mon Mar 26 16:03:30 2018 -0700 fix merge error commit 1294a58 Author: Wenjun Wu <wenjun.wu@live.com> Date: Fri Mar 16 18:13:44 2018 -0700 skip create initial role binding due to bulit-in RBAC support. (#19) commit 375d0c0 Merge: 295461d 7d91a71 Author: chshou <shou3301@outlook.com> Date: Fri Mar 9 12:09:00 2018 -0800 resolve conflicts commit 7d91a71 Author: Wenjun Wu <wenjun.wu@live.com> Date: Fri Mar 9 11:29:11 2018 -0800 disable heapster config (#18) commit 295461d Author: chshou <shou3301@outlook.com> Date: Thu Mar 8 18:23:57 2018 -0800 remove more unnecessary commit 87f7746 Author: chshou <shou3301@outlook.com> Date: Thu Mar 8 17:55:42 2018 -0800 remove unnecessary commit ae0caf8 Author: chshou <shou3301@outlook.com> Date: Wed Mar 7 17:53:45 2018 -0800 fix 2 more missed error commit efa144e Author: chshou <shou3301@outlook.com> Date: Wed Mar 7 17:32:09 2018 -0800 a miss commit 8d96a93 Merge: fd2a409 e3587cb Author: chshou <shou3301@outlook.com> Date: Wed Mar 7 16:54:22 2018 -0800 merged from upstream master commit fd2a409 Author: Jess Frazelle <jessfraz@users.noreply.github.com> Date: Tue Jan 16 23:49:48 2018 -0500 k8s/script: allow parallelizing custom script without clear-containers (Azure#2067) Signed-off-by: Jess Frazelle <acidburn@microsoft.com> (cherry picked from commit cdd2832) Signed-off-by: Jess Frazelle <acidburn@microsoft.com> commit 1e10c0d Author: Jess Frazelle <jessfraz@users.noreply.github.com> Date: Tue Jan 16 19:09:32 2018 -0500 clear containers (Azure#1945) * clear-containers: add runtime to api and pass through parameters Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: add scripts Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: add example Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: fix variables Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: add docs Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: update install script Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: fix script Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: update example Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: update features docs Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * clear-containers: make test linters happy Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * setKubeletOpts to work better with kubeconfig Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * whitespace cruft * more whitespace fun (cherry picked from commit 8bd7c2c) commit fa3d6ff Author: Wenjun Wu <wenjun.wu@live.com> Date: Mon Feb 12 19:24:35 2018 -0800 Squashed commit of the following: commit 203efbf Author: Jiangtian Li <JiangtianLi@users.noreply.github.com> Date: Fri Jan 19 09:07:14 2018 -0800 Extend windows os drive size when customized OSDiskSizeGB is used (Azure#2097) commit 88ec2fb Author: Robbie Zhang <junjiez@microsoft.com> Date: Thu Jan 11 13:49:44 2018 -0800 Update the kube-dns addon commit 217ad8d Merge: 530bedb d8856c8 Author: Wenjun Wu <wenjun.wu@live.com> Date: Mon Jan 8 16:22:56 2018 -0800 Merge remote-tracking branch 'origin/migration' into migration commit d8856c8 Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Jan 5 15:39:28 2018 -0800 Remove the Allow SSH and RDP Rules from NSG commit 530bedb Merge: f3389a6 5070934 Author: Wenjun Wu <wenjun.wu@live.com> Date: Fri Jan 5 15:38:54 2018 -0800 Merge tag 'v0.9.4' into migration commit f3389a6 Author: Wenjun Wu <wenjun.wu@live.com> Date: Fri Dec 15 11:11:13 2017 -0800 remove agent customscript and service file (#13) * remove agent specific custom script and service file. * remove cloud provider from windows start ps1 commit c2eda57 Merge: 8ef4f2b 004145c Author: Wenjun Wu <wenjun.wu@live.com> Date: Tue Dec 12 18:05:13 2017 -0800 Merge commit '004145cba163' into migration commit 004145c Author: Wenjun Wu <wenjun.wu@live.com> Date: Tue Dec 12 18:03:36 2017 -0800 fix merge error: azure storage classes yaml commit 8ef4f2b Merge: adbc1cf bd006fc Author: Wenjun Wu <wenjun.wu@live.com> Date: Mon Nov 27 18:24:06 2017 -0800 Merge tag 'v0.9.3' into migration commit adbc1cf Merge: f8da501 7957245 Author: Wenjun Wu <wenjun.wu@live.com> Date: Wed Oct 25 14:36:24 2017 -0700 Merge tag 'v0.8.0' into migration commit f8da501 Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Sep 1 16:38:00 2017 -0700 Disable Windows Update commit ac83868 Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Sep 1 16:37:36 2017 -0700 Use kubelet v1.6.6.1 for Windows agent commit 5424f14 Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Sep 1 16:36:47 2017 -0700 Set master AvailabilitySet FaultDomainCount and UpdateDomainCount to 1 commit 5b1fbb0 Author: Robbie Zhang <junjiez@microsoft.com> Date: Tue Aug 15 12:23:41 2017 -0700 Enable StorageAccount Encryption and Enforce HTTPS commit 12fd01d Author: Harry He <zhedahht@hotmail.com> Date: Fri Jul 7 10:16:03 2017 -0700 Remove Resource Requests from kube-proxy (#5) Previously kube-proxy requested 100m CPU. It prevented containers requesting 1 CPU from being deployed onto nodes with 1 CPU, because there is only 900m CPU left. This change remove resource requests from kube-proxy. commit 5241639 Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Jul 7 14:23:32 2017 -0700 Set the default CloudProvider backoff values commit 549a4c2 Merge: 0506730 8a47cbd Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Jul 7 16:14:12 2017 -0700 Merge with v0.3.0 commit 0506730 Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Jul 7 13:01:18 2017 -0700 Disable Automatic Windows Update commit 8eb8afe Merge: 639e36a fb09cdf Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Jul 7 12:07:03 2017 -0700 Merge from upstream release v0.2.0 commit 639e36a Author: Robbie Zhang <junjiez@microsoft.com> Date: Mon Jul 3 11:05:10 2017 -0700 Remove azure.json from Windows Agent commit c9d0704 Merge: bae0a8b 579e8b8 Author: Robbie Zhang <junjiez@microsoft.com> Date: Mon Jun 19 10:13:37 2017 -0700 Merge tag 'v0.1.2' into migration commit bae0a8b Author: Raghu Shantha [MSFT] <raghus@microsoft.com> Date: Thu Jun 15 11:36:03 2017 -0700 Enable Firewall on Node, Add Windows Firewall rules for required ports (#2) * Enable Firewall on Node, Add Windows Firewall rules for required ports * Added comments for firewall rules * Allow all traffic; lockdown kubectl Node ports to Master only * Remove & and single quote in comment section resource group deployment parser does not like these chars in the comment section commit af24ad6 Author: Robbie Zhang <junjiez@microsoft.com> Date: Tue Jun 6 18:20:40 2017 -0700 Enable RBAC on APIServer commit e648d3d Merge: 380bc58 cc95f47 Author: Robbie Zhang <junjiez@microsoft.com> Date: Wed May 24 11:01:11 2017 -0700 Merge branch 'master' into migration commit 380bc58 Author: Robbie Zhang <junjiez@microsoft.com> Date: Mon May 15 11:39:43 2017 -0700 Fix: add the size map for F1 commit e64b446 Merge: 87c56c3 253dd41 Author: Wenjun Wu <wenjun.wu@live.com> Date: Sun May 14 15:47:20 2017 -0700 Merge branch 'master' into migration commit 87c56c3 Author: Robbie Zhang <junjiez@microsoft.com> Date: Fri Apr 14 12:55:21 2017 -0700 Private Commit for Azure Console Shell Remove SPN secrets from agent node Remove the Kube Dashboard and Heapster Addons Add agentpool label on the agent nodes Use static IP address for system and agentpool1 commit 9fa6a69 Author: Jack Francis <jack.francis@microsoft.com> Date: Wed Jan 31 17:07:00 2018 -0800 for loop and --retry-connrefused not avail commit 0dda4bb Author: Jack Francis <jack.francis@microsoft.com> Date: Wed Jan 31 16:44:47 2018 -0800 retry etcd download commit bb4b9bc Author: Jack Francis <jack.francis@microsoft.com> Date: Wed Jan 31 15:37:34 2018 -0800 addresses etcd startup race condition commit 2f1bfe6 Author: Jack Francis <jack.francis@microsoft.com> Date: Wed Jan 31 15:25:33 2018 -0800 cloud-init does not respect {1..5} expression commit 069d9e4 Author: CecileRobertMichon <cerobert@microsoft.com> Date: Mon Jan 22 11:41:38 2018 -0800 Add fix to upgrade backwards compatibility commit 030e5dc Author: Jack Francis <jack.francis@microsoft.com> Date: Mon Jan 22 10:42:35 2018 -0800 add support for Kubernetes v1.8.7 commit 7d19218 Author: Jack Francis <jack.francis@microsoft.com> Date: Fri Jan 19 12:31:12 2018 -0800 lint commit 9ed1610 Author: Jack Francis <jack.francis@microsoft.com> Date: Fri Jan 19 12:26:58 2018 -0800 restore properties to KubernetesConfig commit 93589b4 Author: Jack Francis <jack.francis@microsoft.com> Date: Thu Jan 18 11:48:11 2018 -0800 re-enable read-only port on kubelet fixes heapster connection issues
1 parent e3587cb commit 4e1a300

14 files changed

+245
-120
lines changed

‎examples/k8s-upgrade/v1.7.9-hybrid.json.env

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
ACSE_POSTDEPLOY=examples/k8s-upgrade/k8s-upgrade.sh
2-
EXPECTED_ORCHESTRATOR_VERSION=1.8.8
2+
EXPECTED_ORCHESTRATOR_VERSION=1.8.8

‎examples/kubernetes-aci.json

+90
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
{
2+
"apiVersion": "vlabs",
3+
"plan": {},
4+
"properties": {
5+
"provisioningState": "",
6+
"orchestratorProfile": {
7+
"orchestratorType": "Kubernetes",
8+
"orchestratorVersion": "1.8.2",
9+
"kubernetesConfig": {
10+
"networkPolicy": "none",
11+
"kubeletConfig": {
12+
"--cloud-provider": "",
13+
"--cloud-config": "",
14+
"--azure-container-registry-config": ""
15+
},
16+
"addons": [
17+
{
18+
"name": "tiller",
19+
"enabled" : false
20+
},
21+
{
22+
"name": "kubernetes-dashboard",
23+
"enabled" : false
24+
}
25+
]
26+
}
27+
},
28+
"masterProfile": {
29+
"count": 1,
30+
"dnsPrefix": "caas-test-eastus-linux-03",
31+
"vmSize": "Standard_D2_v2",
32+
"firstConsecutiveStaticIP": "10.240.255.5"
33+
},
34+
"agentPoolProfiles": [
35+
{
36+
"name": "system",
37+
"count": 2,
38+
"vmSize": "Standard_F1",
39+
"availabilityProfile": "AvailabilitySet",
40+
"storageProfile": "StorageAccount",
41+
"osType": "Linux"
42+
},
43+
{
44+
"name": "agentpool1",
45+
"count": 2,
46+
"vmSize": "Standard_F2",
47+
"availabilityProfile": "AvailabilitySet",
48+
"storageProfile": "StorageAccount",
49+
"osType": "Linux"
50+
},
51+
{
52+
"name": "agentpool2",
53+
"count": 3,
54+
"vmSize": "Standard_F1",
55+
"availabilityProfile": "AvailabilitySet",
56+
"storageProfile": "StorageAccount",
57+
"osType": "Linux",
58+
"osDiskSizeGB": 50
59+
},
60+
{
61+
"name": "agentpool3",
62+
"count": 3,
63+
"vmSize": "Standard_F1",
64+
"availabilityProfile": "AvailabilitySet",
65+
"storageProfile": "StorageAccount",
66+
"osType": "Linux",
67+
"osDiskSizeGB": 50
68+
}
69+
],
70+
"linuxProfile": {
71+
"adminUsername": "azureuser",
72+
"ssh": {
73+
"publicKeys": [
74+
{
75+
"keyData": ""
76+
}
77+
]
78+
}
79+
},
80+
"windowsProfile": {
81+
"adminUsername": "",
82+
"adminPassword": ""
83+
},
84+
"servicePrincipalProfile": {
85+
"clientId": "",
86+
"secret": ""
87+
},
88+
"certificateProfile": {}
89+
}
90+
}

‎parts/k8s/addons/kubernetesmasteraddons-kube-dns-deployment.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,11 @@ spec:
4343
matchLabels:
4444
k8s-app: kube-dns
4545
version: v20
46+
strategy:
47+
rollingUpdate:
48+
maxSurge: 1
49+
maxUnavailable: 1
50+
type: RollingUpdate
4651
template:
4752
metadata:
4853
annotations:
@@ -168,3 +173,4 @@ spec:
168173
serviceAccountName: kube-dns
169174
nodeSelector:
170175
beta.kubernetes.io/os: linux
176+
agentpool: system

‎parts/k8s/addons/kubernetesmasteraddons-kube-proxy-daemonset.yaml

-3
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,6 @@ spec:
2828
- "--feature-gates=ExperimentalCriticalPodAnnotation=true"
2929
image: "<kubernetesHyperkubeSpec>"
3030
name: kube-proxy
31-
resources:
32-
requests:
33-
cpu: 100m
3431
securityContext:
3532
privileged: true
3633
volumeMounts:

‎parts/k8s/kubernetesagentcustomdata.yml

+3-5
Original file line numberDiff line numberDiff line change
@@ -129,8 +129,9 @@ AGENT_ARTIFACTS_CONFIG_PLACEHOLDER
129129
# SNAT outbound traffic from pods to destinations outside of VNET.
130130
iptables -t nat -A POSTROUTING -m iprange ! --dst-range 168.63.129.16 -m addrtype ! --dst-type local ! -d {{WrapAsVariable "vnetCidr"}} -j MASQUERADE
131131
{{end}}
132-
133-
exit 0
132+
{{if not EnablePodSecurityPolicy}}
133+
sed -i "s|apparmor_parser|d|g" "/etc/systemd/system/kubelet.service"
134+
{{end}}
134135

135136
- path: "/opt/azure/containers/provision.sh"
136137
permissions: "0744"
@@ -168,9 +169,6 @@ coreos:
168169
[Service]
169170
ExecStart=/opt/azure/containers/provision-setup.sh
170171
{{else}}
171-
{{if not EnablePodSecurityPolicy}}
172-
sed -i "s|apparmor_parser|d|g" "/etc/systemd/system/kubelet.service"
173-
{{end}}
174172
runcmd:
175173
- echo `date`,`hostname`, startruncmd>>/opt/m
176174
# the first arg is the number of retries, the second arg is the wait duration between two retries and the rest of the args are the cmd to run

‎parts/k8s/kubernetesagentresourcesvmas.t

+40-2
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,15 @@
2727
{{if eq $seq 1}}
2828
"primary": true,
2929
{{end}}
30+
{{if eq $.Name "system"}}
31+
"privateIPAddress": "[concat(variables('masterFirstAddrPrefix'), copyIndex(add(50, int(variables('masterFirstAddrOctet4')))))]",
32+
"privateIPAllocationMethod": "Static",
33+
{{else if eq $.Name "agentpool1"}}
34+
"privateIPAddress": "[concat(variables('masterFirstAddrPrefix'), copyIndex(add(100, int(variables('masterFirstAddrOctet4')))))]",
35+
"privateIPAllocationMethod": "Static",
36+
{{else}}
3037
"privateIPAllocationMethod": "Dynamic",
38+
{{end}}
3139
"subnet": {
3240
"id": "[variables('{{$.Name}}VnetSubnetID')]"
3341
}
@@ -71,10 +79,25 @@
7179
],
7280
{{end}}
7381
{{end}}
82+
"kind": "Storage",
7483
"location": "[variables('location')]",
7584
"name": "[concat(variables('storageAccountPrefixes')[mod(add(copyIndex(),variables('{{.Name}}StorageAccountOffset')),variables('storageAccountPrefixesCount'))],variables('storageAccountPrefixes')[div(add(copyIndex(),variables('{{.Name}}StorageAccountOffset')),variables('storageAccountPrefixesCount'))],variables('{{.Name}}AccountName'))]",
7685
"properties": {
77-
"accountType": "[variables('vmSizesMap')[variables('{{.Name}}VMSize')].storageAccountType]"
86+
"encryption": {
87+
"keySource": "Microsoft.Storage",
88+
"services": {
89+
"blob": {
90+
"enabled": true
91+
},
92+
"file": {
93+
"enabled": true
94+
}
95+
}
96+
},
97+
"supportsHttpsTrafficOnly": true
98+
},
99+
"sku": {
100+
"name": "[variables('vmSizesMap')[variables('{{.Name}}VMSize')].storageAccountType]"
78101
},
79102
"type": "Microsoft.Storage/storageAccounts"
80103
},
@@ -92,10 +115,25 @@
92115
],
93116
{{end}}
94117
{{end}}
118+
"kind": "Storage",
95119
"location": "[variables('location')]",
96120
"name": "[concat(variables('storageAccountPrefixes')[mod(add(copyIndex(variables('dataStorageAccountPrefixSeed')),variables('{{.Name}}StorageAccountOffset')),variables('storageAccountPrefixesCount'))],variables('storageAccountPrefixes')[div(add(copyIndex(variables('dataStorageAccountPrefixSeed')),variables('{{.Name}}StorageAccountOffset')),variables('storageAccountPrefixesCount'))],variables('{{.Name}}DataAccountName'))]",
97121
"properties": {
98-
"accountType": "[variables('vmSizesMap')[variables('{{.Name}}VMSize')].storageAccountType]"
122+
"encryption": {
123+
"keySource": "Microsoft.Storage",
124+
"services": {
125+
"blob": {
126+
"enabled": true
127+
},
128+
"file": {
129+
"enabled": true
130+
}
131+
}
132+
},
133+
"supportsHttpsTrafficOnly": true
134+
},
135+
"sku": {
136+
"name": "[variables('vmSizesMap')[variables('{{.Name}}VMSize')].storageAccountType]"
99137
},
100138
"type": "Microsoft.Storage/storageAccounts"
101139
},

‎parts/k8s/kubernetesmastercustomdata.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,7 @@ MASTER_ARTIFACTS_CONFIG_PLACEHOLDER
188188
sed -i "s|<kubernetesHyperkubeSpec>|{{WrapAsVariable "kubernetesHyperkubeSpec"}}|g" "/etc/kubernetes/manifests/kube-scheduler.yaml"
189189
sed -i "s|<kubernetesHyperkubeSpec>|{{WrapAsVariable "kubernetesHyperkubeSpec"}}|g; s|<kubeClusterCidr>|{{WrapAsVariable "kubeClusterCidr"}}|g" "/etc/kubernetes/addons/kube-proxy-daemonset.yaml"
190190
sed -i "s|<kubernetesKubeDNSSpec>|{{WrapAsVariable "kubernetesKubeDNSSpec"}}|g; s|<kubernetesDNSMasqSpec>|{{WrapAsVariable "kubernetesDNSMasqSpec"}}|g; s|<kubernetesExecHealthzSpec>|{{WrapAsVariable "kubernetesExecHealthzSpec"}}|g; s|<kubernetesKubeletClusterDomain>|{{WrapAsVariable "kubernetesKubeletClusterDomain"}}|g; s|<kubeDNSServiceIP>|{{WrapAsVariable "kubeDNSServiceIP"}}|g" "/etc/kubernetes/addons/kube-dns-deployment.yaml"
191-
sed -i "s|<kubernetesHeapsterSpec>|{{WrapAsVariable "kubernetesHeapsterSpec"}}|g; s|<kubernetesAddonResizerSpec>|{{WrapAsVariable "kubernetesAddonResizerSpec"}}|g" "/etc/kubernetes/addons/kube-heapster-deployment.yaml"
191+
# sed -i "s|<kubernetesHeapsterSpec>|{{WrapAsVariable "kubernetesHeapsterSpec"}}|g; s|<kubernetesAddonResizerSpec>|{{WrapAsVariable "kubernetesAddonResizerSpec"}}|g" "/etc/kubernetes/addons/kube-heapster-deployment.yaml"
192192

193193
{{if .OrchestratorProfile.KubernetesConfig.IsDashboardEnabled}}
194194
sed -i "s|<kubernetesDashboardSpec>|{{WrapAsVariable "kubernetesDashboardSpec"}}|g" "/etc/kubernetes/addons/kubernetes-dashboard-deployment.yaml"

‎parts/k8s/kubernetesmastercustomscript.sh

+22-11
Original file line numberDiff line numberDiff line change
@@ -56,11 +56,7 @@ echo `date`,`hostname`, startscript>>/opt/m
5656
# A delay to start the kubernetes processes is necessary
5757
# if a reboot is required. Otherwise, the agents will encounter issue:
5858
# https://github.com/kubernetes/kubernetes/issues/41185
59-
if [ -f /var/run/reboot-required ]; then
60-
REBOOTREQUIRED=true
61-
else
62-
REBOOTREQUIRED=false
63-
fi
59+
REBOOTREQUIRED=false
6460

6561
if [[ ! -z "${MASTER_NODE}" ]]; then
6662
echo "executing master node provision operations"
@@ -138,15 +134,21 @@ touch "${APISERVER_PUBLIC_KEY_PATH}"
138134
chmod 0644 "${APISERVER_PUBLIC_KEY_PATH}"
139135
chown root:root "${APISERVER_PUBLIC_KEY_PATH}"
140136

141-
AZURE_JSON_PATH="/etc/kubernetes/azure.json"
142-
touch "${AZURE_JSON_PATH}"
143-
chmod 0600 "${AZURE_JSON_PATH}"
144-
chown root:root "${AZURE_JSON_PATH}"
145-
146137
set +x
147138
echo "${KUBELET_PRIVATE_KEY}" | base64 --decode > "${KUBELET_PRIVATE_KEY_PATH}"
148139
echo "${APISERVER_PUBLIC_KEY}" | base64 --decode > "${APISERVER_PUBLIC_KEY_PATH}"
149-
cat << EOF > "${AZURE_JSON_PATH}"
140+
set -x
141+
142+
if [[ ! -z "${MASTER_NODE}" ]]; then
143+
echo "MASTER_NODE is non-empty, master node, configure azure json."
144+
145+
AZURE_JSON_PATH="/etc/kubernetes/azure.json"
146+
touch "${AZURE_JSON_PATH}"
147+
chmod 0600 "${AZURE_JSON_PATH}"
148+
chown root:root "${AZURE_JSON_PATH}"
149+
150+
set +x
151+
cat << EOF > "${AZURE_JSON_PATH}"
150152
{
151153
"cloud":"${TARGET_ENVIRONMENT}",
152154
"tenantId": "${TENANT_ID}",
@@ -173,6 +175,9 @@ cat << EOF > "${AZURE_JSON_PATH}"
173175
"useInstanceMetadata": ${USE_INSTANCE_METADATA}
174176
}
175177
EOF
178+
else
179+
echo "MASTER_NODE is empty, worker node, skip azure json."
180+
fi
176181

177182
###########################################################
178183
# END OF SECRET DATA
@@ -754,6 +759,12 @@ fi
754759

755760
echo "Install complete successfully"
756761

762+
if [ -f /var/run/reboot-required ]; then
763+
REBOOTREQUIRED=true
764+
else
765+
REBOOTREQUIRED=false
766+
fi
767+
757768
if $REBOOTREQUIRED; then
758769
# wait 1 minute to restart node, so that the custom script extension can complete
759770
echo 'reboot required, rebooting node in 1 minute'

‎parts/k8s/kubernetesmasterresources.t

+18-33
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@
55
"name": "[variables('masterAvailabilitySet')]",
66
"properties":
77
{
8-
"platformFaultDomainCount": "2",
9-
"platformUpdateDomainCount": "3",
8+
"platformFaultDomainCount": "1",
9+
"platformUpdateDomainCount": "1",
1010
"managed" : "true"
1111
},
1212
"type": "Microsoft.Compute/availabilitySets"
@@ -26,10 +26,25 @@
2626
"[concat('Microsoft.Network/publicIPAddresses/', variables('masterPublicIPAddressName'))]"
2727
],
2828
{{end}}
29+
"kind": "Storage",
2930
"location": "[variables('location')]",
3031
"name": "[variables('masterStorageAccountName')]",
3132
"properties": {
32-
"accountType": "[variables('vmSizesMap')[variables('masterVMSize')].storageAccountType]"
33+
"encryption": {
34+
"keySource": "Microsoft.Storage",
35+
"services": {
36+
"blob": {
37+
"enabled": true
38+
},
39+
"file": {
40+
"enabled": true
41+
}
42+
}
43+
},
44+
"supportsHttpsTrafficOnly": true
45+
},
46+
"sku": {
47+
"name": "[variables('vmSizesMap')[variables('masterVMSize')].storageAccountType]"
3348
},
3449
"type": "Microsoft.Storage/storageAccounts"
3550
},
@@ -79,36 +94,6 @@
7994
"name": "[variables('nsgName')]",
8095
"properties": {
8196
"securityRules": [
82-
{{if .HasWindows}}
83-
{
84-
"name": "allow_rdp",
85-
"properties": {
86-
"access": "Allow",
87-
"description": "Allow RDP traffic to master",
88-
"destinationAddressPrefix": "*",
89-
"destinationPortRange": "3389-3389",
90-
"direction": "Inbound",
91-
"priority": 102,
92-
"protocol": "Tcp",
93-
"sourceAddressPrefix": "*",
94-
"sourcePortRange": "*"
95-
}
96-
},
97-
{{end}}
98-
{
99-
"name": "allow_ssh",
100-
"properties": {
101-
"access": "Allow",
102-
"description": "Allow SSH traffic to master",
103-
"destinationAddressPrefix": "*",
104-
"destinationPortRange": "22-22",
105-
"direction": "Inbound",
106-
"priority": 101,
107-
"protocol": "Tcp",
108-
"sourceAddressPrefix": "*",
109-
"sourcePortRange": "*"
110-
}
111-
},
11297
{
11398
"name": "allow_kube_tls",
11499
"properties": {

‎parts/k8s/kubernetesmastervars.t

+4-4
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,7 @@
188188
"sshKeyPath": "[concat('/home/',variables('username'),'/.ssh/authorized_keys')]",
189189

190190
{{if .HasStorageAccountDisks}}
191-
"apiVersionStorage": "2015-06-15",
191+
"apiVersionStorage": "2016-12-01",
192192
"maxVMsPerStorageAccount": 20,
193193
"maxStorageAccountsPerAgent": "[div(variables('maxVMsPerPool'),variables('maxVMsPerStorageAccount'))]",
194194
"dataStorageAccountPrefixSeed": 97,
@@ -211,10 +211,10 @@
211211
{{end}}
212212
"provisionScript": "{{GetKubernetesB64Provision}}",
213213
"mountetcdScript": "{{GetKubernetesB64Mountetcd}}",
214-
"provisionScriptParametersCommon": "[concat('TENANT_ID=',variables('tenantID'),' APISERVER_PUBLIC_KEY=',variables('apiserverCertificate'),' SUBSCRIPTION_ID=',variables('subscriptionId'),' RESOURCE_GROUP=',variables('resourceGroup'),' LOCATION=',variables('location'),' SUBNET=',variables('subnetName'),' NETWORK_SECURITY_GROUP=',variables('nsgName'),' VIRTUAL_NETWORK=',variables('virtualNetworkName'),' VIRTUAL_NETWORK_RESOURCE_GROUP=',variables('virtualNetworkResourceGroupName'),' ROUTE_TABLE=',variables('routeTableName'),' PRIMARY_AVAILABILITY_SET=',variables('primaryAvailabilitySetName'),' SERVICE_PRINCIPAL_CLIENT_ID=',variables('servicePrincipalClientId'),' SERVICE_PRINCIPAL_CLIENT_SECRET=',variables('singleQuote'),variables('servicePrincipalClientSecret'),variables('singleQuote'),' KUBELET_PRIVATE_KEY=',variables('clientPrivateKey'),' TARGET_ENVIRONMENT=',variables('targetEnvironment'),' NETWORK_POLICY=',variables('networkPolicy'),' FQDNSuffix=',variables('fqdnEndpointSuffix'),' VNET_CNI_PLUGINS_URL=',variables('vnetCniLinuxPluginsURL'),' CNI_PLUGINS_URL=',variables('cniPluginsURL'),' MAX_PODS=',variables('maxPods'),' CLOUDPROVIDER_BACKOFF=',variables('cloudProviderBackoff'),' CLOUDPROVIDER_BACKOFF_RETRIES=',variables('cloudProviderBackoffRetries'),' CLOUDPROVIDER_BACKOFF_EXPONENT=',variables('cloudProviderBackoffExponent'),' CLOUDPROVIDER_BACKOFF_DURATION=',variables('cloudProviderBackoffDuration'),' CLOUDPROVIDER_BACKOFF_JITTER=',variables('cloudProviderBackoffJitter'),' CLOUDPROVIDER_RATELIMIT=',variables('cloudProviderRatelimit'),' CLOUDPROVIDER_RATELIMIT_QPS=',variables('cloudProviderRatelimitQPS'),' CLOUDPROVIDER_RATELIMIT_BUCKET=',variables('cloudProviderRatelimitBucket'),' USE_MANAGED_IDENTITY_EXTENSION=',variables('useManagedIdentityExtension'),' USE_INSTANCE_METADATA=',variables('useInstanceMetadata'),' CONTAINER_RUNTIME=',variables('containerRuntime'),' KUBECONFIG_SERVER=',variables('kubeconfigServer'))]",
214+
"provisionScriptParametersCommon": "[concat('KUBELET_PRIVATE_KEY=',variables('clientPrivateKey'),' NETWORK_POLICY=',variables('networkPolicy'),' APISERVER_PUBLIC_KEY=',variables('apiserverCertificate'),' MAX_PODS=',variables('maxPods'),' CONTAINER_RUNTIME=',variables('containerRuntime'))]",
215215

216216
{{if not IsHostedMaster}}
217-
"provisionScriptParametersMaster": "[concat('MASTER_NODE=true TOTAL_NODES=',variables('totalNodes'),' APISERVER_PRIVATE_KEY=',variables('apiServerPrivateKey'),' CA_CERTIFICATE=',variables('caCertificate'),' CA_PRIVATE_KEY=',variables('caPrivateKey'),' MASTER_FQDN=',variables('masterFqdnPrefix'),' KUBECONFIG_CERTIFICATE=',variables('kubeConfigCertificate'),' KUBECONFIG_KEY=',variables('kubeConfigPrivateKey'),' ETCD_SERVER_CERTIFICATE=',variables('etcdServerCertificate'),' ETCD_CLIENT_CERTIFICATE=',variables('etcdClientCertificate'),' ETCD_SERVER_PRIVATE_KEY=',variables('etcdServerPrivateKey'),' ETCD_CLIENT_PRIVATE_KEY=',variables('etcdClientPrivateKey'),' ETCD_PEER_CERTIFICATES=',string(variables('etcdPeerCertificates')),' ETCD_PEER_PRIVATE_KEYS=',string(variables('etcdPeerPrivateKeys')),' ADMINUSER=',variables('username'))]",
217+
"provisionScriptParametersMaster": "[concat('MASTER_NODE=true TOTAL_NODES=',variables('totalNodes'),' TENANT_ID=',variables('tenantID'),' SUBSCRIPTION_ID=',variables('subscriptionId'),' RESOURCE_GROUP=',variables('resourceGroup'),' LOCATION=',variables('location'),' SUBNET=',variables('subnetName'),' NETWORK_SECURITY_GROUP=',variables('nsgName'),' VIRTUAL_NETWORK=',variables('virtualNetworkName'),' VIRTUAL_NETWORK_RESOURCE_GROUP=',variables('virtualNetworkResourceGroupName'),' ROUTE_TABLE=',variables('routeTableName'),' PRIMARY_AVAILABILITY_SET=',variables('primaryAvailabilitySetName'),' SERVICE_PRINCIPAL_CLIENT_ID=',variables('servicePrincipalClientId'),' SERVICE_PRINCIPAL_CLIENT_SECRET=',variables('servicePrincipalClientSecret'),' TARGET_ENVIRONMENT=',variables('targetEnvironment'),' FQDNSuffix=',variables('fqdnEndpointSuffix'),' VNET_CNI_PLUGINS_URL=',variables('vnetCniLinuxPluginsURL'),' CNI_PLUGINS_URL=',variables('cniPluginsURL'),' CLOUDPROVIDER_BACKOFF=',variables('cloudProviderBackoff'),' CLOUDPROVIDER_BACKOFF_RETRIES=',variables('cloudProviderBackoffRetries'),' CLOUDPROVIDER_BACKOFF_EXPONENT=',variables('cloudProviderBackoffExponent'),' CLOUDPROVIDER_BACKOFF_DURATION=',variables('cloudProviderBackoffDuration'),' CLOUDPROVIDER_BACKOFF_JITTER=',variables('cloudProviderBackoffJitter'),' CLOUDPROVIDER_RATELIMIT=',variables('cloudProviderRatelimit'),' CLOUDPROVIDER_RATELIMIT_QPS=',variables('cloudProviderRatelimitQPS'),' CLOUDPROVIDER_RATELIMIT_BUCKET=',variables('cloudProviderRatelimitBucket'),' USE_MANAGED_IDENTITY_EXTENSION=',variables('useManagedIdentityExtension'),' USE_INSTANCE_METADATA=',variables('useInstanceMetadata'),' APISERVER_PRIVATE_KEY=',variables('apiServerPrivateKey'),' CA_CERTIFICATE=',variables('caCertificate'),' CA_PRIVATE_KEY=',variables('caPrivateKey'),' MASTER_FQDN=',variables('masterFqdnPrefix'),' KUBECONFIG_CERTIFICATE=',variables('kubeConfigCertificate'),' KUBECONFIG_KEY=',variables('kubeConfigPrivateKey'),' ETCD_SERVER_CERTIFICATE=',variables('etcdServerCertificate'),' ETCD_CLIENT_CERTIFICATE=',variables('etcdClientCertificate'),' ETCD_SERVER_PRIVATE_KEY=',variables('etcdServerPrivateKey'),' ETCD_CLIENT_PRIVATE_KEY=',variables('etcdClientPrivateKey'),' ETCD_PEER_CERTIFICATES=',string(variables('etcdPeerCertificates')),' ETCD_PEER_PRIVATE_KEYS=',string(variables('etcdPeerPrivateKeys')),' ADMINUSER=',variables('username'),' KUBECONFIG_SERVER=',variables('kubeconfigServer'))]",
218218
{{end}}
219219
"generateProxyCertsScript": "{{GetKubernetesB64GenerateProxyCerts}}",
220220
"orchestratorNameVersionTag": "{{.OrchestratorProfile.OrchestratorType}}:{{.OrchestratorProfile.OrchestratorVersion}}",
@@ -269,7 +269,7 @@
269269
"nsgName": "[concat(variables('agentNamePrefix'), 'nsg')]",
270270
{{end}}
271271
"nsgID": "[resourceId('Microsoft.Network/networkSecurityGroups',variables('nsgName'))]",
272-
"primaryAvailabilitySetName": "[concat('{{ (index .AgentPoolProfiles 0).Name }}-availabilitySet-',variables('nameSuffix'))]",
272+
"primaryAvailabilitySetName": "[concat('{{ (index .AgentPoolProfiles 1).Name }}-availabilitySet-',variables('nameSuffix'))]",
273273
{{if not IsHostedMaster }}
274274
{{if IsPrivateCluster}}
275275
"kubeconfigServer": "[concat('https://', variables('kubernetesAPIServerIP'), ':443')]",

0 commit comments

Comments
 (0)