OSDOCS#5174: Azure: Document installing OpenShift in a restricted network [Installer-provisioned]

Author: xenolinux

_topic_maps/_topic_map.yml

@@ -215,6 +215,8 @@ Topics:
     File: installing-azure-government-region
   - Name: Installing a cluster on Azure using ARM templates
     File: installing-azure-user-infra
+  - Name: Installing a cluster on Azure in a restricted network
+    File: installing-restricted-networks-azure-installer-provisioned  
   - Name: Installing a three-node cluster on Azure
     File: installing-azure-three-node
   - Name: Uninstalling a cluster on Azure

installing/installing_azure/installing-restricted-networks-azure-installer-provisioned.adoc

@@ -0,0 +1,78 @@
+:_content-type: ASSEMBLY
+[id="installing-restricted-networks-azure-installer-provisioned"]
+= Installing a cluster on Azure in a restricted network
+include::_attributes/common-attributes.adoc[]
+:context: installing-restricted-networks-azure-installer-provisioned
+
+toc::[]
+
+In {product-title} version {product-version}, you can install a cluster on Microsoft Azure in a restricted network by creating an internal mirror of the installation release content on an existing Azure Virtual Network (VNet).
+
+[IMPORTANT]
+====
+You can install an {product-title} cluster by using mirrored installation release content, but your cluster requires internet access to use the Azure APIs.
+====
+
+[id="prerequisites_installing-restricted-networks-azure-installer-provisioned"]
+== Prerequisites
+
+* You reviewed details about the xref:../../architecture/architecture-installation.adoc#architecture-installation[{product-title} installation and update] processes.
+* You read the documentation on xref:../../installing/installing-preparing.adoc#installing-preparing[selecting a cluster installation method and preparing it for users].
+* You xref:../../installing/installing_azure/installing-azure-account.adoc#installing-azure-account[configured an Azure account] to host the cluster and determined the tested and validated region to deploy the cluster.
+* You xref:../../installing/disconnected_install/installing-mirroring-installation-images.adoc#installation-about-mirror-registry_installing-mirroring-installation-images[mirrored the images for a disconnected installation] to your registry and obtained the `imageContentSources` data for your version of {product-title}.
++
+[IMPORTANT]
+====
+Because the installation media is on the mirror host, you can use that computer to complete all installation steps.
+====
+* You have an existing VNet in Azure. While installing a cluster in a restricted network that uses installer-provisioned infrastructure, you cannot use the installer-provisioned VNet. You must use a user-provisioned VNet that satisfies one of the following requirements:
+** The VNet contains the mirror registry
+** The VNet has firewall rules or a peering connection to access the mirror registry hosted elsewhere
+* If you use a firewall, you xref:../../installing/install_config/configuring-firewall.adoc#configuring-firewall[configured it to allow the sites] that your cluster requires access to.
+* If the cloud identity and access management (IAM) APIs are not accessible in your environment, or if you do not want to store an administrator-level credential secret in the kube-system namespace, you can xref:../../installing/installing_azure/manually-creating-iam-azure.adoc#manually-creating-iam-azure[manually create and maintain IAM credentials].
+* If you use customer-managed encryption keys, you xref:../../installing/installing_azure/enabling-user-managed-encryption-azure.adoc#enabling-user-managed-encryption-azure[prepared your Azure environment for encryption].
+
+include::modules/installation-about-restricted-network.adoc[leveloffset=+1]
+
+include::modules/installation-azure-user-defined-routing.adoc[leveloffset=+2]
+
+include::modules/installation-about-custom-azure-vnet.adoc[leveloffset=+1]
+
+include::modules/cluster-entitlements.adoc[leveloffset=+1]
+
+include::modules/ssh-agent-using.adoc[leveloffset=+1]
+
+include::modules/installation-initializing.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+.Additional resources
+* xref:../../installing/installing_azure/installation-config-parameters-azure.adoc#installation-config-parameters-azure[Installation configuration parameters for Azure]
+
+include::modules/installation-minimum-resource-requirements.adoc[leveloffset=+2]
+
+include::modules/installation-azure-tested-machine-types.adoc[leveloffset=+2]
+
+include::modules/installation-azure-arm-tested-machine-types.adoc[leveloffset=+2]
+
+include::modules/installation-azure-config-yaml.adoc[leveloffset=+2]
+
+include::modules/installation-configure-proxy.adoc[leveloffset=+2]
+
+include::modules/installation-launching-installer.adoc[leveloffset=+1]
+
+include::modules/cli-installing-cli.adoc[leveloffset=+1]
+
+include::modules/cli-logging-in-kubeadmin.adoc[leveloffset=+1]
+
+include::modules/cluster-telemetry.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+.Additional resources
+
+* See xref:../../support/remote_health_monitoring/about-remote-health-monitoring.adoc#about-remote-health-monitoring[About remote health monitoring] for more information about the Telemetry service
+
+== Next steps
+
+* xref:../../post_installation_configuration/cluster-tasks.adoc#available_cluster_customizations[Customize your cluster].
+* If necessary, you can
+xref:../../support/remote_health_monitoring/opting-out-of-remote-health-reporting.adoc#opting-out-remote-health-reporting_opting-out-remote-health-reporting[opt out of remote health reporting].

modules/cli-installing-cli.adoc

@@ -49,6 +49,7 @@
 // * installing/installing_ibm_powervs/installing-ibm-power-vs-private-cluster.adoc
 // * installing/installing_ibm_powervs/installing-restricted-networks-ibm-power-vs.adoc
 // * installing/installing_ibm_powervs/installing-ibm-powervs-vpc.adoc
+// * installing/installing-restricted-networks-azure-installer-provisioned.adoc
 // AMQ docs link to this; do not change anchor
 
 ifeval::["{context}" == "mirroring-ocp-image-repository"]

modules/cli-logging-in-kubeadmin.adoc

@@ -50,7 +50,7 @@
 // * installing/installing_vsphere/installing-vsphere-installer-provisioned-network-customizations.adoc
 // * installing/installing_vsphere/installing-restricted-networks-installer-provisioned-vsphere.adoc
 // * installing/installing_ibm_z/installing-ibm-z.adoc
-
+// * installing/installing-restricted-networks-azure-installer-provisioned.adoc
 
 :_content-type: PROCEDURE
 [id="cli-logging-in-kubeadmin_{context}"]

modules/cluster-entitlements.adoc

@@ -68,6 +68,8 @@
 // * installing/installing_azure_stack_hub/installing-azure-stack-hub-network-customizations.adoc
 // * architecture/architecture.adoc
 // * installing/installing_nutanix/installing-nutanix-installer-provisioned.adoc
+// * installing/installing_azure/installing-restricted-networks-azure-installer-provisioned.adoc
+
 
 ifeval::["{context}" == "installing-restricted-networks-bare-metal"]
 :restricted:
@@ -105,6 +107,9 @@ endif::[]
 ifeval::["{context}" == "installing-restricted-networks-gcp"]
 :restricted:
 endif::[]
+ifeval::["{context}" == "installing-restricted-networks-azure-installer-provisioned"]
+:restricted:
+endif::[]
 
 :_content-type: CONCEPT
 [id="cluster-entitlements_{context}"]
@@ -173,3 +178,6 @@ endif::[]
 ifeval::["{context}" == "installing-restricted-networks-gcp"]
 :!restricted:
 endif::[]
+ifeval::["{context}" == "installing-restricted-networks-azure-installer-provisioned"]
+:!restricted:
+endif::[]

modules/cluster-telemetry.adoc

@@ -66,6 +66,7 @@
 // * installing/installing_ibm_powervs/installing-ibm-powervs-vpc.adoc
 // * installing/installing-nutanix-installer-provisioned.adoc
 // * installing/installing-restricted-networks-nutanix-installer-provisioned.adoc
+// * installing/installing-restricted-networks-azure-installer-provisioned.adoc
 
 :_content-type: CONCEPT
 [id="cluster-telemetry_{context}"]

modules/installation-about-custom-azure-vnet.adoc

@@ -3,6 +3,7 @@
 // * installing/installing_azure/installing-azure-government-region.adoc
 // * installing/installing_azure/installing-azure-private.adoc
 // * installing/installing_azure/installing-azure-vnet.adoc
+// * installing/installing_azure/installing-restricted-networks-azure-installer-provisioned.adoc
 
 ifeval::["{context}" == "installing-azure-government-region"]
 :azure:
@@ -13,6 +14,10 @@ endif::[]
 ifeval::["{context}" == "installing-azure-vnet"]
 :azure:
 endif::[]
+ifeval::["{context}" == "installing-restricted-networks-azure-installer-provisioned"]
+:azure:
+:restricted:
+endif::[]
 
 :_content-type: CONCEPT
 [id="installation-about-custom-azure-vnet_{context}"]
@@ -103,16 +108,34 @@ The network security group rules must be in place before you install the cluster
 |Allows internal communication to the machine config server for provisioning machines
 |x
 |
+
+ifdef::restricted[]
+|`*`
+a|Allows connections to Azure APIs. You must set a Destination Service Tag to `AzureCloud`. ^[1]^
+|x
+|x
+
+|`*`
+a|Denies connections to the internet. You must set a Destination Service Tag to `Internet`. ^[1]^
+|x
+|x
+endif::restricted[]
 |===
+[.small]
+--
+1. If you are using Azure Firewall to restrict the internet access, then xref:../../installing/install_config/configuring-firewall.adoc#configuring-firewall[you can configure Azure Firewall to allow the Azure APIs]. A network security group rule is not needed.
+--
 
 include::snippets/mcs-endpoint-limitation.adoc[]
 
 Because cluster components do not modify the user-provided network security groups, which the Kubernetes controllers update, a pseudo-network security group is created for the Kubernetes controller to modify without impacting the rest of the environment.
 
+[role="_additional-resources"]
 .Additional resources
 
 * xref:../../networking/openshift_sdn/about-openshift-sdn.adoc#about-openshift-sdn[About the OpenShift SDN network plugin]
 
+* xref:../../installing/install_config/configuring-firewall.adoc#configuring-firewall[Configuring your firewall]
 
 [id="installation-about-custom-azure-permissions_{context}"]
 == Division of permissions
@@ -135,3 +158,17 @@ If you deploy {product-title} to an existing network, the isolation of cluster s
 * Control plane TCP 6443 ingress (Kubernetes API) is allowed to the entire network.
 * Control plane TCP 22623 ingress (MCS) is allowed to the entire network.
 ////
+
+ifeval::["{context}" == "installing-azure-government-region"]
+:!azure:
+endif::[]
+ifeval::["{context}" == "installing-azure-private"]
+:!azure-private:
+endif::[]
+ifeval::["{context}" == "installing-azure-vnet"]
+:!azure:
+endif::[]
+ifeval::["{context}" == "installing-restricted-networks-azure-installer-provisioned"]
+:!azure:
+:!restricted:
+endif::[]

modules/installation-about-restricted-network.adoc

@@ -11,6 +11,7 @@
 // * installing/installing_ibm_power/installing-restricted-networks-ibm-power.adoc
 // * installing/installing_ibm_powervs/installing-restricted-networks-ibm-power-vs.adoc
 // * installing/installing-restricted-networks-nutanix-installer-provisioned.adoc
+// * installing/installing-restricted-networks-azure-installer-provisioned.adoc
 
 ifeval::["{context}" == "installing-ibm-power"]
 :ibm-power:
@@ -36,6 +37,9 @@ endif::[]
 ifeval::["{context}" == "installing-restricted-networks-nutanix-installer-provisioned"]
 :ipi:
 endif::[]
+ifeval::["{context}" == "installing-restricted-networks-azure-installer-provisioned"]
+:ipi:
+endif::[]
 
 :_content-type: CONCEPT
 [id="installation-about-restricted-networks_{context}"]
@@ -102,3 +106,6 @@ endif::[]
 ifeval::["{context}" == "installing-restricted-networks-nutanix-installer-provisioned"]
 :!ipi:
 endif::[]
+ifeval::["{context}" == "installing-restricted-networks-azure-installer-provisioned"]
+:!ipi:
+endif::[]

modules/installation-azure-arm-tested-machine-types.adoc

@@ -7,6 +7,7 @@
 // installing/installing_azure/installing-azure-private.adoc
 // installing/installing_azure/installing-azure-user-infra.adoc
 // installing/installing_azure/installing-azure-vnet.adoc
+// * installing/installing-restricted-networks-azure-installer-provisioned.adoc
 
 [id="installation-azure-arm-tested-machine-types_{context}"]
 = Tested instance types for Azure on 64-bit ARM infrastructures

modules/installation-azure-config-yaml.adoc

@@ -5,6 +5,7 @@
 // * installing/installing_azure/installing-azure-network-customizations.adoc
 // * installing/installing_azure/installing-azure-private.adoc
 // * installing/installing_azure/installing-azure-vnet.adoc
+// * installing/installing-restricted-networks-azure-installer-provisioned.adoc
 
 ifeval::["{context}" == "installing-azure-network-customizations"]
 :with-networking:
@@ -21,6 +22,9 @@ endif::[]
 ifeval::["{context}" == "installing-azure-government-region"]
 :gov:
 endif::[]
+ifeval::["{context}" == "installing-restricted-networks-azure-installer-provisioned"]
+:restricted:
+endif::[]
 
 [id="installation-azure-config-yaml_{context}"]
 = Sample customized install-config.yaml file for Azure
@@ -100,18 +104,21 @@ ifdef::gov[]
     region: usgovvirginia
 endif::gov[]
     resourceGroupName: existing_resource_group <9>
-ifdef::vnet,private,gov[]
+ifdef::vnet,private,gov,restricted[]
     networkResourceGroupName: vnet_resource_group <10>
     virtualNetwork: vnet <11>
     controlPlaneSubnet: control_plane_subnet <12>
     computeSubnet: compute_subnet <13>
-endif::vnet,private,gov[]
-ifndef::private,gov[]
+endif::vnet,private,gov,restricted[]
+ifndef::private,gov,restricted[]
     outboundType: Loadbalancer
-endif::private,gov[]
+endif::private,gov,restricted[]
 ifdef::private,gov[]
     outboundType: UserDefinedRouting <14>
 endif::private,gov[]
+ifdef::restricted[]
+    outboundType: UserDefinedRouting <14>
+endif::restricted[]
 ifndef::gov[]
     cloudName: AzurePublicCloud
 endif::gov[]
@@ -148,10 +155,28 @@ ifdef::openshift-origin[]
 sshKey: ssh-ed25519 AAAA... <16>
 endif::openshift-origin[]
 endif::gov[]
+ifdef::restricted[]
+fips: false <15>
+sshKey: ssh-ed25519 AAAA... <16>
+additionalTrustBundle: | <17>
+    -----BEGIN CERTIFICATE-----
+    <MY_TRUSTED_CA_CERT>
+    -----END CERTIFICATE-----
+imageContentSources: <18>
+- mirrors:
+  - <local_registry>/<local_repository_name>/release
+  source: quay.io/openshift-release-dev/ocp-release
+- mirrors:
+  - <local_registry>/<local_repository_name>/release
+  source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
+publish: Internal <19>
+endif::restricted[]
 ifndef::vnet,private,gov[]
 ifndef::openshift-origin[]
+ifndef::restricted[]
 fips: false <10>
 sshKey: ssh-ed25519 AAAA... <11>
+endif::restricted[]
 endif::openshift-origin[]
 ifdef::openshift-origin[]
 sshKey: ssh-ed25519 AAAA... <10>
@@ -195,18 +220,21 @@ If you disable simultaneous multithreading, ensure that your capacity planning a
 <7> The cluster network plugin to install. The supported values are `OVNKubernetes` and `OpenShiftSDN`. The default value is `OVNKubernetes`.
 <8> Specify the name of the resource group that contains the DNS zone for your base domain.
 <9> Specify the name of an already existing resource group to install your cluster to. If undefined, a new resource group is created for the cluster.
-ifdef::vnet,private,gov[]
+ifdef::vnet,private,gov,restricted[]
 <10> If you use an existing VNet, specify the name of the resource group that contains it.
 <11> If you use an existing VNet, specify its name.
 <12> If you use an existing VNet, specify the name of the subnet to host the control plane machines.
 <13> If you use an existing VNet, specify the name of the subnet to host the compute machines.
-endif::vnet,private,gov[]
+endif::vnet,private,gov,restricted[]
 ifdef::private,gov[]
 <14> You can customize your own outbound routing. Configuring user-defined routing prevents exposing external endpoints in your cluster. User-defined routing for egress requires deploying your cluster to an existing VNet.
 endif::private,gov[]
 ifdef::gov[]
 <15> Specify the name of the Azure cloud environment to deploy your cluster to. Set `AzureUSGovernmentCloud` to deploy to a Microsoft Azure Government (MAG) region. The default value is `AzurePublicCloud`.
 endif::gov[]
+ifdef::restricted[]
+<14> When using Azure Firewall to restrict Internet access, you must configure outbound routing to send traffic through the Firewall. Configuring user-defined routing prevents exposing external endpoints in your cluster.
+endif::restricted[]
 ifdef::vnet[]
 ifndef::openshift-origin[]
 <14> Whether to enable or disable FIPS mode. By default, FIPS mode is not enabled. If FIPS mode is enabled, the {op-system-first} machines that {product-title} runs on bypass the default Kubernetes cryptography suite and use the cryptography modules that are provided with {op-system} instead.
@@ -268,6 +296,11 @@ endif::vnet,private,gov[]
 ====
 For production {product-title} clusters on which you want to perform installation debugging or disaster recovery, specify an SSH key that your `ssh-agent` process uses.
 ====
+ifdef::restricted[]
+<17> Provide the contents of the certificate file that you used for your mirror registry.
+<18> Provide the `imageContentSources` section from the output of the command to mirror the repository.
+<19> How to publish the user-facing endpoints of your cluster. When using Azure Firewall to restrict Internet access, set `publish` to `Internal` to deploy a private cluster. The user-facing endpoints then cannot be accessed from the internet. The default value is `External`.
+endif::restricted[]
 ifdef::private[]
 ifndef::openshift-origin[]
 <17> How to publish the user-facing endpoints of your cluster. Set `publish` to `Internal` to deploy a private cluster, which cannot be accessed from the internet. The default value is `External`.
@@ -285,6 +318,7 @@ ifdef::openshift-origin[]
 endif::openshift-origin[]
 endif::gov[]
 
+
 ifeval::["{context}" == "installing-azure-network-customizations"]
 :!with-networking:
 endif::[]
@@ -300,3 +334,6 @@ endif::[]
 ifeval::["{context}" == "installing-azure-government-region"]
 :!gov:
 endif::[]
+ifeval::["{context}" == "installing-restricted-networks-azure-installer-provisioned"]
+:!restricted:
+endif::[]