Initial commit

Signed-off-by: Adam Fowler <adam@adamfowler.org>

Author: adamfowleruk

.gitignore

@@ -1,4 +1,26 @@
-# Ignore everything in this directory
-*
-# Except this file
-!.gitignore
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+/bin
+
+# Test binary, built with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+# Dependency directories (remove the comment below to include it)
+# vendor/
+
+# Terraform example
+.terraform
+.terraform.lock.hcl
+terraform.tfstate
+terraform.tfstate.backup
+.terraform.tfstate.lock.info
+.terraform.tfstate.swp
+
+*~

CONTRIBUTING.md

@@ -1,5 +1,7 @@
 # Contributing to terraform-provider-namespace-management
 
+TODO replace with CLA (As it's Apache 2.0)
+
 The terraform-provider-namespace-management project team welcomes contributions from the community. Before you start working with terraform-provider-namespace-management, please
 read our [Developer Certificate of Origin](https://cla.vmware.com/dco). All contributions to this repository must be
 signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on

Makefile

@@ -0,0 +1,38 @@
+TEST?=$$(go list ./... | grep -v 'vendor')
+HOSTNAME=vmware.com
+NAMESPACE=vcenter
+NAME=namespace-management
+BINARY=terraform-provider-${NAME}
+VERSION=0.1
+OS_ARCH=darwin_amd64
+
+default: install
+
+build:
+	go build -o ${BINARY}
+
+release:
+	GOOS=darwin GOARCH=amd64 go build -o ./bin/${BINARY}_${VERSION}_darwin_amd64
+	GOOS=darwin GOARCH=arm64 go build -o ./bin/${BINARY}_${VERSION}_darwin_arm64
+	GOOS=freebsd GOARCH=386 go build -o ./bin/${BINARY}_${VERSION}_freebsd_386
+	GOOS=freebsd GOARCH=amd64 go build -o ./bin/${BINARY}_${VERSION}_freebsd_amd64
+	GOOS=freebsd GOARCH=arm go build -o ./bin/${BINARY}_${VERSION}_freebsd_arm
+	GOOS=linux GOARCH=386 go build -o ./bin/${BINARY}_${VERSION}_linux_386
+	GOOS=linux GOARCH=amd64 go build -o ./bin/${BINARY}_${VERSION}_linux_amd64
+	GOOS=linux GOARCH=arm go build -o ./bin/${BINARY}_${VERSION}_linux_arm
+	GOOS=openbsd GOARCH=386 go build -o ./bin/${BINARY}_${VERSION}_openbsd_386
+	GOOS=openbsd GOARCH=amd64 go build -o ./bin/${BINARY}_${VERSION}_openbsd_amd64
+	GOOS=solaris GOARCH=amd64 go build -o ./bin/${BINARY}_${VERSION}_solaris_amd64
+	GOOS=windows GOARCH=386 go build -o ./bin/${BINARY}_${VERSION}_windows_386
+	GOOS=windows GOARCH=amd64 go build -o ./bin/${BINARY}_${VERSION}_windows_amd64
+
+install: build
+	mkdir -p ~/.terraform.d/plugins/${HOSTNAME}/${NAMESPACE}/${NAME}/${VERSION}/${OS_ARCH}
+	mv ${BINARY} ~/.terraform.d/plugins/${HOSTNAME}/${NAMESPACE}/${NAME}/${VERSION}/${OS_ARCH}
+
+test: 
+	go test -i $(TEST) || exit 1                                                   
+	echo $(TEST) | xargs -t -n4 go test $(TESTARGS) -timeout=30s -parallel=4                    
+
+testacc: 
+	TF_ACC=1 go test $(TEST) -v $(TESTARGS) -timeout 120m   

README.md

@@ -1,14 +1,168 @@
-# terraform-provider-namespace-management
+# Namespace Management Terraform provider
 
-## Overview
+This Terraform Provider enables control of vSphere Workload Management.
+This includes enabling or disabling workload management 
+(effectively creating, and destroying, Tanzu Supervisor Clusters), 
+and enabling or disabling supervisor cluster services.
+
+The project is called namespace-management rather than tanzu-supervisor 
+to be consistent with the vSphere API name which it wraps.
+
+Note that Tanzu Workload Clusters should be requested by sending YAML 
+configuration via kubectl to the appropriate Supervisor Cluster's 
+vSphere namespace. There is no internal vSphere REST API for this by design.
+(This is internalised in CAPV - Cluster API for vSphere).
+
+Likewise, creating the underlying infrastructure in vSphere can be 
+accomplished by using the 
+[HashiCorp vSphere provider](https://registry.terraform.io/providers/hashicorp/vsphere)
+and the [Avi Terraform Provider]().
+An example of these being used alongside our Workload Management 
+Terraform provider is provided in the 
+examples/full_esxi_tanzu_cluster sample.
+
+## Pre initial release sprints
+
+- Alpha 1
+   - Single Avi controller instance, Avi Essentials configuration only, v21.1.2
+   - vDS networking 7.0u2 and 7.0u3 support for vCenter
+   - Works with latest photon build for TKGS (see concourse/combinations/README.md)
+   - Tested against h2o.vmware.com and my own homelab nested esxi environment
+   - No automated CI/CD testing
+   - Manual uploading of OVAs and manual Content Library creation
+   - Add all necessary repo files (Update CLA from DCO, CONTRIBUTING changes for this too)
+   - Support manual build only (Provider not yet added to Terraform registry)
+   - Govmomi bug fixes and enhancements contributed back to project
+   - REQUEST REPO BE MADE PUBLIC
+- Alpha 2
+   - Functional validation tests post cluster creation (Node up, node reachable)
+   - Overarching Concourse tests for develop branch
+   - Concourse loads environment combinations and runs multiple env pipelines in order using Terraform
+     - Support n-2 photon versions
+     - Automate testing on h20 (7.0u2) and homelab (7.0u3) using Concourse remote workers
+     - This is a total of 6 combinations
+   - Include initial vDS creation
+   - Include file upload from staging to datastore
+   - Include content library creation and uploading of TKR releases
+   - Produce test report summary files for develop branch
+   - Project introduction video
+   - Support manual build only (Provider not yet added to Terraform registry)
+- Beta 1
+   - Beta builds submitted automatically to Terraform registry on tag and release (main branch)
+   - Full sample documentation
+   - Include support for TkgServiceConfiguration customisation
+   - Include support for Custom ingress and egress CIDR ranges, CA certs
+   - Include restriction of Certs used for EC P-256
+   - Multi-node Avi controller support
+   - Avi Enterprise support (including license key upload)
+   - Add more version combinations
+     - n-2 Avi version support
+     - This is a total of 12 combinations
+   - Bootstrap Harbor VM support
+   - Helm Harbor services cluster support and sample
+   - Node/pod communication check tests (VMs, Pods)
+   - More detailed Concourse build success reports
+- Beta 2
+   - Add more version and environment combinations
+     - Include basic Workload Cluster creation for photon and ubuntu TKR at n-2 (Only 2 supported currently)
+     - Latest NSX-T support with own load balancer
+     - This is 48 combinations in total
+   - NSX-T support intro video
+   - Built in Harbor support
+   - Support for shared and standalone prometheus, grafana
+- Initial full release
+   - Add new environment and versions
+     - NSX-T n-2 version support
+     - NSX-T support with Avi load balancer
+     - Latest ESXi/vSphere version tested (currently 7.0u3d)
+     - n-2 tests for Avi Load Balancer, Avi Terraform Plugin (Matched to Avi), NSX-T and NSX-T Terraform Plugin (Matched to NSX-T)
+     - This is 576 combinations
+   - Full suite of tests (main and develop branches) with all latest minor releases of k8s TKRs
+   - Tanzu Standard on top of vSphere for Tanzu, with restricted psp/opa
+   - Istio with ingress, egress, istio-cni, minimum extra permissions (just the CNI pod)
+   - Kiali support for istio configuration validation/manual checking
+   - Full release documentation
+   - Launch video
+
+
+## Status (PRIOR TO INITIAL PUBLIC RELEASE ONLY, THEN REMOVED FROM HERE)
+
+- data_source_clusters 
+  - clustersRead()
+    - lists clusters with id, name, k8s status, config status
+    - Uses GET /api/vcenter/namespace-management/clusters
+    - Working, see examples/03_basic_list/clusters/main.tf
+    - Returns { clusters: [ {id: "domain-c1005", name:"Cluster01", kubernetes_status:"READY", config_status:"RUNNING"}, ... ] }
+- data_source_cluster
+  - clusterRead()
+    - Given a cluster NAME (NOT id) like 'Cluster01' returns the cluster's Tanzu Supervisor Cluster summary
+    - Uses GET /api/vcenter/namespace-management/clusters
+    - Summary includes (only) id, name, kubernetes_status, config_status
+    - Working, see see examples/02_basic_read/clusters/main.tf
+    - Returns {id: "domain-c1005", name:"Cluster01", kubernetes_status:"READY", config_status:"RUNNING"}
+- resource_cluster
+  - clusterCreate()
+    - Given a vSphere cluster ID (NOT name) like 'domain-c1005', enables workload management
+    - Uses POST /api/vcenter/namespace-management/clusters/{cluster}?action=enable
+    - Implemented, untested, see examples/01_basic_create/clusters/main.tf
+    - Uses hardcoded cluster enable spec data today
+    - Limited to NSX-T today rather than full information due to missing govmomi features: https://github.com/vmware/govmomi/issues/2860
+    - Warning: Due to the above, the workload cluster NTP source(s) will not be set, which will cause your workload clusters to not spin up successfully until you manually add this configuration element via vCenter
+  - clusterRead()
+    - Given a cluster NAME (NOT id) like 'Cluster01' returns the cluster's Tanzu Supervisor Cluster summary
+    - Uses List method as data_clusters clusterRead today
+    - Working, see see examples/02_basic_read/clusters/main.tf
+    - Limited to cluster summary today rather than full information due to missing govmomi feature: https://github.com/vmware/govmomi/issues/2860
+  - clusterUpdate()
+    - Given a vSphere cluster ID (NOT name) like 'domain-c1005', replaces the current cluster enable spec with a new full spec
+    - Not implemented today
+  - clusterDelete()
+    - Given a vSphere cluster ID (NOT name) like 'domain-c1005', disables workload management
+    - Doesn't actually delete the vSphere cluster, just the Tanzu Supervisor Cluster
+    - Not implemented
 
 ## Try it out
 
 ### Prerequisites
 
-* Prereq 1
-* Prereq 2
-* Prereq 3
+* You must have Terraform installed on your system
+* You must have a Go runtime installed with corresponding build tools
+* You must have a vSphere 7.0 update 2 (7.0.2) system configured with a vCenter and at least two hosts (ideally 3 or more)
+
+## Building the provider
+
+Run the following command to build the provider
+
+```shell
+go build -o terraform-provider-namespace-management
+```
+
+## Test sample configuration
+
+First, build and install the provider.
+
+```shell
+make install
+```
+
+Download the simulator from here: 
+
+TODO REWORK THIS SECTION TO NOT USE THE SIMULATOR
+
+Now unpack and run the VMware simulator
+```shell
+cat ~/Downloads/vcsim_PLATFORM_ARCH.tar.gz | sudo tar -C /usr/local/bin -xzvf - vcsim
+vcsim &
+```
+
+This will report `export GOVC_URL=https://user:pass@127.0.0.1:8989/sdk GOVC_SIM_PID=69867` when running
+
+Then, run the following command to initialize the workspace and apply the sample configuration.
+
+```shell
+cd examples/SOME_EXAMPLE
+terraform init && terraform apply
+```
 
 ### Build & Run
 
@@ -20,10 +174,13 @@
 
 ## Contributing
 
+TODO REPLACE WITH CLA (As it's Apache 2)
+
 The terraform-provider-namespace-management project team welcomes contributions from the community. Before you start working with terraform-provider-namespace-management, please
 read our [Developer Certificate of Origin](https://cla.vmware.com/dco). All contributions to this repository must be
 signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on
 as an open-source patch. For more detailed information, refer to [CONTRIBUTING.md](CONTRIBUTING.md).
 
 ## License
 
+This project is licensed under the terms of the Apache-2.0 license and is Copyright VMware, Inc. 2022. See the LICENSE file for full details.

concourse/README.md

@@ -0,0 +1,82 @@
+# Concourse automation
+
+Due to the nature of complex multi-host testing it is not possible
+to use standard GitHub CI automation to test everything within
+this project's repositories.
+
+It is insufficient to test just the provider code or to
+use the vcsim software to validate this terraform module.
+This is because vcsim's code is created to precisely match
+the vCenter API Go wrapper in the same repository - so it is
+much more likely any bug in the Go API wrapper has been duplicated
+in the vcsim software that was created for it.
+
+There is also the problem of regressions. As new software comes
+out we need to run tests against this new infrastructure.
+GitHub CI only tests the code at the point when code is created
+or a Pull Request submitted. So we need another way to monitor
+and detect upstream software changes - as they happen - and test
+various supported combinations of VMware software against these
+Terraform modules. We also need to do this against multiple
+platforms many of which are on different networks.
+
+This is where Concourse, the generic thing do-er, comes in.
+Concourse is a Free and Open Source Software (FOSS) project
+that allows for a declarative event-driven approach to testing
+software as new builds are released. Concourse proactively
+monitors a set of dependencies and if they change will kick
+off a build.
+
+This folder holds the tests and supported platform combinations
+and configurations for both the development branch (/concourse/develop)
+and main branch (/concourse/main) code repositories.
+
+When changes occur, the Concourse runtime on multiple servers will
+initiate, perform a FRESH infra rollout and automate any additional
+practical checks and tests. If any issues are found they will be 
+summarised and added to a GitHub Issue automatically.
+
+## Contents
+
+* main folder - tests that run against the current release and prior releases
+* develop folder - tests that run against the current develop branch
+* FOLDER/pipeline-*.yaml - A single pipeline in Concourse
+* FOLDER/settings-*.yaml - Settings that automated CI may update
+* combinations folder - One file per supported product combination
+
+### Common question: Shouldn't the develop tests live in the develop branch?
+
+No. The develop branch is for 'changes undergoing testing' Thus the develop
+branch of the main folder is 'changes we're trying out to tests we want
+to apply to current and previous releases' and the develop branch of the
+develop folder is 'changes we're trying out to the tests of the things
+we're adding/building in the develop branch'. So whilst the naming
+is similar, the reasoning for using BOTH folder names and branches is sound.
+
+## How this works
+
+We maintain a set of settings in the settings-combinations.yaml file. This
+uses the [experimental instance groups feature](https://concourse-ci.org/instanced-pipelines.html)
+of Concourse to instantiate a pipeline per target test environment.
+This means a single pipeline definition is instantiated for each version combination
+and settings, enabling us to test upcoming versions we've not specifically
+coded for upon release.
+
+## TODO items
+
+In order of decreasing value to this project and its users:-
+
+Immediate / Proof of Concept:-
+
+* As a new develop merge occurs, run the real tests against a new rollout on Adam's Homelab on 7.0.3 with vDS and Avi
+
+Before v1.0 release:-
+
+* Support for NSX-T deployments
+* Support for NSX-T with Avi deployments
+* Most secure by default (See ../security/README.md for details) settings and tests (poor man's pen testing)
+* Automation for current supported (Beyond Sep 2022) version combinations of vSphere with Tanzu (See combinations/ for details)
+
+Futures:-
+
+* Help other teams deliver the same for TKGm (Multicloud) and TKGI (integrated) on different cloud platforms