Created scripts for provisioning Azure resources with Terraform and a preliminary README

Author: hvalfangst

README.md

@@ -0,0 +1,61 @@
+# Azure Function ETL with Pandas and CI/CD
+
+
+This repository hosts a pair of Azure Functions designed for Extract, Transform, Load (ETL) operations:
+
+    1. HTTP-triggered Function:
+        Handles CSV file uploads from request body to the Blob 'in/input.csv' under container 'hvalfangstcontainer'.
+
+    2. Blob-triggered Function:
+        Listens for changes to aforementioned Blob 'in/input.csv' and responds by loading the CSV into a Pandas dataframe on changes.
+        Conducts calculations to determine correlations between specific columns.
+        Stores the resulting correlations in a dictionary and dumps said dictionary to JSON.
+        It then proceeds to upload the JSON contents to the Blob 'out/statistics.json'.
+
+
+
+A CI/CD pipeline has been implemented utilizing a GitHub Actions Workflow script,
+which enabled automatic deployment to Azure Function App on repository pushes. Azure resources are provisioned with Terraform via shell scripts 'up' and 'down'.
+
+## Requirements
+
+* x86-64
+* Linux/Unix
+* [Python](https://www.python.org/downloads/)
+
+## Creating resources
+
+The shell script 'up' allocates Azure resources with Terraform.
+
+## Deleting resources
+
+The shell script 'down' deallocates Azure resources.
+
+## Guide
+
+### 1. Provision Azure Resources
+
+- Run the 'up' script to provision Azure resources with Terraform.
+
+
+### 2. Access Azure Portal
+
+- Open your browser and navigate to the Azure Portal.
+
+
+### 3. Function App Publish Profile
+
+- Navigate to the newly created Function App 'hvalfangstlinuxfunctionapp'
+- Click on 'Get publish profile' to download a file.
+- The associated file contents will be used in the next step.
+
+### 4. GitHub Repository Secrets
+
+- Open the 'Settings' tab of your GitHub repository.
+- Click on 'Actions' under 'Security' -> 'Secrets and variables'.
+- Create the following repository secret:
+    - PUBLISH_PROFILE: Copy value from downloaded file obtained in step #3
+
+### 5. Deploy Workflow
+
+Our Azure Function will be deployed on repository pushes by utilizing a GitHub Actions workflow script.

down.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+
+# Exits immediately if a command exits with a non-zero status
+set -e
+
+echo "Deleting Azure resources..."
+terraform -chdir=infra destroy;

infra/terraform.tf

@@ -0,0 +1,66 @@
+terraform {
+  required_providers {
+    azurerm = {
+      source  = "hashicorp/azurerm"
+      version = "3.49.0"
+    }
+  }
+}
+
+provider "azurerm" {
+  features {}
+}
+
+resource "azurerm_resource_group" "hvalfangst" {
+  name     = "hvalfangstresourcegroup"
+  location = "West Europe"
+}
+
+resource "azurerm_storage_account" "hvalfangst" {
+  name                     = "hvalfangststorageaccount"
+  resource_group_name      = azurerm_resource_group.hvalfangst.name
+  location                 = azurerm_resource_group.hvalfangst.location
+  account_tier             = "Standard"
+  account_replication_type = "LRS"
+}
+
+resource "azurerm_storage_container" "hvalfangst" {
+  name                  = "hvalfangstcontainer"
+  storage_account_name  = azurerm_storage_account.hvalfangst.name
+  container_access_type = "private"
+}
+
+resource "azurerm_service_plan" "hvalfangst" {
+  name = "hvalfangstserviceplan"
+  location = azurerm_resource_group.hvalfangst.location
+  resource_group_name = azurerm_resource_group.hvalfangst.name
+  os_type = "Linux"
+  sku_name = "Y1"
+}
+
+resource "azurerm_application_insights" "hvalfangst" {
+  name                = "hvalfangstapplicationinsights"
+  resource_group_name = azurerm_resource_group.hvalfangst.name
+  location            = azurerm_resource_group.hvalfangst.location
+  application_type    = "other"
+}
+
+resource "azurerm_linux_function_app" "hvalfangst" {
+  name                = "hvalfangstlinuxfunctionapp"
+  resource_group_name = azurerm_resource_group.hvalfangst.name
+  location            = azurerm_resource_group.hvalfangst.location
+  storage_account_name       = azurerm_storage_account.hvalfangst.name
+  storage_account_access_key = azurerm_storage_account.hvalfangst.primary_access_key
+  service_plan_id            = azurerm_service_plan.hvalfangst.id
+  site_config {
+    application_insights_key = azurerm_application_insights.hvalfangst.instrumentation_key
+    application_insights_connection_string = azurerm_application_insights.hvalfangst.connection_string
+    application_stack{
+      python_version = "3.10"
+    }
+  }
+  app_settings = {
+    "APPINSIGHTS_INSTRUMENTATIONKEY" = azurerm_application_insights.hvalfangst.instrumentation_key
+    "AzureWebJobsFeatureFlags" = "EnableWorkerIndexing"
+  }
+}

up.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+
+# Exits immediately if a command exits with a non-zero status
+set -e
+
+echo "Initializing Terraform..."
+terraform -chdir=infra init;
+
+echo "Planning Azure resource provisioning..."
+terraform -chdir=infra plan;
+
+echo "Applying planned Azure resource provisioning..."
+terraform -chdir=infra apply;