[DevOps] Automate the release process

[flanakin]

⚠️ Problem

The monthly release process is overly manual, takes a lot of time, and is error prone.

🛠️ Solution

Automate the following tasks in the release process.

Note

Each of these should be done in a separate, small PR. Please don't try to solve everything in one PR.

• Automate building PowerShell for open data PRs #916
  • Whenever a .csv or .json file is updated in the /src/open-data folder in a PR, run Build-OpenData -PowerShell and push the results to the PR.
• Run open data tests when open data files are updated: Build-OpenData -Test. #915
• Automatically comment on Open Data PR's when unit tests fail #961
• Create a script to update open data files and run it weekly:
  • Update the PricingUnits, Regions, and Services files by running a Kusto query and saving results as CSV.
  • Run an existing script to update the ResourceTypes files.
  • Submit a PR.
• [DevOps] Create a test to check for broken links #1690
  • Bonus points for adding this as a Pester test
• Create an manual action/script that sets up a new milestone:
  • Create an empty "yyyy-MM - Mmmm (v#.#)" milestone.
  • Create a corresponding release discussion thread.
• Create an action to update feature branches using an existing script when code is pushed to dev and create an issue to track updating branches that fail to merge. If any branches haven't been updated in over 1mo, create an issue to consider deleting the branch.
• Create a weekly action that runs all PowerShell unit, integration, and lint tests and creates issues for any failures to catch breaking changes from dependencies or breaks missed in earlier PRs.
  • Side note: We've never resolved all lint errors, so that's another set of small tasks that can go in one at a time. It's hard to get others to care about those
• Configure a CI/CD test environment for each of the templates (one is already done) and run them when PRs update code for the respective solutions.
• ⭐ Update the Package-Toolkit script to use a UI automation library to prepare Power BI project files:
  • Save Power BI project files as PBIX.
  • Remove queries that are not needed per report.
  • Set sensitivity to Public.
  • Save PBIX again.
  • Save as PBIT.
• Create a manual action/script that finalizes a release:
  • Run the Update-Version script.
  • Update the Bicep CLI, if applicable.
  • Build all templates, run all tests, and create issues for any failures.
  • If no failures, merge dev to main.
  • Tag the latest in the main branch with "v".
  • Run the Package-Toolkit script and push changes back to the main branch. (It updates some versioned files.)
  • Create a draft release called "Mmm yyyy (v#.#)" based on a template, copy changes from the changelog, update the list of contributors, and upload the files from Package-Toolkit.
  • Update the related release discussion forum, if applicable.
  • Add the Status: Released label to all issues in the milestone and remove any other Status labels.
    • Note: Removing other status labels could be done via an issue policy. I don't think GitOps issue policies support milestone change triggers.
• Create an action that creates PRs in external repos when there's a new release:
  • All templates go to the Azure Quickstart Templates repo (partially implemented).
  • Bicep modules go to the Bicep Registry repo (partially implemented).
  • Some workbooks go to the Azure Monitor workbook repo.
  • Some templates go to the Azure Verified Modules repo.
• Create an action that runs (updates?) version-based integration tests when there's a new release and submit a PR.
  • Note: This just needs to add expected/unexpected files.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.