Add docs for upgrading to Caracal (#1250)

* Improve docs for upgrading to Caracal

* Update docs to include designate-sink change

* Apply suggestions from code review

Co-authored-by: Matt Crees <mattc@stackhpc.com>

* Fix wording in upgrade docs

---------

Co-authored-by: Matt Crees <mattc@stackhpc.com>

Author: Alex-Welsh

doc/source/operations/index.rst

@@ -10,7 +10,6 @@ This guide is for operators of the StackHPC Kayobe configuration project.
    hotfix-playbook
    nova-compute-ironic
    octavia
-   rabbitmq
    secret-rotation
    tempest
    upgrading-openstack

doc/source/operations/rabbitmq.rst

@@ -35,6 +35,64 @@ Notable changes in the |current_release| Release
 There are many changes in the OpenStack |current_release| release described in
 the release notes for each project. Here are some notable ones.
 
+RabbitMQ SLURP upgrade
+----------------------
+
+Because this is a SLURP upgrade, RabbitMQ must be upgraded manually from 3.11,
+to 3.12, then to 3.13 on Antelope before the Caracal upgrade. This upgrade
+should not cause an API outage (though it should still be considered "at
+risk").
+
+There are two prerequisites:
+
+1. Kolla-Ansible should be upgraded to the latest version:
+
+   .. code-block:: bash
+
+      cd $KOLLA_SOURCE_PATH
+      git fetch && git pull
+      $KOLLA_VENV_PATH/bin/pip install .
+
+2. The RabbitMQ container image tag must be equal to or newer than
+   ``20240823T101942``. Check the timestamps in
+   ``etc/kayobe/kolla-image-tags.yml``.
+
+Once complete, upgrade RabbitMQ:
+
+.. code-block:: bash
+
+   kayobe overcloud service configuration generate --node-config-dir /tmp/ignore -kt none
+   kayobe kolla ansibe run "rabbitmq-upgrade 3.12"
+   kayobe kolla ansibe run "rabbitmq-upgrade 3.13"
+
+RabbitMQ quorum queues
+----------------------
+
+In Caracal, quorum queues are enabled by default for RabbitMQ. This is
+different to Antelope which used HA queues. Before upgrading to Caracal, it is
+strongly recommended that you migrate from HA to quorum queues. The migration
+is automated using a script.
+
+.. warning::
+   This migration will stop all services using RabbitMQ and cause an
+   extended API outage while queues are migrated. It should only be
+   performed in a pre-agreed maintenance window.
+
+Set the following variables in your kolla globals file (i.e.
+``$KAYOBE_CONFIG_PATH/kolla/globals.yml`` or
+``$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/kolla/globals.yml``):
+
+.. code-block:: yaml
+
+      om_enable_rabbitmq_high_availability: false
+      om_enable_rabbitmq_quorum_queues: true
+
+Then execute the migration script:
+
+.. code-block:: bash
+
+   $KAYOBE_CONFIG_PATH/../../tools/rabbitmq-quorum-migration.sh
+
 Heat disabled by default
 ------------------------
 
@@ -54,6 +112,24 @@ using Heat, and disable the service.
 
 TODO: guide for disabling Heat
 
+Designate sink disabled by default
+----------------------------------
+
+Designate sink is optional designate service which listens for event
+Notifications, primarily from Nova and Neutron. It is disabled by default (when
+designate is enabled) in Caracal. It is not required for Designate to function.
+
+If you still wish to use it, you should set the flag manually:
+
+.. code-block:: yaml
+   :caption: ``kolla/globals.yml``
+
+   designate_enable_notifications_sink: true
+
+If you are using Designate and do not make this change, the Antelope
+``designate-sink`` container will remain on the controllers after the upgrade.
+It must be removed manually.
+
 Grafana Volume
 --------------
 The Grafana container volume is no longer used. If you wish to automatically
@@ -85,7 +161,16 @@ configuration must change the names of those files in
 Known issues
 ============
 
-* None!
+* OVN breaks on Rocky 9 deployments where hostnames are FQDNs.
+  Before upgrading, you must make sure no compute or controller nodes have any
+  ``.`` characters in their hostnames. Run the command below to check:
+
+  .. code-block:: bash
+
+     kayobe overcloud host command run --command "grep -v \'\.\' /etc/hostname" --show-output
+
+  There is currently no known fix for this issue aside from reprovisioning. A
+  patch will be developed soon.
 
 Security baseline
 =================