Skip to content

added probes for RabbitMQ with graceful shutdown #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
henzigo wants to merge 1 commit into
base: jg/redis-order
Choose a base branch
from
Open

added probes for RabbitMQ with graceful shutdown #75

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions UPGRADE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## Upgrade from v5.0.0 to v5.1.0

- added probes for RabbitMQ with graceful shutdown ([#75](https://github.com/shopsys/deployment/pull/75))
- changed container order for Redis ([#74](https://github.com/shopsys/deployment/pull/74))
- cache warmup is now run as an init container instead of a `postStart` lifecycle hook ([#73](https://github.com/shopsys/deployment/pull/73))
- cron pod is now always deleted gracefully ([#72](https://github.com/shopsys/deployment/pull/72))
Expand Down
32 changes: 32 additions & 0 deletions kubernetes/deployments/rabbitmq.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ metadata:
spec:
serviceName: rabbitmq
replicas: 1
updateStrategy:
type: RollingUpdate
rollingUpdate:
partition: 0
selector:
matchLabels:
app: rabbitmq
Expand All @@ -30,6 +34,9 @@ spec:
image: rabbitmq:4.1-management-alpine
ports:
- name: rabbitmq
containerPort: 5672
protocol: TCP
- name: rabbitmq-management
containerPort: 15672
protocol: TCP
- name: exporter
Expand All @@ -40,12 +47,37 @@ spec:
value: "{{RABBITMQ_DEFAULT_USER}}"
- name: RABBITMQ_DEFAULT_PASS
value: "{{RABBITMQ_DEFAULT_PASS}}"
lifecycle:
preStop:
exec:
command: [ "sh", "-c", "rabbitmqctl stop_app && sleep 5" ]

Copilot AI Apr 15, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preStop uses rabbitmqctl stop_app && sleep 5, so if stop_app exits non-zero (e.g., node already stopping / CLI can't connect), the sleep won’t run and the hook won’t provide the intended drain delay. Consider making the hook resilient (e.g., run stop_app best-effort and always sleep).

Suggested change
command: [ "sh", "-c", "rabbitmqctl stop_app && sleep 5" ]
command: [ "sh", "-c", "rabbitmqctl stop_app || true; sleep 5" ]

Copilot uses AI. Check for mistakes.
resources:
requests:
cpu: "20m"
memory: "256Mi"
limits:
memory: "512Mi"
startupProbe:
exec:
command: [ "rabbitmq-diagnostics", "ping" ]
failureThreshold: 30
periodSeconds: 10

Copilot AI Apr 15, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

startupProbe doesn’t set timeoutSeconds, so it defaults to 1s; exec probes can easily exceed that under load (especially during cold start), causing unnecessary restarts. Set an explicit timeoutSeconds (typically matching liveness/readiness) to avoid flaky startups.

Suggested change
periodSeconds: 10
periodSeconds: 10
timeoutSeconds: 15

Copilot uses AI. Check for mistakes.
livenessProbe:
exec:
command: ["rabbitmq-diagnostics", "ping"]
periodSeconds: 30
timeoutSeconds: 15
failureThreshold: 5
readinessProbe:
exec:
command: [ "rabbitmq-diagnostics", "-q", "check_running" ]
periodSeconds: 10
timeoutSeconds: 10
failureThreshold: 3
volumeMounts:
- name: rabbitmq-data
mountPath: /var/lib/rabbitmq
terminationGracePeriodSeconds: 90
imagePullSecrets:
- name: dockerregistry
volumeClaimTemplates:
Expand Down
44 changes: 43 additions & 1 deletion tests/scenarios/basic-production/expected/migrate-continuous-deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,22 +330,64 @@ spec:
- name: RABBITMQ_DEFAULT_PASS
value: rabbitmq-password
image: rabbitmq:4.1-management-alpine
lifecycle:
preStop:
exec:
command:
- sh
- -c
- rabbitmqctl stop_app && sleep 5
livenessProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 5
periodSeconds: 30
timeoutSeconds: 15
name: rabbitmq
ports:
- containerPort: 15672
- containerPort: 5672
name: rabbitmq
protocol: TCP
- containerPort: 15672
name: rabbitmq-management
protocol: TCP
- containerPort: 15692
name: exporter
protocol: TCP
readinessProbe:
exec:
command:
- rabbitmq-diagnostics
- -q
- check_running
failureThreshold: 3
periodSeconds: 10
timeoutSeconds: 10
resources:
limits:
memory: 512Mi
requests:
cpu: 20m
memory: 256Mi
startupProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 30
periodSeconds: 10
volumeMounts:
- mountPath: /var/lib/rabbitmq
name: rabbitmq-data
imagePullSecrets:
- name: dockerregistry
terminationGracePeriodSeconds: 90
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- metadata:
name: rabbitmq-data
Expand Down
44 changes: 43 additions & 1 deletion tests/scenarios/basic-production/expected/migrate-first-deploy-with-demo-data.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,22 +330,64 @@ spec:
- name: RABBITMQ_DEFAULT_PASS
value: rabbitmq-password
image: rabbitmq:4.1-management-alpine
lifecycle:
preStop:
exec:
command:
- sh
- -c
- rabbitmqctl stop_app && sleep 5
livenessProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 5
periodSeconds: 30
timeoutSeconds: 15
name: rabbitmq
ports:
- containerPort: 15672
- containerPort: 5672
name: rabbitmq
protocol: TCP
- containerPort: 15672
name: rabbitmq-management
protocol: TCP
- containerPort: 15692
name: exporter
protocol: TCP
readinessProbe:
exec:
command:
- rabbitmq-diagnostics
- -q
- check_running
failureThreshold: 3
periodSeconds: 10
timeoutSeconds: 10
resources:
limits:
memory: 512Mi
requests:
cpu: 20m
memory: 256Mi
startupProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 30
periodSeconds: 10
volumeMounts:
- mountPath: /var/lib/rabbitmq
name: rabbitmq-data
imagePullSecrets:
- name: dockerregistry
terminationGracePeriodSeconds: 90
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- metadata:
name: rabbitmq-data
Expand Down
44 changes: 43 additions & 1 deletion tests/scenarios/basic-production/expected/migrate-first-deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,22 +330,64 @@ spec:
- name: RABBITMQ_DEFAULT_PASS
value: rabbitmq-password
image: rabbitmq:4.1-management-alpine
lifecycle:
preStop:
exec:
command:
- sh
- -c
- rabbitmqctl stop_app && sleep 5
livenessProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 5
periodSeconds: 30
timeoutSeconds: 15
name: rabbitmq
ports:
- containerPort: 15672
- containerPort: 5672
name: rabbitmq
protocol: TCP
- containerPort: 15672
name: rabbitmq-management
protocol: TCP
- containerPort: 15692
name: exporter
protocol: TCP
readinessProbe:
exec:
command:
- rabbitmq-diagnostics
- -q
- check_running
failureThreshold: 3
periodSeconds: 10
timeoutSeconds: 10
resources:
limits:
memory: 512Mi
requests:
cpu: 20m
memory: 256Mi
startupProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 30
periodSeconds: 10
volumeMounts:
- mountPath: /var/lib/rabbitmq
name: rabbitmq-data
imagePullSecrets:
- name: dockerregistry
terminationGracePeriodSeconds: 90
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- metadata:
name: rabbitmq-data
Expand Down
44 changes: 43 additions & 1 deletion tests/scenarios/development-single-domain/expected/migrate-continuous-deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,22 +211,64 @@ spec:
- name: RABBITMQ_DEFAULT_PASS
value: rabbitmq-password
image: rabbitmq:4.1-management-alpine
lifecycle:
preStop:
exec:
command:
- sh
- -c
- rabbitmqctl stop_app && sleep 5
livenessProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 5
periodSeconds: 30
timeoutSeconds: 15
name: rabbitmq
ports:
- containerPort: 15672
- containerPort: 5672
name: rabbitmq
protocol: TCP
- containerPort: 15672
name: rabbitmq-management
protocol: TCP
- containerPort: 15692
name: exporter
protocol: TCP
readinessProbe:
exec:
command:
- rabbitmq-diagnostics
- -q
- check_running
failureThreshold: 3
periodSeconds: 10
timeoutSeconds: 10
resources:
limits:
memory: 512Mi
requests:
cpu: 20m
memory: 256Mi
startupProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 30
periodSeconds: 10
volumeMounts:
- mountPath: /var/lib/rabbitmq
name: rabbitmq-data
imagePullSecrets:
- name: dockerregistry
terminationGracePeriodSeconds: 90
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- metadata:
name: rabbitmq-data
Expand Down
44 changes: 43 additions & 1 deletion tests/scenarios/development-single-domain/expected/migrate-first-deploy-with-demo-data.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,22 +211,64 @@ spec:
- name: RABBITMQ_DEFAULT_PASS
value: rabbitmq-password
image: rabbitmq:4.1-management-alpine
lifecycle:
preStop:
exec:
command:
- sh
- -c
- rabbitmqctl stop_app && sleep 5
livenessProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 5
periodSeconds: 30
timeoutSeconds: 15
name: rabbitmq
ports:
- containerPort: 15672
- containerPort: 5672
name: rabbitmq
protocol: TCP
- containerPort: 15672
name: rabbitmq-management
protocol: TCP
- containerPort: 15692
name: exporter
protocol: TCP
readinessProbe:
exec:
command:
- rabbitmq-diagnostics
- -q
- check_running
failureThreshold: 3
periodSeconds: 10
timeoutSeconds: 10
resources:
limits:
memory: 512Mi
requests:
cpu: 20m
memory: 256Mi
startupProbe:
exec:
command:
- rabbitmq-diagnostics
- ping
failureThreshold: 30
periodSeconds: 10
volumeMounts:
- mountPath: /var/lib/rabbitmq
name: rabbitmq-data
imagePullSecrets:
- name: dockerregistry
terminationGracePeriodSeconds: 90
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- metadata:
name: rabbitmq-data
Expand Down
Loading