Add config keys for controlling public/private template secondary storage replica counts by Damans227 · Pull Request #12877 · apache/cloudstack

Damans227 · 2026-03-23T16:12:29Z

Description

Adds two new operator-level configuration keys to control the number of secondary storage copies made for public and private templates, decoupling replica count from template visibility.

secstorage.public.template.copy.max (default: 0 = all stores, preserving existing behavior)
secstorage.private.template.copy.max (default: 1, preserving existing behavior)

Doc PR: apache/cloudstack-documentation#643

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

How Has This Been Tested?

Unit tests in HypervisorTemplateAdapterTest covering the new replica-limit logic in isZoneAndImageStoreAvailable.
Manually verified on a zone with multiple secondary storage pools.

…rage replica counts

codecov · 2026-03-23T16:22:13Z

Codecov Report

❌ Patch coverage is 20.37915% with 168 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.10%. Comparing base (b744824) to head (f1e5884).
⚠️ Report is 222 commits behind head on main.

Files with missing lines	Patch %	Lines
.../cloudstack/storage/image/TemplateServiceImpl.java	20.49%	121 Missing and 7 partials ⚠️
...n/java/com/cloud/template/TemplateManagerImpl.java	0.00%	25 Missing ⚠️
...com/cloud/storage/ImageStoreUploadMonitorImpl.java	0.00%	6 Missing ⚠️
.../com/cloud/template/HypervisorTemplateAdapter.java	0.00%	5 Missing ⚠️
...n/java/com/cloud/template/TemplateAdapterBase.java	66.66%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12877      +/-   ##
============================================
+ Coverage     18.02%   18.10%   +0.08%     
- Complexity    16450    16761     +311     
============================================
  Files          5968     6037      +69     
  Lines        537086   542985    +5899     
  Branches      65961    66502     +541     
============================================
+ Hits          96819    98326    +1507     
- Misses       429347   433608    +4261     
- Partials      10920    11051     +131

Flag	Coverage Δ
uitests	`3.51% <ø> (-0.02%)`	⬇️
unittests	`19.27% <20.37%> (+0.07%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…vate templates

Damans227 · 2026-03-23T16:48:59Z

@blueorangutan package

blueorangutan · 2026-03-23T16:50:05Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-03-23T17:33:35Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17224

Copilot

Pull request overview

This PR introduces new template-related configuration keys intended to control how many secondary storage replicas are created for public vs private templates, decoupling replication behavior from template visibility.

Changes:

Adds two new Zone-scoped ConfigKey<Integer> settings: public.template.secstorage.copy and private.template.secstorage.copy.
Reworks secondary storage allocation logic to enforce a per-zone replica limit using a zone→count map.
Updates unit tests to match the new method signatures/replica-limit behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`engine/components-api/src/main/java/com/cloud/template/TemplateManager.java`	Adds new Zone-scoped config keys for public/private template replica limits.
`server/src/main/java/com/cloud/template/TemplateManagerImpl.java`	Registers the new config keys via `getConfigKeys()`.
`server/src/main/java/com/cloud/template/TemplateAdapterBase.java`	Replaces zone-set visibility logic with zone copy counting + replica limit enforcement.
`server/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java`	Threads replica-limit and zone copy count through template creation/allocation paths.
`server/src/test/java/com/cloud/template/HypervisorTemplateAdapterTest.java`	Updates tests for the new signatures and adds coverage for replica-limit behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-25T16:17:19Z

    /**
     * If the template/ISO is marked as private, then it is allocated to a random secondary storage; otherwise, allocates to every storage pool in every zone given by the
     * {@link TemplateProfile#getZoneIdList()}.
     */
    protected void postUploadAllocation(List<DataStore> imageStores, VMTemplateVO template, List<TemplateOrVolumePostUploadCommand> payloads) {


This method JavaDoc no longer matches the behavior: allocation is now governed by replica-limit config keys (public/private can both be limited), not strictly by template visibility (private=random vs public=all). Please update the comment to reflect the new replica-limit semantics (and that the limits are zone-scoped).

Copilot · 2026-03-25T16:17:20Z

+    ConfigKey<Integer> PublicTemplateSecStorageCopy = new ConfigKey<Integer>("Advanced", Integer.class,
+            PublicTemplateSecStorageCopyCK, "0",
+            "Maximum number of secondary storage pools to which a public template is copied. " +
+            "0 means copy to all secondary storage pools (default behavior).",
+            true, ConfigKey.Scope.Zone);
+
+    ConfigKey<Integer> PrivateTemplateSecStorageCopy = new ConfigKey<Integer>("Advanced", Integer.class,
+            PrivateTemplateSecStorageCopyCK, "1",
+            "Maximum number of secondary storage pools to which a private template is copied. " +
+            "Default is 1 to preserve existing behavior.",
+            true, ConfigKey.Scope.Zone);


The config key descriptions don’t mention that the limits are evaluated per-zone (and the keys themselves are zone-scoped). Consider clarifying the wording (e.g., “per zone”) so operators understand how replica limits are applied across multiple zones/image stores.

…c values

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Damans227 · 2026-04-15T11:46:44Z

@blueorangutan package

blueorangutan · 2026-04-15T11:48:04Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-15T12:47:54Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17501

sureshanaparti · 2026-04-27T12:35:10Z

@Damans227 update the description with update config names.

sureshanaparti

clgtm

Damans227 · 2026-04-27T13:40:36Z

@Damans227 update the description with update config names.

Done. Thanks!

…limit logic and update related tests

Damans227 · 2026-05-22T20:49:06Z

@blueorangutan package

blueorangutan · 2026-05-22T20:50:04Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-05-22T21:35:02Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17996

Damans227 · 2026-05-25T18:42:54Z

@blueorangutan package

blueorangutan · 2026-05-25T18:44:04Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-05-25T19:26:50Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18021

… template creation

Damans227 · 2026-05-27T10:04:52Z

@blueorangutan package

blueorangutan · 2026-05-27T10:06:05Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-05-27T10:49:25Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18056

…limits and schedule additional copies for cross-zone templates

Damans227 · 2026-05-27T15:22:50Z

@blueorangutan package

blueorangutan · 2026-05-27T15:24:06Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-05-27T16:14:12Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18063

…apacity during template management

Damans227 · 2026-05-28T12:42:56Z

@blueorangutan package

blueorangutan · 2026-05-28T12:44:05Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-05-28T13:37:03Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18078

kiranchavala · 2026-06-08T13:01:16Z

@blueorangutan test

blueorangutan · 2026-06-08T13:02:03Z

@kiranchavala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-06-08T13:57:38Z

[SF] Trillian Build Failed (tid-16260)

kiranchavala · 2026-06-11T07:00:49Z

@blueorangutan test

blueorangutan · 2026-06-11T07:02:03Z

@kiranchavala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-06-11T21:44:38Z

[SF] Trillian test result (tid-16286)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49957 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12877-t16286-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

kiranchavala

LGTM

Tested the feature manually.

The Public and private Templates honor the global setting value of secstorage.public.template.copy.max and secstorage.private.template.copy.max and copies the template to the respective secondary storage.

validate secstorage.public.template.copy.max at global and zone level	Pass
validate secstorage.private.template.copy.max at global and zone level	Pass
Validate publlic and private template creation operation from volume	Pass
Validate publlic and private template creation operation from a volume snapshot	Pass
Validate Upload Template operation from Local	Pass
Validate Copy Template operation from one zone to another	Pass
Validate Update Template operation from public to private and vice versa	Pass

weizhouapache · 2026-06-12T07:40:10Z

LGTM

Tested the feature manually.

The Public and private Templates honor the global setting value of secstorage.public.template.copy.max and secstorage.private.template.copy.max and copies the template to the respective secondary storage.

validate secstorage.public.template.copy.max at global and zone level Pass
validate secstorage.private.template.copy.max at global and zone level Pass
Validate publlic and private template creation operation from volume Pass
Validate publlic and private template creation operation from a volume snapshot Pass
Validate Upload Template operation from Local Pass
Validate Copy Template operation from one zone to another Pass
Validate Update Template operation from public to private and vice versa Pass

great, thanks @kiranchavala

@harikrishna-patnala
will you review the feature before merge ?

Add config keys for controlling public/private template secondary sto…

40b9e4d

…rage replica counts

Daman Arora added 2 commits March 23, 2026 12:36

Allow configurable secondary storage replica count for public and pri…

f997444

…vate templates

Add unit tests

0708236

DaanHoogland requested a review from Copilot March 25, 2026 16:12

Copilot started reviewing on behalf of DaanHoogland March 25, 2026 16:13 View session

Copilot AI reviewed Mar 25, 2026

View reviewed changes

winterhazel added this to the 4.23.0 milestone Mar 28, 2026

Damans227 mentioned this pull request Apr 15, 2026

Add documentation for Template replication on secondary storage apache/cloudstack-documentation#643

Open

Refactor secondary storage replica limit handling to use zone-specifi…

e5bfeab

…c values

Damans227 requested a review from sureshanaparti April 15, 2026 09:06

sureshanaparti reviewed Apr 15, 2026

View reviewed changes

Comment thread engine/components-api/src/main/java/com/cloud/template/TemplateManager.java Outdated

sureshanaparti reviewed Apr 15, 2026

View reviewed changes

Comment thread engine/components-api/src/main/java/com/cloud/template/TemplateManager.java Outdated

sureshanaparti reviewed Apr 15, 2026

View reviewed changes

Comment thread server/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java Outdated

sureshanaparti reviewed Apr 15, 2026

View reviewed changes

Comment thread server/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java Outdated

sureshanaparti requested a review from Copilot April 15, 2026 09:24

Copilot started reviewing on behalf of sureshanaparti April 15, 2026 09:25 View session

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Comment thread server/src/main/java/com/cloud/template/TemplateAdapterBase.java Outdated

Refactor as per pr comments

6dbeb52

Damans227 requested a review from sureshanaparti April 16, 2026 06:33

sureshanaparti approved these changes Apr 27, 2026

View reviewed changes

sureshanaparti added the status:needs-testing label Apr 27, 2026

Refactor template storage handling to improve secondary storage copy …

f3fa8f8

…limit logic and update related tests

Add template copy limit check to enforce secondary storage constraints

cb36bef

Implement template replication up to capacity after upload and during…

df1ad66

… template creation

Enhance template replication logic to respect secondary storage copy …

97569b9

…limits and schedule additional copies for cross-zone templates

Enforce secondary storage copy limits and replicate templates up to c…

f1e5884

…apacity during template management

kiranchavala approved these changes Jun 12, 2026

View reviewed changes

weizhouapache added status:ready-for-merge and removed status:needs-testing labels Jun 12, 2026

Conversation

Damans227 commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

How Has This Been Tested?

Uh oh!

codecov Bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Damans227 commented Mar 23, 2026

Uh oh!

blueorangutan commented Mar 23, 2026

Uh oh!

blueorangutan commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Damans227 commented Apr 15, 2026

Uh oh!

blueorangutan commented Apr 15, 2026

Uh oh!

blueorangutan commented Apr 15, 2026

Uh oh!

sureshanaparti commented Apr 27, 2026

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

Damans227 commented Apr 27, 2026

Uh oh!

Damans227 commented May 22, 2026

Uh oh!

blueorangutan commented May 22, 2026

Uh oh!

blueorangutan commented May 22, 2026

Uh oh!

Damans227 commented May 25, 2026

Uh oh!

blueorangutan commented May 25, 2026

Uh oh!

blueorangutan commented May 25, 2026

Uh oh!

Damans227 commented May 27, 2026

Uh oh!

blueorangutan commented May 27, 2026

Uh oh!

blueorangutan commented May 27, 2026

Uh oh!

Damans227 commented May 27, 2026

Uh oh!

blueorangutan commented May 27, 2026

Uh oh!

blueorangutan commented May 27, 2026

Uh oh!

Damans227 commented May 28, 2026

Damans227 commented Mar 23, 2026 •

edited

Loading

codecov Bot commented Mar 23, 2026 •

edited

Loading