Skip to content

DAOS-19173 pool: destroy timeout tiering and serialization#18523

Draft
kccain wants to merge 1 commit into
release/2.8from
kccain/daos_19173_rel2p8
Draft

DAOS-19173 pool: destroy timeout tiering and serialization#18523
kccain wants to merge 1 commit into
release/2.8from
kccain/daos_19173_rel2p8

Conversation

@kccain

@kccain kccain commented Jun 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

For MD on SSD configurations, engine local SCM capacity based corpc timeout configuration:

  • Add pool_destroy_local_scm_size() get engine-local SCM size
  • Add pool_destroy_rpc_timeout() to customize destroy CoRPC timeout for ds_mgmt_tgt_pool_destroy_ranks()
  • non-MD-on-SSD case falls back to default timeout

And serialize pool target destroy handling for when a previous handler invocation (whose CoRPC initiator timed out) is still busy performing expensive subtree destruction / file unlinking:

  • Refactor ds_pooltgts with dual synchronization domains: Create: dpt_create_mutex/cv + dpt_creates_ht (existing create-cancel) Destroy: dpt_destroy_mutex/cv + dpt_destroys_ht (new serialization)
  • Add ds_pooltgts_destroy_rec for destroy-in-flight tracking hash table
  • Add to ds_mgmt_hdlr_tgt_destroy() the destroy serialization

faults-enabled: false
Features: pool

Steps for the author:

  • Commit message follows the guidelines.
  • Appropriate Features or Test-tag pragmas were used.
  • Appropriate Functional Test Stages were run.
  • At least two positive code reviews including at least one code owner from each category referenced in the PR.
  • Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

  • Gatekeeper requested (daos-gatekeeper added as a reviewer).

@github-actions

github-actions Bot commented Jun 23, 2026 •
edited
Loading

Copy link
Copy Markdown

Ticket title is 'dmg pool destroy very slow'
Status is 'In Progress'
Labels: 'request_for_2.8'
https://daosio.atlassian.net/browse/DAOS-19173

@kccain kccain force-pushed the kccain/daos_19173_rel2p8 branch from 2a34b97 to ab8f393 Compare June 23, 2026 13:48
@daosbuild3

daosbuild3 commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Test stage NLT completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18523/2/testReport/

@Michael-Hennecke Michael-Hennecke self-requested a review June 23, 2026 14:42
@daosbuild3

daosbuild3 commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Test stage Functional on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18523/2/execution/node/1116/log

@kccain kccain marked this pull request as ready for review July 6, 2026 13:54
@kccain kccain requested review from liw and wangshilong July 6, 2026 13:54
@kccain

kccain commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

reviewing the patch in parallel to @Michael-Hennecke patch deployment and test. Potentially a candidate for 2.8-rc2 (subject to approval to merge)

@kccain kccain removed request for liw and wangshilong July 6, 2026 14:05
@kccain kccain marked this pull request as draft July 6, 2026 14:06
@kccain kccain mentioned this pull request Jul 6, 2026
Merged
6 tasks
@kccain kccain marked this pull request as ready for review July 10, 2026 12:23
@kccain kccain marked this pull request as draft July 10, 2026 12:29
@kccain
DAOS-19173 pool: destroy timeout tiering and serialization
7da9c16 
For MD on SSD configurations, engine local SCM capacity
based corpc timeout configuration:
- Add pool_destroy_local_scm_size() get engine-local SCM size
- Add pool_destroy_rpc_timeout() to customize destroy CoRPC timeout
  for ds_mgmt_tgt_pool_destroy_ranks()
- non-MD-on-SSD case falls back to default timeout

And serialize pool target destroy handling for when a previous
handler invocation (whose CoRPC initiator timed out) is still
busy performing expensive subtree destruction / file unlinking:
- Refactor ds_pooltgts with dual synchronization domains:
  Create: dpt_create_mutex/cv + dpt_creates_ht (existing create-cancel)
  Destroy: dpt_destroy_mutex/cv + dpt_destroys_ht (new serialization)
- Add ds_pooltgts_destroy_rec for destroy-in-flight tracking hash table
- Add to ds_mgmt_hdlr_tgt_destroy() the destroy serialization

Features: pool

Signed-off-by: Kenneth Cain <kenneth.cain@hpe.com>
@kccain kccain force-pushed the kccain/daos_19173_rel2p8 branch from ab8f393 to 7da9c16 Compare July 10, 2026 12:31
@daosbuild3

daosbuild3 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Test stage Functional Hardware Large MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18523/3/execution/node/1649/log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Michael-Hennecke Michael-Hennecke Awaiting requested review from Michael-Hennecke

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kccain @daosbuild3