Skip to content

Conversation

@GitHK
Copy link
Contributor

@GitHK GitHK commented Jul 18, 2025

What do these changes do?

Brings the ability to mount S3 state folders for services. Each state folder gets a separate mount command.
NOTE The current setting of rclone are tuned so it's a good balance between handling lots of small files, middle sized files and big files. With current settings the rclone mount is bound to max 1 cpu and and I've never seen it go over 100MB of ram in all my tests.

Extras:

  • 🗑️ removes unused code relative to the docker volumes plugin
  • ⬆️ upgraded rclone to 1.70.3
  • ♻️ Agent only backs up volumes that it likes
  • 🐛 director-v2 did not start volume removal at the proper time causing an volumes to remain on disk.

Related issue/s

How to test

Dev-ops

  • merge this MR before releasing

@GitHK GitHK self-assigned this Jul 18, 2025
@GitHK GitHK added this to the Engage milestone Jul 18, 2025
@sonarqubecloud
Copy link

@codecov
Copy link

codecov bot commented Jul 18, 2025

Codecov Report

❌ Patch coverage is 54.84581% with 205 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.90%. Comparing base (2497b29) to head (daf5a8a).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8128      +/-   ##
==========================================
- Coverage   87.52%   85.90%   -1.62%     
==========================================
  Files        2021     2018       -3     
  Lines       79516    79154     -362     
  Branches     1389     1408      +19     
==========================================
- Hits        69593    67996    -1597     
- Misses       9527    10763    +1236     
+ Partials      396      395       -1     
Flag Coverage Δ
integrationtests 63.88% <48.95%> (-0.05%) ⬇️
unittests 86.77% <86.11%> (+0.42%) ⬆️
Components Coverage Δ
pkg_aws_library 95.26% <ø> (ø)
pkg_celery_library 81.31% <ø> (ø)
pkg_dask_task_models_library 79.37% <ø> (ø)
pkg_models_library 92.89% <ø> (ø)
pkg_notifications_library 85.52% <ø> (ø)
pkg_postgres_database 88.02% <100.00%> (+<0.01%) ⬆️
pkg_service_integration 72.86% <ø> (ø)
pkg_service_library 71.27% <0.00%> (-0.10%) ⬇️
pkg_settings_library 90.76% <100.00%> (+0.52%) ⬆️
pkg_simcore_sdk 69.65% <45.48%> (-14.87%) ⬇️
agent 93.51% <100.00%> (+0.06%) ⬆️
api_server 91.35% <ø> (ø)
autoscaling 95.25% <ø> (ø)
catalog 92.06% <ø> (ø)
clusters_keeper 99.14% <ø> (ø)
dask_sidecar 91.72% <ø> (ø)
datcore_adapter 97.95% <ø> (ø)
director 75.45% <ø> (-0.09%) ⬇️
director_v2 78.80% <92.85%> (-12.47%) ⬇️
dynamic_scheduler 96.54% <ø> (ø)
dynamic_sidecar 88.19% <ø> (-2.56%) ⬇️
efs_guardian 89.83% <ø> (ø)
invitations 90.90% <ø> (ø)
payments 92.70% <ø> (ø)
resource_usage_tracker 92.06% <ø> (+0.47%) ⬆️
storage 86.70% <ø> (+0.11%) ⬆️
webclient ∅ <ø> (∅)
webserver 86.67% <ø> (-0.01%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2497b29...daf5a8a. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mergify
Copy link
Contributor

mergify bot commented Aug 20, 2025

🧪 CI Insights

Here's what we observed from your CI run for daf5a8a.

❌ Job Failures

Pipeline Job Health on master Retries 🔍 CI Insights 📄 Logs
CI integration-tests Healthy 0 View View
unit-tests Healthy 0 View View

@sonarqubecloud
Copy link

@GitHK GitHK changed the title ⬆️ Upgrading rclone to 1.70.3 ⬆️ Adding rclount mount capability to dynamic-sidecar Aug 29, 2025
@GitHK GitHK changed the title ⬆️ Adding rclount mount capability to dynamic-sidecar ✨ Adding rclount mount capability to dynamic-sidecar Aug 29, 2025
@GitHK GitHK modified the milestones: Engage, Voyager Sep 1, 2025
@GitHK GitHK modified the milestones: Voyager, Cheops Sep 3, 2025
@GitHK GitHK modified the milestones: Cheops, Delirium Noel Dec 10, 2025
@sonarqubecloud
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
4 Security Hotspots

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant