Skip to content

u/yusra/DM-53485: Demo only #346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
yalsayyad wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

u/yusra/DM-53485: Demo only #346

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
52f50cf
Add compression yaml
yalsayyad Dec 8, 2025
7d518f5
Enable composed compression and caching butler configs
yalsayyad Dec 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion bps/caching/LSSTCam/usdf/DRP-caching.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,15 @@

subDirTemplate: "{label}/{detector}"

includeConfigs:
# DAF_BUTLER_CONFIG_PATH is set here:
- ${DRP_PIPE_DIR}/bps/daf_butler_config_path/compose.yaml

environment:
# Default caching config
# Each pipetask creates its own local cache folder,
# $TMPDIR/butler-cache-dir-*
DAF_BUTLER_CONFIG_PATH: ${DRP_PIPE_DIR}/bps/caching/LSSTCam/usdf/config:${DAF_BUTLER_CONFIG_PATH}
CACHING_CONFIG_PATH: ${DRP_PIPE_DIR}/bps/caching/LSSTCam/usdf/config
DAF_BUTLER_CACHE_EXPIRATION_MODE: datasets=500

cluster:
Expand Down
15 changes: 15 additions & 0 deletions bps/compression/LSSTCam/DRP/DRP-compression.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# There are recommended configs for compression for LSSTCam DRP
#
# Use it by adding
#
# includeConfigs:
# - ${DRP_PIPE_DIR}/bps/compression/LSSTCam/DRP/DRP-compression.yaml
#
# (with no outer indentation) to your BPS config file.

includeConfigs:
# DAF_BUTLER_CONFIG_PATH is set here:
- ${DRP_PIPE_DIR}/bps/daf_butler_config_path/compose.yaml

environment:
COMPRESSION_CONFIG_PATH: ${DRP_PIPE_DIR}/bps/compression/LSSTCam/DRP/config
17 changes: 17 additions & 0 deletions bps/compression/LSSTCam/DRP/config/datastores/fileDatastore.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
formatters:
preliminary_visit_image:
formatter: lsst.obs.base.formatters.fitsExposure.FitsExposureFormatter
parameters:
recipe: lossy16
visit_image:
formatter: lsst.obs.base.formatters.fitsExposure.FitsExposureFormatter
parameters:
recipe: lossy16
deep_coadd:
formatter: lsst.obs.base.formatters.fitsExposure.FitsExposureFormatter
parameters:
recipe: lossy16
template_coadd:
formatter: lsst.obs.base.formatters.fitsExposure.FitsExposureFormatter
parameters:
recipe: lossy16
2 changes: 2 additions & 0 deletions bps/daf_butler_config_path/compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
environment:
DAF_BUTLER_CONFIG_PATH: "{COMPRESSION_CONFIG_PATH}:{CACHING_CONFIG_PATH}:${DAF_BUTLER_CONFIG_PATH}"
Loading