Skip to content

Add persistent global configuration via python-dotenv#352

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/add-configuration-file
Draft

Add persistent global configuration via python-dotenv#352
Copilot wants to merge 2 commits intomainfrom
copilot/add-configuration-file

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 19, 2026

Adds support for persistent, layered configuration files so users can set default RunConfig options globally or per-dataset without repeating CLI flags on every invocation.

Design

Priority order (highest → lowest):

  1. Explicit CLI flags / Python API kwargs
  2. NWB2BIDS_* environment variables
  3. Local config: <bids_directory>/.nwb2bids/.env
  4. Global config: ~/.nwb2bids/.env
  5. RunConfig model defaults

Changes

  • pyproject.toml — adds python-dotenv as a core dependency
  • src/nwb2bids/_core/_global_config.py (new) — loading/merging logic: reads both .env files, applies env var overrides, parses typed RunConfig kwargs
  • src/nwb2bids/_converters/_run_config.py — adds RunConfig.from_dotenv_files(**explicit_kwargs) class method
  • src/nwb2bids/_command_line_interface/_main.py — CLI uses RunConfig.from_dotenv_files(...) instead of RunConfig(...) directly; sanitization config is only included in explicit kwargs when --sanitization flags are passed (prevents silent override of dotenv sanitization defaults)

Supported config keys

Key RunConfig field
NWB2BIDS_FILE_MODE file_mode
NWB2BIDS_CACHE_DIRECTORY cache_directory
NWB2BIDS_ARCHIVE_TARGET archive_target
NWB2BIDS_SPACE space
NWB2BIDS_SANITIZATION_SUB_LABELS sanitization_config.sub_labels
NWB2BIDS_SANITIZATION_SES_LABELS sanitization_config.ses_labels

Example

~/.nwb2bids/.env:

NWB2BIDS_FILE_MODE=symlink
NWB2BIDS_SANITIZATION_SUB_LABELS=true
# Picks up file_mode=symlink and sub_labels=True from ~/.nwb2bids/.env
run_config = RunConfig.from_dotenv_files(bids_directory="/data/my_bids")

# Explicit kwarg wins — file_mode=copy regardless of .env
run_config = RunConfig.from_dotenv_files(bids_directory="/data/my_bids", file_mode="copy")

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.dandiarchive.org
    • Triggering command: /usr/bin/python python -m pytest tests/unit/ -v --tb=short (dns block)
  • rig.mit.edu
    • Triggering command: /usr/bin/python python -m pytest tests/unit/ -v --tb=short (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Add configuration file</issue_title>
<issue_description>For persistent global options

This will include

  1. default number of jobs to use in parallelization
  2. default sanitization level
  3. add more here</issue_description>

Comments on the Issue (you are @copilot in this section)

@CodyCBakerPhD The start of this in-memory is in #164

Follow-up will allow setting perment global config on disk</comment_new>
<comment_new>@CodyCBakerPhD
Plan:

  • use python-dotenv package for allowing persistent (dataset-specific or system-specific; or local vs. global, in order of priority) files for the RunConfig model
  • this also allows environment variables of the specific convention of that package to be used to override items from the persistent files
  • the best source of ground truth is still the manually passed flags to the CLI or API </comment_new>

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: CodyCBakerPhD <51133164+CodyCBakerPhD@users.noreply.github.com>
Copilot AI changed the title [WIP] Add configuration file for persistent global options Add persistent global configuration via python-dotenv Mar 19, 2026
Copilot AI requested a review from CodyCBakerPhD March 19, 2026 04:00
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 19, 2026

Codecov Report

❌ Patch coverage is 91.52542% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.05%. Comparing base (782176c) to head (631af18).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
src/nwb2bids/_command_line_interface/_main.py 0.00% 5 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #352      +/-   ##
==========================================
+ Coverage   78.96%   86.05%   +7.09%     
==========================================
  Files          40       42       +2     
  Lines        1578     1678     +100     
==========================================
+ Hits         1246     1444     +198     
+ Misses        332      234      -98     
Flag Coverage Δ
unittests 86.05% <91.52%> (+7.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/nwb2bids/_converters/_run_config.py 96.07% <100.00%> (+0.95%) ⬆️
src/nwb2bids/_core/_global_config.py 100.00% <100.00%> (ø)
src/nwb2bids/_command_line_interface/_main.py 0.00% <0.00%> (ø)

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add configuration file

2 participants