perf: autobump snakemake wrappers - #183
Conversation
35b25ee to
b5ad7d4
Compare
📝 WalkthroughWalkthroughThe workflow updates Snakemake wrapper references across annotation, comparison, download, and utility rules. The changes cover VEP, tabix, bcftools, samtools, BWA-MEM, and Picard wrappers. ChangesWorkflow wrapper refresh
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@workflow/rules/annotation.smk`:
- Line 27: Update the release value used by both get_vep_cache rules to "116" so
it matches the ensembl-vep=116.0 pins in the v9.9.0/bio/vep/cache and
v9.14.0/bio/vep/annotate environments. Keep the existing cache-generation
behavior otherwise unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 736fa1ac-740d-4a3d-8678-83cd124faa54
📒 Files selected for processing (4)
workflow/rules/annotation.smkworkflow/rules/compare-vcfs.smkworkflow/rules/download.smkworkflow/rules/utils.smk
| release="115", | ||
| wrapper: | ||
| "v7.6.0/bio/vep/cache" | ||
| "v9.9.0/bio/vep/cache" |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
release="$(sed -n 's/.*release="\([^"]*\)".*/\1/p' workflow/rules/annotation.smk | head -n1)"
cache_vep="$(curl -fsSL https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.9.0/bio/vep/cache/environment.yaml | sed -n 's/.*ensembl-vep =\([0-9.]*\).*/\1/p')"
annotate_vep="$(curl -fsSL https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.14.0/bio/vep/annotate/environment.yaml | sed -n 's/.*ensembl-vep =\([0-9.]*\).*/\1/p')"
cache_major="${cache_vep%%.*}"
annotate_major="${annotate_vep%%.*}"
if [[ "$release" != "$cache_major" || "$release" != "$annotate_major" ]]; then
printf 'VEP mismatch: cache release=%s, cache VEP=%s, annotate VEP=%s\n' \
"$release" "$cache_vep" "$annotate_vep"
exit 1
fiRepository: snakemake-workflows/dna-seq-benchmark
Length of output: 244
🌐 Web query:
Ensembl VEP cache version must match VEP release cache formats incompatible newer versions
💡 Result:
Ensembl strongly recommends using a VEP cache version that strictly matches your installed Ensembl VEP software version [1][2]. For instance, VEP version 116 should be used with the version 116 cache [1]. This requirement exists because the cache data content and internal structure are generated specifically for each Ensembl release to align with corresponding data updates and API changes [1][2]. Consequently, cache data formats can change between releases, potentially rendering older caches incompatible with newer versions of the VEP tool [1][2]. Using mismatched versions can lead to discrepancies, unexpected behavior, or errors during annotation [3][4]. If you must use a different cache version (e.g., when working with Ensembl Genomes, where version numbers differ from Ensembl Vertebrates/VEP versions), you can use the --cache_version option to explicitly specify the version [1][5][6]. In standard workflows, however, it is best practice to keep the VEP and cache versions synchronized to ensure stability and accuracy [3][4].
Citations:
- 1: https://www.ensembl.org/info/docs/tools/vep/script/vep_cache.html
- 2: https://oct2024.archive.ensembl.org/info/docs/tools/vep/script/vep_cache.html
- 3: Cache version flag defaulting to latest Ensembl/ensembl-vep#943
- 4: Vep does not find cache file Ensembl/ensembl-vep#920
- 5: https://www.ensembl.org/info/docs/tools/vep/script/vep_options.html
- 6: https://mart.ensembl.org/info/docs/tools/vep/script/vep_options.html
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
printf '--- annotation.smk VEP-related lines ---\n'
rg -n 'get_vep_cache|v9\.9\.0/v9\.14\.0/vep|release=|release=' workflow/rules/annotation.smk
printf '\n--- referenced wrapper environment files ---\n'
for url in \
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.9.0/bio/vep/cache/environment.yaml" \
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.14.0/bio/vep/annotate/environment.yaml"
do
printf '\nURL: %s\n' "$url"
sed -n '/^conda:/,/^\s*dependencies:/p; /^conda:/,/^[^[:space:]]/p' "$url" || true
sed -n 's/^.*:\s*`{0,1}ensembl-vep\s*[:=]\s*{0,1}[^`"]*$/\0/p' "$url" || true
done
printf '\n--- local dependency files mentioning VEP ---\n'
rg -n "ensembl-vep|VEP|vep" -S --glob '!*.ipynb' --glob '!.snakemake/**' .Repository: snakemake-workflows/dna-seq-benchmark
Length of output: 266
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- annotation.smk VEP-related lines ---'
rg -n 'get_vep_cache|v9\.9\.0/v9\.14\.0/vep|release=|release=' workflow/rules/annotation.smk || true
printf '%s\n' ''
printf '%s\n' '--- referenced wrapper environment files ---'
for url in \
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.9.0/bio/vep/cache/environment.yaml" \
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.14.0/bio/vep/annotate/environment.yaml"
do
printf '%s\n' ""
printf '%s\n' "URL: ${url}"
sed -n '1,220p' "$url" | sed -n '1,80p' || true
rg -n '^[\s-]*`?ensembl-vep' "$url" || true
done
printf '%s\n' ''
printf '%s\n' '--- local dependency files mentioning VEP ---'
rg -n "ensembl-vep|VEP|vep" -S --glob '!*.ipynb' --glob '!.snakemake/**' . || trueRepository: snakemake-workflows/dna-seq-benchmark
Length of output: 6818
🌐 Web query:
snakemake wrappers v9.9.0 bio vep cache environment.yaml ensembl-vep
💡 Result:
In Snakemake wrappers version 9.9.0, the bio/vep/cache wrapper is designed to manage the downloading and installation of Ensembl VEP caches [1]. Key details for usage in Snakemake 9.9.0 include: Environment and Dependencies The environment for the bio/vep/cache wrapper is managed via a corresponding environment.yaml file [2]. This environment typically includes ensembl-vep, which is kept up to date through automated maintenance in the snakemake-wrappers repository [2][3]. For example, recent updates have seen ensembl-vep versions around 115.2, paired with specific compatible versions of tools like bcftools to resolve environment installation issues (e.g., LibMambaUnsatisfiableError) [4][5]. Cache Wrapper Configuration The bio/vep/cache wrapper supports parameters to define the specific VEP cache to download [1]: - species: The target species (e.g., saccharomyces_cerevisiae). - build: The genome assembly version (e.g., R64-1-1). - release: The Ensembl release version (e.g., 114). - indexed: A boolean parameter (default is True) that determines whether to download a pre-indexed cache [1][6]. Releases 114 and newer are supported only for indexed VEP caches [1]. General Usage Pattern When using the wrapper in a workflow, the cache directory produced by the bio/vep/cache wrapper can be passed directly as input to the bio/vep/annotate wrapper using the cache parameter [7][1]. Using the cache: "omit-software" directive within your rule is recommended to save space and enable efficient between-workflow caching [1]. Always refer to the official Snakemake wrappers documentation for the most accurate and up-to-date parameter definitions for version 9.9.0 [7][1].
Citations:
- 1: https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/bio/vep/cache.html
- 2: snakemake/snakemake-wrappers@v9.8.0...v9.9.0
- 3: snakemake/snakemake-wrappers@fd74454
- 4: fix: downgrade bcftools in vep environment snakemake/snakemake-wrappers#4713
- 5: snakemake/snakemake-wrappers@f6f3104
- 6: https://www.github.com/snakemake/snakemake-wrappers/issues/3102
- 7: https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/bio/vep/annotate.html
🌐 Web query:
snakemake wrappers v9.14.0 bio vep annotate environment.yaml ensembl-vep
💡 Result:
The Ensembl VEP wrapper in the Snakemake wrappers repository (bio/vep/annotate) relies on an automatically managed Conda environment [1][2]. As of recent versions, the environment definition (environment.yaml) typically includes ensembl-vep, bcftools, and associated Perl dependencies [1][3]. Key points regarding the bio/vep/annotate environment and usage: 1. Environment Management: The environment is subject to regular automatic updates (autobumps) within the snakemake-wrappers repository [4][5][6]. Users should rely on the version of the environment provided with the specific wrapper version they are using (e.g., via the wrapper URL referencing a specific release or master branch) [2][7]. 2. Troubleshooting and Compatibility: If you encounter issues (e.g., LibMambaUnsatisfiableError), they are often related to version conflicts between ensembl-vep and supporting tools like bcftools [1]. The repository maintainers have historically addressed these by adjusting dependency versions (e.g., pinning bcftools or specific Perl packages) [1][3]. 3. Usage Requirements: * The wrapper supports both VEP cache-based annotation and custom GFF/Fasta annotation [8]. * When using custom GFF files, they must be sorted, bgzipped, and tabix-indexed [8][9]. The official documentation provides an example rule for preparing these files using tabix and bgzip [8]. * Plugins can be specified in the wrapper params, and any required auxiliary files for those plugins must be defined within the wrapper input [8]. For the most accurate environment.yaml file, refer to the specific version of the wrapper you are utilizing on the official Snakemake wrappers documentation or GitHub repository [8][4]. If you are working with a specific tag (e.g., v9.14.0), navigate to the corresponding directory in the source tree to inspect the precise dependencies defined at that time [2].
Citations:
- 1: fix: downgrade bcftools in vep environment snakemake/snakemake-wrappers#4713
- 2: snakemake/snakemake-wrappers@v9.8.0...v9.9.0
- 3: snakemake/snakemake-wrappers@f6f3104
- 4: https://github.com/snakemake/snakemake-wrappers/releases
- 5: https://snakemake-wrappers.readthedocs.io/en/stable/changelog.html
- 6: perf: autobump bio/vep/annotate snakemake/snakemake-wrappers#4155
- 7: https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/bio/vep/cache.html
- 8: https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/bio/vep/annotate.html
- 9: fix: more verbose docs for VEP usage snakemake/snakemake-wrappers#4608
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- fetch wrapper environment headers ---'
for url in \
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.9.0/bio/vep/cache/wrapper.py" \
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.9.0/bio/vep/cache/environment.yaml" \
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.14.0/bio/vep/annotate/wrapper.py" \
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.14.0/bio/vep/annotate/environment.yaml"
do
printf '%s\n' ""
printf '%s\n' "URL: ${url}"
if curl -fsSIL "$url" >/dev/null; then
curl -fsSL "$url" | sed -n '1,40p'
else
printf '%s\n' "FETCH FAILED"
fi
done
python3 - <<'PY'
import urllib.request
urls = [
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.9.0/bio/vep/cache/environment.yaml",
"https://raw.githubusercontent.com/snakemake/snakemake-wrappers/v9.14.0/bio/vep/annotate/environment.yaml",
]
for url in urls:
text = urllib.request.urlopen(url, timeout=20).read().decode()
lines = [line.strip() for line in text.splitlines() if "ensembl-vep" in line]
print(url, "=>", "; ".join(lines) or "NO ensembl-vep line")
PYRepository: snakemake-workflows/dna-seq-benchmark
Length of output: 5931
Align the downloaded VEP cache release with the wrapper VEP version.
get_vep_cache downloads release 115, but the v9.9.0/bio/vep/cache environment pins ensembl-vep=116.0 and the v9.14.0/bio/vep/annotate environments also pin ensembl-vep=116.0. Ensembl recommends matching the cache release to the installed VEP release because cache formats can differ between versions. Update release to "116" in both cache-producing rules, or keep release 115 by pinning both wrappers to VEP 115.2-compatible versions.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@workflow/rules/annotation.smk` at line 27, Update the release value used by
both get_vep_cache rules to "116" so it matches the ensembl-vep=116.0 pins in
the v9.9.0/bio/vep/cache and v9.14.0/bio/vep/annotate environments. Keep the
existing cache-generation behavior otherwise unchanged.
2 participants
Automated changes by create-pull-request GitHub action
Summary by CodeRabbit