-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
julesFor Jules AI to work onFor Jules AI to work on
Description
Summary
Add ability to read URLs from a file and collect them all in batch.
Use Case
Skills have registries with many URLs. Need to collect all of them efficiently.
Proposed Commands
# From file
borg collect batch urls.txt
# From JSON registry
borg collect batch registry.json --jq '.papers[].url'
# Stdin
cat urls.txt | borg collect batch -File Formats
# urls.txt - one URL per line
https://example.com/paper1.pdf
https://example.com/paper2.pdf
# Or with metadata
https://example.com/paper1.pdf|paper1|cryptography
Options
--parallel N- concurrent downloads--delay 1s- delay between requests--output-dir- base output directory--continue- skip already collected
Acceptance Criteria
- Read URLs from text file
- Read URLs from JSON with jq filter
- Support stdin piping
- Parallel downloads with configurable concurrency
- Rate limiting
- Progress reporting
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
julesFor Jules AI to work onFor Jules AI to work on