Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Humans: Please describe what this PR does and why it's needed.
## Checklist (all PRs)

- [ ] My code follows the **Agent Code of Conduct**.
- [ ] I have run `python -m flake8 .`, `pytest skills/`, and `pytest tests/` locally (or the subset relevant to this change).
- [ ] I have run `python -m flake8 .`, `pytest skills/` (or `skillware test`), and `pytest tests/` locally (or the subset relevant to this change).
- [ ] `CHANGELOG.md` updated under `[Unreleased]` if this PR changes user-visible behavior.
- [ ] `examples/README.md` is updated if this PR adds, renames, or removes a runnable script under `examples/`.

Expand Down
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,13 @@ Contributors add user-facing entries under `[Unreleased]` in the same PR. Mainta

- **Tests**: `tests/test_skill_issuer.py` now requires `test_skill.py` for every registry skill under `skills/` (#160).
- **Documentation**: Clarified that bundle tests must mock network calls and model downloads in CI (#170).
- **Documentation**: Added a **Status** section to [TESTING.md](docs/TESTING.md) summarizing the current testing model and planned CLI work (#179).
- **Documentation**: Added a **Status** section to [TESTING.md](docs/TESTING.md) summarizing the current testing model (#179).
- **Documentation**: Post-release alignment — category tables, Python 3.10+ badge, dev install (`[dev,all]` vs `[dev]`), README configuration via `.env.example`, DeFi env vars in `.env.example`, framework env vars in [api_keys.md](docs/usage/api_keys.md) (#154).

### Added

- **CLI**: `skillware test` runs bundle tests via pytest — all roots, by skill ID, or by `--category`; supports `-v` and `--no-header`. Documented in [cli.md](docs/usage/cli.md), [TESTING.md](docs/TESTING.md), and contributor guides (#83).

## [0.3.7] - 2026-06-22

### Added
Expand Down
10 changes: 9 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,12 +127,16 @@ Follow the [Agent Code of Conduct](CODE_OF_CONDUCT.md): deterministic skill outp
python -m pytest tests/
```

Bundle tests can also be run with `skillware test` (see [CLI reference](docs/usage/cli.md#skillware-test)); requires `[dev]` or `[dev,all]`.

For a single skill:

```bash
python -m pytest skills/<category>/<skill_name>/test_skill.py
```

Or: `skillware test <category>/<skill_name>`.

- Install packages from that skill's `manifest.yaml` `requirements` when they are not covered by `[all]`. After adding a skill with new third-party deps, update the matching optional extra and `[all]` in `pyproject.toml` (see [Packaging](#packaging-pypi-and-pip-install)).
- Wait for GitHub Actions CI to pass before requesting review.

Expand Down Expand Up @@ -160,13 +164,17 @@ Agents must follow [Agent Contribution Workflow](docs/contributing/ai_native_wor
pytest tests/
```

Or `skillware test` for bundle tests (see [CLI reference](docs/usage/cli.md#skillware-test)).

For skill work, also run:

```bash
pytest skills/<category>/<skill_name>/test_skill.py
pytest tests/test_skill_issuer.py
```

Or `skillware test <category>/<skill_name>` for the bundle test only.

5. **Commit** — Clear imperative message, no emojis; include issue reference when appropriate. Do not add AI tools in `Co-authored-by:` trailers (see [Agent Code of Conduct](CODE_OF_CONDUCT.md#contribution-process)).
6. **Changelog** — If the PR is user-visible, add lines under `[Unreleased]` in [CHANGELOG.md](CHANGELOG.md) before opening the PR.
7. **Push** to your fork and open a PR into `ARPAHLS/skillware` `main`.
Expand Down Expand Up @@ -257,7 +265,7 @@ The primary guide for the host LLM.
- **Required** for every new registry skill (template: `templates/python_skill/test_skill.py`; enforced by `tests/test_skill_issuer.py`).
- Unit tests for schema compliance and deterministic execution paths (offline; mock externals).
- Ships inside the skill bundle via `pip install skillware`.
- Run: `pytest skills/<category>/<skill_name>/test_skill.py`
- Run: `pytest skills/<category>/<skill_name>/test_skill.py` or `skillware test <category>/<skill_name>`
- Optional extra depth for maintainers: `tests/skills/<category>/test_<skill_name>.py` — see [TESTING.md](docs/TESTING.md).
- Mock network calls and first-run model downloads in bundle tests.

Expand Down
16 changes: 13 additions & 3 deletions docs/TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Tests fall into four layers: **bundle**, **framework**, **maintainer**, and **ex
| Bundle tests mock network and model downloads in CI | Done |
| Maintainer tests under `tests/skills/` (optional per skill) | Done |
| `[all]` extra covers bundle-test runtime deps | Done |
| CLI `skillware test` for bundle discovery | Planned |
| CLI `skillware test` for bundle discovery | Done |

Every pull request runs `black --check`, `flake8`, `pytest skills/`, and `pytest tests/`. Bundle tests gate merge the same as framework and maintainer tests.

Expand Down Expand Up @@ -156,6 +156,16 @@ python -m pytest skills/
python -m pytest tests/
```

Or use the CLI (same bundle paths; requires `pip install -e ".[dev]"` or `[dev,all]`):

```bash
skillware test
skillware test <category>/<skill_name>
skillware test --category <category>
```

See [CLI reference](usage/cli.md#skillware-test).

Single skill bundle test:

```bash
Expand Down Expand Up @@ -183,6 +193,6 @@ Before pushing your code, run the following commands:
1. `skillware list` (verify install and path resolution)
2. `python -m black --check .` (verify formatting; use `python -m black .` to fix)
3. `python -m flake8 .` (check quality)
4. `python -m pytest skills/` (bundle tests — same scope as CI)
4. `python -m pytest skills/` or `skillware test` (bundle tests — same scope as CI)
5. `python -m pytest tests/` (framework + maintainer tests — same scope as CI)
6. `python -m pytest skills/<category>/<skill_name>/test_skill.py` when you want a single-skill subset
6. `python -m pytest skills/<category>/<skill_name>/test_skill.py` or `skillware test <category>/<skill_name>` for a single skill
4 changes: 3 additions & 1 deletion docs/contributing/ai_native_workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,8 @@ pytest skills/
pytest tests/
```

Bundle tests: `skillware test` is equivalent for `skills/**/test_skill.py` (see [CLI reference](../usage/cli.md#skillware-test)).

For a single skill:

```bash
Expand Down Expand Up @@ -263,7 +265,7 @@ Complete the checklist that matches your issue during Stage 5.
- [ ] `skill.py`: deterministic, JSON-serializable returns, safe error handling
- [ ] `instructions.md`: when to use, how to interpret output, limitations
- [ ] `card.json`: `issuer` matches manifest
- [ ] `test_skill.py` (bundle test) passes — `pytest skills/<category>/<skill_name>/test_skill.py`
- [ ] `test_skill.py` (bundle test) passes — `pytest skills/<category>/<skill_name>/test_skill.py` or `skillware test <category>/<skill_name>`
- [ ] Bundle tests mock all network calls and model downloads; CI does not download models.
- [ ] `docs/skills/<skill_name>.md` and catalog row in `docs/skills/README.md`
- [ ] **Usage Examples** on the catalog page (all five providers per [skill usage template](../usage/skill_usage_template.md)); link to `docs/usage/` and list skill `env_vars` without duplicating [api_keys.md](../usage/api_keys.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/usage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ How to load Skillware skills and connect them to language models. Each guide cov

For pip-installed apps, keep project skills in `./skills/<category>/<name>/` or set `SKILLWARE_SKILL_PATH` to your skills root.

To list all locally available skills from the terminal, see the [CLI reference](cli.md).
To list locally available skills or run bundle tests from the terminal, see the [CLI reference](cli.md).

| Provider | Adapter | Guide | Agent API key (typical) |
| :--- | :--- | :--- | :--- |
Expand All @@ -21,7 +21,7 @@ To list all locally available skills from the terminal, see the [CLI reference](
| OpenAI (ChatGPT) | `to_openai_tool()` | [openai.md](openai.md) | `OPENAI_API_KEY` |
| DeepSeek | `to_deepseek_tool()` | [deepseek.md](deepseek.md) | `DEEPSEEK_API_KEY` |
| Ollama (prompt mode) | `to_ollama_prompt()` | [ollama.md](ollama.md) | (local; no cloud key) |
| CLI | `skillware list` | [cli.md](cli.md) | |
| CLI | `skillware list`, `skillware test` | [cli.md](cli.md) | pytest in `[dev]` for `test` |

Skill-specific **Usage Examples** (sample prompts and execute payloads) live on each [skill catalog page](../skills/README.md).

Expand Down
31 changes: 29 additions & 2 deletions docs/usage/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ After installation, the `skillware` command is available directly:

skillware
skillware list
skillware test
skillware --version

If `skillware` is not recognized, Python's `Scripts` directory may not be on
Expand All @@ -35,6 +36,7 @@ as Python is installed):

python -m skillware
python -m skillware list
python -m skillware test finance/wallet_screening
python -m skillware list --category compliance
python -m skillware --help

Expand Down Expand Up @@ -76,8 +78,8 @@ Available commands:
| Input | Command | Status |
| :--- | :--- | :--- |
| `1` / `list` | List all locally installed skills | Available |
| `2` / `paths` | Show and repair skill directory resolution paths | Coming in #81 |
| `3` / `test` | Run test_skill.py for one or all skills | Coming in #83 |
| `2` / `paths` | Show and repair skill directory resolution paths | Coming soon |
| `3` / `test` | Run bundle tests (`test_skill.py`) for one or all skills | Available |
| `4` / `help` | Print rich-formatted help with commands, flags, and examples | Available |

## Commands
Expand Down Expand Up @@ -114,6 +116,31 @@ Sample output:
# Use a custom skills directory
skillware list --skills-root /path/to/my/skills

### skillware test

Run skill **bundle tests** (`test_skill.py`) via pytest. Uses the same skill roots as `skillware list` (`SKILLWARE_SKILL_PATH`, `--skills-root`, cwd `skills/`, bundled registry).

Requires pytest (`pip install -e ".[dev]"` or `pip install -e ".[dev,all]"`).

skillware test
skillware test finance/wallet_screening
skillware test --category compliance
skillware test --verbose
skillware test office/pdf_form_filler --no-header

#### Arguments and flags

| Input | Description |
| :--- | :--- |
| *(no args)* | Run bundle tests under all resolved skill roots |
| `<category>/<skill_name>` | Run one skill's `test_skill.py` |
| `--category <name>` | Run all bundle tests in a category directory |
| `--skills-root <path>` | Override the skills directory for this command |
| `-v` / `--verbose` | Pass `-v` to pytest |
| `--no-header` | Pass `--no-header` to pytest |

Exit code matches pytest (non-zero on failures or missing test paths).

## Path resolution

`skillware list` searches for skills in the same order as `SkillLoader`:
Expand Down
4 changes: 2 additions & 2 deletions docs/vision.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Multi-layer screening runs locally in one `execute()` call. No generated scraper
Skillware is designed so agents and their operators can discover, vet, and integrate capabilities without reinventing the wheel.

- **Manifests** declare inputs, outputs, dependencies, and constitution in `manifest.yaml`.
- **`skillware list`** (CLI) surfaces the local registry with categories, issuers, and descriptions.
- **`skillware list`** and **`skillware test`** (CLI) surface the local registry and run bundle tests.
- **[Examples index](../examples/README.md)** maps runnable provider scripts to skills.
- **[Usage guides](usage/README.md)** show the same load / tool-call / execute loop for Gemini, Claude, OpenAI, DeepSeek, and Ollama.
- **[Agent contribution workflow](contributing/ai_native_workflow.md)** documents how supervised agents propose scoped changes and open PRs.
Expand Down Expand Up @@ -106,7 +106,7 @@ Honest snapshot for **v0** (current v0.3.x line):

- **Registry**: Skills under `skills/` with docs in [docs/skills/](skills/README.md).
- **Loader**: Dynamic import, dependency checks, and adapters for major LLM tool formats.
- **CLI**: `skillware list` and an interactive menu, included with `pip install skillware`.
- **CLI**: `skillware list`, `skillware test`, and an interactive menu, included with `pip install skillware`.
- **Active work**: Wallet screening enhancements ([RFC #115](https://github.com/ARPAHLS/skillware/issues/115)), CLI polish, contributor docs, and good first issues across docs and framework.

Browse [open good first issues](https://github.com/ARPAHLS/skillware/issues?q=is%3Aopen+label%3A%22good+first+issue%22) if you want a low-risk entry point.
Expand Down
Loading
Loading