feat: Marketplace integration -- read marketplace.json for plugin discovery + governance

## Problem

APM uses Git URLs directly for plugin installation (`apm install owner/repo`). Meanwhile, the plugin ecosystem converges on marketplaces -- Copilot CLI and Claude Code both use `marketplace.json` as a discovery catalog. Today, plugins are discovered in a marketplace, then separately installed and governed via APM. This two-tool workflow makes governance optional and easily skipped.

APM should read existing `marketplace.json` files for discovery, resolve plugins to Git URLs, then apply its full governance layer (lockfile, SHA pinning, audit trail, dev/prod separation) on top -- collapsing a two-tool workflow into one.

### Supporting documentation

- [Copilot CLI Plugin Reference](https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference) -- `plugin.json` spec
- [Copilot CLI Plugin Reference: marketplace.json](https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference#marketplacejson) -- marketplace.json spec
- [Copilot CLI: Creating a plugin marketplace](https://docs.github.com/en/copilot/tutorials/copilot-cli-tutorial/creating-a-plugin-marketplace-for-copilot-cli) -- marketplace authoring guide
- [Copilot CLI: Finding and installing plugins](https://docs.github.com/en/copilot/how-to/copilot-cli/finding-and-installing-copilot-cli-plugins) -- marketplace consumer workflow
- [Claude Code: Plugins overview](https://docs.anthropic.com/en/docs/claude-code/plugins) -- Claude plugin spec
- [Claude Code: Discover plugins](https://docs.anthropic.com/en/docs/claude-code/plugins/discover-plugins) -- Claude marketplace discovery
- [Claude Code: Plugin marketplaces](https://docs.anthropic.com/en/docs/claude-code/plugins/plugin-marketplaces) -- Claude marketplace.json format and source types

---

## Design Decisions

Informed by a 5-expert analysis panel (UX, npm ecosystem, product strategy, Python architecture, Anthropic spec compatibility).

### 1. Marketplace = discovery layer, Git = source of truth

APM reads marketplace.json as-is (both `.github/plugin/marketplace.json` and `.claude-plugin/marketplace.json`). The marketplace resolves to a Git coordinate. APM never depends on a marketplace being online for installs after initial resolution. This is the Go modules model (where `pkg.go.dev` is discovery but `go get` always resolves to VCS), not the npm model (where the registry IS the source).

### 2. Git ref in apm.yml, marketplace provenance in lockfile

When `apm install security-checks@acme-tools` runs:
1. Query marketplace -> resolve to `owner/plugin-repo#v1.3.0`
2. Store `owner/plugin-repo#v1.3.0` in apm.yml (self-contained, portable)
3. Store `discovered_via: acme-tools` + `marketplace_plugin_name: security-checks` in apm.lock.yaml

If the marketplace disappears, existing installs still work. Future `apm deps update --marketplace` will use lockfile provenance to re-query.

**Why not store `NAME@MARKETPLACE` in apm.yml?** Storing the resolved Git ref makes apm.yml fully self-contained -- it does not require the recipient to have the same marketplace configured, and it survives marketplace deletion or migration.

### 3. No DependencyReference modifications (critical)

`DependencyReference` represents a resolved Git coordinate. A marketplace name is an indirect reference that gets resolved INTO a `DependencyReference`, not stored AS one. Use a pre-parse intercept in `install.py` before `DependencyReference.parse()` is called.

### 4. `@` disambiguation: slash-less detection

**Rule**: If input matches `^[a-zA-Z0-9._-]+@[a-zA-Z0-9._-]+$` (no `/`, no `:`, no `git@`), it is a marketplace ref. Everything else goes to existing parse.

**Why this is safe**: `owner/repo@alias` has `/` -> rejected. `git@host:...` has `:` -> rejected. Only `word@word` matches. These inputs previously raised `ValueError` ("Use 'user/repo' format"), so this is a **backward-compatible grammar extension** -- no existing valid specifiers change behavior.

### 5. Bare name install rejected in v1

`apm install security-checks` (without `@MARKETPLACE`) is rejected with a helpful error suggesting `@MARKETPLACE` syntax and `apm search`. This prevents dependency confusion attacks and non-deterministic resolution when multiple marketplaces are configured.

### 6. No default marketplaces in v1

A governance tool should not auto-trust third-party sources. `apm marketplace add` is frictionless. Future phase: suggest popular marketplaces during `apm init`.

### 7. User-scoped marketplace config at `~/.apm/`

Registered marketplaces in `~/.apm/marketplaces.json`. Marketplaces are a personal discovery preference, not a project dependency. Follows existing `~/.apm/config.json` pattern.

### 8. Cache with TTL + stale-while-revalidate

`~/.apm/cache/marketplace/` with 1h TTL. Network failure serves stale cache. `marketplace add` and `marketplace browse` force refresh.

### 9. Reuse AuthResolver unchanged

`AuthResolver.try_with_fallback(unauth_first=True)` handles public-first with fallback for private marketplace repos. No new auth code needed.

### 10. Support 4 of 5 Anthropic source types

Support: relative path, github, url, git-subdir. Skip: npm (out of APM's Git-native model).

---

## Architecture

### New module: `src/apm_cli/marketplace/`

```
src/apm_cli/marketplace/
  __init__.py          # Public exports
  models.py            # Frozen dataclasses: MarketplaceSource, MarketplacePlugin, MarketplaceManifest
  client.py            # MarketplaceClient: fetch, parse, cache marketplace.json via GitHub API
  registry.py          # Manage registered marketplaces in ~/.apm/marketplaces.json
  resolver.py          # parse_marketplace_ref() + resolve NAME -> canonical owner/repo string
  errors.py            # MarketplaceError, MarketplaceNotFoundError, PluginNotFoundError
  commands.py          # Click command group: marketplace add/list/browse/update/remove + search
```

### Resolution flow

```
apm install security-checks@acme-tools
  |
  v
Pre-parse intercept in install.py _validate_and_add_packages_to_apm_yml() L130
  -> parse_marketplace_ref("security-checks@acme-tools") returns ("security-checks", "acme-tools")
  |
  v
resolve_marketplace_plugin("security-checks", "acme-tools")
  -> registry.get_marketplace_by_name("acme-tools") -> MarketplaceSource
  -> client.fetch_or_cache(source) -> MarketplaceManifest
  -> Find plugin: { name: "security-checks", source: { source: "github", repo: "owner/repo", ref: "v1.3.0" } }
  |
  v
Resolve source to canonical string: "owner/repo#v1.3.0"
  |
  v
Replace package variable -> existing install flow continues unchanged
  -> DependencyReference.parse("owner/repo#v1.3.0") -> validate -> apm.yml -> lockfile
```

### CLI commands

```
apm marketplace add OWNER/REPO          # Register a marketplace
apm marketplace list                    # Rich table of registered marketplaces
apm marketplace browse NAME             # Rich table of plugins in a marketplace
apm marketplace update [NAME]           # Refresh cache (one or all)
apm marketplace remove NAME             # Unregister with confirmation

apm search QUERY                        # Search across all registered marketplaces
apm install NAME@MARKETPLACE            # Resolve via marketplace, full governance
```

### Storage layout

```
~/.apm/
  config.json                           # Existing
  marketplaces.json                     # NEW: registered marketplace sources
  cache/
    marketplace/
      acme-tools.json                   # Cached marketplace.json content
      acme-tools.meta.json              # { fetched_at, ttl_seconds }
```

### Data models

```python
@dataclass(frozen=True)
class MarketplaceSource:
    name: str               # Display name (e.g., "acme-tools")
    owner: str              # GitHub owner
    repo: str               # GitHub repo
    host: str = "github.com"
    branch: str = "main"
    path: str = "marketplace.json"  # Auto-detected on add

@dataclass(frozen=True)
class MarketplacePlugin:
    name: str               # Plugin name (unique within marketplace)
    source: Any             # String (relative) or dict (github/url/git-subdir)
    description: str = ""
    version: str = ""
    tags: tuple = ()
    source_marketplace: str = ""

@dataclass(frozen=True)
class MarketplaceManifest:
    name: str
    plugins: tuple = ()     # Tuple of MarketplacePlugin
    owner_name: str = ""
    description: str = ""
```

### Lockfile provenance (new optional fields on LockedDependency)

```yaml
- repo_url: owner/plugin-repo
  resolved_commit: abc123def456
  resolved_ref: v1.3.0
  content_hash: sha256:2800e9f...
  discovered_via: acme-tools                # NEW
  marketplace_plugin_name: security-checks  # NEW
```

### `@` disambiguation regex

```python
_MARKETPLACE_RE = re.compile(r'^([a-zA-Z0-9._-]+)@([a-zA-Z0-9._-]+)$')

def parse_marketplace_ref(specifier):
    if '/' in specifier or ':' in specifier:
        return None
    match = _MARKETPLACE_RE.match(specifier.strip())
    return (match.group(1), match.group(2)) if match else None
```

### Modified existing files

- `src/apm_cli/commands/install.py` L130: pre-parse intercept (~10 lines)
- `src/apm_cli/deps/lockfile.py`: 2 optional fields on LockedDependency + serialization
- `src/apm_cli/cli.py` L71: register marketplace command group (2 lines)

---

## Implementation Plan

### Wave 1 -- Foundation (parallelizable, no dependencies)

#### `marketplace-models`
Create `src/apm_cli/marketplace/models.py` and `errors.py`:
- Frozen dataclasses: `MarketplaceSource`, `MarketplacePlugin`, `MarketplaceManifest`
- `parse_marketplace_json(data, source_name)` parser for both Copilot CLI and Claude Code marketplace.json formats
- Error classes: `MarketplaceError`, `MarketplaceNotFoundError`, `PluginNotFoundError`, `MarketplaceFetchError`
- Also create `__init__.py` with exports

#### `lockfile-provenance`
Edit `src/apm_cli/deps/lockfile.py`:
- Add `discovered_via` and `marketplace_plugin_name` optional fields to `LockedDependency`
- Update `to_dict()` and `from_dict()` (backward compatible -- None by default)

### Wave 2 -- Registry

#### `marketplace-registry`
Create `src/apm_cli/marketplace/registry.py`:
- CRUD for `~/.apm/marketplaces.json`: `get_registered_marketplaces()`, `add_marketplace()`, `remove_marketplace()`, `get_marketplace_by_name()`
- Process-lifetime cache, atomic writes, uses `config.ensure_config_exists()`
- Depends on: `marketplace-models`

### Wave 3 -- Client

#### `marketplace-client`
Create `src/apm_cli/marketplace/client.py`:
- `MarketplaceClient` with `AuthResolver` integration
- `fetch_marketplace()` via GitHub Contents API with `unauth_first=True`
- `fetch_or_cache()` with 1h TTL, stale-while-revalidate on network errors
- `search_plugins()` across all registered marketplaces
- Auto-detect marketplace.json location (`.claude-plugin/` vs `.github/plugin/` vs repo root)
- Cache at `~/.apm/cache/marketplace/`
- Depends on: `marketplace-models`, `marketplace-registry`

### Wave 4 -- Resolver

#### `marketplace-resolver`
Create `src/apm_cli/marketplace/resolver.py`:
- `parse_marketplace_ref()` regex for `NAME@MARKETPLACE` detection
- `resolve_marketplace_plugin()` -> canonical `owner/repo#ref` string
- Handles 4 source types: relative path, github, url, git-subdir
- Rejects npm source with clear error message
- Actionable error messages for missing marketplace/plugin
- Depends on: `marketplace-models`, `marketplace-client`, `marketplace-registry`

### Wave 5 -- CLI + Install Hook (parallelizable)

#### `marketplace-cli`
Create `src/apm_cli/commands/marketplace.py`:
- Click group with: `add`, `list`, `browse`, `update`, `remove` subcommands
- Top-level `search` command
- Rich tables with colorama fallbacks, `STATUS_SYMBOLS`
- Follow `mcp.py` pattern exactly
- Depends on: `marketplace-client`, `marketplace-resolver`, `marketplace-registry`

#### `install-marketplace-hook`
Edit `src/apm_cli/commands/install.py` `_validate_and_add_packages_to_apm_yml()` at L130:
- Pre-parse intercept: `parse_marketplace_ref()` before the `"/" not in package` check
- Resolve to canonical string, replace `package` variable
- Pass marketplace provenance to lockfile writer
- Lazy imports to avoid overhead when marketplace is unused
- Depends on: `marketplace-resolver`, `lockfile-provenance`

### Wave 6 -- Wiring

#### `register-and-wire`
- Register marketplace command group in `cli.py` (2 lines)
- Finalize `__init__.py` exports
- Depends on: `marketplace-cli`

### Wave 7 -- Tests + Docs (parallelizable)

#### `tests-marketplace`
~90 unit tests across 7 test files:
- `test_marketplace_models.py` (~15): dataclass behavior, JSON parsing, source types
- `test_marketplace_resolver.py` (~20): regex positive/negative cases, resolution for all 4 source types
- `test_marketplace_client.py` (~15): HTTP mock, caching, TTL, auth, auto-detection
- `test_marketplace_registry.py` (~10): CRUD with `tmp_path` isolation
- `test_marketplace_commands.py` (~15): `CliRunner` for each command, empty states, errors
- `test_marketplace_install_integration.py` (~10): install flow with mocked marketplace
- `test_lockfile_provenance.py` (~5): serialization round-trip, backward compat
- Depends on: all above

#### `docs-marketplace`
- New: `docs/src/content/docs/guides/marketplaces.md` (user guide)
- Update: CLI reference with marketplace commands + `NAME@MARKETPLACE` syntax
- Update: plugins guide with marketplace integration section
- Depends on: all above

---

## Todo list

- [ ] `marketplace-models` -- Frozen dataclasses + JSON parser + error hierarchy (Wave 1)
- [ ] `lockfile-provenance` -- 2 optional fields on LockedDependency (Wave 1)
- [ ] `marketplace-registry` -- `~/.apm/marketplaces.json` CRUD (Wave 2)
- [ ] `marketplace-client` -- GitHub API fetch, 1h TTL cache, AuthResolver reuse (Wave 3)
- [ ] `marketplace-resolver` -- `parse_marketplace_ref()` regex + source type resolution (Wave 4)
- [ ] `marketplace-cli` -- Click group: add/list/browse/update/remove + search (Wave 5)
- [ ] `install-marketplace-hook` -- ~10 lines in install.py L130 (Wave 5)
- [ ] `register-and-wire` -- 2 lines in cli.py (Wave 6)
- [ ] `tests-marketplace` -- ~90 tests across 7 files (Wave 7)
- [ ] `docs-marketplace` -- New guide + CLI reference updates (Wave 7)

## Dependencies between todos

```
marketplace-models            -- no deps (Wave 1)
lockfile-provenance           -- no deps (Wave 1)
marketplace-registry          -- depends on: marketplace-models (Wave 2)
marketplace-client            -- depends on: marketplace-models, marketplace-registry (Wave 3)
marketplace-resolver          -- depends on: marketplace-models, marketplace-client, marketplace-registry (Wave 4)
marketplace-cli               -- depends on: marketplace-client, marketplace-resolver, marketplace-registry (Wave 5)
install-marketplace-hook      -- depends on: marketplace-resolver, lockfile-provenance (Wave 5)
register-and-wire             -- depends on: marketplace-cli (Wave 6)
tests-marketplace             -- depends on: all above (Wave 7)
docs-marketplace              -- depends on: all above (Wave 7)
```

---

## Phase 2 (future, not in this plan)

- Claude `settings.json` integration (`enabledPlugins` + `extraKnownMarketplaces`)
- `apm publish` for self-hosted marketplaces
- NPM source type support
- Bare name install with disambiguation
- `apm deps update --marketplace` using lockfile provenance
- `apm deps outdated` with marketplace update info
- Default marketplace suggestions in `apm init`
- CI seed directory generation (`CLAUDE_CODE_PLUGIN_SEED_DIR`)


feat: Marketplace integration -- read marketplace.json for plugin discovery + governance #501

Description

Problem

Supporting documentation

Design Decisions

1. Marketplace = discovery layer, Git = source of truth

2. Git ref in apm.yml, marketplace provenance in lockfile

3. No DependencyReference modifications (critical)

4. @ disambiguation: slash-less detection

5. Bare name install rejected in v1

6. No default marketplaces in v1

7. User-scoped marketplace config at ~/.apm/

8. Cache with TTL + stale-while-revalidate

9. Reuse AuthResolver unchanged

10. Support 4 of 5 Anthropic source types

Architecture

New module: src/apm_cli/marketplace/

Resolution flow

CLI commands

Storage layout

Data models

Lockfile provenance (new optional fields on LockedDependency)

@ disambiguation regex

Modified existing files

Implementation Plan

Wave 1 -- Foundation (parallelizable, no dependencies)

marketplace-models

lockfile-provenance

Wave 2 -- Registry

marketplace-registry

Wave 3 -- Client

marketplace-client

Wave 4 -- Resolver

marketplace-resolver

Wave 5 -- CLI + Install Hook (parallelizable)

marketplace-cli

install-marketplace-hook

Wave 6 -- Wiring

register-and-wire

Wave 7 -- Tests + Docs (parallelizable)

tests-marketplace

docs-marketplace

Todo list

Dependencies between todos

Phase 2 (future, not in this plan)

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

4. `@` disambiguation: slash-less detection

7. User-scoped marketplace config at `~/.apm/`

New module: `src/apm_cli/marketplace/`

`@` disambiguation regex

`marketplace-models`

`lockfile-provenance`

`marketplace-registry`

`marketplace-client`

`marketplace-resolver`

`marketplace-cli`

`install-marketplace-hook`

`register-and-wire`

`tests-marketplace`

`docs-marketplace`