Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ebce94d
feat: add Compass catalog manifests for ocp-admin pack
r2dedios Jul 3, 2026
c008708
feat: add Compass manifests for all 7 agentic packs
r2dedios Jul 6, 2026
f8e52a9
feat: add Domain and System hierarchy for Compass UI navigation
r2dedios Jul 6, 2026
656cfe1
refactor: deduplicate MCP manifests by technology
r2dedios Jul 6, 2026
df03ed4
fix: remove Domain entity from Location targets
r2dedios Jul 6, 2026
3deefb1
feat: add TechDocs annotation to all System manifests
r2dedios Jul 6, 2026
3bf6e63
feat: add rh-agentic-plugins-mcps namespace to MCP manifests
r2dedios Jul 7, 2026
d0532ae
feat: enrich Compass manifests with MCP System, relations, and annota…
r2dedios Jul 7, 2026
d884c97
feat: add TechDocs annotations to MCP manifests
r2dedios Jul 7, 2026
5125295
feat: add skill-to-skill dependsOn relations to Compass manifests
r2dedios Jul 8, 2026
a69a5fd
fix: remove techdocs-ref annotations that break Compass TechDocs builds
r2dedios Jul 8, 2026
7e301fc
refactor: convert pack System entities to AiResource plugins
r2dedios Jul 10, 2026
b913638
refactor: replace spec.system with dependsOn in skill manifests
r2dedios Jul 10, 2026
c6b8d80
feat: unify under single System and restructure Location hierarchy
r2dedios Jul 10, 2026
b476511
docs: add Compass manifest documentation to CLAUDE.md
r2dedios Jul 10, 2026
8f60a02
fix: qualify system reference with default namespace
r2dedios Jul 10, 2026
8c2289e
fix: add explicit hasPart relations to System entity
r2dedios Jul 10, 2026
77c4cb3
revert: remove explicit hasPart relations (not supported by Compass)
r2dedios Jul 10, 2026
386f8a0
refactor: reuse canonical MCP servers for Lightspeed and Security
r2dedios Jul 10, 2026
ac06903
chore: change owner group to ai5-marketplace
r2dedios Jul 13, 2026
4ca7656
feat: add distribution: external label to MCPServer manifests
r2dedios Jul 27, 2026
44e705a
fix: replace env var interpolation with static placeholder URLs in MC…
r2dedios Jul 27, 2026
25c03b5
fix: replaced link by the proper annotation
r2dedios Jul 27, 2026
6aa42bb
refactor: move agentic-plugins.redhat.com labels to annotations
r2dedios Jul 27, 2026
c3ace00
feat: add bidirectional semantic relations to Compass manifests
r2dedios Jul 28, 2026
a40e79b
fix: replace partOf/hasPart with dependsOn/dependencyOf in Compass ma…
r2dedios Jul 28, 2026
d312972
feat: document primitives for all MCP servers and fix Compass manifes…
r2dedios Aug 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 122 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,16 @@ Contributors work here to create, improve, and validate skills. An internal proc

```
agentic-plugins/
├── catalog-info.yaml # Root Location — entry point for Compass ingestion
├── system.yaml # System entity (agentic-plugins) — models the whole repo
├── rh-sre/ # Site Reliability Engineering pack (reference implementation)
├── rh-developer/ # Developer tools pack
├── ocp-admin/ # OpenShift administration pack
├── rh-virt/ # Virtualization management pack
├── rh-basic/ # Getting started pack
├── rh-ai-engineer/ # AI Engineering pack
├── rh-automation/ # Automation pack
├── rh-support-engineer/ # Support engineering pack
├── mcps/ # MCP server Compass manifests (System + 9 MCPServer entities)
├── eval/ # Skill evaluation reports (report.json + report.md per skill)
├── scripts/ # Validation and CI helper scripts
├── catalog/ # JSON Schema for .catalog/collection.yaml validation
Expand All @@ -37,6 +39,8 @@ Each pack is persona-specific and follows this structure:

```
<pack-name>/
├── catalog-info.yaml # Location entity — indexes this pack's Compass manifests
├── <pack-name>-plugin.yaml # AiResource (type: plugin) — defines the pack itself
├── AGENTS.md # AI Context Module instruction routing (persona, skills, rules)
├── README.md # Pack description, persona, target marketplaces
├── mcps.json # MCP server configurations (uses env vars for credentials)
Expand All @@ -45,14 +49,120 @@ Each pack is persona-specific and follows this structure:
│ └── collection.json # Deterministic JSON mirror of collection.yaml
├── skills/ # Specialized task executors (including orchestration skills)
│ └── <skill>/
│ └── SKILL.md # Skill definition with YAML frontmatter
│ ├── SKILL.md # Skill definition with YAML frontmatter
│ └── catalog-info.yaml # AiResource (type: skill) — Compass manifest
└── docs/ # AI-optimized knowledge base (optional, rh-sre reference)
```

### Relationship with the Catalog

Each pack's `.catalog/` directory contains metadata that describes the pack for the marketplace. This metadata stays here, alongside the skills it describes. The catalog build process reads it from this repo to assemble the unified marketplace. The golden sources are always `SKILL.md`, `AGENTS.md`, `README.md`, and `mcps.json` — `.catalog/` is derived from them, never the other way around.

### Compass / Backstage Manifests

The repository is registered in [Red Hat Compass](https://compass.redhat.com) (internal Backstage instance) through a hierarchy of Backstage entity manifests. All manifests use `apiVersion: backstage.io/v1alpha1` except MCPServer entities which use `apiVersion: mcp/v1beta1`.

#### Entity Kinds

| Kind | Purpose | spec.type | Count |
|------|---------|-----------|-------|
| **Location** | Index that references other manifest files | — | 9 (1 root + 7 packs + 1 mcps) |
| **System** | Top-level grouping for the repository and MCP servers | — | 2 (`agentic-plugins`, `rh-agentic-plugins-mcps`) |
| **AiResource** | Skills and pack definitions | `plugin` (packs) / `skill` (skills) / `rule` | 7 packs + 75 skills |
| **MCPServer** | MCP server configurations | `local` | 9 |

#### Location Hierarchy

Compass ingests a single root Location. Everything else is discovered through delegation:

```
catalog-info.yaml (root Location)
├── system.yaml → System: agentic-plugins
├── ocp-admin/catalog-info.yaml → Location → ocp-admin-plugin.yaml + 7 skills
├── rh-sre/catalog-info.yaml → Location → rh-sre-plugin.yaml + 13 skills
├── rh-virt/catalog-info.yaml → Location → rh-virt-plugin.yaml + 10 skills
├── rh-developer/catalog-info.yaml → Location → rh-developer-plugin.yaml + 17 skills
├── rh-basic/catalog-info.yaml → Location → rh-basic-plugin.yaml + 6 skills
├── rh-ai-engineer/catalog-info.yaml → Location → rh-ai-engineer-plugin.yaml + 11 skills
├── rh-automation/catalog-info.yaml → Location → rh-automation-plugin.yaml + 11 skills
└── mcps/catalog-info.yaml → Location → system.yaml + 9 MCPServers
```

#### Entity Relationships

All relationships use `dependsOn` / `dependencyOf`:
Comment thread
r2dedios marked this conversation as resolved.

> **Why not `partOf`/`hasPart`?** Compass only processes `partOf`/`hasPart` for standard Backstage kinds (Component, API, Resource). Custom kinds like `AiResource` and `MCPServer` can store these fields in `spec`, but they are not processed into the relation graph. Only `dependsOn`/`dependencyOf` and `ownedBy`/`ownerOf` generate actual relations for custom kinds.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems a bug and we should instead try to fix it rather than finding a workaround. WDYT?


Concrete relationships in this repo:

- **Skill → Plugin**: `dependsOn` / `dependencyOf` (a skill belongs to its plugin)
- **Plugin → System**: `dependsOn` / `dependencyOf` (a plugin belongs to the system)
- **MCPServer → System**: `dependsOn` / `dependencyOf` (an MCP server belongs to the system)
- **Skill → MCPServer**: `dependsOn` / `dependencyOf` (a skill uses an MCP server)
- **Plugin → MCPServer**: `dependsOn` / `dependencyOf` (a plugin uses an MCP server)
- **Skill → Skill**: `dependsOn` / `dependencyOf` (orchestration skills invoke other skills)
- **All entities → Group**: `spec.owner: group:redhat/ai5-marketplace`

> **Bidirectional declaration policy:** Compass does not auto-generate inverse relations for custom entity kinds such as `AiResource` and `MCPServer` (tracked as COMPASS-1288). Until this is resolved upstream, we explicitly declare **both directions** of every relationship in our manifests. For example, if a skill declares `dependsOn: [airesource:rh-sre/rh-sre]`, the plugin must also declare `dependencyOf: [airesource:rh-sre/<skill>]`. When adding or modifying a relationship, always update both the source and target manifests.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of them are not needed and may require hard maintenance, please consider this to define your design.


#### Namespaces

Each pack uses its own namespace matching the pack name (e.g., `rh-sre`, `ocp-admin`). All MCP servers share the namespace `agentic-plugins`. The root System `agentic-plugins` uses the `default` namespace.

#### Entity Reference Formats

- Skills: `airesource:<pack-namespace>/<skill-name>`
- Pack plugins: `airesource:<pack-namespace>/<pack-name>`
- MCP servers: `mcpserver:agentic-plugins/<server-name>`

#### Adding Compass Manifests for a New Skill
Comment thread
r2dedios marked this conversation as resolved.

When adding a skill, create `skills/<skill-name>/catalog-info.yaml`:
```yaml
apiVersion: backstage.io/v1alpha1
kind: AiResource
metadata:
name: <skill-name>
namespace: <pack-name>
title: <Skill Title>
description: >
<skill description>
labels:
distribution: external
annotations:
backstage.io/source-location: >-
url:https://github.com/RHEcosystemAppEng/agentic-plugins/blob/main/<pack>/skills/<skill>/SKILL.md
tags:
- ai-skill
links:
- url: https://github.com/RHEcosystemAppEng/agentic-plugins
title: Source Repository
icon: github
spec:
type: skill
lifecycle: beta
owner: group:redhat/ai5-marketplace
disciplines:
- <discipline>
categories:
- <category>
agents:
- claude-code
- opencode
- cursor
dependsOn:
- airesource:<pack-name>/<pack-name>
# Add mcpserver and airesource (skill) dependencies as needed
- mcpserver:agentic-plugins/<server-name>
```

Then update **both sides** of every relationship:
1. Add the file as a target in the pack's `catalog-info.yaml` Location
2. Add the skill to the plugin's `dependencyOf` list in `<pack>-plugin.yaml`
3. Add the skill to each referenced MCP server's `dependencyOf` list in `mcps/<server>.yaml`
4. If the skill depends on other skills, add `dependencyOf` entries in those skills' manifests

## Contributing

Skills are added directly to this repository, inside an existing pack. The contributor opens a PR, skills are reviewed and merged, and maintainers own them from that point. Use `/agentic-contribution-skill` in Claude Code or follow [CONTRIBUTING.md](CONTRIBUTING.md).
Expand Down Expand Up @@ -165,6 +275,7 @@ last_updated: YYYY-MM-DD

### Files
- Skills: `skills/<skill-name>/SKILL.md` (uppercase SKILL.md)
- Compass manifests: `catalog-info.yaml` (Locations), `<pack-name>-plugin.yaml` (pack AiResource), `system.yaml` (System entities)
- Docs: Lowercase with dashes, categorized by directory

## Development Workflow
Expand All @@ -176,7 +287,11 @@ last_updated: YYYY-MM-DD
3. Add `AGENTS.md` with persona, skill-first rule, intent routing table, MCP servers, and global rules (see [rh-ai-engineer/AGENTS.md](rh-ai-engineer/AGENTS.md) for reference)
4. Create `skills/` directory
5. Add `mcps.json` when the pack integrates MCP servers (use `${VAR}` for secrets)
6. Update main `README.md` table with link
6. Create Compass manifests:
- `<pack-name>-plugin.yaml` — AiResource with `type: plugin`, `system: agentic-plugins`
- `catalog-info.yaml` — Location targeting the plugin file and all skill catalog-info.yaml files
7. Add the pack's `catalog-info.yaml` as a target in the root `catalog-info.yaml`
8. Update main `README.md` table with link

### Adding a Skill

Expand All @@ -192,8 +307,10 @@ last_updated: YYYY-MM-DD
- Dependencies declaration
4. Include concrete examples and complete error handling
5. Update the pack's `AGENTS.md` intent routing table to include the new skill
6. Test with `Skill` tool invocation
7. Validate with `uv run python scripts/validate_skills_tier1.py <pack>/skills/<skill-name>/SKILL.md`
6. Create `skills/<skill-name>/catalog-info.yaml` Compass manifest (see "Adding Compass Manifests for a New Skill")
7. Add the skill's `catalog-info.yaml` as a target in the pack's `catalog-info.yaml` Location
8. Test with `Skill` tool invocation
9. Validate with `uv run python scripts/validate_skills_tier1.py <pack>/skills/<skill-name>/SKILL.md`

**Collection-Specific Standards:**
- **rh-virt**: Follow `rh-virt/SKILL_TEMPLATE.md` for enhanced quality standards including mandatory Common Issues and Example Usage sections
Expand Down
26 changes: 26 additions & 0 deletions catalog-info.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
apiVersion: backstage.io/v1alpha1
kind: Location
metadata:
name: agentic-plugins
title: Agentic Plugins
description: >
Root catalog entry point for the agentic-plugins repository.
Points to all agentic pack locations and MCP server manifests
for Compass ingestion.
annotations:
agentic-plugins.redhat.com/version: "1.0"
agentic-plugins.redhat.com/repo: "agentic-plugins"
spec:
targets:
# Repository-level system
- ./system.yaml
# Agentic Pack Locations
- ./ocp-admin/catalog-info.yaml
- ./rh-sre/catalog-info.yaml
- ./rh-virt/catalog-info.yaml
- ./rh-developer/catalog-info.yaml
- ./rh-basic/catalog-info.yaml
- ./rh-ai-engineer/catalog-info.yaml
- ./rh-automation/catalog-info.yaml
# MCP Servers Location
- ./mcps/catalog-info.yaml
181 changes: 181 additions & 0 deletions mcps/ai-observability.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
apiVersion: mcp/v1beta1
kind: MCPServer
metadata:
namespace: agentic-plugins
name: ai-observability
title: AI Observability MCP Server
description: >
AI observability MCP server for monitoring and debugging AI/ML model
inference, performance metrics, and serving runtime health on
Red Hat OpenShift AI.
labels:
distribution: external
annotations:
backstage.io/source-location: >-
url:https://github.com/RHEcosystemAppEng/agentic-plugins/blob/main/rh-ai-engineer/mcps.json
tags:
- ai
- observability
- machine-learning
- monitoring
- mcp-server
links:
- url: https://github.com/rh-ai-quickstart/ai-observability-summarizer
title: Upstream Repository
icon: github
spec:
type: remote
lifecycle: beta
owner: group:redhat/ai5-marketplace
system: default/agentic-plugins
primitives:
- type: tool
name: list_models
description: List all available AI models for analysis.
- type: tool
name: list_vllm_namespaces
description: Get list of monitored vLLM Kubernetes namespaces with observability data.
- type: tool
name: get_model_config
description: Get available LLM models configured for summarization and analysis.
- type: tool
name: get_vllm_metrics_tool
description: Get available vLLM metrics with friendly names and PromQL queries.
- type: tool
name: fetch_vllm_metrics_data
description: Fetch vLLM metrics data with time-series for dashboard display.
- type: tool
name: analyze_vllm
description: Analyze vLLM metrics and generate an AI summary using an LLM.
- type: tool
name: calculate_metrics
description: Calculate statistics (avg, min, max, latest) for provided metrics data.
- type: tool
name: list_summarization_models
description: List all configured summarization models with metadata from runtime configuration.
- type: tool
name: get_gpu_info
description: Get cluster GPU information from Prometheus DCGM metrics.
- type: tool
name: get_deployment_info
description: Get model deployment details for a model in a namespace.
- type: tool
name: chat_vllm
description: Chat about vLLM metrics by asking follow-up questions on analyzed data.
- type: tool
name: analyze_openshift
description: Analyze OpenShift metrics for a category and scope with LLM-generated summary.
- type: tool
name: fetch_openshift_metrics_data
description: Fetch OpenShift metrics data for dashboard visualization without LLM analysis.
- type: tool
name: list_openshift_namespaces
description: Get list of all OpenShift namespaces observed in Prometheus.
- type: tool
name: list_openshift_metric_groups
description: Return available OpenShift metric group categories for cluster-wide analysis.
- type: tool
name: list_openshift_namespace_metric_groups
description: Return OpenShift metric groups that support namespace-scoped analysis.
- type: tool
name: chat_openshift
description: Chat about OpenShift metrics for a specific category and scope.
- type: tool
name: search_metrics
description: Search for Prometheus metrics by pattern using semantic understanding.
- type: tool
name: get_metric_metadata
description: Get detailed metadata for a specific Prometheus metric.
- type: tool
name: get_label_values
description: Get all possible values for a specific label of a Prometheus metric.
- type: tool
name: execute_promql
description: Execute a PromQL query and return structured results.
- type: tool
name: explain_results
description: Explain PromQL query results in natural language.
- type: tool
name: suggest_queries
description: Suggest related PromQL queries based on user intent.
- type: tool
name: select_best_metric
description: Select the best metric for a user question using semantic analysis.
- type: tool
name: find_best_metric_with_metadata_v2
description: Enhanced metric selection with improved semantic analysis and query suggestions.
- type: tool
name: find_best_metric_with_metadata
description: Find the best metric for a user question using comprehensive metadata analysis.
- type: tool
name: get_metrics_categories
description: Get all metric categories with summary counts, priorities, and example queries.
- type: tool
name: search_metrics_by_category
description: Search metrics filtered by category IDs and priority levels.
- type: tool
name: get_category_metrics_detail
description: Get category summaries or detailed metrics catalog for a single category.
- type: tool
name: convert_time_to_promql_duration
description: Convert decimal hours to Prometheus duration format for use in PromQL queries.
- type: tool
name: query_tempo_tool
description: Search Tempo traces by TraceQL query, service, operation, and time range.
- type: tool
name: get_trace_details_tool
description: Get detailed trace information including spans by trace ID.
- type: tool
name: chat_tempo_tool
description: Conversational interface for Tempo trace analysis using natural language questions.
- type: tool
name: korrel8r_query_objects
description: Execute a Korrel8r domain query and return correlated objects.
- type: tool
name: korrel8r_get_correlated
description: Return correlated objects (alerts, logs, traces, metrics) for a Korrel8r query.
- type: tool
name: get_correlated_logs
description: Fetch application and infrastructure logs for a namespace or pod via Korrel8r.
- type: tool
name: chat
description: Chat with AI models using observability tools with progress tracking.
- type: tool
name: validate_api_key
description: Validate an API key for a given provider by making a minimal server-side request.
- type: tool
name: save_api_key
description: Save or update a provider API key in a Kubernetes Secret.
- type: tool
name: check_provider_secret
description: Check if a provider API key secret exists in the cluster.
- type: tool
name: delete_provider_secret
description: Delete a provider API key secret from the cluster.
- type: tool
name: list_provider_models
description: Query a provider API to list available models for configuration.
- type: tool
name: add_model_to_config
description: Add a new model to the model configuration by updating the Kubernetes ConfigMap.
- type: tool
name: update_maas_model_api_key
description: Update API key and optionally endpoint for an existing MaaS model.
packages: []
remotes:
- url: https://ai-observability.example.com/mcp
dependsOn:
- system:default/agentic-plugins
dependencyOf:
Comment thread
r2dedios marked this conversation as resolved.
- airesource:rh-ai-engineer/ai-observability
- airesource:rh-ai-engineer/debug-inference
- airesource:rh-ai-engineer/ds-project-setup
- airesource:rh-ai-engineer/guardrails-config
- airesource:rh-ai-engineer/model-deploy
- airesource:rh-ai-engineer/model-monitor
- airesource:rh-ai-engineer/model-registry
- airesource:rh-ai-engineer/nim-setup
- airesource:rh-ai-engineer/pipeline-manage
- airesource:rh-ai-engineer/rh-ai-engineer
- airesource:rh-ai-engineer/serving-runtime-config
- airesource:rh-ai-engineer/workbench-manage
Loading
Loading