Skip to content

Conversation

@tekgrunt
Copy link
Contributor

@tekgrunt tekgrunt commented Dec 10, 2025

Summary

Refactors the reporting skill to use a declarative JSON template system, making it easier to add new report types and maintain consistent data collection patterns.

Changes

Template System

  • Added JSON schema templates that define report structure, inputs, and data sources
  • Templates are self-documenting with field descriptions and API call documentation
  • Supports scope parameter for single-org or all-org reports

Report Templates

  • billing-report.json - Invoice-focused billing report with SKU breakdowns
  • mssp-executive-report.json - Fleet health overview for MSSP leadership
  • detection-analytics-report.json - Detection volume, categories, and trends
  • customer-health-report.json - Combined sensor coverage, detection activity, and health scoring

Key Features

  • Consistent metadata structure across all templates (generated_at, time_window, organizations)
  • Rollup aggregation for multi-tenant reports
  • Error handling with per-org error tracking
  • Health status thresholds (healthy ≥90%, warning 70-89%, critical <70%)

Test plan

  • Run /lc-essentials:reporting-templates to list available templates
  • Generate billing report for single org and all orgs
  • Generate MSSP executive report and verify sensor/detection metrics
  • Generate customer health report and verify health status calculations

🤖 Generated with Claude Code

tekgrunt and others added 4 commits December 9, 2025 17:47
- Replace monolithic reporting skill with clean template-driven design
- Add billing-report.json template defining input/output schema
- Create billing-collector agent for parallel multi-org data collection
- Skill now calls MCP tools directly (like detection-engineering pattern)
- Single-org: direct MCP calls from skill
- Multi-org: parallel billing-collector agents

Templates define contracts, skill handles orchestration, agents handle
data collection. Never fabricates data - all values from API responses.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove MSSP Executive Summary, Customer Health Dashboard, and Detection Analytics templates
- Keep only Monthly Billing Report as the single template option
- Delete unused schemas: mssp-report.json, custom-report.json, security-overview.json
- Delete unused HTML templates: mssp-dashboard.html.j2, custom-report.html.j2, security-overview.html.j2
- Enhance billing-summary.json schema with explicit examples for all variables
- Add comprehensive field descriptions and example values throughout schema

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New JSON templates for the reporting skill:
- mssp-executive-report.json: Fleet health overview with sensor status,
  detections, and per-org health metrics
- detection-analytics-report.json: Detection volume, categories, trends,
  and rule performance across tenants

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Combines sensor coverage, detection activity, and health scoring
into a single customer success tracking report. Includes:
- Fleet-wide health rollup with online/offline sensor counts
- Per-customer health status (healthy/warning/critical/inactive)
- Detection severity breakdown and top categories
- Attention items highlighting issues requiring follow-up

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@tekgrunt tekgrunt changed the title Simplify reporting templates to billing-only Refactor reporting skill to use template-based architecture Dec 10, 2025
tekgrunt and others added 7 commits December 10, 2025 20:32
- Add 3 new schemas: fleet-health, detection-analytics, customer-health
- Add 3 new Jinja2 templates for visualizing report data
- Update billing-summary schema/template to match reporting skill output
- Update SKILL.md with new template documentation
- Update reporting-templates.md with execution flows for all 4 reports

Templates use schema-based decoupling - no direct dependency on reporting skill.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add explicit 3-step wizard flow with mandatory markers to prevent
the model from skipping the time period question and assuming defaults.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Problem:
The billing report had redundant sections - a summary table showing
per-org costs AND a separate expandable card section showing SKU
details. Users saw the same organization data twice.

Solution:
Consolidated into a single "Per-Organization Breakdown" section with
expandable cards that show both summary info and SKU details:

- All 25 orgs displayed in one list (sorted by cost, highest first)
- Card header shows: org name, status badge, line item count, % of total
- Click to expand reveals SKU breakdown with amounts
- Visual distinction by status:
  - Success: normal styling
  - Permission Denied: red left border, grayed out
  - No Invoice: gray left border, grayed out
- Summary footer with total org count and combined amount

Also updated render-html.py validation to expect data.tenants and
data.rollup instead of deprecated data.usage structure.

Tested with live billing data from 25 LimaCharlie organizations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace bar charts with detailed list views for better readability:
- Organization Health: Now shows full org names, health %, color-coded
  progress bars, and online/total sensor counts
- Top Detection Categories: List with severity badges, percentages,
  affected orgs/hosts counts
- Organizations by Detection Volume: List with sensor counts and
  top category info

Convert Organization Details table to expandable cards showing:
- Platform breakdown with online/offline counts per platform
- Top detections with severity badges
- Top offline sensors with days offline

Add CSS for:
- Severity badges (critical/high/medium/low/info)
- Expandable card components
- Detail list items with progress bars
- Limited access status styling

Handle null health values properly for orgs with limited permissions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add new fields to align with the refactored template:

metadata.organizations:
- Add limited_access count for orgs with partial permissions

data.organizations[]:
- Add status: "limited_access" enum value
- Add status_reason for explaining limited/error status
- Add platforms[] with per-org platform breakdown (online/offline)
- Add top_detections[] with severity and host
- Add top_offline_sensors[] with hostname, platform, days_offline

data.top_categories[]:
- Add severity (critical/high/medium/low/info)
- Add affected_orgs and affected_hosts counts

data.top_orgs_by_detections[]:
- Add top_category and sensors count

data.offline_summary (new):
- total_offline, offline_7_plus_days, offline_30_plus_days
- by_platform breakdown

Updated example to demonstrate all new fields including
limited_access org scenario.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Focus on JSON data output to templates rather than auto-opening HTML.
Users can choose to open generated reports themselves.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove invalid ternary operator in for loop filter that was causing
template rendering failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants