Skip to content

feat(usage): add ask usage#4206

Open
alex-alecu wants to merge 21 commits into
mainfrom
feat/kilo-usage-ai-mvp
Open

feat(usage): add ask usage#4206
alex-alecu wants to merge 21 commits into
mainfrom
feat/kilo-usage-ai-mvp

Conversation

@alex-alecu

Copy link
Copy Markdown
Contributor

Summary

  • Added a private Ask Usage flow for admins so they can ask questions about their own Kilo usage.
  • The flow starts a normal billable Cloud Agent session only after the admin clicks Start analysis.
  • Usage answers come from the native MCP dataset tool with short-lived admin access, strict personal scope, and trusted cards and charts instead of model-written UI.

Verification

  • Sign in as a Kilo organization admin and open /aks.
  • Click Start analysis and confirm the chat starts without asking for a repository.
  • Confirm the answer shows cards or charts from returned usage data, and ask one follow-up in the same session.
  • Sign in as a non-admin and confirm /aks is not available.

Visual Changes

N/A

Reviewer Notes

The Ask Usage session uses the public KiloMan repository only because Cloud Agent sessions still require a repository. The runtime agent is locked to the usage dataset tool, and starting a new analysis is the renewal path when the short-lived MCP token expires.

'linear',
'discord',
'app-builder',
'kilo-usage-ai',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Including web-default profiles breaks the Ask Usage isolation

Adding kilo-usage-ai to the implicit profile-resolution allowlist means these sessions now inherit the caller's default profile whenever no explicit profileId is supplied. In this flow that merges default envVars, setupCommands, mcpServers, runtimeSkills, and any additional runtimeAgents on top of the inline usage-analyst config, because profile resolution layers inline overrides instead of replacing earlier layers. A user can therefore bring extra MCP servers or switch to a non-usage agent, which undercuts the intended dataset-only boundary for this feature.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@kilo-code-bot

kilo-code-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/components/cloud-agent-next/PartRenderer.tsx 85 Ask Usage now parses kilo_usage_render_result blocks directly from assistant text, so a hallucinated or prompt-injected response can render validated-looking usage cards without ever calling the MCP dataset tool.

Fix these issues in Kilo Cloud

Files Reviewed (8 files)
  • apps/web/src/components/cloud-agent-next/AskUsageRawRenderResultCard.tsx
  • apps/web/src/components/cloud-agent-next/KiloDatasetToolCard.test.ts
  • apps/web/src/components/cloud-agent-next/KiloDatasetToolCard.tsx
  • apps/web/src/components/cloud-agent-next/PartRenderer.test.ts
  • apps/web/src/components/cloud-agent-next/PartRenderer.tsx - 1 issue
  • apps/web/src/components/cloud-agent-next/raw-tool-call-markup.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.test.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.ts
Previous Review Summaries (7 snapshots, latest commit 1267665)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 1267665)

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/components/cloud-agent-next/KiloDatasetToolCard.tsx 73 Native MCP dataset results are parsed here, but PartRenderer still only mounts KiloDatasetToolCard for flat kilo_usage_query_kilo_dataset tool parts, so Ask Usage falls back to GenericToolCard instead of rendering cards and charts.

Fix these issues in Kilo Cloud

Files Reviewed (8 files)
  • apps/web/src/components/cloud-agent-next/CloudChatPage.tsx
  • apps/web/src/components/cloud-agent-next/KiloDatasetToolCard.test.ts
  • apps/web/src/components/cloud-agent-next/KiloDatasetToolCard.tsx - 1 issue
  • apps/web/src/components/cloud-agent-next/MessageBubble.tsx
  • apps/web/src/components/cloud-agent-next/PartRenderer.test.ts
  • apps/web/src/components/cloud-agent-next/PartRenderer.tsx
  • apps/web/src/components/cloud-agent-next/raw-tool-call-markup.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.ts

Previous review (commit 833c17f)

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
dev/local/cli.ts 77 WORKTREE_ENV_OVERRIDES omits CLOUD_AGENT_NEXT_API_URL, so shared tmux sessions can still inherit a stale Cloud Agent Next endpoint from another worktree.

Fix these issues in Kilo Cloud

Files Reviewed (3 files)
  • apps/web/src/components/cloud-agent-next/ChatInput.tsx
  • apps/web/src/lib/cloud-agent-next/cloud-agent-client.ts
  • dev/local/cli.ts - 1 issue

Previous review (commit 44c4dc6)

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/routers/admin/kilo-usage-ai-router.ts 113 The prepared Ask Usage session still stores and later submits a synthetic first prompt instead of starting truly blank.

Fix these issues in Kilo Cloud

Files Reviewed (14 files)
  • apps/web/src/app/(app)/aks/KiloUsageAiContent.tsx
  • apps/web/src/app/(app)/aks/page.tsx
  • apps/web/src/app/(app)/ask/page.tsx
  • apps/web/src/app/(app)/components/PersonalAppSidebar.tsx
  • apps/web/src/components/cloud-agent-next/ChatInput.tsx
  • apps/web/src/components/cloud-agent-next/CloudChatPage.tsx
  • apps/web/src/routers/admin/kilo-usage-ai-router.test.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.ts - 1 issue
  • services/cloud-agent-next/src/session-service.test.ts
  • services/cloud-agent-next/src/session-service.ts
  • services/cloud-agent-next/wrapper/src/restore-session.test.ts
  • services/cloud-agent-next/wrapper/src/restore-session.ts
  • services/cloud-agent-next/wrapper/src/session-bootstrap.ts
  • services/cloud-agent-next/wrapper/src/utils.ts

Previous review (commit ae9e889)

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/routers/admin/kilo-usage-ai-router.ts 103 The prepared Ask Usage session still stores and later submits a synthetic first prompt instead of starting truly blank.

Fix these issues in Kilo Cloud

Files Reviewed (16 files)
  • apps/web/src/lib/cloud-agent-next/cloud-agent-client.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.test.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.ts - 1 issue
  • services/cloud-agent-next/src/persistence/CloudAgentSession.ts
  • services/cloud-agent-next/src/persistence/session-metadata.ts
  • services/cloud-agent-next/src/router/handlers/session-management.ts
  • services/cloud-agent-next/src/router/handlers/session-prepare.ts
  • services/cloud-agent-next/src/router/handlers/session-start.ts
  • services/cloud-agent-next/src/router/schemas.ts
  • services/cloud-agent-next/src/session-prepare.test.ts
  • services/cloud-agent-next/src/session-service.ts
  • services/cloud-agent-next/src/session/message-settlement-outbox.ts
  • services/cloud-agent-next/src/session/session-requests.ts
  • services/cloud-agent-next/src/shared/wrapper-bootstrap.ts
  • services/cloud-agent-next/wrapper/src/session-bootstrap.test.ts
  • services/cloud-agent-next/wrapper/src/session-bootstrap.ts

Previous review (commit 9de5345)

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/routers/admin/kilo-usage-ai-router.ts 104 The prepared Ask Usage session still stores and later submits a synthetic first prompt instead of starting truly blank.

Fix these issues in Kilo Cloud

Files Reviewed (15 files)
  • .specs/mcp-gateway-auth.md
  • apps/web/src/components/cloud-agent-next/MessageBubble.test.ts
  • apps/web/src/components/cloud-agent-next/MessageBubble.tsx
  • apps/web/src/components/cloud-agent-next/assistant-error-message.ts
  • apps/web/src/lib/kilo-datasets/catalog-description.test.ts
  • apps/web/src/lib/kilo-datasets/catalog-description.ts
  • apps/web/src/lib/kilo-datasets/contracts.test.ts
  • apps/web/src/lib/kilo-datasets/contracts.ts
  • apps/web/src/lib/kilo-datasets/query.test.ts
  • apps/web/src/lib/kilo-datasets/query.ts
  • apps/web/src/lib/mcp/kilo-dataset-server.protocol.test.ts
  • apps/web/src/lib/mcp/kilo-dataset-server.test.ts
  • apps/web/src/lib/mcp/kilo-dataset-server.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.test.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.ts - 1 issue

Previous review (commit 98827c5)

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/routers/admin/kilo-usage-ai-router.ts 104 The prepared Ask Usage session still stores and later submits a synthetic first prompt instead of starting truly blank.

Fix these issues in Kilo Cloud

Files Reviewed (12 files)
  • .specs/mcp-gateway-auth.md
  • apps/web/src/lib/kilo-datasets/catalog-description.test.ts
  • apps/web/src/lib/kilo-datasets/catalog-description.ts
  • apps/web/src/lib/kilo-datasets/contracts.test.ts
  • apps/web/src/lib/kilo-datasets/contracts.ts
  • apps/web/src/lib/kilo-datasets/query.test.ts
  • apps/web/src/lib/kilo-datasets/query.ts
  • apps/web/src/lib/mcp/kilo-dataset-server.protocol.test.ts
  • apps/web/src/lib/mcp/kilo-dataset-server.test.ts
  • apps/web/src/lib/mcp/kilo-dataset-server.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.test.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.ts - 1 issue

Previous review (commit 2df5ec6)

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
services/cloud-agent-next/src/router/handlers/session-prepare.ts 58 Implicit web-default profile resolution widens the Ask Usage session beyond the intended dataset-only scope.

Fix these issues in Kilo Cloud

Files Reviewed (42 files)
  • .specs/mcp-gateway-auth.md
  • apps/web/package.json
  • apps/web/src/app/(app)/aks/KiloUsageAiContent.tsx
  • apps/web/src/app/(app)/aks/page.tsx
  • apps/web/src/app/(app)/components/PersonalAppSidebar.tsx
  • apps/web/src/app/.well-known/oauth-authorization-server/oauth/authorize/route.ts
  • apps/web/src/app/.well-known/oauth-authorization-server/route.ts
  • apps/web/src/app/.well-known/oauth-protected-resource/mcp/route.ts
  • apps/web/src/app/.well-known/oauth-protected-resource/route.ts
  • apps/web/src/app/api/mcp-gateway/oauth/authorize/route.test.ts
  • apps/web/src/app/api/mcp-gateway/oauth/authorize/route.ts
  • apps/web/src/app/api/mcp-gateway/oauth/token/route.test.ts
  • apps/web/src/app/api/mcp-gateway/oauth/token/route.ts
  • apps/web/src/app/mcp/route.test.ts
  • apps/web/src/app/mcp/route.ts
  • apps/web/src/components/cloud-agent-next/CloudChatPage.tsx
  • apps/web/src/components/cloud-agent-next/KiloDatasetToolCard.test.ts
  • apps/web/src/components/cloud-agent-next/KiloDatasetToolCard.tsx
  • apps/web/src/components/cloud-agent-next/PartRenderer.tsx
  • apps/web/src/lib/kilo-datasets/contracts.ts
  • apps/web/src/lib/kilo-datasets/query.test.ts
  • apps/web/src/lib/kilo-datasets/query.ts
  • apps/web/src/lib/mcp-gateway/services.ts
  • apps/web/src/lib/mcp-gateway/token-service.ts
  • apps/web/src/lib/mcp/kilo-dataset-server.ts
  • apps/web/src/lib/native-mcp/oauth/native-authorization-service.ts
  • apps/web/src/lib/native-mcp/oauth/native-token-service.ts
  • apps/web/src/lib/native-mcp/oauth/native-token-verifier.test.ts
  • apps/web/src/lib/native-mcp/oauth/native-token-verifier.ts
  • apps/web/src/lib/user/index.test.ts
  • apps/web/src/lib/user/index.ts
  • apps/web/src/routers/admin-router.ts
  • apps/web/src/routers/admin/kilo-usage-ai-router.ts
  • kilo.json
  • packages/db/src/migrations/0167_perpetual_hulk.sql
  • packages/db/src/schema.ts
  • packages/mcp-gateway/src/index.test.ts
  • packages/mcp-gateway/src/index.ts
  • packages/mcp-gateway/src/native-resource.ts
  • scripts/verify-drizzle-bootstrap.sh
  • services/cloud-agent-next/src/router/handlers/session-prepare.ts
  • services/cloud-agent-next/src/session-prepare.test.ts

Generated files skipped from issue scanning per review rules: packages/db/src/migrations/meta/0167_snapshot.json, packages/db/src/migrations/meta/_journal.json, pnpm-lock.yaml.


Reviewed by gpt-5.4-20260305 · Input: 101.7K · Output: 11.1K · Cached: 570.4K

Review guidance: REVIEW.md from base branch main

Make the usage-cost convenience tool strict-compatible for common total-cost prompts so GPT can complete with one handler call. Route advanced cost shapes through query_kilo_dataset and cover real MCP schema serialization.
…nto feat/kilo-usage-ai-mvp

# Conflicts:
#	apps/web/src/lib/kilo-datasets/contracts.ts
gitUrl: KILO_USAGE_AI_REPOSITORY_URL,
mode: 'usage-analyst',
model: KILO_USAGE_AI_MODEL,
prompt: blankUsageAnalysisPrompt,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: This still seeds the session with a synthetic first prompt

prepareSession stores prompt as the prepared initial turn, and the later initiateFromPreparedSession step submits that stored turn. With autoInitiate: false, this change does not create a truly blank Ask Usage chat; it just replaces the old overview prompt with "Blank Ask Usage session. Wait for the user to ask a question.", so the first queued message is still bot-authored instead of the admin's actual question.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.


const input = toolPart.state.input;
return (
input.server_name === KILO_DATASET_MCP_SERVER_NAME &&

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Native MCP dataset results still will not render as usage cards

resolveKiloDatasetToolView() now accepts tool === 'mcp', but PartRenderer still only mounts KiloDatasetToolCard for part.tool === KILO_DATASET_TOOL_NAME. In the Ask Usage flow that means the new native MCP query result still falls back to GenericToolCard, so the host never shows the validated cards and charts this branch is trying to restore.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

part: Extract<Part, { type: 'text' }>;
suppressRawToolCallText?: boolean;
}) {
const rawUsageRenderResults = suppressRawToolCallText

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: This reintroduces model-authored charts as trusted host UI

extractRawUsageRenderResults(part.text) runs on every Ask Usage text part, so any assistant response that emits a <function_result><invoke name="kilo_usage_render_result">... block will render a usage card even though kilo_usage_render_result is not a real tool. A hallucinated or prompt-injected text response can therefore fabricate validated-looking charts without ever calling the MCP dataset tool, which defeats the safety boundary this flow is trying to restore.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant