Skip to content

Add invocation logging and 1st-class filtering to the bash tool#13

Open
cramforce wants to merge 4 commits intomainfrom
malte/invocation-log
Open

Add invocation logging and 1st-class filtering to the bash tool#13
cramforce wants to merge 4 commits intomainfrom
malte/invocation-log

Conversation

@cramforce
Copy link
Contributor

Summary

  • Add invocation logging feature to store full command output for later retrieval
  • Add output filtering to both bash and readFile tools for efficient large output handling

Motivation

When agents execute commands that produce large outputs, they often only need a portion of it (e.g., last 50 lines of a build log, grep for errors). Currently, the full output is returned and truncated if too large, losing potentially useful data.

This feature allows agents to:

  1. Request filtered output (e.g., tail -50, grep -i error) to reduce token usage
  2. Store the full unfiltered output in a log file
  3. Re-query the stored output with different filters via readFile

Changes

New Options (CreateBashToolOptions)

Option Type Default Description
enableInvocationLog boolean false Enable storing command invocations
invocationLogPath string .bash-tool/commands Custom path for log files

Bash Tool

  • New outputFilter parameter in schema for filtering stdout before returning
  • Writes invocation log with full output when enabled
  • Returns invocationLogPath in response when logging is enabled

ReadFile Tool

  • New outputFilter parameter that uses cat file | filter for efficient streaming
  • Auto-detects .invocation files and extracts stdout content
  • Supports filtering invocation file content with different filters

Invocation Log Format

Uses a grep/tail-friendly text format instead of JSON:

# timestamp: 2024-01-15T10:30:45.123Z
# command: npm run build
# exitCode: 1
# outputFilter: tail -50
---STDOUT---
<full stdout content here>
---STDERR---
<full stderr content here>

Example Usage

const { tools } = await createBashTool({
  enableInvocationLog: true,
});

// Agent runs a build, requesting only last 50 lines
const result = await tools.bash.execute({
  command: "npm run build",
  outputFilter: "tail -50"
});
// result.stdout = last 50 lines
// result.invocationLogPath = "/workspace/.bash-tool/commands/2024-01-15T10-30-45.123Z.invocation"

// Later, agent wants to grep for errors in the full output
const errors = await tools.readFile.execute({
  path: result.invocationLogPath,
  outputFilter: "grep -i error"
});

Test Plan

  • pnpm validate passes (lint, knip, typecheck, tests)
  • Unit tests for invocation logging in bash.test.ts
  • Unit tests for output filtering in bash.test.ts
  • Unit tests for readFile with .invocation files and filtering
  • Verify log format is grep/tail friendly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant