Skip to content

TESTING.md

Latest commit

 

History

History
286 lines (230 loc) · 11.2 KB

TESTING.md

File metadata and controls

286 lines (230 loc) · 11.2 KB
Raw

CLOVE — Module-by-Module Test Plan

Test every component, API, and tech stack integration independently before running end-to-end.

1. Venice AI

1a. Strategy Compiler

curl -X POST http://localhost:3000/api/agent/compile \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Daily check, 25 USDC budget, protect from risk, rebalance to best yield, notify Telegram"}'

Pass: Returns JSON with nodes[], edges[], summary. Nodes include risk-check and compare-apy since prompt mentions risk + rebalance. Status: ✅ PASS — 8 nodes, has risk-check and compare-apy

1b. Yield Analyst (intelligence gate)

# Expect 402 — no payment
curl http://localhost:3000/api/intelligence

# Expect yield data — demo signature
curl http://localhost:3000/api/intelligence -H "PAYMENT-SIGNATURE: demo-test"

Pass: First returns HTTP 402 with PAYMENT-REQUIRED header. Second returns { bestApy, recommended, yields, marketIntel }. Status: ✅ PASS — 402 gate works, bestApy ~9.3% on morpho

2. MongoDB (Agent Memory)

2a. Connection

curl "http://localhost:3000/api/agent/memory/prompt?wallet=0xTEST"

Pass: { "prompt": "AGENT MEMORY: No previous runs..." } — not an error. Status: ✅ PASS

2b. Save a run

curl -X POST http://localhost:3000/api/agent/memory/run \
  -H "Content-Type: application/json" \
  -d '{"walletAddress":"0xTEST","runId":"t1","success":true,"protocol":"morpho","action":"deposit","amount":"10.00","apy":9.11,"riskLevel":"LOW","txHash":null,"costPaid":0.01,"veniceReason":"Best yield","durationMs":5000,"yields":{"morpho":9.11,"sky":6.1,"aerodrome":12.3,"lido":3.8,"uniswap":0}}'

Pass: { "saved": true } Status: ✅ PASS

2c. Memory prompt reflects saved run

curl "http://localhost:3000/api/agent/memory/prompt?wallet=0xTEST"

Pass: Prompt includes "Current position: $10.00 in morpho @ 9.11%" and last run entry. Status: ✅ PASS

2d. Save workflow (via compile with walletAddress)

curl -X POST http://localhost:3000/api/agent/compile \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Hourly check, Morpho vault","walletAddress":"0xTEST"}'

Pass: Workflow saved to MongoDB workflows collection. Verify in Atlas console. Status: ✅ PASS

2e. Save schedule

curl -X POST http://localhost:3000/api/agent/schedule \
  -H "Content-Type: application/json" \
  -d '{"walletAddress":"0xTEST","enabled":true,"interval":"1hour","cron":"0 * * * *","timezone":"UTC"}'

Pass: { "ok": true }. Verify in Atlas schedules collection. Status: ✅ PASS

3. 1Shot API

3a. Session wallet address

curl http://localhost:3000/api/session/address

Pass: { "address": "0x...", "source": "1shot" } (or "local" if 1Shot creds expired) Status: ⚠️ PARTIAL — returns source: "local" because 1Shot credentials are returning 403 Forbidden. Note: The 1Shot API key/secret pair is invalid. Needs regeneration at dashboard.1shotapi.com. The fallback (source: "local") is functional for all demo operations.

3b. Store delegation (will fail without real permissionsContext — expected)

curl -X POST http://localhost:3000/api/x402/store-delegation \
  -H "Content-Type: application/json" \
  -d '{"permissionsContext":"0xDEADBEEF","expiresAt":9999999999}'

Pass: Either { "delegationId": "..." } or { "stored": false, "reason": "..." } — both are acceptable (non-fatal). Status: ✅ PASS — returns { stored: false } gracefully

4. x402 Payment Flow

4a. Full x402 pay with demo context

curl -X POST http://localhost:3000/api/x402/pay \
  -H "Content-Type: application/json" \
  -d '{"endpoint":"/api/intelligence","permissionsContext":"0xdemo","delegationManager":"0x"}'

Pass: Returns { bestApy, recommended, _clove: { paid: true, via: "..." } }. The via field shows whether 1Shot or demo was used. Status: ✅ PASS — paid: true, via: "demo", bestApy ~9.26%

5. Venice ReAct Agent (Full Loop)

curl -X POST http://localhost:3000/api/agent/run-ai \
  -H "Content-Type: application/json" \
  -d '{"walletAddress":"0x317914bc4db3f61c0cba933a3e00d7a8bed124a5","budgetUsdc":"10.00","goal":"Find best yield and deposit"}'

Pass (takes 30-90 seconds):

  • success: true
  • steps array includes: checkYields, checkRisk, executeDefi or rebalance, notifyUser
  • bestApy and protocol populated
  • Run saved to MongoDB (verify via memory prompt)
  • Telegram message received

Status: ✅ PASS — all 4 steps execute in order, ~70s runtime

Test memory-driven HOLD: Run again immediately — second run should show in terminal:

Memory: currently in morpho @ 9.11%
Venice: HOLD — already in best position

Steps should NOT include executeDefi.

Status: ⚠️ PARTIAL — memory is stored correctly. Venice sometimes still calls executeDefi even when already in Morpho (LLM non-determinism). The malformed <function=...> text output is now parsed and handled via the parseTextFunctionCall() fallback.

6. DeFi Execution (Calldata Preparation)

All 5 protocols:

# Morpho
curl -X POST http://localhost:3000/api/execute/defi \
  -H "Content-Type: application/json" \
  -d '{"action":"morpho-vault-deposit","protocol":"morpho","nodeConfig":{"amount":"10.00","platform":"Morpho","action":"deposit"},"permissionsContext":"0xdemo","delegationManager":"0x","walletAddress":"0x317914bc4db3f61c0cba933a3e00d7a8bed124a5"}'

# Sky
curl -X POST http://localhost:3000/api/execute/defi \
  -H "Content-Type: application/json" \
  -d '{"action":"sky-deposit","protocol":"sky","nodeConfig":{"amount":"10.00"},"permissionsContext":"0xdemo","delegationManager":"0x","walletAddress":"0x317914bc4db3f61c0cba933a3e00d7a8bed124a5"}'

# Lido
curl -X POST http://localhost:3000/api/execute/defi \
  -H "Content-Type: application/json" \
  -d '{"action":"lido-wrap","protocol":"lido","nodeConfig":{"amount":"10.00"},"permissionsContext":"0xdemo","delegationManager":"0x","walletAddress":"0x317914bc4db3f61c0cba933a3e00d7a8bed124a5"}'

# Uniswap
curl -X POST http://localhost:3000/api/execute/defi \
  -H "Content-Type: application/json" \
  -d '{"action":"uniswap-swap-exact-input","protocol":"uniswap","nodeConfig":{"amount":"10.00"},"permissionsContext":"0xdemo","delegationManager":"0x","walletAddress":"0x317914bc4db3f61c0cba933a3e00d7a8bed124a5"}'

# Aerodrome
curl -X POST http://localhost:3000/api/execute/defi \
  -H "Content-Type: application/json" \
  -d '{"action":"aerodrome-swap-exact-tokens","protocol":"aerodrome","nodeConfig":{"amount":"10.00"},"permissionsContext":"0xdemo","delegationManager":"0x","walletAddress":"0x317914bc4db3f61c0cba933a3e00d7a8bed124a5"}'

Pass: { "prepared": true, "calldata": "0x...", "contractAddress": "0x...", "functionName": "..." } Note: submitted: true only if 1Shot contract method UUIDs are configured in dashboard.

Status: ✅ PASS ALL 5 — All protocols return valid calldata:

  • Morpho: deposit(assets, receiver) → Moonwell USDC vault
  • Sky: deposit(assets, receiver) → sUSDS vault
  • Lido: wrap(_stETHAmount) → wstETH
  • Uniswap: exactInputSingle(params) → SwapRouter02
  • Aerodrome: swapExactTokensForTokens(amountIn, amountOutMin, routes, to, deadline) → Router

7. Telegram Notification

curl -X POST http://localhost:3000/api/notify/telegram \
  -H "Content-Type: application/json" \
  -d '{"message":"Test from CLOVE test suite — agent memory working"}'

Pass: { "sent": true } + message appears in Telegram from @clove_erc7715bot. Status: ✅ PASS

8. Schedule + Cron

8a. Cron (local test — no auth needed in dev)

curl http://localhost:3000/api/agent/cron

Pass: { "ran": 0, "reason": "No enabled schedules" } if no schedules saved, or runs agent for each enabled wallet. Status: ✅ PASS

8b. Schedule manager (browser)

  • Open http://localhost:3000/dashboard
  • Right panel → Schedule tab
  • Set "Every 5 minutes", click Enable, Save
  • Wait 5 min → agent auto-fires → terminal shows "⏰ Scheduled trigger fired"

9. Frontend (Browser tests)

9a. AI prompt in sidebar

  • Open dashboard
  • Left sidebar → "Ask AI" textarea
  • Type: "Protect my USDC during market panic"
  • Click Generate Workflow
  • Pass: Canvas shows nodes including intelligence-tavily and risk-check

9b. Protocol sidebar

  • Right panel → Protocols tab
  • Click [+ Add] on Morpho
  • Pass: New defi-lend node appears on canvas connected to last node

9c. ERC-7715 permission (MetaMask required — NO FLASK NEEDED)

  • Connect MetaMask to dashboard (regular MetaMask v12+)
  • Right panel → Properties → Grant ERC-7715 Permission
  • Pass: Terminal shows "ERC-7715 permission granted!" with context hash

9d. Code tab

  • Compile any strategy
  • Right panel → Code tab
  • Pass: TypeScript code rendered with line numbers, matches compiled nodes

9e. Runs tab

  • Run the agent
  • Pass: After completion, auto-switches to Runs tab showing step-by-step results

10. Full End-to-End (Happy Path)

  1. Open dashboard → type "Hourly rebalance to best yield" in AI prompt → Generate
  2. Canvas shows: trigger → budget → intelligence → compare-apy → defi-lend → notify
  3. Connect MetaMask → Grant ERC-7715 permission (50 USDC / 30 days)
  4. Schedule tab → Every hour → Enable → Save
  5. Hit Run
  6. Watch terminal: trigger → budget verified → x402 paid → Venice decides → calldata prepared → Telegram sent
  7. Check MongoDB Atlas: agent_runs has a new document, agent_positions shows Morpho
  8. Hit Run again: Venice says "HOLD — already in best position" (memory working)
  9. Check Telegram: two messages received

Known Limitations (Not Bugs)

  • DeFi execution is prepared not submitted: 1Shot method UUIDs not configured — calldata is correct but not sent on-chain
  • x402 with real ERC-7715: Regular MetaMask returns near-empty permissionsContext; via: "demo" is used in this case
  • 1Shot API credentials invalid: wI0kYgjSLtsoWbktVsIm3nb3v3gY3SZx returns 403. Regenerate at dashboard.1shotapi.com to enable real on-chain execution
  • Venice latency: 30-90s per agent run — acceptable for background automation, slow for live demo
  • APY data: Venice simulates yields (no live oracle) unless TAVILY_API_KEY is set
  • Venice text-mode tool calls: llama-3.3-70b occasionally outputs <function=name,{...}> as text instead of proper tool_calls. Handled via parseTextFunctionCall() fallback in the ReAct loop.

Test Results Summary (as of 2026-05-23)

Module Test Status
Venice Compiler Strategy → nodes ✅
Intelligence API 402 gate ✅
Intelligence API Yield data ✅
MongoDB Connection ✅
MongoDB Save run ✅
MongoDB Memory prompt ✅
MongoDB Save workflow ✅
MongoDB Save schedule ✅
1Shot Session address ⚠️ (local fallback)
1Shot Store delegation ⚠️ (non-fatal fail)
x402 Full pay flow ✅
Venice ReAct Full loop ✅
Morpho Calldata ✅
Sky Calldata ✅
Lido Calldata ✅
Uniswap Calldata ✅
Aerodrome Calldata ✅
Telegram Notification ✅
Cron Trigger ✅