Skip to content

Commit 8630be1

Browse files
f3rdyf3rdy
committed
Add Anthropic Provider test guide
1 parent ac5afe3 commit 8630be1

1 file changed

Lines changed: 250 additions & 0 deletions

File tree

‎ANTHROPIC_TEST.md‎

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
# Anthropic Provider Test Guide
2+
3+
## Setup (Once)
4+
5+
### 1. Get API Key
6+
1. Go to https://console.anthropic.com/settings/keys
7+
2. Click "Create API Key"
8+
3. Name it "WorkshopForge"
9+
4. Copy the key (starts with `sk-ant-...`)
10+
5. Purchase initial credits (e.g. $5, $20, $50)
11+
12+
### 2. Set Environment Variable
13+
```bash
14+
# Add to your ~/.bashrc or ~/.zshrc for persistence
15+
export ANTHROPIC_API_KEY=sk-ant-your-key-here
16+
17+
# Or set for current session only
18+
export ANTHROPIC_API_KEY=sk-ant-...
19+
```
20+
21+
### 3. Verify Setup
22+
```bash
23+
cd ~/git/cdds/workshopforge
24+
uv run workshopforge ai plan --provider anthropic --goal "test" 2>&1 | head -20
25+
```
26+
27+
**Expected:** Should NOT show "ANTHROPIC_API_KEY environment variable not set"
28+
29+
---
30+
31+
## Test 1: Simple Plan Generation
32+
33+
```bash
34+
cd ~/git/cdds/lab-terraform-basics
35+
36+
# Generate plan for theory module
37+
workshopforge ai plan \
38+
--provider anthropic \
39+
--goal "Generate Marp slides for terraform-basics module based on spec objectives and legacy slides"
40+
```
41+
42+
**Expected Output:**
43+
- Plan displayed in terminal
44+
- Log created in `ai_logs/<timestamp>-<goal>/`
45+
- Files: `plan.json`, `prelude.txt`, `prompt.txt`, `response.txt`
46+
47+
**Verify:**
48+
```bash
49+
ls -lh ai_logs/
50+
cat ai_logs/*/response.txt | head -50
51+
```
52+
53+
---
54+
55+
## Test 2: Apply Plan (Generate Content)
56+
57+
```bash
58+
cd ~/git/cdds/lab-terraform-basics
59+
60+
# IMPORTANT: Make sure you're in lab-terraform-basics, NOT workshopforge!
61+
62+
# Generate slides
63+
workshopforge ai apply \
64+
--provider anthropic \
65+
--goal "Generate Marp slides for terraform-basics module. Create out/instructor/instructor/slides/day1-terraform-basics.md with:
66+
1. Title slide with module info
67+
2. Section: What is Infrastructure as Code?
68+
3. Section: Why Terraform?
69+
4. Section: HCL Syntax Basics
70+
5. Section: Terraform Workflow (init/plan/apply/destroy)
71+
6. Section: State Management Intro
72+
Use Marp syntax with --- separators. Reference spec/modules.yml objectives."
73+
```
74+
75+
**Expected:**
76+
- Plan generated
77+
- Policy checks run
78+
- Content written to `out/instructor/instructor/slides/day1-terraform-basics.md`
79+
- Compliance report in `reports/compliance.{json,md}`
80+
81+
**Verify:**
82+
```bash
83+
# Check if slide file was created
84+
ls -lh out/instructor/instructor/slides/
85+
86+
# View first 50 lines
87+
head -50 out/instructor/instructor/slides/day1-terraform-basics.md
88+
89+
# Check compliance
90+
cat reports/compliance.md
91+
```
92+
93+
---
94+
95+
## Test 3: Compliance Check
96+
97+
```bash
98+
cd ~/git/cdds/lab-terraform-basics
99+
100+
# Run compliance check
101+
workshopforge ai check --target-dir out/instructor
102+
```
103+
104+
**Expected:**
105+
- Policy engine runs
106+
- Report generated in `reports/compliance.{json,md}`
107+
- Shows any violations (missing deliverables, guideline issues)
108+
109+
**Verify:**
110+
```bash
111+
cat reports/compliance.md
112+
```
113+
114+
---
115+
116+
## Test 4: Generate Lab Content
117+
118+
```bash
119+
cd ~/git/cdds/lab-terraform-basics
120+
121+
# Generate instructor notes for existing lab
122+
workshopforge ai apply \
123+
--provider anthropic \
124+
--goal "Generate comprehensive instructor notes for Lab A (dev-sandbox). Create out/instructor/instructor/notes/lab-a-notes.md with:
125+
- Learning objectives recap
126+
- Common student mistakes
127+
- Troubleshooting tips
128+
- Time management (60min total)
129+
- Extension ideas for advanced students
130+
Reference existing lab files in out/instructor/labs/lab-a-dev-sandbox/"
131+
```
132+
133+
**Expected:**
134+
- Notes file created at `out/instructor/instructor/notes/lab-a-notes.md`
135+
- Content references actual lab code
136+
- Follows instructor notes structure
137+
138+
---
139+
140+
## Cost Monitoring
141+
142+
After each test, check your credit usage:
143+
1. Go to https://console.anthropic.com/settings/billing
144+
2. View "Usage" tab
145+
146+
**Estimated costs for tests above:**
147+
- Test 1 (plan only): ~$0.01-0.05
148+
- Test 2 (slide generation): ~$0.20-0.50
149+
- Test 3 (compliance check): Free (local policy engine)
150+
- Test 4 (notes generation): ~$0.30-0.80
151+
152+
**Total for all tests:** ~$0.50-1.50
153+
154+
---
155+
156+
## Troubleshooting
157+
158+
### Error: "ANTHROPIC_API_KEY environment variable not set"
159+
**Fix:** Export the key in your current shell:
160+
```bash
161+
export ANTHROPIC_API_KEY=sk-ant-...
162+
```
163+
164+
### Error: "anthropic package not installed"
165+
**Fix:** Reinstall WorkshopForge:
166+
```bash
167+
cd ~/git/cdds/workshopforge
168+
uv sync
169+
uv tool uninstall workshopforge
170+
uv tool install .
171+
```
172+
173+
### Error: "401 Unauthorized" or "403 Forbidden"
174+
**Fix:**
175+
- Verify API key is correct
176+
- Check if credits are available at https://console.anthropic.com
177+
178+
### Error: "429 Too Many Requests"
179+
**Fix:** Wait 60 seconds, API rate limit exceeded
180+
181+
### Generation quality issues
182+
**Tips:**
183+
- Be more specific in your goal statement
184+
- Reference spec files explicitly ("based on spec/modules.yml objectives")
185+
- Mention output format (Marp, Markdown, code comments)
186+
- Reference existing files to maintain consistency
187+
188+
---
189+
190+
## Next Steps After Testing
191+
192+
If all tests pass:
193+
194+
1. **Generate remaining theory modules:**
195+
```bash
196+
cd ~/git/cdds/lab-terraform-basics
197+
198+
# Variables/Outputs/State module
199+
workshopforge ai apply --provider anthropic --goal "..."
200+
201+
# Modules module
202+
workshopforge ai apply --provider anthropic --goal "..."
203+
204+
# Remote State module
205+
workshopforge ai apply --provider anthropic --goal "..."
206+
207+
# Best Practices module
208+
workshopforge ai apply --provider anthropic --goal "..."
209+
```
210+
211+
2. **Generate instructor notes for Labs B-D**
212+
213+
3. **Generate reference materials:**
214+
- Terraform workflow cheatsheet
215+
- Variable types reference
216+
- Backend configuration guide
217+
- Security checklist
218+
- Troubleshooting guide
219+
- Cost optimization tips
220+
221+
4. **Build PDFs:**
222+
```bash
223+
make slides
224+
```
225+
226+
5. **Create student version:**
227+
```bash
228+
workshopforge promote out/instructor out/student
229+
```
230+
231+
---
232+
233+
## Success Criteria
234+
235+
✅ **WorkshopForge + Anthropic is working if:**
236+
- `ai plan` shows intelligent, context-aware plans
237+
- `ai apply` generates high-quality content
238+
- Generated content follows spec objectives
239+
- Compliance checks pass (or violations are expected)
240+
- Content is usable without major manual editing
241+
242+
---
243+
244+
**Happy Testing!** 🚀
245+
246+
If you encounter issues, check:
247+
1. API key is set correctly
248+
2. Credits are available
249+
3. You're in the correct directory (lab-terraform-basics, not workshopforge)
250+
4. Specs are valid (`workshopforge validate`)

0 commit comments

Comments
 (0)