diff --git a/migration-to-aws/POWER.md b/migration-to-aws/POWER.md index f3f14587..e24fc026 100644 --- a/migration-to-aws/POWER.md +++ b/migration-to-aws/POWER.md @@ -1,21 +1,20 @@ --- name: "migration-to-aws" displayName: "GCP to AWS Migration Advisor" -description: "Expert guidance for migrating workloads from Google Cloud Platform to AWS. This no-cost tool assesses your current cloud provider's usage, geography, and billing data to estimate and compare AWS services and pricing, and recommends migration or continued use of your current provider. AWS pricing is based on current published pricing and may vary over time. The tool may generate a .migration folder containing comparison and migration execution data, which you may delete upon completion or use to migrate to AWS." -keywords: ["gcp", "aws", "migration", "cloud migration", "terraform", "re-platform", "cost estimation", "architecture"] +description: "Migrate workloads from Google Cloud Platform to AWS — including AI and agentic workloads regardless of cloud provider. Triggers on: migrate from GCP, GCP to AWS, move off Google Cloud, migrate Terraform to AWS, migrate Cloud SQL to RDS, migrate GKE to EKS, migrate Cloud Run to Fargate, Google Cloud migration, migrate from OpenAI to Bedrock, move off OpenAI, switch from ChatGPT API to AWS, migrate from Gemini to Bedrock, migrate LangChain to Bedrock, migrate LangGraph to AWS, migrate agentic workloads to AWS, move AI workloads to AWS, migrate my AI app to AWS. Runs a 6-phase process: discover GCP resources from Terraform files, app code, or billing exports, clarify migration requirements, design AWS architecture, estimate costs, generate migration artifacts, and collect optional feedback. Clarify must finish before Design, Estimate, or Generate. Includes AI provider migration guidance (for example, OpenAI to Amazon Bedrock) by selecting closest-fit Bedrock model families for required modality, latency/quality targets, context windows, and cost constraints. Model mapping is compatibility-guided, not 1:1 parity; validate prompts, tool-calling behavior, and eval metrics before cutover. Do not use for: Azure or on-premises migrations to AWS, AWS-to-GCP reverse migration, general AWS architecture advice without migration intent, GCP-to-GCP refactoring, or multi-cloud deployments that do not involve migrating off GCP." +keywords: ["gcp", "aws", "migration", "cloud migration", "terraform", "re-platform", "cost estimation", "architecture", "bedrock", "openai", "gemini", "anthropic"] author: "AWS" --- # GCP-to-AWS Migration Advisor -Migrate workloads from Google Cloud Platform to AWS with a 5-phase guided process: Terraform infrastructure discovery, requirements clarification, AWS architecture design, cost estimation, and execution planning. This no-cost tool assesses your current cloud provider's usage, geography, and billing data to estimate and compare AWS services and pricing, and recommends migration or continued use of your current provider. AWS pricing is based on current published pricing and may vary over time. The tool may generate a .migration folder containing comparison and migration execution data, which you may delete upon completion or use to migrate to AWS. - - ## Philosophy - **Re-platform by default**: Select AWS services that match GCP workload types (e.g., Cloud Run → Fargate, Cloud SQL → RDS). - **Dev sizing unless specified**: Default to development-tier capacity (e.g., db.t4g.micro, single AZ). Upgrade only on user direction. +- **No human one-time migration costs**: Do not present human labor, professional services, or people-time work as dollar estimates or "one-time migration cost" budget categories. Vendor charges grounded in data (for example GCP data transfer egress in the infra estimate when billing exists) are allowed. - **Multi-signal approach**: Design phase adapts based on available inputs — Terraform IaC for infrastructure, billing data for service mapping, and app code for AI workload detection. +- **BigQuery / `google_bigquery_*`**: The power **does not** recommend a specific AWS analytics or warehouse service. During **Clarify**, if discovery shows BigQuery (IaC `google_bigquery_*` and/or billing rows for BigQuery), you **must** surface the specialist advisory **before** Design (see `steering/clarify.md`). Design output uses **`Deferred — specialist engagement`**; keep directing the user to their **AWS account team** and/or a **data analytics migration partner** through Design, Estimate, and docs (see `steering/design-infra.md` BigQuery specialist gate). --- @@ -26,9 +25,50 @@ Migrate workloads from Google Cloud Platform to AWS with a 5-phase guided proces --- +## Context Loading Rules + +Each phase loads reference files on demand. To keep per-turn context manageable and prevent instruction-following degradation: + +- **Budget:** Each phase should load no more than ~800 lines of instructions (excluding user artifacts like JSON profiles and MCP tool results). +- **Conditional loading:** Reference files with trigger conditions (e.g., `agentic_profile.is_agentic == true`) MUST NOT be loaded unless the condition is met. Do not speculatively load files. +- **No duplication:** Model mapping tables, pricing data, and shared warnings exist in one canonical file. Other files reference them; they do not copy them inline. +- **Progressive depth:** Phase orchestrators (`design.md`, `generate.md`) contain short routing logic that points to detailed sub-files. Load the sub-file only when its path is selected. + +**Conditional reference files (load ONLY when condition is true):** + +| File | Condition | +| ------------------------------------------------ | -------------------------------------------------------------------------------------------------- | +| `steering/design-ref-ai-gemini-to-bedrock.md` | `ai-workload-profile.json` exists AND `summary.ai_source` = `"gemini"` or `"both"` | +| `steering/design-ref-ai-openai-to-bedrock.md` | `ai-workload-profile.json` exists AND `summary.ai_source` = `"openai"` or `"both"` | +| `steering/design-ref-ai-anthropic-to-bedrock.md` | `ai-workload-profile.json` exists AND `summary.ai_source` = `"anthropic"` | +| `steering/design-ref-ai.md` | `ai-workload-profile.json` exists AND `summary.ai_source` = `"other"` | +| `steering/design-ref-harness.md` | `agentic_profile.is_agentic == true` AND `ai_constraints.agentic.migration_approach == "harness"` | +| `steering/design-ref-agentic-to-agentcore.md` | `agentic_profile.is_agentic == true` AND `ai_constraints.agentic.migration_approach == "strands"` | +| `steering/retarget-gotchas.md` | `agentic_profile.is_agentic == true` AND `ai_constraints.agentic.migration_approach == "retarget"` | + +When adding new reference files, verify the phase's total loaded instructions remain under budget. If a new file would exceed ~800 lines when combined with other loaded refs, split it or make it conditional. + +**Hybrid stack budget warning:** + +When both `gcp-resource-inventory.json` AND `ai-workload-profile.json` exist, the combined design refs will approach the ~800-line budget. Output this warning to the user **before** loading the AI design refs: + +> "⚠️ This is a large hybrid stack (infrastructure + AI workloads). To ensure complete and accurate recommendations, consider running the migration in two separate passes: +> +> **Pass 1 — Infrastructure:** Run with only your Terraform files to get infra mapping, Terraform generation, and cost estimates. +> +> **Pass 2 — AI workloads:** Run with only your application code to get Bedrock model recommendations, provider adapters, and AI migration artifacts. +> +> Continue with the combined run? (Y/N)" + +If the user chooses to continue, proceed with the combined run. Load AI refs **after** infra refs to preserve infra instruction fidelity. If the user declines, stop and instruct them to re-run with a single input source type. + +**This warning is advisory only** — it does not block the run. + +--- + ## Prerequisites -**GCP sources** — User must provide at least one: +User must provide at least one GCP source: - **Terraform IaC**: `.tf` files (with optional `.tfvars`, `.tfstate`) - **Application code**: Source files with GCP SDK or AI framework imports @@ -51,20 +91,40 @@ If none of the above are found, stop and ask user to provide at least one source This is the execution controller. After completing each phase, consult this table to determine the next action. -| Current State | Condition | Next Action | -| --------------- | --------- | ------------------------------- | -| `start` | always | Load `steering/discover.md` | -| `discover_done` | always | Load `steering/clarify.md` | -| `clarify_done` | always | Load `steering/design.md` | -| `design_done` | always | Load `steering/estimate.md` | -| `estimate_done` | always | Load `steering/generate.md` | -| `generate_done` | always | Migration planning complete | +| Current State | Condition | Next Action | +| ------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | +| `discover` | `phases.discover != "completed"` | Load `steering/discover.md` | +| `clarify` | `phases.discover == "completed"` AND `phases.clarify != "completed"` | Load `steering/clarify.md` | +| `design` | `phases.clarify == "completed"` AND `phases.design != "completed"` | Load `steering/design.md` | +| `estimate` | `phases.design == "completed"` AND `phases.estimate != "completed"` | Load `steering/estimate.md` | +| `generate` | `phases.estimate == "completed"` AND `phases.generate != "completed"` | Load `steering/generate.md` | +| `complete` | `phases.generate == "completed"` AND `phases.feedback == "pending"` | Set `phases.feedback` to `"completed"` (user had two chances), then migration complete | +| `complete` | `phases.generate == "completed"` AND `phases.feedback == "completed"` | Migration planning complete | + +**How to determine current state (deterministic):** -**How to determine current state:** Read `$MIGRATION_DIR/.phase-status.json` → check `phases` object → find the last phase with value `"completed"`. +1. Read `$MIGRATION_DIR/.phase-status.json` +2. If `current_phase` exists, use it (must match one of: discover, clarify, design, estimate, generate, complete) +3. Otherwise use ordered phase evaluation: `discover` → `clarify` → `design` → `estimate` → `generate` +4. Pick the **first** phase in that order where `phases. != "completed"`; if none, state is `complete` **Phase gate checks**: If prior phase incomplete, do not advance (e.g., cannot enter estimate without completed design). -**Feedback checkpoints**: Feedback is not a sequential phase — it is offered at two interleaved checkpoints (after Discover and after Estimate). See step 7 of **Workflow Execution** below for details. +**Clarify is mandatory:** Do not load `steering/design.md`, `steering/estimate.md`, or `steering/generate.md` unless `$MIGRATION_DIR/.phase-status.json` exists and `phases.clarify` is exactly `"completed"`. A `preferences.json` file alone is **not** sufficient proof that Clarify ran. If the user asks to skip Clarify or jump straight to Design, cost estimate, or artifact generation, refuse briefly, then load `steering/clarify.md` and run Phase 2. There is no exception for "quick" or "obvious" migrations. + +**Feedback checkpoints**: Feedback is not a sequential phase — it is offered at two interleaved checkpoints (after Discover and after Estimate). See step 8 of **Workflow Execution** below for details. + +### Handoff Gate Orchestration (Fail Closed) + +Load `steering/handoff-gates.md` when executing any phase completion step. + +1. **Single `$MIGRATION_DIR`**: Use one run directory for the entire migration. Do not mix artifacts across `.migration/*/` sessions. +2. **Re-read from disk**: Before each phase (and before each handoff gate), Read required artifacts from `$MIGRATION_DIR/`. Do not rely on chat memory. +3. **Advance only on `HANDOFF_OK`**: A phase is complete only when its orchestrator emits `HANDOFF_OK | phase= | artifacts=...`. Do not load the next phase without it. +4. **On `GATE_FAIL`**: Output the failure line(s) to the user in plain language. **Do NOT modify artifacts** to pass the gate. **Do NOT continue** to the next phase. Tell the user which phase to re-run. +5. **Re-entry**: Re-running an earlier phase after downstream phases completed requires explicit user confirmation; downstream phases must be reset to `"pending"`. See `handoff-gates.md` re-entry table. + +Generate phase additionally loads `steering/validate-artifacts.md` before writing `migration-report.html`. --- @@ -75,7 +135,10 @@ When reading `$MIGRATION_DIR/.phase-status.json`, validate before proceeding: 1. **Multiple sessions**: If multiple directories exist under `.migration/`, list them with their phase status and ask: [A] Resume latest, [B] Start fresh, [C] Cancel. 2. **Invalid JSON**: If `.phase-status.json` fails to parse, STOP. Output: "State file corrupted (invalid JSON). Delete the file and restart the current phase." 3. **Unrecognized phase**: If `phases` object contains a phase not in {discover, clarify, design, estimate, generate, feedback}, STOP. Output: "Unrecognized phase: [value]. Valid phases: discover, clarify, design, estimate, generate, feedback." -4. **Unrecognized status**: If any `phases.*` is not in {pending, in_progress, completed}, STOP. Output: "Unrecognized status: [value]. Valid values: pending, in_progress, completed." +4. **Unrecognized status**: If any `phases.*` value is not in {pending, in_progress, completed}, STOP. Output: "Unrecognized status: [value]. Valid values: pending, in_progress, completed." +5. **Invalid `current_phase`** (if present): If `current_phase` is not in {discover, clarify, design, estimate, generate, complete}, STOP. Output: "Unrecognized current_phase: [value]. Valid values: discover, clarify, design, estimate, generate, complete." +6. **Out-of-order completion**: For ordered phases [discover, clarify, design, estimate, generate], if any later phase is `"completed"` while an earlier phase is not `"completed"`, STOP. Output: "Inconsistent phase ordering detected. Reconcile `.phase-status.json` before resuming." +7. **Multiple active phases**: Across core phases {discover, clarify, design, estimate, generate}, at most one phase may be `"in_progress"`. If >1, STOP. Output: "Multiple phases are in_progress. Keep only one active phase before resuming." --- @@ -88,7 +151,8 @@ Migration state lives in `$MIGRATION_DIR` (`.migration/[MMDD-HHMM]/`), created b ```json { "migration_id": "0226-1430", - "last_updated": "2026-02-26T15:35:22-05:00", + "last_updated": "2026-02-26T15:35:22Z", + "current_phase": "design", "phases": { "discover": "completed", "clarify": "completed", @@ -101,21 +165,28 @@ Migration state lives in `$MIGRATION_DIR` (`.migration/[MMDD-HHMM]/`), created b ``` **Status values:** `"pending"` → `"in_progress"` → `"completed"`. Never goes backward. +For core phases (discover, clarify, design, estimate, generate), at most one phase may be `"in_progress"` at any time. +`current_phase` is optional but recommended; when present it is authoritative. The `.migration/` directory is automatically protected by a `.gitignore` file created in Phase 1. ---- +### Phase Status Update Protocol -## Phase Status Update Protocol +Use **read-merge-write** updates for `.phase-status.json`: -**Do not Read `.phase-status.json` before updating it.** You already know the current state because you are executing phases sequentially. Use the Write tool to write the **complete file** in the same turn as your final phase work (e.g., the output message announcing phase completion). +1. Read the current file before every update. +2. Change only the phase keys being advanced and `last_updated`. +3. Keep prior completed phases unchanged. +4. Set `current_phase` to the next deterministic phase (or `complete` after generate). +5. Write the full file in the same turn as your final phase work message. Example — after completing the Clarify phase, write `$MIGRATION_DIR/.phase-status.json` with: ```json { "migration_id": "MMDD-HHMM", - "last_updated": "2026-02-26T15:35:22-05:00", + "last_updated": "2026-02-26T15:35:22Z", + "current_phase": "design", "phases": { "discover": "completed", "clarify": "completed", @@ -127,9 +198,7 @@ Example — after completing the Clarify phase, write `$MIGRATION_DIR/.phase-sta } ``` -Replace `MMDD-HHMM` with the actual migration ID, generate the `last_updated` ISO 8601 timestamp with local timezone offset yourself, and set each phase to its correct status at that point. - -**Read `.phase-status.json` ONLY during session resume** (Step 0 of discover.md when checking for existing runs) or the feedback prerequisite check. +Replace `MMDD-HHMM` with the actual migration ID, generate the `last_updated` ISO 8601 UTC timestamp yourself, and set each phase to its correct status at that point. --- @@ -147,25 +216,14 @@ This applies to all files written during any phase, including JSON artifacts, Te ## Phase Summary Table -| Phase | Inputs | Outputs | Reference | -| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | -| **Discover** | `.tf` files, app source code, and/or billing exports (at least one required) | `gcp-resource-inventory.json`, `gcp-resource-clusters.json`, `ai-workload-profile.json`, `billing-profile.json`, `.phase-status.json` updated (outputs vary by input) | `steering/discover.md` | -| **Clarify** | Discovery artifacts (`gcp-resource-inventory.json`, `gcp-resource-clusters.json`, `ai-workload-profile.json`, `billing-profile.json` — whichever exist) | `preferences.json`, `.phase-status.json` updated | `steering/clarify.md` | -| **Design** | `preferences.json` + discovery artifacts | `aws-design.json` (infra), `aws-design-ai.json` (AI), `aws-design-billing.json` (billing-only) | `steering/design.md` | -| **Estimate** | `aws-design.json` or `aws-design-billing.json` or `aws-design-ai.json`, `preferences.json` | `estimation-infra.json` or `estimation-ai.json` or `estimation-billing.json`, `.phase-status.json` updated | `steering/estimate.md` | -| **Generate** | `estimation-infra.json` or `estimation-ai.json` or `estimation-billing.json`, `aws-design.json` or `aws-design-billing.json` or `aws-design-ai.json`, `preferences.json` | `generation-infra.json` or `generation-ai.json` or `generation-billing.json` + `terraform/`, `scripts/`, `ai-migration/`, `MIGRATION_GUIDE.md`, `README.md`, `.phase-status.json` updated | `steering/generate.md` | -| **Feedback** | `.phase-status.json` (discover completed minimum), all existing migration artifacts | `feedback.json`, `trace.json`, `.phase-status.json` updated | `steering/feedback.md` | - ---- - -## Defaults - -- **IaC output**: Terraform configurations, migration scripts, AI migration code, and documentation -- **Region**: `us-east-1` (unless user specifies, or GCP region → AWS region mapping suggests otherwise) -- **Sizing**: Development tier (e.g., `db.t4g.micro` for databases, 0.5 CPU for Fargate) -- **Migration mode**: Adapts based on available inputs (infrastructure, AI, or billing-only) -- **Cost currency**: USD -- **Timeline assumption**: 8-12 weeks total +| Phase | Inputs | Outputs | Reference | +| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | +| **Discover** | `.tf` files, app source code, and/or billing exports (at least one required) | `gcp-resource-inventory.json`, `gcp-resource-clusters.json`, `ai-workload-profile.json`, `billing-profile.json`, `.phase-status.json` updated (outputs vary by input) | `steering/discover.md` | +| **Clarify** | Discovery artifacts (`gcp-resource-inventory.json`, `gcp-resource-clusters.json`, `ai-workload-profile.json`, `billing-profile.json` — whichever exist) | `preferences.json`, `.phase-status.json` updated | `steering/clarify.md` | +| **Design** | `preferences.json` + discovery artifacts | `aws-design.json` (infra), `aws-design-ai.json` (AI), `aws-design-billing.json` (billing-only) | `steering/design.md` | +| **Estimate** | `aws-design.json` or `aws-design-billing.json` or `aws-design-ai.json`, `preferences.json` | `estimation-infra.json` or `estimation-ai.json` or `estimation-billing.json`, `.phase-status.json` updated | `steering/estimate.md` | +| **Generate** | `estimation-infra.json` or `estimation-ai.json` or `estimation-billing.json`, `aws-design.json` or `aws-design-billing.json` or `aws-design-ai.json`, `preferences.json` | `generation-infra.json` or `generation-ai.json` or `generation-billing.json` + `terraform/`, `scripts/`, `ai-migration/`, `validation-report.json` (when infra route active), `MIGRATION_GUIDE.md`, `README.md`, `.phase-status.json` updated | `steering/generate.md` | +| **Feedback** | `.phase-status.json` (discover completed minimum), all existing migration artifacts | `feedback.json`, `trace.json`, `.phase-status.json` updated | `steering/feedback.md` | --- @@ -175,7 +233,7 @@ This applies to all files written during any phase, including JSON artifacts, Te - Provides `get_pricing`, `get_pricing_service_codes`, `get_pricing_service_attributes` tools - Only needed during Estimate phase. Discover and Design do not require it. -- Primary pricing source: `steering/cached-prices.md` (cached rates, ±5-10% for infra, ±15-25% for AI). MCP is secondary — used only for services not found in the cache. +- Primary pricing source: `steering/cached-prices.md` (cached 2026 rates, ±5-10% for infrastructure, ±15-25% for AI models). MCP is secondary — used only for services not found in the cache. **Recommended setup** (better accuracy): @@ -186,72 +244,10 @@ This applies to all files written during any phase, including JSON artifacts, Te --- -## Workflow Execution - -When invoked, the agent **MUST follow this exact sequence**: - -1. **Load phase status**: Read `.phase-status.json` from `.migration/*/`. - - If missing: Initialize for Phase 1 (Discover) - - If exists: Determine current phase based on phase field and status value - -2. **Determine phase to execute**: - - If status is `in_progress`: Resume that phase (read corresponding steering file) - - If status is `completed`: Advance to next phase (read next steering file) - - Phase mapping for advancement: - - discover (completed) → Execute clarify (read `steering/clarify.md`) - - clarify (completed) → Execute design (read `steering/design.md`) - - design (completed) → Execute estimate (read `steering/estimate.md`) - - estimate (completed) → Execute generate (read `steering/generate.md`) - - generate (completed) → Migration complete - -3. **Read phase reference**: Load the full steering file for the target phase. - -4. **Execute ALL steps in order**: Follow every numbered step in the steering file. **Do not skip, optimize, or deviate.** - -5. **Validate outputs**: Confirm all required output files exist with correct schema before proceeding. - -6. **Update phase status**: Use the Phase Status Update Protocol (Write tool, no Read) in the same turn as the phase's final output message. - -7. **Feedback checkpoint**: After a phase completes, check if feedback should be offered. This runs **before** advancing to the next phase. - - - **After Discover** (if `phases.feedback` is `"pending"`): Output to user: - "Would you like to share quick feedback (5 optional questions + anonymized usage data) to help improve this tool? Your data never includes resource names, file paths, or account IDs. - [A] Send feedback now - [B] Wait until after the Estimate phase" - - If user picks **A** → Load `steering/feedback.md`, execute it, then continue to Clarify. - - If user picks **B** → Continue to Clarify (feedback stays `"pending"`). - - - **After Estimate** (if `phases.feedback` is `"pending"`): Output to user: - "Would you like to share quick feedback now? (5 optional questions + anonymized usage data) - [A] Yes, share feedback - [B] No thanks, continue to Generate" - - If user picks **A** → Load `steering/feedback.md`, execute it, then continue to Generate. - - If user picks **B** → Set `phases.feedback` to `"completed"`, update `last_updated`. Continue to Generate. - - - **After Generate**: No feedback offer. If `phases.feedback` is still `"pending"`, set it to `"completed"` and update `last_updated` (user had two chances and chose to defer/skip). - -8. **Display summary**: Show user what was accomplished, highlight next phase, or confirm migration completion. - -**Critical constraint**: Agent must strictly adhere to the steering file's workflow. If unable to complete a step, stop and report the exact step that failed. - -User can invoke the power again to resume from last completed phase. - ---- - -## Error Conditions - -| Condition | Action | -| ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | -| No GCP sources found (no `.tf`, no app code, no billing data) | Stop. Output: "No GCP sources detected. Provide at least one source type (Terraform files, application code, or billing exports) and try again." | -| `.phase-status.json` missing phase gate | Stop. Output: "Cannot enter Phase X: Phase Y-1 not completed. Start from Phase Y or resume Phase Y-1." | -| awspricing unavailable after 3 attempts | Display user warning about ±5-25% accuracy. Use `steering/cached-prices.md`. Add `pricing_source: "cached"` to estimation.json. | -| User skips questions or says "use all defaults" | Apply documented defaults from each category file. Phase 2 completes either way. | -| `aws-design.json` missing required clusters | Stop Phase 4. Output: "Re-run Phase 3 to generate missing cluster designs." | - ---- - ## Files in This Power +Kiro powers require a **flat** `steering/` directory — nested subdirectories are not loaded. All reference files live directly under `steering/`. + ``` migration-to-aws/ ├── POWER.md ← You are here (orchestrator + state machine) @@ -268,14 +264,15 @@ migration-to-aws/ ├── feedback.md # Phase 6: Feedback orchestrator │ ├── # Discover sub-files - ├── discover-iac.md # IaC discovery (Terraform/K8s/Docker) + ├── discover-iac.md # Terraform/IaC discovery ├── discover-app-code.md # App code discovery (SDK imports, AI detection) ├── discover-billing.md # Billing data discovery + ├── discover-preview.md # Discovery preview for user review │ ├── # Clarify sub-files ├── clarify-global.md # Category A: Global/Strategic (Q1-Q7) ├── clarify-compute.md # Categories B+C: Config Gaps + Compute (Q8-Q11) - ├── clarify-database.md # Category D: Database (Q12-Q13) + ├── clarify-database.md # Category D: Database (Q12–Q13b) ├── clarify-ai.md # Category F: AI/Bedrock (Q14-Q22) ├── clarify-ai-only.md # Standalone AI-only migration flow │ @@ -298,6 +295,7 @@ migration-to-aws/ ├── generate-artifacts-ai.md # Provider adapter + test harness ├── generate-artifacts-billing.md # Skeleton Terraform with TODO markers ├── generate-artifacts-docs.md # MIGRATION_GUIDE.md + README.md + ├── generate-artifacts-report.md # migration-report.html │ ├── # Feedback sub-files ├── feedback-trace.md # Anonymized trace builder @@ -306,39 +304,125 @@ migration-to-aws/ ├── design-ref-index.md # Lookup table: GCP type → design-ref file ├── design-ref-fast-path.md # Deterministic 1:1 mappings (Pass 1) ├── design-ref-compute.md # Compute mappings (Cloud Run, GCE, GKE, App Engine) - ├── design-ref-database.md # Database mappings (Cloud SQL, Firestore, BigQuery, Redis) - ├── design-ref-storage.md # Storage mappings (GCS → S3) + ├── design-ref-database.md # Database mappings (Cloud SQL, Spanner, Firestore, Redis) + ├── design-ref-storage.md # Storage mappings (GCS, Filestore) ├── design-ref-networking.md # Networking mappings (VPC, LB, DNS, Interconnect) ├── design-ref-messaging.md # Messaging mappings (Pub/Sub, Cloud Tasks) - ├── design-ref-ai.md # AI/ML mappings (traditional ML → SageMaker) + ├── design-ref-security.md # Security baseline (GuardDuty, CloudTrail, IMDSv2) + ├── design-ref-ai.md # AI/ML mappings (Vertex AI → Bedrock) ├── design-ref-ai-gemini-to-bedrock.md # Gemini → Bedrock model selection guide ├── design-ref-ai-openai-to-bedrock.md # OpenAI → Bedrock model selection guide + ├── design-ref-ai-anthropic-to-bedrock.md # Anthropic → Bedrock model selection guide + ├── design-ref-harness.md # AgentCore Harness migration path + ├── design-ref-agentic-to-agentcore.md # Agentic → AgentCore / Strands path │ ├── # Clustering algorithm files ├── clustering-classification-rules.md # Primary/secondary classification rules - ├── clustering-algorithm.md # Cluster formation rules (6 priority rules) + ├── clustering-algorithm.md # Cluster formation rules ├── depth-calculation.md # Topological depth (Kahn's algorithm) ├── typed-edges-strategy.md # Edge type assignment (HCL reference parsing) │ - ├── # Schema files + ├── # Shared / cross-phase ├── schema-phase-status.md # .phase-status.json schema ├── schema-discover-iac.md # gcp-resource-inventory + clusters schemas ├── schema-discover-ai.md # ai-workload-profile schema ├── schema-discover-billing.md # billing-profile schema ├── schema-estimate-infra.md # estimation-infra.json schema + ├── handoff-gates.md # Fail-closed handoff protocol (GATE_FAIL / HANDOFF_OK) + ├── validate-artifacts.md # Pre-report validation (Generate Step 0; read-only) + ├── terraform-validation.md # Terraform artifact validation rules + ├── migration-complexity.md # Complexity tier definitions (small/medium/large) + ├── cached-prices.md # Cached AWS + source provider pricing (±5-25%, primary) + ├── pricing-fallback.md # MCP fallback rules when cache missing + ├── bedrock-quotas.md # Bedrock TPM/RPM quota awareness + ├── ai-migration-guardrails.md # AI migration safety rails + ├── ai-model-lifecycle.md # Bedrock model lifecycle & deprecation policy + ├── retarget-gotchas.md # Retarget path pitfalls (agentic) │ - ├── # Pricing files - └── cached-prices.md # Pre-fetched live AWS pricing (±5-10%, primary) + ├── # Data + └── sdk-capability-map.json # Deterministic SDK method → AI capability map ``` --- +## Error Conditions + +| Condition | Action | +| ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | +| No GCP sources found (no `.tf`, no app code, no billing data) | Stop. Output: "No GCP sources detected. Provide at least one source type (Terraform files, application code, or billing exports) and try again." | +| `.phase-status.json` missing phase gate | Stop. Output: "Cannot enter Phase X: Phase Y-1 not completed. Start from Phase Y or resume Phase Y-1." | +| awspricing unavailable after 3 attempts | Display user warning about ±5-25% accuracy. Use `cached-prices.md`. Add `pricing_source: "cached_fallback"` to the applicable `estimation-*.json` file. | +| User skips questions or says "use defaults for the rest" | Apply documented defaults for remaining questions in the current batch and all subsequent batches. Phase 2 completes either way. | +| `aws-design.json` missing required clusters | Stop Phase 4. Output: "Re-run Phase 3 to generate missing cluster designs." | + +--- + +## Defaults + +- **IaC output**: Terraform configurations, migration scripts, AI migration code, and documentation +- **Region**: `us-east-1` (unless user specifies, or GCP region → AWS region mapping suggests otherwise) +- **Sizing**: Development tier (e.g., `db.t4g.micro` for databases, 0.5 CPU for Fargate) +- **Migration mode**: Adapts based on available inputs (infrastructure, AI, or billing-only) +- **Cost currency**: USD +- **Timeline assumption**: 2-16 weeks depending on migration complexity — small (2-6 weeks), medium (6-12 weeks), large (12-18 weeks). See `steering/migration-complexity.md` for tier definitions. + +--- + +## Workflow Execution + +When invoked, the agent **MUST follow this exact sequence**: + +1. **Load phase status**: Read `.phase-status.json` from `.migration/*/`. + - If missing: Initialize for Phase 1 (Discover) + - If exists: Determine current phase using deterministic rules in **State Machine** + +2. **Determine phase to execute**: + - If `current_phase` exists: execute that phase. + - Otherwise execute the first non-completed phase in ordered list: discover → clarify → design → estimate → generate. + - If all ordered phases are completed: migration is complete (with feedback finalization rule). + +3. **Read phase reference**: Load the full reference file for the target phase. + +4. **Execute ALL steps in order**: Follow every numbered step in the reference file. **Do not skip, optimize, or deviate.** + +5. **Validate outputs**: Confirm all required output files exist with correct schema before proceeding. Phase orchestrators run **Completion Handoff Gate** checks per `steering/handoff-gates.md`. + +6. **Handoff gate**: Emit `HANDOFF_OK` or `GATE_FAIL` per `steering/handoff-gates.md`. On `GATE_FAIL`, stop — do not update phase status or load the next phase. + +7. **Update phase status**: Only after `HANDOFF_OK`. Use the Phase Status Update Protocol (read-merge-write) in the same turn as the phase's final output message. + +8. **Feedback checkpoint**: After a phase completes, check if feedback is due (see rules below). This runs **before** advancing to the next phase. + + - **After Discover** (if `phases.feedback` is `"pending"`): Output to user: + "Would you like to share quick feedback (5 optional questions + anonymized usage data) to help improve this tool? Your data never includes resource names, file paths, or account IDs. + [A] Send feedback now + [B] Wait until after the Estimate phase" + - If user picks **A** → Load `steering/feedback.md`, execute it, then continue to Clarify. + - If user picks **B** → Continue to Clarify (feedback stays `"pending"`). + + - **After Estimate** (if `phases.feedback` is `"pending"`): Output to user: + "Would you like to share quick feedback now? (5 optional questions + anonymized usage data) + [A] Yes, share feedback + [B] No thanks, continue to Generate" + - If user picks **A** → Load `steering/feedback.md`, execute it, then continue to Generate. + - If user picks **B** → Use the Phase Status Update Protocol to set `phases.feedback` to `"completed"`. Continue to Generate. + + - **After Generate**: No feedback offer. If `phases.feedback` is still `"pending"`, use the Phase Status Update Protocol to set it to `"completed"` (user had two chances and chose to defer/skip). + +9. **Display summary**: Show user what was accomplished, highlight next phase, or confirm migration completion. + +**Critical constraint**: Agent must strictly adhere to the reference file's workflow. If unable to complete a step, stop and report the exact step that failed. + +User can invoke the power again to resume from `current_phase` (or deterministic ordered evaluation when `current_phase` is absent). + +--- + ## Scope Notes **v1.0 includes:** - Terraform infrastructure discovery -- App code scanning (AI workload detection — Gemini, OpenAI, and other providers) +- App code scanning (AI workload detection — Gemini, OpenAI, Anthropic, and other providers) - Billing data import from GCP - User requirement clarification (adaptive questions by category) - Multi-path Design (infrastructure, AI workloads, billing-only fallback) diff --git a/migration-to-aws/steering/ai-migration-guardrails.md b/migration-to-aws/steering/ai-migration-guardrails.md new file mode 100644 index 00000000..99dfdcec --- /dev/null +++ b/migration-to-aws/steering/ai-migration-guardrails.md @@ -0,0 +1,114 @@ +# AI Migration Guardrails + +Shared warnings and constraints for all agentic migration paths. Loaded once by `design-ai.md` when `agentic_profile.is_agentic == true`. Path-specific design references (Harness, Strands, retarget) should NOT duplicate these — reference this file instead. + +--- + +## AgentCore Regional Availability + +AgentCore services have different regional footprints. Always validate via `get_regional_availability` from the `awsknowledge` MCP server before recommending. + +**As of May 2026:** + +| Service | Availability | Regions | +| --------------------------- | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| AgentCore Runtime (GA) | 15 regions | us-east-1, us-east-2, us-west-2, us-west-1, ap-southeast-1, ap-southeast-2, ap-northeast-1, ap-northeast-2, ap-south-1, eu-central-1, eu-west-1, eu-west-2, eu-north-1, sa-east-1, ca-central-1 | +| AgentCore Harness (Preview) | 4 regions | us-west-2, us-east-1, ap-southeast-2, eu-central-1 | +| AgentCore Memory (GA) | 15 regions | Same as Runtime | +| AgentCore Gateway (GA) | 15 regions | Same as Runtime | + +**IMPORTANT:** These lists go stale. The `get_regional_availability` MCP call is the source of truth. Use the table above only as a fallback if the MCP call fails. + +**If target region is unavailable for a recommended service:** + +1. Flag prominently in `aws-design-ai.json` → `regional_warnings[]` +2. Suggest nearest available region as alternative +3. Note in user summary: "[Service] is not yet available in [target region]. Nearest available: [alternative]." + +--- + +## Bedrock Mantle Throughput Limits (Shared Account) + +Bedrock Mantle provides OpenAI-compatible endpoints on Bedrock. It runs on a **shared account limit of 10,000 RPM** across all Mantle users in a region — this is not a per-customer quota. + +**Risk table:** + +| Workload Volume | Risk Level | Guidance | +| ---------------------- | ---------- | ---------------------------------------------------------------------------------------- | +| Low (< 100 RPM) | Low | Mantle is a good fit; shared limit is not a concern | +| Medium (100–1,000 RPM) | Medium | Monitor for 429s at peak; have a fallback ready | +| High (> 1,000 RPM) | High | Use `bedrock-runtime` (Converse API) directly — not subject to the shared Mantle RPM cap | + +**When to use `bedrock-runtime` instead of Mantle:** + +- Production workloads with sustained high request rates +- Latency-sensitive workloads where shared-limit throttling is unacceptable +- Workloads that need per-customer quota increases via Service Quotas + +**Source:** [AWS Bedrock Mantle scaling throughput best practices](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) + +--- + +## Bedrock Mantle TPM Limits (Claude Models) + +Claude models on Mantle have an additional **output TPM cap** that differs by model generation: + +| Model Family | Output TPM Cap | Notes | +| ----------------------- | ----------------------------- | --------------------------------- | +| Claude 4.7+ | 2,000,000 output TPM | Per-model cap applies | +| All other Claude models | No per-model output TPM limit | Standard account TPM limits apply | + +**Impact for migration decisions:** + +- For Claude migrations at medium/high volume: the 2M output TPM cap on Claude 4.7+ is the binding constraint, not the 10K RPM limit +- For gpt-oss migrations (OpenAI model architecture on Bedrock): check whether the target model is Claude 4.7+ and flag the output TPM cap in the design +- When output-heavy workloads (long JSON, tool outputs, multi-step reasoning) are detected, flag this cap prominently and recommend `bedrock-runtime` for production + +--- + +## AgentCore Harness Preview Caveats + +- Harness is in **public preview** — not GA. Production workloads should evaluate stability. +- No separate Harness charge — pay only for underlying AgentCore capabilities (Runtime, Memory, Gateway). +- Harness is powered by Strands Agents internally. Custom orchestration can switch from config-based to code-defined harness without rearchitecting. +- Harness supports Bedrock, OpenAI, and Google Gemini models. Third-party API keys stored in AgentCore Identity token vault. + +--- + +## Model Lifecycle Checks + +Before recommending any Bedrock model in an agentic design: + +1. Check `steering/ai-model-lifecycle.md` for model status +2. Do NOT recommend Legacy models as primary selections +3. If a model is approaching EOL, note the date and suggest the Active successor + +--- + +## Pricing Source Rules + +For agentic workload cost estimation: + +1. **Primary:** `steering/cached-prices.md` (±5-10% accuracy) +2. **Secondary:** `awspricing` MCP server (±5-10%, real-time) +3. **Tertiary:** `steering/pricing-fallback.md` (±15-25%, broad coverage) + +AgentCore Runtime and Harness pricing: consumption-based, no upfront cost. Include in estimate only if the user selects Harness or Strands path. + +--- + +## Effort Estimation Rules + +Do NOT output fixed week estimates for agentic migrations. Output ranges with drivers: + +**Format:** "[low]–[high] weeks depending on [driver 1] ([value]), [driver 2] ([value]), [driver 3] ([value])" + +**Drivers to include:** + +- Agent count (from `agentic_profile.agent_count`) +- Tool count (from `agentic_profile.tool_count`) +- Orchestration complexity (from `agentic_profile.orchestration_pattern`) +- Framework familiarity (team's experience with target framework) +- Test coverage (existing tests reduce migration risk) + +**Example:** "2–5 weeks depending on agent count (3), tool count (8), and graph complexity (hierarchical with conditional routing)" diff --git a/migration-to-aws/steering/ai-model-lifecycle.md b/migration-to-aws/steering/ai-model-lifecycle.md new file mode 100644 index 00000000..81b0b520 --- /dev/null +++ b/migration-to-aws/steering/ai-model-lifecycle.md @@ -0,0 +1,137 @@ +# Bedrock Model Lifecycle Awareness + +Reference: [Amazon Bedrock Model Lifecycle](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html) + +Models on Bedrock move through three states: **Active** → **Legacy** (minimum 6 months before EOL) → **End-of-Life (EOL)**. After EOL, the model is unavailable and requests fail. + +For models with EOL dates after February 1, 2026, a **public extended access** period begins at least 3 months into the Legacy state. During this period pricing may increase at the model provider's discretion. + +--- + +## Lifecycle States (Not the Same Thing) + +| State | What it means | Usable? | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------- | +| **Active** | Provider is actively maintaining the model. Full feature access. | Yes | +| **Legacy** | Deprecated. Still works for existing users, but new Provisioned Throughput cannot be created, new customers cannot onboard, pricing may increase during public extended access, and the model is on a countdown to removal. | Yes, with restrictions | +| **EOL** | Model is removed. All inference requests fail. | **No** | + +**Legacy does not mean unavailable** — it means the model still functions today but has a firm expiration date. EOL means unavailable. + +--- + +## Selection Rules + +### Rule 1: Active models only for new migrations + +**New migrations must target Active models only.** Do not recommend a Legacy or EOL model as the primary selection for any new migration, even if it is cheaper. + +### Rule 2: 90-day exclusion zone + +**Models within 90 days of their EOL date must be excluded from all recommendation and comparison tables.** A migration takes weeks or months to plan, test, and deploy. Recommending a model that will be unavailable before the migration is production-ready is harmful. + +- **Excluded** = do not list in "Best Bedrock Match" columns, tiered strategy tables, or `recommended_model` / `backup_model` fields. +- These models may still appear in the pricing cache (for reference by users already on them) but must be marked `excluded (EOL YYYY-MM-DD)` in the Status column. + +### Rule 3: Legacy models outside the 90-day zone + +Legacy models with >90 days until EOL may appear in comparison tables **with annotation** (`Legacy — EOL YYYY-MM-DD`), but never as `recommended_model` or "Best Bedrock Match" when an Active alternative exists. + +### Applying the rules + +On each run, compute `days_to_eol = EOL date − today` for every model in the Legacy/EOL table below. Then: + +1. `days_to_eol ≤ 0` → EOL. Remove from all tables. +2. `0 < days_to_eol ≤ 90` → **Exclusion zone.** Remove from recommendation/comparison tables. Mark `excluded` in pricing cache. +3. `days_to_eol > 90` and Legacy → Annotate, never recommend as primary. +4. Active → No restrictions. + +--- + +## Legacy / EOL Models (as of June 14, 2026) + +Check the [model lifecycle page](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html) for the authoritative list. The table below captures models referenced elsewhere in this plugin. **Recompute the Status column on each run** using `days_to_eol = EOL date − today`. + +| Model | Model ID | EOL Date | Days to EOL | Status | Active Replacement | +| ------------------------ | ------------------------------------------- | ------------ | ----------- | ------------ | ------------------------ | +| Titan Image Generator v2 | `amazon.titan-image-generator-v2:0` | Jun 30, 2026 | 16 | **excluded** | Stability AI (see note) | +| Llama 3.2 (all sizes) | `meta.llama3-2-*-instruct-v1:0` | Jul 7, 2026 | 23 | **excluded** | Llama 4 Scout / Maverick | +| Llama 3.1 405B Instruct | `meta.llama3-1-405b-instruct-v1:0` | Jul 7, 2026 | 23 | **excluded** | Llama 4 Maverick | +| Claude 3 Sonnet | `anthropic.claude-3-sonnet-20240229-v1:0` | Jul 30, 2026 | 46 | **excluded** | Claude Sonnet 4.5 / 4.6 | +| Claude 3.5 Sonnet v1 | `anthropic.claude-3-5-sonnet-20240620-v1:0` | Jul 30, 2026 | 46 | **excluded** | Claude Sonnet 4.5 / 4.6 | +| Claude 3.5 Sonnet v2 | `anthropic.claude-3-5-sonnet-20241022-v2:0` | Jul 30, 2026 | 46 | **excluded** | Claude Sonnet 4.5 / 4.6 | +| Command R / R+ | `cohere.command-r-v1:0` / `plus` | Aug 19, 2026 | 66 | **excluded** | — | +| Claude 3 Haiku | `anthropic.claude-3-haiku-20240307-v1:0` | Sep 10, 2026 | 88 | **excluded** | Claude Haiku 4.5 | +| Nova Premier v1 | `amazon.nova-premier-v1:0` | Sep 14, 2026 | 92 | legacy | Nova 2 Pro (Preview) | +| Nova Sonic v1 | `amazon.nova-sonic-v1:0` | Sep 14, 2026 | 92 | legacy | Nova 2 Sonic | +| Nova Canvas v1 | `amazon.nova-canvas-v1:0` | Sep 30, 2026 | 108 | legacy | Stability AI (see note) | +| Nova Reel v1 | `amazon.nova-reel-v1:0` / `v1:1` | Sep 30, 2026 | 108 | legacy | — | +| Claude Sonnet 4 | `anthropic.claude-sonnet-4-20250514-v1:0` | Oct 14, 2026 | 122 | legacy | Claude Sonnet 4.5 / 4.6 | +| Jamba 1.5 Large | `ai21.jamba-1-5-large-v1:0` | Nov 26, 2026 | 165 | legacy | — | +| Jamba 1.5 Mini | `ai21.jamba-1-5-mini-v1:0` | Nov 26, 2026 | 165 | legacy | — | + +**Status key:** `excluded` = ≤90 days to EOL, must not appear in any recommendation. `legacy` = >90 days to EOL, annotate but do not recommend as primary. + +**⚠️ Image generation — Active successor is Stability AI:** Nova Canvas v1 is Legacy (EOL Sep 30, 2026). The Active image generation models on Bedrock are **Stability AI** models: + +| Model | Model ID | Pricing | Tier | Use case | +| -------------------------- | ----------------------------------- | ------------- | -------- | ------------------------------------- | +| Stable Image Ultra | `stability.stable-image-ultra-v1:0` | ~$0.08/image | premium | Photorealistic, high-end visuals | +| Stable Diffusion 3.5 Large | `stability.sd3-5-large-v1:0` | ~$0.065/image | flagship | High volume creative assets | +| Stable Image Core | `stability.stable-image-core-v1:0` | ~$0.04/image | fast | Rapid, affordable generation at scale | + +When `image_generation` capability is detected: + +1. Recommend **Stability AI** models as the primary Active target (not Nova Canvas). +2. Note the pricing model difference: Stability AI charges **per image**, not per token. Direct cost comparison with source provider (DALL-E, Imagen) requires converting to per-image equivalents. +3. If the user's source workload is DALL-E or Imagen, map to Stable Image Ultra (quality-first) or Stable Image Core (cost-first) based on `quality_vs_cost` preference in `preferences.json`. +4. Nova Canvas may appear as a Legacy fallback annotation but must not be `recommended_model`. + +--- + +## Integration Points + +### Design Phase (`design-ai.md`) + +After selecting a Bedrock model for each workload: + +1. Check the Legacy/EOL table above (or the lifecycle page). +2. If the model is in the **exclusion zone** (≤90 days to EOL) or EOL: reject it. Use the Active replacement. +3. If the model is Legacy but >90 days from EOL: replace with Active replacement if one exists. If no Active replacement exists, note the EOL date and recommend the user plan a follow-up migration. +4. If Active: proceed normally. + +### Estimate Phase (`estimate-ai.md`) + +When building the model comparison table: + +- **Exclusion zone models**: omit entirely from `model_comparison`. Do not include in `recommended_model` or `backup_model`. +- **Legacy (>90 days)**: include with `(Legacy — EOL YYYY-MM-DD)` annotation. Never use as `recommended_model` if an Active alternative exists. +- **Active**: no restrictions. + +### Pricing Cache (`cached-prices.md`) + +The multi-provider quick reference table includes a `Status` column: + +| Status value | Meaning | +| --------------------------- | --------------------------------------------------------------------------------------- | +| `active` | No restrictions | +| `legacy (EOL YYYY-MM-DD)` | Legacy, >90 days from EOL. Listed for reference, annotated. | +| `excluded (EOL YYYY-MM-DD)` | ≤90 days from EOL. Kept for existing users but must not be selected for new migrations. | + +When refreshing the cache, recompute `days_to_eol` and update the Status column from the [model lifecycle page](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html). + +### Mapping Guides (`design-ref-ai-openai-to-bedrock.md`, `design-ref-ai-gemini-to-bedrock.md`) + +- "Best Bedrock Match" columns must only contain Active models. +- Exclusion-zone models must not appear in any recommendation row. +- Legacy models (>90 days) may appear in notes or legacy-source mapping rows but never as the primary recommendation. + +--- + +## Refresh Cadence + +**On every design run:** The agent MUST recompute `days_to_eol = EOL date − today` for every row in the table above and apply the four rules in "Applying the rules" before making any model recommendation. The static Days to EOL column in this file is a snapshot only — do not use it directly without recomputing. + +**Periodic table refresh:** When the table itself needs updating (new models added, EOL dates changed by AWS, or past-EOL rows to remove), update this file and `cached-prices.md` together. The authoritative source is always the [Bedrock model lifecycle page](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html). + +**Past-EOL rows:** Once `days_to_eol ≤ 0`, remove the row from this table entirely on the next periodic refresh — past-EOL models serve no reference value and create confusion. diff --git a/migration-to-aws/steering/bedrock-quotas.md b/migration-to-aws/steering/bedrock-quotas.md new file mode 100644 index 00000000..9c54a53e --- /dev/null +++ b/migration-to-aws/steering/bedrock-quotas.md @@ -0,0 +1,118 @@ +# Bedrock Quota Awareness — Migration Decision Logic + +For raw quota documentation, use the AWS Documentation MCP server or see: [Quotas for Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html) | [How tokens are counted](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-token-burndown.html) + +--- + +## Why This Matters for Migrations + +OpenAI and Gemini APIs have high default rate limits for paying customers (OpenAI Tier 4/5: 2M+ TPM). Bedrock default quotas start lower and scale with account history. The quota risk assessment below works **without an AWS account** — it uses the user's current token volume and selected model to flag whether a quota increase will be needed, so teams can plan the request into their migration timeline. + +--- + +## The 5× Burndown Trap + +Claude models (3.7+) on Bedrock consume quota at **5× the rate for output tokens**. This is the single most common surprise for teams migrating from OpenAI (which has no output multiplier). + +**Effective TPM by workload type** (at 200K TPM quota on Claude): + +| Workload Type | Input:Output Ratio | Effective TPM | vs OpenAI 2M TPM | +| ------------------ | ------------------ | ------------- | -------------------------- | +| RAG / long context | 80:20 | ~111K | Need 18× increase to match | +| Balanced chat | 60:40 | ~77K | Need 26× increase to match | +| Code generation | 20:80 | ~48K | Need 42× increase to match | + +Formula: `Effective TPM = TPM_quota / (input_ratio + output_ratio × 5)` + +Nova, Llama, DeepSeek, and Mistral models have **1× burndown** (no multiplier). Output-heavy workloads that hit Claude quota limits may benefit from routing to these models for applicable tasks. + +--- + +## Quota Risk Assessment + +Apply during Design phase after model selection. Uses `ai_token_volume` from `preferences.json` and the selected model family. + +| `ai_token_volume` | Model Family | `quota_risk` | Surface to User | +| ------------------------- | ---------------------------------- | ------------ | ----------------------------------------------------------------------------- | +| `"high"` or `"very_high"` | Any | `"high"` | "Request Bedrock quota increase before migration (allow 1–5 business days)" | +| `"medium"` | Claude (5× burndown) | `"medium"` | "Monitor TPM usage during parallel run; quota increase may be needed at peak" | +| `"medium"` | Nova / Llama / other (1× burndown) | `"low"` | No action | +| `"low"` | Any | `"low"` | No action | + +Include `quota_risk` in `aws-design-ai.json` → `ai_architecture`. + +--- + +## `max_tokens` Reservation Trap + +Bedrock deducts `max_tokens` from TPM quota **at request start**, before any tokens are generated. Unused quota is returned after response completes. + +If code sets `max_tokens: 4096` (common OpenAI default) but typical responses are 200 tokens, each request reserves 20× more quota than needed — dramatically reducing concurrency. + +**Migration action:** Set `max_tokens` to ~1.5× expected output length. This is a code change that should be flagged in the migration guide when `quota_risk` is `"medium"` or `"high"`. + +--- + +## Cross-Region Inference as Quota Mitigation + +Cross-region inference (CRIS) profiles have **separate, higher quotas** because traffic distributes across multiple regions. For production migrations with `quota_risk` = `"high"`: + +- Recommend CRIS as the default deployment mode +- No code changes required (use the CRIS inference profile ID instead of the regional model ID) +- Trade-off: slightly higher latency variance, ~10% price premium in some regions + +--- + +## Pre-Migration Quota Checklist + +Surface these items in the production readiness checklist (generate phase) when `quota_risk` ≥ `"medium"`: + +- [ ] Quota risk assessed (this can be done before having an AWS account — it's based on token volume and model selection) +- [ ] AWS account created and Bedrock model access enabled in target region +- [ ] Current TPM quota checked via Service Quotas console for target model + region +- [ ] Quota increase requested if peak TPM exceeds default (allow 1–5 business days) +- [ ] `max_tokens` set to ~1.5× expected output (not model maximum) +- [ ] Cross-region inference profile evaluated for high-volume workloads +- [ ] Retry configuration uses adaptive mode (`botocore.config.Config(retries={"mode": "adaptive"})`) +- [ ] CloudWatch alarm configured on `ThrottlingException` metric > 0 + +--- + +## How to Request a Quota Increase + +Include this guidance in migration artifacts when `quota_risk` ≥ `"medium"`. Users may not have an AWS account yet when the plugin assesses risk — these instructions tell them what to do once they do. + +**Where:** AWS Console → Service Quotas → Amazon Bedrock → search for the model name (e.g., "Claude Sonnet 4" or "tokens per minute") + +**What to request:** + +- Find the quota named like: `Tokens per minute for [Model Name]` or `Cross-region model inference tokens per minute for [Model Name]` +- Click "Request increase at account level" +- Enter desired value based on the effective TPM calculation above + +**How to calculate the target value:** + +``` +Target TPM = (peak_tokens_per_minute_on_source_provider) × burndown_multiplier +``` + +Where `burndown_multiplier` = 5 for Claude models (because output tokens consume 5× quota), 1 for all others. If the user doesn't know their peak TPM, estimate from daily volume: + +``` +Estimated peak TPM = (daily_tokens / active_hours / 60) × 3 (3× for peak headroom) +``` + +**Timeline:** Allow 1–5 business days for approval. Request increases during the setup/provisioning stage of migration, not during cutover. + +**Alternative — cross-region inference:** If the increase isn't approved in time or the requested value is very high, cross-region inference profiles provide higher aggregate capacity without a quota increase request (traffic distributes across regions automatically). + +--- + +## Integration Points + +| Phase | File | What to do | +| -------- | -------------------------- | ------------------------------------------------------------------------------------------------------------- | +| Design | `design-ai.md` | Compute `quota_risk` per table above; include in `aws-design-ai.json` | +| Estimate | `estimate-ai.md` | If `quota_risk` = `"high"`, add to `complexity_factors[]`: "Bedrock quota increase required before migration" | +| Generate | `generate-artifacts-ai.md` | Add quota check step to `setup_bedrock.sh`; include checklist items above in production readiness | +| Generate | `generate-ai.md` | If `quota_risk` ≥ `"medium"`, add "Request quota increase" to Week 1 activities | diff --git a/migration-to-aws/steering/cached-prices.md b/migration-to-aws/steering/cached-prices.md index d814f89c..f105ee8a 100644 --- a/migration-to-aws/steering/cached-prices.md +++ b/migration-to-aws/steering/cached-prices.md @@ -1,11 +1,12 @@ # AWS Pricing Cache -**Last updated:** 2026-03-07 +**Last updated:** 2026-06-14 **Region:** us-east-1 **Currency:** USD **Accuracy:** ±5-10% for infrastructure services (sourced from AWS Price List API), ±15-25% for AI models (sourced from public pricing pages) -> Prices may vary by region and change over time. Use for estimation only. For real-time pricing, fall back to the AWS Pricing MCP server. +> Prices may vary by region and change over time. Use for estimation only. For real-time pricing, fall back to the AWS Pricing MCP server. **Amazon Nova** figures in the Bedrock subsection often reference **US East (Ohio)** and **inference mode** (global vs geo); other services in this file default to **us-east-1** unless noted. +> **Staleness warning:** If today's date is more than 30 days after the **Last updated** date above, treat AI model prices as potentially stale (±15-25% accuracy may widen). Infrastructure prices (Fargate, RDS, S3, etc.) change rarely and remain reliable longer. When staleness is detected, set `pricing_source: "cached_stale"` in the estimate output and note: "Pricing cache is more than 30 days old — AI model prices may have changed. Verify via the AWS Pricing MCP server or [aws.amazon.com/bedrock/pricing](https://aws.amazon.com/bedrock/pricing/)." --- @@ -247,12 +248,37 @@ VPC itself is free. Add-ons: ### CloudWatch -| Metric | Rate | -| ----------------------------- | ------ | -| Log ingestion per GB | $0.50 | -| Log storage per GB-month | $0.03 | -| Insights query per GB scanned | $0.005 | -| Custom metric per month | $0.30 | +| Metric | Rate | Notes | +| ---------------------------------------- | ------ | -------------------------------------------------------------------------------- | +| Log ingestion per GB (Standard) | $0.50 | | +| Log ingestion per GB (Infrequent Access) | $0.25 | 50% cheaper than Standard; no Live Tail, subscription filters, or metric filters | +| Log storage per GB-month | $0.03 | Same for both Standard and Infrequent Access | +| Insights query per GB scanned | $0.005 | Same for both log classes | +| Custom metric per month (≤10K) | $0.30 | Flat rate at startup scale; $0.10 for 10K–250K, $0.05 for 250K+ | +| Standard alarm per month | $0.10 | | +| High-resolution alarm per month | $0.30 | | +| Dashboard per month (first 3 free) | $3.00 | | + +Free tier (not subtracted in estimates — startup apps often exceed quickly): + +- 5 GB log ingestion + archive + Insights queries +- 10 custom metrics + 10 standard alarms +- 3 dashboards (50 metrics each) + +### X-Ray + +| Metric | Rate | Notes | +| ---------------------------- | ----- | --------------------- | +| Traces recorded per million | $5.00 | First 100K free/month | +| Traces retrieved per million | $0.50 | First 1M free/month | +| Traces scanned per million | $0.50 | First 1M free/month | + +### CloudWatch Container Insights (ECS/Fargate) + +| Metric | Rate | Notes | +| ---------------------------- | ----- | ------------------------------------------ | +| Per-task performance log/GB | $0.50 | Same as standard log ingestion | +| Cluster/service/task metrics | $0.30 | Per custom metric — can accumulate quickly | ### SQS @@ -317,32 +343,358 @@ Serverless inference: $0.0000200 per second per GB memory. ## Bedrock Models (On-Demand) -Prices per 1M tokens. Prompt caching available for Claude models (90% reduction on cached portions). Long Context variants activate automatically when input exceeds 200K tokens — 2x input price, 1.5x output price. - -| Model | Model ID | Provider | Input $/1M | Output $/1M | Context | Tier | -| -------------------------------- | ---------------------------------------- | --------- | ---------- | ----------- | ------- | --------- | -| Claude Sonnet 4.6 | anthropic.claude-sonnet-4-6 | Anthropic | 3.00 | 15.00 | 200K | flagship | -| Claude Sonnet 4.6 — Long Context | anthropic.claude-sonnet-4-6 | Anthropic | 6.00 | 22.50 | >200K | flagship | -| Claude Opus 4.6 | anthropic.claude-opus-4-6-v1 | Anthropic | 5.00 | 25.00 | 200K | premium | -| Claude Opus 4.6 — Long Context | anthropic.claude-opus-4-6-v1 | Anthropic | 10.00 | 37.50 | >200K | premium | -| Claude Haiku 4.5 | anthropic.claude-haiku-4-5-20251001-v1:0 | Anthropic | 1.00 | 5.00 | 200K | fast | -| Llama 4 Maverick | meta.llama4-maverick-17b-instruct-v1:0 | Meta | 0.24 | 0.97 | 1M | mid | -| Llama 4 Scout | meta.llama4-scout-17b-instruct-v1:0 | Meta | 0.17 | 0.66 | 10M | efficient | -| Llama 3.3 70B | meta.llama3-3-70b-instruct-v1:0 | Meta | 0.72 | 0.72 | 128K | mid | -| Llama 3.2 90B | meta.llama3-2-90b-instruct-v1:0 | Meta | 0.72 | 0.72 | 128K | mid | -| Nova 2 Lite | amazon.nova-2-lite-v1:0 | Amazon | 0.33 | 2.75 | 1M | mid | -| Nova 2 Pro | amazon.nova-2-pro-v1:0 | Amazon | 1.38 | 11.00 | 1M | flagship | -| Nova Pro | amazon.nova-pro-v1:0 | Amazon | 0.80 | 3.20 | 300K | mid | -| Nova Lite | amazon.nova-lite-v1:0 | Amazon | 0.06 | 0.24 | 300K | fast | -| Nova Micro | amazon.nova-micro-v1:0 | Amazon | 0.035 | 0.14 | 128K | budget | -| Nova Premier | amazon.nova-premier-v1:0 | Amazon | 2.50 | 12.50 | 1M | reasoning | -| Mistral Large 3 | mistral.mistral-large-3-675b-instruct | Mistral | 0.50 | 1.50 | 256K | flagship | -| DeepSeek-R1 | deepseek.r1-v1:0 | DeepSeek | 1.35 | 5.40 | 128K | reasoning | -| gpt-oss-20b | openai.gpt-oss-20b-1:0 | OpenAI | 0.07 | 0.30 | 128K | budget | -| gpt-oss-120b | openai.gpt-oss-120b-1:0 | OpenAI | 0.15 | 0.60 | 128K | efficient | -| Gemma 3 4B IT | google.gemma-3-4b-it | Google | 0.04 | 0.08 | 128K | budget | -| Gemma 3 12B IT | google.gemma-3-12b-it | Google | 0.09 | 0.29 | 128K | budget | -| Gemma 3 27B IT | google.gemma-3-27b-it | Google | 0.23 | 0.38 | 128K | efficient | +**Anthropic Claude (Standard on-demand)** figures below match **US East (N. Virginia)** on [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) as of cache refresh. **Claude Opus 4.7** lists the same headline on-demand input/output as **Opus 4.6** on that page; confirm **batch** availability per model (Opus 4.7 batch was **not** listed on the global cross-region table when this row was added). **Batch**, **prompt cache** (5m / 1h write + cache read), and **geo / in-region cross-region** rows on that page can differ; e.g. **US East (Ohio)** cross-region inference for Claude Sonnet 4.6 is listed at **$3.30 / $16.50** per 1M input/output (≈10% above N. Virginia). Long-context SKUs **do not** all use the same multiplier: **Sonnet 4.6** and **Opus 4.6** long-context modes share the same on-demand rates as the non–long-context rows on the standard table; **Sonnet 4.5** and **Sonnet 4** long-context rows are priced higher on that same table. + +### Multi-provider quick reference (per 1M tokens) + +See `steering/ai-model-lifecycle.md` for lifecycle details. **Do not recommend Legacy models for new migrations.** + +| Model | Model ID | Provider | Input $/1M | Output $/1M | Context | Tier | Status | +| -------------------------------- | ---------------------------------------- | --------- | ---------- | ----------- | ------- | --------- | -------------------------- | +| Claude Sonnet 4.6 | anthropic.claude-sonnet-4-6 | Anthropic | 3.00 | 15.00 | 200K | flagship | active | +| Claude Sonnet 4.6 — Long Context | anthropic.claude-sonnet-4-6 | Anthropic | 3.00 | 15.00 | 200K+ | flagship | active | +| Claude Opus 4.6 | anthropic.claude-opus-4-6-v1 | Anthropic | 5.00 | 25.00 | 200K | premium | active | +| Claude Opus 4.6 — Long Context | anthropic.claude-opus-4-6-v1 | Anthropic | 5.00 | 25.00 | 200K+ | premium | active | +| Claude Opus 4.5 | — | Anthropic | 5.00 | 25.00 | 200K | premium | active | +| Claude Sonnet 4.5 | — | Anthropic | 3.00 | 15.00 | 200K | flagship | active | +| Claude Sonnet 4.5 — Long Context | — | Anthropic | 6.00 | 22.50 | 200K+ | flagship | active | +| Claude Haiku 4.5 | anthropic.claude-haiku-4-5-20251001-v1:0 | Anthropic | 1.00 | 5.00 | 200K | fast | active | +| Claude Sonnet 4 | anthropic.claude-sonnet-4-20250514-v1:0 | Anthropic | 3.00 | 15.00 | 200K | flagship | legacy (EOL Oct 14, 2026) | +| Llama 4 Maverick | meta.llama4-maverick-17b-instruct-v1:0 | Meta | 0.24 | 0.97 | 1M | mid | active | +| Llama 4 Scout | meta.llama4-scout-17b-instruct-v1:0 | Meta | 0.17 | 0.66 | 10M | efficient | active | +| Llama 3.3 70B | meta.llama3-3-70b-instruct-v1:0 | Meta | 0.72 | 0.72 | 128K | mid | active | +| Llama 3.2 90B | meta.llama3-2-90b-instruct-v1:0 | Meta | 0.72 | 0.72 | 128K | mid | excluded (EOL Jul 7, 2026) | +| Nova 2 Lite | amazon.nova-2-lite-v1:0 | Amazon | 0.33 | 2.75 | 1M | mid | active | +| Nova 2 Pro (Preview) | amazon.nova-2-pro-v1:0 | Amazon | 1.375 | 11.00 | 1M | flagship | active | +| Nova Pro | amazon.nova-pro-v1:0 | Amazon | 0.80 | 3.20 | 300K | mid | active | +| Nova Pro (latency optimized) | — | Amazon | 1.00 | 4.00 | 300K | mid | active | +| Nova Lite | amazon.nova-lite-v1:0 | Amazon | 0.06 | 0.24 | 300K | fast | active | +| Nova Micro | amazon.nova-micro-v1:0 | Amazon | 0.035 | 0.14 | 128K | budget | active | +| Nova Premier | amazon.nova-premier-v1:0 | Amazon | 2.50 | 12.50 | 1M | reasoning | legacy (EOL Sep 14, 2026) | +| Mistral Large 3 | mistral.mistral-large-3-675b-instruct | Mistral | 0.50 | 1.50 | 256K | flagship | active | +| DeepSeek-R1 | deepseek.r1-v1:0 | DeepSeek | 1.35 | 5.40 | 128K | reasoning | active | +| DeepSeek-V3.1 | — | DeepSeek | 0.58 | 1.68 | — | mid | active (Sydney only) | +| gpt-oss-20b | openai.gpt-oss-20b-1:0 | OpenAI | 0.07 | 0.30 | 128K | budget | active | +| gpt-oss-120b | openai.gpt-oss-120b-1:0 | OpenAI | 0.15 | 0.60 | 128K | efficient | active | +| Gemma 3 4B IT | google.gemma-3-4b-it | Google | 0.04 | 0.08 | 128K | budget | active | +| Gemma 3 12B IT | google.gemma-3-12b-it | Google | 0.09 | 0.29 | 128K | budget | active | +| Gemma 3 27B IT | google.gemma-3-27b-it | Google | 0.23 | 0.38 | 128K | efficient | active | +| Jamba 1.5 Large | ai21.jamba-1-5-large-v1:0 | AI21 Labs | 2.00 | 8.00 | — | mid | legacy (EOL Nov 26, 2026) | +| Jamba 1.5 Mini | ai21.jamba-1-5-mini-v1:0 | AI21 Labs | 0.20 | 0.40 | — | efficient | legacy (EOL Nov 26, 2026) | +| Jurassic-2 Mid | — | AI21 Labs | 12.50 | 12.50 | — | legacy | legacy | +| Jurassic-2 Ultra | — | AI21 Labs | 18.80 | 18.80 | — | legacy | legacy | +| Jamba-Instruct | — | AI21 Labs | 0.50 | 0.70 | — | mid | active | + +### Stability AI — Image Generation (per image, US East) + +Active image generation models on Bedrock. Priced **per image** (not per token). Use for `image_generation` capability workloads. + +| Model | Model ID | Price/image | Resolution | Tier | Status | +| -------------------------- | --------------------------------- | ----------- | ---------- | -------- | ------ | +| Stable Image Ultra | stability.stable-image-ultra-v1:0 | $0.08 | up to 4MP | premium | active | +| Stable Diffusion 3.5 Large | stability.sd3-5-large-v1:0 | $0.065 | up to 1MP | flagship | active | +| Stable Image Core | stability.stable-image-core-v1:0 | $0.04 | up to 1MP | fast | active | + +Image editing services (inpaint, erase, upscale, etc.) are priced at $0.03–$0.60/operation. See [Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) for full list. + +> **Cost comparison note:** DALL-E 3 (OpenAI) charges $0.04–$0.12/image depending on resolution. Google Imagen charges per character of prompt. When comparing, use per-image cost directly — do not convert Stability AI prices to per-token equivalents. + +### Anthropic Claude — batch & prompt cache (Standard, US East N. Virginia) + +Per 1M tokens unless noted. See [Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) for full regional and tier tables. + +| Model | Batch in | Batch out | 5m cache write | 1h cache write | Cache read | +| ------------------------ | -------- | --------- | -------------- | -------------- | ---------- | +| Claude Sonnet 4.6 (+ LC) | 1.50 | 7.50 | 3.75 | 6.00 | 0.30 | +| Claude Opus 4.6 (+ LC) | 2.50 | 12.50 | 6.25 | 10.00 | 0.50 | +| Claude Opus 4.5 | 2.50 | 12.50 | 6.25 | 10.00 | 0.50 | +| Claude Haiku 4.5 | 0.50 | 2.50 | 1.25 | 2.00 | 0.10 | +| Claude Sonnet 4.5 | 1.50 | 7.50 | 3.75 | 6.00 | 0.30 | +| Claude Sonnet 4.5 — LC | 3.00 | 11.25 | 7.50 | 12.00 | 0.60 | + +### AI21 Labs + +**On-demand**, **US East (N. Virginia)** per [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/). Prices per 1M input / output tokens. + +| Model | Input $/1M | Output $/1M | +| ---------------- | ---------- | ----------- | +| Jamba 1.5 Large | 2.00 | 8.00 | +| Jamba 1.5 Mini | 0.20 | 0.40 | +| Jurassic-2 Mid | 12.50 | 12.50 | +| Jurassic-2 Ultra | 18.80 | 18.80 | +| Jamba-Instruct | 0.50 | 0.70 | + +_Quick-reference rows use **—** for **model ID** and **context**; resolve in the Bedrock console or AWS model documentation._ + +### Mistral AI + +Per [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) (Mistral AI). **Priority** tier ≈ **+75%** vs Standard; **Flex** ≈ **−50%** vs Standard on the same page. + +#### US East (Ohio) — Pixtral (on-demand + batch header from page) + +| Model | Input $/1M | Output $/1M | Batch in | Batch out | +| --------------------- | ---------- | ----------- | -------- | --------- | +| Pixtral Large (25.02) | 2.00 | 6.00 | N/A | N/A | + +#### On-demand — US East (N. Virginia), US East (Ohio), US West (Oregon) + +Prices per 1M input / output tokens. + +| Model | Input $/1M | Output $/1M | +| ------------------- | ---------- | ----------- | +| Devstral 2 123B | 0.40 | 2.00 | +| Magistral Small 1.2 | 0.50 | 1.50 | +| Voxtral Mini 1.0 | 0.04 | 0.04 | +| Voxtral Small 1.0 | 0.10 | 0.30 | +| Ministral 3B 3.0 | 0.10 | 0.10 | +| Ministral 8B 3.0 | 0.15 | 0.15 | +| Ministral 14B 3.0 | 0.20 | 0.20 | +| Mistral Large 3 | 0.50 | 1.50 | + +_Rates differ in **Asia Pacific (Mumbai)**, **São Paulo**, **Tokyo**, **Europe (Ireland/Milan/London)**, **Sydney**, **Jakarta**, **Frankfurt**, **Stockholm** (e.g. Mistral Large 3 Mumbai **$0.59 / $1.76**). **Devstral 2 123B** appears in Jakarta / Frankfurt / Stockholm at **$0.48 / $2.40** on the same page._ + +### DeepSeek (Amazon Bedrock) + +Per [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) (DeepSeek). + +#### US East (Ohio) — Standard tier + +> **Note:** DeepSeek-V3.1 is no longer listed for US East regions on the Bedrock pricing page as of May 2026. Use DeepSeek-V3.2 for US East deployments. DeepSeek-V3.1 remains available in Asia Pacific (Sydney). + +| Model | Input $/1M | Output $/1M | +| ----------- | ---------- | ----------- | +| DeepSeek-R1 | 1.35 | 5.40 | + +#### US East (N. Virginia), US East (Ohio), US West (Oregon) + +| Model | Input $/1M | Output $/1M | +| ------------- | ---------- | ----------- | +| DeepSeek v3.2 | 0.62 | 1.85 | + +#### Other regions (examples on same page) + +- **Asia Pacific (Mumbai)**, **São Paulo**, **Jakarta**, **Tokyo**, **Stockholm**: DeepSeek v3.2 **$0.74 / $2.22** per 1M. +- **Asia Pacific (Sydney)**: DeepSeek v3.1 **$0.5974 / $1.7304**; v3.2 **$0.6386 / $1.9055** (and **Priority** / **Flex** tiers listed separately on the page). + +### OpenAI on Bedrock (gpt-oss) + +**Standard tier** per [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) (OpenAI). + +| Model | Region | Input $/1M | Output $/1M | +| ------------ | --------------------- | ---------- | ----------- | +| gpt-oss-20b | US East (Ohio) | 0.07 | 0.30 | +| gpt-oss-120b | US East (Ohio) | 0.15 | 0.60 | +| gpt-oss-20b | Asia Pacific (Sydney) | 0.0721 | 0.3090 | +| gpt-oss-120b | Asia Pacific (Sydney) | 0.1545 | 0.6180 | + +_The Bedrock page also lists **Flex**, **Priority**, **Batch**, **GPT OSS Safeguard**, and **model customization** pricing for other regions._ + +### Meta Llama (Amazon Bedrock) + +Per [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) (Meta). Prices per 1M input / output tokens unless noted. + +#### Llama 4 — US East (Ohio), on-demand and batch + +| Model | On-demand in | On-demand out | Batch in | Batch out | +| -------------------- | ------------ | ------------- | -------- | --------- | +| Llama 4 Maverick 17B | 0.24 | 0.97 | 0.12 | 0.485 | +| Llama 4 Scout 17B | 0.17 | 0.66 | 0.085 | 0.33 | + +#### Llama 3.3 — US East (Ohio), on-demand and batch + +| Model | On-demand in | On-demand out | Batch in | Batch out | +| ------------------------ | ------------ | ------------- | -------- | --------- | +| Llama 3.3 Instruct (70B) | 0.72 | 0.72 | 0.36 | 0.36 | + +#### Llama 3.2 — US East (Ohio), on-demand and batch + +> **Lifecycle note:** All Llama 3.2 Instruct models are in the **exclusion zone** (EOL Jul 7, 2026, ≤90 days). Do not select for new migrations. Use **Llama 4 Scout / Maverick** instead. See `steering/ai-model-lifecycle.md`. + +| Model | On-demand in | On-demand out | Batch in | Batch out | +| ------------------------ | ------------ | ------------- | -------- | --------- | +| Llama 3.2 Instruct (1B) | 0.10 | 0.10 | N/A | N/A | +| Llama 3.2 Instruct (3B) | 0.15 | 0.15 | N/A | N/A | +| Llama 3.2 Instruct (11B) | 0.16 | 0.16 | N/A | N/A | +| Llama 3.2 Instruct (90B) | 0.72 | 0.72 | N/A | N/A | + +#### Llama 3.2 — customization & provisioned throughput (US West Oregon) + +**Fine-tuning** (per 1M tokens trained / month storage / no-commit PT per model unit-hour): + +| Model | Train 1M tokens | Store / month | Infer / unit-hr (no commit) | +| ------------------------ | --------------- | ------------- | --------------------------- | +| Llama 3.2 Instruct (1B) | $0.50 | $1.95 | $23.00 | +| Llama 3.2 Instruct (3B) | $1.10 | $1.95 | $23.00 | +| Llama 3.2 Instruct (11B) | $3.50 | $1.95 | $23.00 | +| Llama 3.2 Instruct (90B) | $7.90 | $1.95 | $30.00 | + +**Provisioned throughput** ($/hour per model unit): + +| Model | No commit | 1-mo commit | 6-mo commit | +| ------------------------ | --------- | ----------- | ----------- | +| Llama 3.2 Instruct (1B) | $23.00 | $20.30 | $12.54 | +| Llama 3.2 Instruct (3B) | $23.00 | $20.30 | $12.54 | +| Llama 3.2 Instruct (11B) | $23.00 | $20.30 | $12.54 | +| Llama 3.2 Instruct (90B) | $30.00 | $26.48 | $16.35 | + +#### Llama 3.1 — US East (Ohio), on-demand and batch + +> **Lifecycle note:** Llama 3.1 405B Instruct is in the **exclusion zone** (EOL Jul 7, 2026, ≤90 days). Do not select for new migrations. Use **Llama 4 Maverick** instead. See `steering/ai-model-lifecycle.md`. + +| Model | On-demand in | On-demand out | Batch in | Batch out | +| -------------------------------------------------- | ------------ | ------------- | -------- | --------- | +| Llama 3.1 Instruct (8B) | 0.22 | 0.22 | 0.11 | 0.11 | +| Llama 3.1 Instruct (70B) | 0.72 | 0.72 | 0.36 | 0.36 | +| Llama 3.1 Instruct (405B) | 2.40 | 2.40 | 1.20 | 1.20 | +| Llama 3.1 Instruct (70B) (latency optimized inf.) | 0.90 | 0.90 | N/A | N/A | +| Llama 3.1 Instruct (405B) (latency optimized inf.) | 3.00 | 3.00 | N/A | N/A | + +#### Llama 3.1 — customization & provisioned throughput (US West Oregon) + +| Model | Train 1M tokens | Store / month | Infer / unit-hr (no commit) | +| ------------------------ | --------------- | ------------- | --------------------------- | +| Llama 3.1 Instruct (8B) | $1.49 | $1.95 | $24.00 | +| Llama 3.1 Instruct (70B) | $7.99 | $1.95 | $24.00 | + +| Model | No commit | 1-mo commit | 6-mo commit | +| ------------------------ | --------- | ----------- | ----------- | +| Llama 3.1 Instruct (8B) | $24.00 | $21.18 | $13.08 | +| Llama 3.1 Instruct (70B) | $24.00 | $21.18 | $13.08 | + +#### Llama 3 — US East (N. Virginia), on-demand + +| Model | Input $/1M | Output $/1M | +| ---------------------- | ---------- | ----------- | +| Llama 3 Instruct (8B) | 0.30 | 0.60 | +| Llama 3 Instruct (70B) | 2.65 | 3.50 | + +#### Llama 2 — US East (N. Virginia) and US West (Oregon), on-demand + +| Model | Input $/1M | Output $/1M | +| ------------------ | ---------- | ----------- | +| Llama 2 Chat (13B) | 0.75 | 1.00 | +| Llama 2 Chat (70B) | 1.95 | 2.56 | + +#### Llama 2 — customization & provisioned throughput + +**Fine-tuning:** Llama 2 Pretrained (13B) **$1.49** per 1M tokens trained, **$1.95**/month storage, **$23.50**/unit-hr no-commit infer; (70B) **$7.99** / **$1.95** / **$23.50**. + +**Provisioned throughput** (per model unit-hour): 13B and 70B — **$21.18** (1-mo commit), **$13.08** (6-mo commit). Pretrained Llama 2 is **provisioned throughput only after customization** per AWS. + +### Amazon Nova + +All figures from [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) (Amazon Nova sections). Nova pricing depends on **inference deployment** (e.g. **Global cross-region** vs **Geo cross-region and in-region**), **tier** (Standard, Priority, Flex, Batch), **modality** (text / image / video / audio), and **AWS Region**. The quick-reference table above uses **Geo cross-region inference and in-region**, **Standard tier**, **US East (Ohio)**, **text + image + video** rates where a single input price applies to all three. Confirm the page before estimates. + +**Cache read (Nova):** AWS states cache read input tokens are **75% below** the on-demand input price for the applicable tier/modality. + +#### US East (Ohio) — Standard tier, text + image + video (Geo cross-region inference and in-region) + +Per 1M input / output tokens unless noted. + +| Model | Input $/1M | Output $/1M | +| ---------------------------------------- | ---------- | ----------- | +| Amazon Nova 2 Lite | 0.33 | 2.75 | +| Amazon Nova Micro | 0.035 | 0.14 | +| Amazon Nova Lite | 0.06 | 0.24 | +| Amazon Nova Pro | 0.80 | 3.20 | +| Amazon Nova Pro (latency optimized inf.) | 1.00 | 4.00 | +| Amazon Nova Premier | 2.50 | 12.50 | + +#### US East (Ohio) — Standard tier, Global cross-region inference (text + image + video) + +| Model | Input $/1M | Output $/1M | +| ------------------ | ---------- | ----------- | +| Amazon Nova 2 Lite | 0.30 | 2.50 | + +#### US East (Ohio) — Standard tier, text / image / video / audio (Geo cross-region and in-region) + +Per 1M tokens. **Nova 2 Omni** and **Nova 2 Pro** are **Preview**. Image column is **output** image pricing where listed. + +| Model | Text in | Image in | Video in | Audio in | Text out | Image out | +| ---------------------------- | ------- | -------- | -------- | -------- | -------- | --------- | +| Amazon Nova 2 Omni (Preview) | 0.30 | 0.30 | 0.30 | 1.10 | 2.80 | 44.00 | +| Amazon Nova 2 Pro (Preview) | 1.375 | 1.375 | 1.375 | 1.375 | 11.00 | N/A | + +**Global cross-region inference (Ohio), Standard** uses different multimodal numbers on the same page — e.g. **Nova 2 Omni (Preview)** audio in **$1.00**, text out **$2.50**, image out **$40.00** (text/image/video in **$0.30** each). Use the Bedrock pricing page for the deployment you select. + +#### US East (Ohio) — Nova 2 Lite by tier (text + image + video) + +##### Global cross-region inference + +| Tier | Input $/1M | Output $/1M | +| -------- | ---------- | ----------- | +| Standard | 0.30 | 2.50 | +| Priority | 0.525 | 4.375 | +| Flex | 0.15 | 1.25 | +| Batch | 0.15 | 1.25 | + +##### Geo cross-region inference and in-region + +| Tier | Input $/1M | Output $/1M | +| -------- | ---------- | ----------- | +| Standard | 0.33 | 2.75 | +| Priority | 0.5775 | 4.8125 | +| Flex | 0.165 | 1.375 | +| Batch | 0.1595 | 1.342 | + +#### US East (Ohio) — Batch tier (text + image + video), selected models + +| Model | Input $/1M | Output $/1M | +| ------------------- | ---------- | ----------- | +| Amazon Nova 2 Lite | 0.1595 | 1.342 | +| Amazon Nova Micro | 0.0175 | 0.07 | +| Amazon Nova Lite | 0.03 | 0.12 | +| Amazon Nova Pro | 0.40 | 1.60 | +| Amazon Nova Premier | 1.25 | 6.25 | + +#### On-demand inference (listed rates, cache read column) + +| Model | Input $/1M | Cache read $/1M | Output $/1M | +| -------------------- | ---------- | --------------- | ----------- | +| Amazon Nova 2.0 Lite | N/A | N/A | N/A | +| Amazon Nova Micro | 0.035 | N/A | 0.14 | +| Amazon Nova Lite | 0.06 | N/A | 0.24 | +| Amazon Nova Pro | 0.80 | N/A | 3.20 | + +#### Built-in tools — Web grounding (US East Ohio) + +| Model | Price | +| ---------------------------- | ---------------------- | +| Amazon Nova 2 Omni (Preview) | $30.00 per 1K requests | +| Amazon Nova 2 Pro (Preview) | $30.00 per 1K requests | +| Amazon Nova Premier | $30.00 per 1K requests | + +#### Creative — US East (N. Virginia) + +> **Lifecycle note:** Nova Canvas v1 is **Legacy** (EOL Sep 30, 2026) and Nova Reel v1 is **Legacy** (EOL Sep 30, 2026). Do not recommend for new migrations. See `steering/ai-model-lifecycle.md`. + +**Amazon Nova Canvas** (on-demand, per image): up to **1024×1024** — Standard **$0.04**, Premium **$0.06**; up to **2048×2048** — Standard **$0.06**, Premium **$0.08**. + +**Model customization (Nova Canvas):** **$0.005** per image seen; **$1.95**/month per custom model stored; provisioned inference per model unit per hour (no commit / 1-mo / 6-mo) **$60.50 / $55.00 / $30.25**. + +**Amazon Nova Reel** (video): **720p, 24 fps** — **$0.08** per second of video generated. + +#### Speech — US East (N. Virginia) + +> **Lifecycle note:** Nova Sonic v1 is **Legacy** (EOL Sep 14, 2026). Prefer **Nova 2 Sonic** for new migrations. See `steering/ai-model-lifecycle.md`. + +Per 1M tokens. + +| Model | Modality | Input $/1M | Output $/1M | Status | +| ------------------- | -------- | ---------- | ----------- | ------------------------- | +| Amazon Nova Sonic | Speech | 3.40 | 13.60 | legacy (EOL Sep 14, 2026) | +| Amazon Nova Sonic | Text | 0.06 | 0.24 | legacy (EOL Sep 14, 2026) | +| Amazon Nova 2 Sonic | Speech | 3.00 | 12.00 | active | +| Amazon Nova 2 Sonic | Text | 0.33 | 2.75 | active | + +#### Multimodal embeddings — US East (N. Virginia) + +| Offering | Text $/1M | Std image / doc image / video sec / audio sec | +| --------------------------------------------- | --------- | ------------------------------------------------ | +| Amazon Nova Multimodal Embeddings (On-demand) | 0.135 | $0.00006 / $0.0006 / $0.0007 / $0.00014 per unit | +| Amazon Nova Multimodal Embeddings (Batch) | 0.0675 | $0.00003 / $0.00048 / $0.00056 / $0.000112 | + +_On-demand inference for **custom Nova** models matches **base Nova** inference pricing per AWS._ --- @@ -352,22 +704,33 @@ Use alongside Bedrock pricing to calculate migration ROI. ### Gemini (Standard Tier) -Prices per 1M tokens. +Prices per 1M tokens. Source: [ai.google.dev/gemini-api/docs/pricing](https://ai.google.dev/gemini-api/docs/pricing), verified May 2026. + +| Model | Input $/1M | Output $/1M | Context | Tier | +| --------------------- | ---------- | ----------- | ------- | -------- | +| Gemini 3.5 Flash | 1.50 | 9.00 | 1M | flagship | +| Gemini 3.1 Pro | 2.00 | 12.00 | 1M | flagship | +| Gemini 3.1 Flash-Lite | 0.25 | 1.50 | 1M | budget | +| Gemini 2.5 Pro | 1.25 | 10.00 | 1M | flagship | +| Gemini 2.5 Flash | 0.30 | 2.50 | 1M | fast | +| Gemini 2.0 Flash | 0.10 | 0.40 | 1M | fast | +| Gemini 2.0 Flash Lite | 0.075 | 0.30 | 1M | budget | -| Model | Input $/1M | Output $/1M | Context | Tier | -| ---------------------- | ---------- | ----------- | ------- | -------- | -| Gemini 3.1 Pro Preview | 2.00 | 12.00 | 1M | flagship | -| Gemini 2.5 Pro | 1.25 | 10.00 | 1M | flagship | -| Gemini 2.5 Flash | 0.30 | 2.50 | 1M | fast | -| Gemini 2.0 Flash | 0.10 | 0.40 | 1M | fast | -| Gemini 2.0 Flash Lite | 0.075 | 0.30 | 1M | budget | +> **Gemini 3.1 Pro breakpoint pricing:** $4.00/$18.00 per 1M for prompts >200k tokens (vs $2.00/$12.00 for ≤200k). Table above uses ≤200k rates. +> **Gemini 3.5 Flash** is now GA and the current flagship Flash model, replacing Gemini 2.5 Flash as the primary Flash-tier recommendation. At $1.50/$9.00 it is 5x more expensive than Gemini 2.5 Flash — the Bedrock cost savings case is significantly stronger against 3.5 Flash. ### OpenAI (Standard Tier) -Prices per 1M tokens. +Prices per 1M tokens. GPT-5.5 and GPT-5.5 Pro use the same breakpoint pricing structure as GPT-5.4 at 272K input tokens. GPT-5.4 and GPT-5.4 Pro use **breakpoint pricing** at 272K input tokens: rates below are for <272K context; above 272K, input is 2x and output is 1.5x. | Model | Input $/1M | Output $/1M | Context | Tier | | ------------ | ---------- | ----------- | ------- | --------- | +| GPT-5.5 | 5.00 | 30.00 | 1M | flagship | +| GPT-5.5 Pro | 30.00 | 180.00 | 1M | premium | +| GPT-5.4 | 2.50 | 15.00 | 1.05M | flagship | +| GPT-5.4 Mini | 0.75 | 4.50 | — | fast | +| GPT-5.4 Nano | 0.20 | 1.25 | — | budget | +| GPT-5.4 Pro | 30.00 | 180.00 | 1.05M | premium | | GPT-5.2 | 1.75 | 14.00 | 200K | flagship | | GPT-5.1 | 1.25 | 10.00 | 200K | flagship | | GPT-5 Mini | 0.25 | 2.00 | 200K | fast | @@ -378,3 +741,55 @@ Prices per 1M tokens. | GPT-4o | 2.50 | 10.00 | 128K | flagship | | o3 | 2.00 | 8.00 | 200K | reasoning | | o4-mini | 1.10 | 4.40 | 200K | reasoning | + +## Security Baseline + +**Per-unit rates verified via AWS Pricing API for us-east-1 on 2026-05-04.** +**Config pricing effective 2025-09-01; Security Hub pricing effective 2026-03-01.** +**Re-verify if migrating to a non-us-east-1 region or if any of these services re-prices.** + +### CloudTrail + +| Metric | Rate | +| ----------------------------------------------- | ------------------------------------------ | +| Management events (first trail per region/type) | $0.00 | +| Management events (additional trails) | $2.00 per 100K events | +| Data events | $0.10 per 100K events (not used by Tier 1) | + +### GuardDuty + +| Metric | Rate | Notes | +| --------------------------- | -------------------------- | ----------------------------------------------- | +| First 30 days | $0.00 | Free trial per account | +| CloudTrail event analysis | $4.00 per 1M events | First 500M/mo; scales down thereafter | +| VPC Flow Log / DNS analysis | $1.00 per GB | First 500 GB/mo | +| DNS query analysis | $1.00 per 1M queries | | +| Small-startup typical | ~$2–25/mo (typical $14/mo) | After free trial, with ~2M CloudTrail events/mo | + +### AWS Config (pricing effective 2025-09-01) + +| Metric | Rate | Notes | +| ---------------------------------- | -------------------------------------- | ---------------------------------------------------------------------- | +| Continuous configuration item | $0.003 per item | Records every change | +| Daily configuration item | $0.012 per daily item | Once-per-day snapshot; cheaper for slow-changing accounts, less signal | +| Small-startup typical (continuous) | ~$2–10/mo | 50–300 CIs/mo continuous | +| Source | AWS Pricing API, us-east-1, 2026-05-04 | | + +### AWS Security Hub (pricing effective 2026-03-01) + +| Metric | Rate | Notes | +| ------------------------------ | -------------------------------------- | -------------------------------------------------------------------- | +| First 30 days | $0.00 | Free trial per account | +| Security checks | $0.001 per check | First 100K checks/mo; tapers above | +| Per-EC2-hour monitoring | $0.0052083/hr | ~$3.80/mo per instance | +| Per-Lambda-function monitoring | $0.000434/hr | ~$0.32/mo per function | +| Per-container-image scanning | $0.0002894/hr | | +| Small-startup typical | ~$1–15/mo | After trial; Fargate-only startups pay nothing for the EC2 dimension | +| Source | AWS Pricing API, us-east-1, 2026-05-04 | | + +### AWS Budgets + +| Metric | Rate | Notes | +| --------------------------- | ------------------------ | ------------------------------------------ | +| First 2 budgets per account | $0.00 | Free tier | +| Additional budgets | $0.02 per budget per day | Tier 1 emits 1 budget, so effectively free | diff --git a/migration-to-aws/steering/clarify-ai-only.md b/migration-to-aws/steering/clarify-ai-only.md index b2f0016a..9e739a72 100644 --- a/migration-to-aws/steering/clarify-ai-only.md +++ b/migration-to-aws/steering/clarify-ai-only.md @@ -2,421 +2,310 @@ **Standalone flow** — Used when ONLY `ai-workload-profile.json` exists (no infrastructure or billing artifacts). Infrastructure stays on GCP; only AI/LLM calls move to AWS Bedrock. -This file replaces the category-based question system for AI-only migrations. It produces the same `preferences.json` output but with `design_constraints` limited to region and compliance, and `ai_constraints` fully populated. +Produces the same `preferences.json` output but with `design_constraints` limited to region and `ai_constraints` fully populated. Questions are presented in **two progressive batches** with an intermediate save — partial answers persist across sessions. --- -## Step 1: Present AI Detection Summary +## Step 0: Prior Run Check -> **AI-Only Migration Detected** -> Your project has AI workloads but no infrastructure artifacts (Terraform, billing). I'll focus on migrating your AI/LLM calls to AWS Bedrock while your infrastructure stays on GCP. -> -> **AI source:** [from `summary.ai_source`] -> **Models detected:** [from `models[].model_id`] -> **Capabilities in use:** [from `integration.capabilities_summary` where true] -> **Integration pattern:** [from `integration.pattern`] via [from `integration.primary_sdk`] -> **Gateway/router:** [from `integration.gateway_type`, or "None (direct SDK)"] -> **Frameworks:** [from `integration.frameworks`, or "None"] +Check `$MIGRATION_DIR/` for existing state: ---- +**Case 1 — Completed preferences exist** (`preferences.json` present): -## Step 2: Ask Questions (Q1–Q10) +> "I found existing migration preferences from a previous run. Would you like to:" +> +> A) Re-use these preferences and skip questions +> B) Start fresh and re-answer all questions -Present all questions at once with a progress indicator. Questions use their own numbering (Q1–Q10), independent of the full migration numbering. +- If A: skip to Step 3 (Validation), proceed with existing file. +- If B: delete `preferences.json`, continue to Step 1. -``` -I have a few questions to tailor your AI migration plan. -You can answer each, skip individual ones (I'll use sensible defaults), -or say "use all defaults" to proceed. -``` +**Case 2 — Draft preferences exist** (`preferences-draft.json` present, no `preferences.json`): ---- +> "I found a partial set of answers from a previous session (1 of 2 batches completed). Would you like to:" +> +> A) Resume from where you left off — I'll pick up the remaining questions +> B) Start fresh and re-answer all questions -### AI Framework & Orchestration (Q1) +- If A: load the draft, skip Batch 1 in Step 2, present Batch 2 directly. +- If B: delete `preferences-draft.json`, continue to Step 1. -Always asked first. The orchestration layer determines the entire migration approach — gateway config change, provider swap, or full agent migration. +**Case 3 — No prior state**: Continue to Step 1. --- -### Q1 — What AI framework or orchestration layer are you using? (select all that apply) - -Same decision logic, auto-detect signals, combination logic, and interpretation as Q14 in `clarify-ai.md`. Refer to that file for full details. - -**Auto-detect signals** — scan application code before asking: - -- No AI framework imports, raw HTTP calls → A -- LiteLLM, OpenRouter, PortKey, Helicone, Kong, Apigee → B -- LangChain/LangGraph/LlamaIndex → C -- CrewAI, AutoGen, custom multi-agent → D -- OpenAI Agents SDK / Swarm → E -- MCP / A2A protocol → F -- Vapi, Bland.ai, Retell, Whisper → G +## Step 1: Present AI Detection Summary -> How your AI calls reach the model determines migration effort. +> **AI-Only Migration Detected** +> Your project has AI workloads but no infrastructure artifacts (Terraform, billing). I'll focus on migrating your AI/LLM calls to AWS Bedrock while your infrastructure stays on GCP. > -> A) No framework — direct API calls -> B) LLM router/gateway (LiteLLM, OpenRouter, PortKey, Kong, Apigee) -> C) LangChain / LangGraph -> D) Multi-agent framework (CrewAI, AutoGen, custom) -> E) OpenAI Agents SDK / custom agent loop -> F) MCP servers or A2A protocol -> G) Voice/conversational agent platform (Vapi, Retell, Bland.ai) - -Interpret: same as Q14 in `clarify-ai.md` → `ai_framework` array. - -Default: _(auto-detect)_ — fall back to `["direct"]`. +> **AI source:** [from `summary.ai_source`] +> **Models detected:** [from `models[].model_id`] +> **Capabilities in use:** [from `integration.capabilities_summary` where true] +> **Integration pattern:** [from `integration.pattern`] via [from `integration.primary_sdk`] +> **Gateway/router:** [from `integration.gateway_type`, or "None (direct SDK)"] --- -### Priority & Cost (Q2–Q3) - -Top-level filters for model selection and credits eligibility. +## Step 1.5: Fast-Path Check ---- +If `migration-preview.json` exists and `ai_complexity_signal == "likely_simple"` (single model, non-agentic, no multi-provider, no multi-model routing): -### Q2 — What matters most for your AI application? +> "Your AI migration looks straightforward — one model swapping to Bedrock. I only need 3 quick answers to complete your migration plan." -Same decision logic as Q16 in `clarify-ai.md`. +Present **only Q2, Q3, Q4** (Q1 framework is extracted; Q5 model is extracted; Q6 capabilities are extracted; Q7–Q10 use defaults). After answering, skip directly to Step 3. -**Context for user:** When asking, help the user think about their actual priority rather than defaulting to "best quality": +If `ai_complexity_signal` is `"standard"` or `"complex"`, or `migration-preview.json` is absent, continue to Step 2 (full question set). -- **Best quality/reasoning** — accuracy and depth matter most; you'd pay more or wait longer for better answers (e.g., legal analysis, complex code generation, medical summarization) -- **Fastest speed** — response time is the primary constraint; users are waiting in real-time (e.g., chat UI, autocomplete, live suggestions) -- **Lowest cost** — volume is high and budget is tight; good-enough quality at scale (e.g., classification, tagging, bulk summarization) -- **Specialized capability** — you rely on a specific feature like vision, function calling, or extended thinking (covered in Q10) -- **Balanced** — no single dimension dominates; you want a solid all-rounder - -> This determines our model selection strategy. -> -> A) Best quality/reasoning -> B) Fastest speed -> C) Lowest cost -> D) Specialized capability (covered in Q10) -> E) Balanced -> F) I don't know +--- -| Answer | Recommendation Impact | -| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| Best quality/reasoning | Claude Sonnet 4.6 (latest, highest reasoning in Sonnet family) — primary; Claude Opus 4.6 for most demanding reasoning tasks | -| Fastest speed | Claude Haiku 4.5 — lowest latency in Claude family; also consider Amazon Nova Micro/Lite for cost-optimized speed | -| Lowest cost | Claude Haiku 4.5 or Amazon Nova Micro — lowest cost per token | -| Specialized capability | Deferred to Q10 to determine which model | -| Balanced | Claude Sonnet 4.6 as default balanced recommendation | +## Step 2: Ask Questions in Progressive Batches (Q1–Q10) -Interpret: same as Q16 → `ai_priority`. +Questions are presented in two batches with a save after the first. The user can skip individual questions (defaults applied), say **"use defaults for the rest"** to apply defaults for all remaining questions and proceed immediately, or answer normally. -Default: E — `ai_priority: "balanced"`. +### Batch 1 — AI Strategy & Setup (Q1–Q5) ---- +Present with this intro: -### Q3 — Approximately how much are you spending on OpenAI or Gemini per month? +``` +Before designing your Bedrock migration, I have two short sections of questions. +You can answer each, skip individual ones (I'll use sensible defaults), +or say "use defaults for the rest" at any point. -Same decision logic as Q15 in `clarify-ai.md` — credits eligibility and cost baseline. +Let's start with your AI strategy and current setup. -> AI spend helps me calculate migration credits eligibility and establish a Bedrock cost baseline. -> -> A) < $500/month -> B) $500–$2,000/month -> C) $2,000–$10,000/month -> D) > $10,000/month -> E) I don't know +--- AI Strategy & Setup --- +``` -Interpret: same as Q15 → `ai_monthly_spend`. +## Q1 — AI framework or orchestration layer (select all that apply) -Default: B — `ai_monthly_spend: "$500-$2K"`. +Same decision logic, auto-detect signals, and interpretation as Q14 in `clarify-ai.md`. ---- +Auto-detect: No framework → A, LiteLLM/OpenRouter/Kong/Apigee → B, LangChain/LangGraph → C, CrewAI/AutoGen → D, OpenAI Agents SDK → E, MCP/A2A → F, Vapi/Bland.ai/Retell → G. -### Cross-Cloud Architecture (Q4) +_Skip when:_ `integration.gateway_type` AND `integration.frameworks` are both populated in `ai-workload-profile.json` — use extracted values with `chosen_by: "extracted"` and do not present this question. -Unique to AI-only migrations where infrastructure stays on GCP while AI calls route to AWS. +> A) No framework — direct API calls | B) LLM router/gateway | C) LangChain / LangGraph | D) Multi-agent framework | E) OpenAI Agents SDK | F) MCP/A2A | G) Voice platform ---- +Interpret → `ai_framework` array. Default: auto-detect, fallback `["direct"]`. -### Q4 — Cross-cloud API call concerns +## Q2 — What matters most for your AI application? -**Rationale:** AI-only migrations keep infrastructure on GCP while moving AI calls to AWS Bedrock. Cross-region API calls add latency and potential egress costs that may affect the recommendation. This question is unique to AI-only migrations. +> A) Best quality/reasoning | B) Fastest speed | C) Lowest cost | D) Specialized capability (→ Q10) | E) Balanced | F) I don't know -**Context for user:** Explain the tradeoff concretely: +| Answer | Model Impact | +| -------- | ----------------------------------------------------------- | +| Quality | Claude Sonnet 4.6 primary; Opus 4.7 / 4.6 for hardest tasks | +| Speed | Claude Haiku 4.5; also Nova Micro/Lite | +| Cost | Claude Haiku 4.5 or Nova Micro | +| Special | Deferred to Q10 | +| Balanced | Claude Sonnet 4.6 | -- **Yes — latency critical** — AI calls are in the hot path and every 20–50ms matters (e.g., real-time chat, autocomplete, live transcription) -- **No — latency acceptable** — AI calls are async or users expect a brief wait (e.g., background processing, batch jobs, report generation) -- **Concerned about egress costs** — sending large payloads (images, documents, audio) between GCP and AWS -- **Want to test first** — run both providers in parallel before committing +Interpret → `ai_priority`. Default: E → `"balanced"`. -> Since your infrastructure stays on GCP, AI calls to Bedrock will cross cloud boundaries. This affects latency and data transfer costs. -> -> A) Yes — latency critical, AI calls are in the hot path -> B) No — latency acceptable, async or users can wait -> C) Concerned about egress costs for large payloads -> D) Want to test first — run both providers in parallel +## Q3 — Monthly AI spend on OpenAI or Gemini? -| Answer | Recommendation Impact | -| ---------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| Latency critical | VPC endpoint for Bedrock strongly recommended; us-east-1 or closest region to GCP deployment; latency benchmarks included | -| Latency acceptable | Standard Bedrock endpoint; region selected for cost/availability | -| Concerned about egress | AWS PrivateLink for Bedrock recommended; egress cost analysis included | -| Want to test first | Phased migration plan; parallel running guidance included | +> A) < $500 | B) $500–$2K | C) $2K–$10K | D) > $10K | E) Don't know -Interpret: +Interpret → `ai_monthly_spend`. Default: B → `"$500-$2K"`. -``` -A -> cross_cloud: "latency-critical" — VPC endpoint; closest region to GCP -B -> cross_cloud: "latency-acceptable" — Standard endpoint; region by cost -C -> cross_cloud: "egress-concerned" — PrivateLink; egress analysis -D -> cross_cloud: "test-first" — Phased migration; parallel running -``` +## Q4 — Cross-cloud API call concerns -Default: B — `cross_cloud: "latency-acceptable"`. +Unique to AI-only: infrastructure stays on GCP while AI calls route to AWS. ---- +> A) Latency critical — AI in hot path | B) Latency acceptable — async/users can wait | C) Concerned about egress costs | D) Want to test first — parallel running -### Model & Modality (Q5–Q6) +| Answer | Impact | +| ---------------- | ---------------------------------------------- | +| Latency critical | VPC endpoint; closest region to GCP deployment | +| Acceptable | Standard endpoint; region by cost | +| Egress concerned | PrivateLink; egress cost analysis | +| Test first | Phased migration; parallel running guidance | -Establish the baseline model recommendation and whether multimodal capabilities are needed. +Interpret → `cross_cloud`. Default: B → `"latency-acceptable"`. ---- +## Q5 — Current model in use? -### Q5 — Which model are you currently using? +Establishes baseline Bedrock recommendation. Override hierarchy: Q10 special features > Q2 priority > Q7/Q8 volume/latency > Q5 baseline. -**Rationale:** The source model establishes the baseline Bedrock recommendation — a like-for-like capability match. This is a starting point only; answers to Q2 (priority), Q7 (volume), Q8 (latency), Q9 (complexity), and Q10 (special features) can all override this baseline. +_Skip when:_ `models[].model_id` is populated in `ai-workload-profile.json` — auto-detect from detected model IDs with `chosen_by: "extracted"` and do not present this question. The detected models are already shown in the Step 1 summary. -**Override hierarchy:** +> A) Gemini Flash | B) Gemini Pro | C) GPT-3.5 Turbo | D) GPT-4/4 Turbo | E) GPT-4o | F) GPT-5.4/Mini/Nano | G) GPT-5/5.x (older) | H) GPT-5.5/Pro | I) o-series | J) Claude (Anthropic SDK) | K) Other/Multiple | L) Don't know -1. Q10 special features — hard overrides (e.g., speech-to-speech forces Nova Sonic regardless of source model) -2. Q2 priority — adjusts up or down within the Claude family (e.g., "lowest cost" downgrades Sonnet → Haiku even if source model was GPT-4) -3. Q7/Q8 volume and latency — may adjust toward provisioned throughput or faster models -4. Q5 source model — baseline only, used when no overrides apply +| Source | Baseline Recommendation | Pricing Context | +| -------------- | --------------------------------- | ---------------------------------- | +| Gemini Flash | Claude Haiku 4.5 ($1/$5) | Strong savings | +| Gemini Pro | Claude Sonnet 4.6 ($3/$15) | Comparable tier | +| GPT-3.5 Turbo | Claude Haiku 4.5 ($1/$5) | Faster and cheaper | +| GPT-4/4 Turbo | Claude Sonnet 4.6 ($3/$15) | Major savings (GPT-4T: $10/$30) | +| GPT-4o | Claude Sonnet 4.6 ($3/$15) | Modest savings on output | +| GPT-5.4 | Claude Sonnet 4.6 ($3/$15) | ~5% cheaper on OpenAI; near parity | +| GPT-5.4 Mini | Nova Lite ($0.06/$0.24) | 94% cheaper on Bedrock | +| GPT-5.4 Nano | Nova Micro ($0.035/$0.14) | 87% cheaper on Bedrock | +| GPT-5.4 Pro | Nova 2 Pro ($1.38/$11) | 94% cheaper on Bedrock | +| GPT-5/5.x | Claude Sonnet 4.6 ($3/$15) | Savings story is quality, not cost | +| GPT-5 flagship | Claude Opus 4.7 / 4.6 ($5/$25) | Cheaper than GPT-5 Pro ($15/$120) | +| o-series | Sonnet 4.6 with extended thinking | o1 $15/$60 → significant savings | +| Claude (any) | Same model on Bedrock | Client swap only — no model change | -> The model you're currently using helps me recommend the closest Bedrock equivalent. -> -> A) Gemini Flash (1.5/2.0/2.5 Flash) -> B) Gemini Pro (1.5/2.5/3 Pro) -> C) GPT-3.5 Turbo -> D) GPT-4 / GPT-4 Turbo -> E) GPT-4o -> F) GPT-5 / GPT-5.x -> G) o-series (o1, o3) -> H) Other / Multiple models -> I) I don't know - -| Source Model | Baseline Bedrock Recommendation | Pricing Context | -| ------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------- | -| Gemini Flash variants | Claude Haiku 4.5 ($1/$5) — speed and cost optimized | Strong savings vs Gemini Flash pricing | -| Gemini Pro variants | Claude Sonnet 4.6 ($3/$15) — quality match | Comparable pricing tier | -| GPT-3.5 Turbo | Claude Haiku 4.5 ($1/$5) — cost-equivalent | Haiku is faster and cheaper | -| GPT-4 / GPT-4 Turbo | Claude Sonnet 4.6 ($3/$15) — quality equivalent | Major savings: GPT-4 Turbo is $10/$30 vs Sonnet $3/$15 | -| GPT-4o | Claude Sonnet 4.6 ($3/$15) — performance equivalent | Modest savings on output; input slightly higher on Bedrock | -| GPT-5 / GPT-5.x | Claude Sonnet 4.6 ($3/$15) — performance equivalent | GPT-5 is $1.25/$10 — savings story is quality/features, not cost | -| GPT-5 (flagship use case) | Claude Opus 4.6 ($5/$25) — flagship-to-flagship | Opus still cheaper than GPT-5 Pro ($15/$120) | -| o-series (o1, o3) | Claude Sonnet 4.6 with extended thinking; Opus 4.6 for most demanding | o1 is $15/$60 — significant savings with Sonnet 4.6 at $3/$15 | - -**Example overrides:** - -- GPT-4 user (baseline: Sonnet 4.6) + Q2=lowest cost → **Haiku 4.5** -- Gemini Flash user (baseline: Haiku 4.5) + Q10=extended thinking → **Sonnet 4.6 with extended thinking** -- GPT-4o user (baseline: Sonnet 4.6) + Q10=real-time speech → **Nova Sonic** (Claude has no speech capability) -- GPT-3.5 user (baseline: Haiku 4.5) + Q9=complex reasoning → **Sonnet 4.6** (task complexity overrides cost-equivalent mapping) -- GPT-5 user (baseline: Opus 4.6) + Q2=balanced → **Sonnet 4.6** (priority overrides flagship-to-flagship mapping) - -Interpret: same as Q19 in `clarify-ai.md` → `ai_model_baseline`. - -Default: _(auto-detect)_ — fall back to Q2 priority-based selection. +Override examples: GPT-4 + Q2=cost → Haiku; Flash + Q10=extended thinking → Sonnet; GPT-4o + Q10=speech → Nova 2 Sonic; GPT-5.5 + Q2=cost → Sonnet 4.6. ---- +Interpret → `ai_model_baseline`. Default: auto-detect, fallback Q2 priority-based. -### Q6 — Do you need vision or just text? +### Batch 1 → Save Draft and Present Batch 2 -**Rationale:** Vision capability narrows the model selection to multimodal-capable models only. +After the user responds to Batch 1: -> Vision capability limits which models are available. -> -> A) Text only -> B) Vision required -> C) Audio/Video inputs needed +1. Interpret all Batch 1 answers (apply interpret rules above; apply defaults for skipped questions). +2. Write `$MIGRATION_DIR/preferences-draft.json` with Batch 1 answers: -| Answer | Recommendation Impact | -| ------------------ | ------------------------------------------------------------------------------------- | -| Text only | Full model catalog available; cheapest/fastest text model per Q2 priority | -| Vision required | Claude Sonnet family (multimodal) required; Haiku excluded for vision tasks | -| Audio/Video inputs | Amazon Nova Reel (video) or Nova Sonic (audio); Claude excluded for audio/video input | +```json +{ + "metadata": { + "draft": true, + "batches_completed": ["ai-strategy"], + "batches_remaining": ["ai-technical"], + "migration_type": "ai-only", + "timestamp": "", + "discovery_artifacts": ["ai-workload-profile.json"], + "questions_asked": ["Q1", "Q2", ...], + "questions_defaulted": [...] + }, + "design_constraints": { ... }, + "ai_constraints": { ... } +} +``` -Interpret: same as Q20 → `ai_vision`. +1. Present Batch 2: -Default: A — no constraint (text only). +``` +Got it — your AI strategy preferences are saved. ---- +Last section — 5 questions about your technical requirements, then we're ready to design. +You can answer each, skip individual ones, or say "use defaults for the rest." -### Workload Characteristics (Q7–Q10) +--- Technical Requirements --- +``` -These questions refine the model recommendation based on actual usage patterns — volume, latency, complexity, and specialized features can all override the baseline from Q5. +**"Use defaults for the rest" handling:** If the user says this during Batch 1, apply defaults for all unanswered Batch 1 questions and all Batch 2 questions, then skip directly to Step 3. ---- +### Batch 2 — Technical Requirements (Q6–Q10) -### Q7 — Monthly AI usage volume +## Q6 — What input types must the model accept: text only, images (vision), or audio/video? -**Rationale:** Volume determines whether on-demand or provisioned throughput is more cost-effective. +_Skip when:_ `integration.capabilities_summary` in `ai-workload-profile.json` has definitive values for `vision` AND (`speech_to_text` or `text_to_speech`) — derive from capabilities with `chosen_by: "extracted"` and do not present this question. Only ask if capabilities are unknown or ambiguous (all false with no evidence either way). -> Volume determines pricing strategy — on-demand vs provisioned throughput. -> -> A) Low (< 1M tokens/month) -> B) Medium (1–10M tokens/month) -> C) High (10–100M tokens/month) -> D) Very high (> 100M tokens/month) -> E) I don't know +> A) Text only | B) Vision required | C) Audio/Video inputs -| Answer | Recommendation Impact | -| --------- | -------------------------------------------------------------------------------------- | -| Low | On-demand pricing; no provisioned throughput needed | -| Medium | On-demand with prompt caching analysis | -| High | Provisioned throughput analysis; prompt caching strongly recommended | -| Very high | Provisioned throughput required for cost control; dedicated capacity planning included | +| Answer | Impact | +| ----------- | --------------------------------------------------------------------------------------------------------------- | +| Text only | Full model catalog | +| Vision | Claude Sonnet or Haiku (both support multimodal vision); Nova Micro excluded (text-only) | +| Audio/Video | Nova 2 Sonic (audio); Nova Reel v1 for video (Legacy — EOL Sep 30, 2026); Claude excluded for audio/video input | -Interpret: +Interpret → `ai_vision`. Default: A → no constraint. -``` -A -> ai_token_volume: "<1M" — On-demand; no provisioned throughput -B -> ai_token_volume: "1M-10M" — On-demand; prompt caching analysis -C -> ai_token_volume: "10M-100M" — Provisioned throughput analysis; prompt caching -D -> ai_token_volume: ">100M" — Provisioned throughput required; capacity planning -E -> same as default (B) -``` +## Q7 — Monthly AI usage volume -Default: B — `ai_token_volume: "1M-10M"`. +> A) < 1M tokens | B) 1–10M | C) 10–100M | D) > 100M | E) Don't know ---- +| Answer | Impact | +| --------- | -------------------------------------------------- | +| Low | On-demand; no provisioned throughput | +| Medium | On-demand with prompt caching analysis | +| High | Provisioned throughput analysis; prompt caching | +| Very high | Provisioned throughput required; capacity planning | -### Q8 — How important is response speed? +Interpret → `ai_token_volume`: A → `"low"`, B → `"medium"`, C → `"high"`, D → `"very_high"`. Default: B → `"medium"`. -**Rationale:** Latency requirements can override cost and quality preferences from Q2. +## Q8 — Response speed importance -**Context for user:** When asking, anchor each option in a real scenario: +Present with concrete anchors: Critical = autocomplete/live chat; Important = chat assistant; Flexible = reports/batch. -- **Critical (< 500ms)** — users are staring at a loading spinner; every millisecond matters (e.g., autocomplete, live chat, real-time transcription) -- **Important (< 2s)** — users expect a quick response but a brief pause is acceptable (e.g., chat assistant, search augmentation, inline suggestions) -- **Flexible (2–10s)** — users submit a request and can wait; background or async is fine (e.g., report generation, batch analysis, email drafting) +> A) Critical (< 500ms) | B) Important (< 2s) | C) Flexible (2–10s) -> Latency requirements can override cost and quality preferences. -> -> A) Critical (< 500ms) -> B) Important (< 2s) -> C) Flexible (2–10s) +| Answer | Impact | +| --------- | ------------------------------------------------------------ | +| Critical | Haiku/Nova Micro; streaming required; provisioned throughput | +| Important | Sonnet 4.6 with streaming; standard on-demand | +| Flexible | Any model; batch inference for cost savings | -| Answer | Recommendation Impact | -| ------------------ | ------------------------------------------------------------------------------------------------- | -| Critical (< 500ms) | Claude Haiku 4.5 or Nova Micro; streaming required; provisioned throughput for consistent latency | -| Important (< 2s) | Claude Sonnet 4.6 with streaming; standard on-demand acceptable | -| Flexible (2–10s) | Any model; batch inference considered for cost savings at high volume | +Interpret → `ai_latency`. Default: B → `"important"`. -Interpret: same as Q21 → `ai_latency`. +## Q9 — AI task complexity -Default: B — `ai_latency: "important"`. +Present with concrete examples: Simple = classify/extract/summarize; Moderate = analyze+JSON/few-shot; Complex = multi-turn reasoning/tool use/agentic. ---- +> A) Simple | B) Moderate | C) Complex -### Q9 — How complex are your AI tasks? +| Answer | Impact | +| -------- | ----------------------------------------------------------------------------- | +| Simple | Haiku/Nova Micro sufficient; significant cost savings | +| Moderate | Sonnet 4.6 recommended; Haiku may suffice with prompt engineering | +| Complex | Sonnet 4.6 required; extended thinking considered; Opus 4.7 / 4.6 for hardest | -**Rationale:** Task complexity determines whether a cheaper/faster model can handle the workload or whether a more capable model is required. +Interpret → `ai_complexity`. Default: B → `"moderate"`. -**Context for user:** When asking, give concrete examples so the user doesn't over- or under-estimate: +## Q10 — Specialized features needed -- **Simple** — single-step tasks with short prompts: classify this text, extract these fields, summarize this paragraph -- **Moderate** — multi-step prompts with examples or structured output: analyze this document and return JSON, generate content following a template, few-shot classification -- **Complex** — multi-turn reasoning, tool use, or long chain-of-thought: agentic workflows, code generation with debugging loops, research tasks that require planning and iteration +Same decision logic as Q17 in `clarify-ai.md`. -> Task complexity determines whether cheaper models can handle your workload. -> -> A) Simple (classification, short summaries, extraction) -> B) Moderate (analysis, structured content, few-shot) -> C) Complex (multi-step reasoning, tool use, agentic workflows) +> A) Function calling | B) Ultra-long context (> 300K) | C) Extended thinking | D) Prompt caching | E) RAG optimization | F) Agentic workflows | G) Real-time speed | H) Image generation | I) Conversational speech | J) None -| Answer | Recommendation Impact | -| -------- | ------------------------------------------------------------------------------------------- | -| Simple | Claude Haiku 4.5 or Nova Micro sufficient; significant cost savings vs larger models | -| Moderate | Claude Sonnet 4.6 recommended; Haiku may suffice with prompt engineering | -| Complex | Claude Sonnet 4.6 required; extended thinking considered; Claude Opus 4.6 for hardest tasks | +Interpret → `ai_critical_feature`. Default: J → no override. -Interpret: same as Q22 → `ai_complexity`. +### Batch 2 Complete -Default: B — `ai_complexity: "moderate"`. +After the user responds to Batch 2, interpret all Batch 2 answers and proceed to Step 3. --- -### Q10 — Specialized features needed +## Step 3: Assemble and Write preferences.json -Same decision logic as Q17 in `clarify-ai.md`. +Assemble all interpreted answers from both batches into the final file. If `preferences-draft.json` exists, use it as the base — merge in Batch 2 answers, remove draft-specific metadata fields (`draft`, `batches_completed`, `batches_remaining`), and set `metadata.timestamp` to the current time. -> Some features are only available in specific models. What's your most critical specialized requirement? -> -> A) Function calling / Tool use -> B) Ultra-long context (> 300K tokens) -> C) Extended thinking / Chain-of-thought -> D) Prompt caching -> E) RAG optimization -> F) Agentic workflows -> G) Real-time speed (< 500ms) -> H) Multimodal with image generation -> I) Real-time conversational speech -> J) None — standard features sufficient +Write `$MIGRATION_DIR/preferences.json`: -Interpret: same as Q17 → `ai_critical_feature`. +**Schema — AI-only structure:** -Default: J — no additional override. +| Field | Path | Notes | +| -------------------------- | ----------------------------------------- | ------------------------------------------- | +| `migration_type` | `metadata.migration_type` | `"ai-only"` — downstream skips infra phases | +| `discovery_artifacts` | `metadata.discovery_artifacts` | `["ai-workload-profile.json"]` | +| `questions_asked` | `metadata.questions_asked` | Array of Q IDs actually presented | +| `questions_defaulted` | `metadata.questions_defaulted` | Array of Q IDs where defaults used | +| `questions_extracted` | `metadata.questions_extracted` | Array of Q IDs skipped via auto-detect | +| `target_region` | `design_constraints.target_region` | Derived from GCP region or cross-cloud pref | +| `ai_framework` | `ai_constraints.ai_framework` | From Q1 | +| `ai_priority` | `ai_constraints.ai_priority` | From Q2 | +| `ai_monthly_spend` | `ai_constraints.ai_monthly_spend` | From Q3 | +| `cross_cloud` | `ai_constraints.cross_cloud` | From Q4 (unique to AI-only) | +| `ai_model_baseline` | `ai_constraints.ai_model_baseline` | From Q5 | +| `ai_vision` | `ai_constraints.ai_vision` | From Q6 | +| `ai_token_volume` | `ai_constraints.ai_token_volume` | From Q7 | +| `ai_latency` | `ai_constraints.ai_latency` | From Q8 | +| `ai_complexity` | `ai_constraints.ai_complexity` | From Q9 | +| `ai_critical_feature` | `ai_constraints.ai_critical_feature` | From Q10 | +| `ai_capabilities_required` | `ai_constraints.ai_capabilities_required` | Derived from `capabilities_summary` | ---- - -## Step 3: Write preferences.json +Each `ai_constraints` field uses `{ "value": ..., "chosen_by": "user"|"extracted"|"derived" }` format. No nulls. All schema rules from `clarify.md` apply. -Write `$MIGRATION_DIR/preferences.json` with AI-only structure: +After writing `preferences.json`, delete `$MIGRATION_DIR/preferences-draft.json` if it exists. -```json -{ - "metadata": { - "timestamp": "", - "migration_type": "ai-only", - "discovery_artifacts": ["ai-workload-profile.json"], - "questions_asked": ["Q1", "Q2", "Q3", "Q4", "Q5", "Q6", "Q7", "Q8", "Q9", "Q10"], - "questions_defaulted": [], - "questions_skipped_extracted": [], - "questions_skipped_not_applicable": [] - }, - "design_constraints": { - "target_region": { "value": "us-east-1", "chosen_by": "derived" } - }, - "ai_constraints": { - "ai_framework": { "value": ["langchain"], "chosen_by": "extracted" }, - "ai_priority": { "value": "balanced", "chosen_by": "user" }, - "ai_monthly_spend": { "value": "$500-$2K", "chosen_by": "user" }, - "cross_cloud": { "value": "latency-acceptable", "chosen_by": "user" }, - "ai_model_baseline": { "value": "claude-sonnet-4-6", "chosen_by": "derived" }, - "ai_vision": { "value": "text-only", "chosen_by": "user" }, - "ai_token_volume": { "value": "1M-10M", "chosen_by": "user" }, - "ai_latency": { "value": "important", "chosen_by": "user" }, - "ai_complexity": { "value": "moderate", "chosen_by": "user" }, - "ai_critical_feature": { "value": "none", "chosen_by": "user" }, - "ai_capabilities_required": { - "value": ["text_generation", "streaming"], - "chosen_by": "derived" - } - } -} -``` +--- -### Schema Notes (AI-Only) +## Step 4: Update Phase Status -1. `metadata.migration_type` is `"ai-only"` — downstream phases use this to skip infrastructure design/estimation. -2. `design_constraints` is minimal — only `target_region` (derived from GCP deployment region or cross-cloud latency preference). -3. `ai_constraints.cross_cloud` is unique to AI-only migrations — not present in full migration preferences. -4. `ai_constraints.ai_token_volume` uses different tiers than full migration Q18 — more granular for AI-only cost analysis. -5. All other schema rules from `clarify.md` apply (value/chosen_by fields, no nulls, derived capabilities). +Before phase completion, enforce output gate: ---- +- `preferences.json` must exist. +- `preferences.json.metadata.migration_type` must equal `"ai-only"`. -## Step 4: Update Phase Status +If either check fails: STOP and output: "AI-only clarify output validation failed. Fix `preferences.json` before completing Phase 2." -Update `$MIGRATION_DIR/.phase-status.json`: +Use the Phase Status Update Protocol (read-merge-write) to update `.phase-status.json` in the same turn as the output message: - Set `phases.clarify` to `"completed"` -- Update `last_updated` to current timestamp +- Set `current_phase` to `"design"` -Output to user: "Clarification complete. Proceeding to Phase 3: Design AI Migration Architecture." +Output: "Clarification complete. Proceeding to Phase 3: Design AI Migration Architecture." diff --git a/migration-to-aws/steering/clarify-ai.md b/migration-to-aws/steering/clarify-ai.md index f570cd8b..acd3c215 100644 --- a/migration-to-aws/steering/clarify-ai.md +++ b/migration-to-aws/steering/clarify-ai.md @@ -2,27 +2,103 @@ _Fire when:_ `ai-workload-profile.json` exists in `$MIGRATION_DIR/`. -These questions refine the AI migration approach — framework/orchestration layer, model selection, cost projections, and performance requirements. - --- ## AI Context Summary -Before presenting questions, show the AI detection context: +Before presenting questions, show: > **AI Context Summary:** > **AI source:** [from `summary.ai_source`: "Gemini", "OpenAI", "Both", or "Other"] -> **Models detected:** [from `models[].model_id`] -> **Capabilities in use:** [from `integration.capabilities_summary` where true] -> **Integration pattern:** [from `integration.pattern`] via [from `integration.primary_sdk`] +> **Profile origin:** [from `metadata.profile_source`: if `iac_vertex` or `summary.inferred_from_iac` is true, state that Terraform was the primary signal and application code did not fully characterize the workload] +> **Models detected:** [from `models[].model_id`; if empty, say **None inferred from code or IaC** — the following questions will pin down models and frameworks] +> **Capabilities in use:** [from `integration.capabilities_summary` where true; if all false or pattern is `unknown`, say **Not inferred — confirm below**] +> **Integration pattern:** [from `integration.pattern`; if `unknown`, say **Unknown (IaC-only)**] via [from `integration.primary_sdk`, or **not determined**] > **Gateway/router:** [from `integration.gateway_type`, or "None (direct SDK)"] > **Frameworks:** [from `integration.frameworks`, or "None"] --- -## Q14 — What AI framework or orchestration layer are you using? (select all that apply) +## Q14 — Framework auto-detection signals + +**Auto-detect signals** — scan IaC and application code before asking: + +--- + +## Multi-Workload Confirmation Table (if `workloads[]` has ≥ 2 entries) + +**Fire when:** `ai-workload-profile.json` contains a non-empty `workloads[]` array with 2 or more entries. This replaces the per-workload Q16–Q22 loop with a single confirmation table. + +Before presenting Q16–Q22, show the detected workloads and proposed Bedrock targets: + +> **Detected AI Workloads:** +> +> | # | Model | SDK Method | Capability | Confidence | Proposed Bedrock Target | +> | - | ----------------------- | ---------------------------- | ----------------- | ---------- | ----------------------------- | +> | 1 | gemini-2.5-flash | generateContent | text_generation | medium | [text-class per Q16 priority] | +> | 2 | gemini-2.5-flash | generateContent (structured) | structured_output | high | [same text-class as row 1] | +> | 3 | imagen-3.0-generate-001 | generateImages | image_generation | high | [image-class model] | +> +> **For each row, you can:** +> +> - **Accept** — keep the proposed mapping +> - **Edit** — change the capability or Bedrock target +> - **Drop** — this isn't an AI workload (false positive) +> +> _(v1: merge and split actions are planned for v2)_ +> +> _Do you accept all mappings? Or type the row number to edit._ + +**Timing:** This table fires AFTER existing global/infra questions complete (Q1–Q15 and Q14–Q15 AI globals). It does not replace or conflict with the master Clarify orchestrator or PR #57 auto-extraction — those run first, and their answers feed into the capability confirmation. + +**Behavior:** + +1. **High-confidence rows (confidence = `high`):** Pre-fill Bedrock target from `capability → Bedrock model` mapping. Do NOT ask Q16–Q22 for these rows unless the user edits. + +2. **Medium/low-confidence rows:** Ask at most 2 questions per row: + - "Is the detected capability correct?" (confirm or select from: text_generation, structured_output, image_generation, embedding, speech_to_text, text_to_speech, unknown) + - "What matters most for this workload?" (Q16 priority: quality/speed/cost/balanced) -**Rationale:** The orchestration layer determines the migration surface area more than the model choice. A LangGraph app with multi-agent tool-calling has a fundamentally different migration path than raw API calls routed through a gateway. Understanding the full AI stack upfront determines whether to migrate orchestration to Bedrock Agents, keep the existing framework with a Bedrock provider swap, adopt a hybrid approach, or simply change a gateway config. +3. **Target mapping** (default, overridden by user edits — look up actual model IDs from design-refs tables, not hardcoded names): + + | Capability | Target Class | Notes | + | ----------------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | + | text_generation | Text/reasoning class | Apply Q16–Q19 override hierarchy | + | structured_output | Text/reasoning class (same as text_generation) | Uses same Bedrock target as text_generation for that workload's priority tier — structured output is a mode, not a different model | + | image_generation | Image generation class | e.g., Nova Canvas | + | embedding | Embedding class | e.g., Titan Embed Text v2 | + | speech_to_text | Speech-to-text class | e.g., Transcribe | + | text_to_speech | Text-to-speech class | e.g., Polly | + | unknown | Ask Q16–Q22 for this workload | Falls back to full questionnaire | + +4. **After confirmation — persist workloads[] to preferences.json (REQUIRED):** + + Immediately after the user confirms (accepts all, or finishes editing individual rows), write the final `workloads[]` array to `preferences.json`. This is the **single source of truth** for all downstream phases (Design, Estimate, Generate). Do NOT rely on `ai-workload-profile.json` downstream — it contains the raw Discover output before user edits/drops. + + **Write rules:** + - Read existing `preferences.json` (preserving all non-AI fields written by earlier Clarify categories) + - Add or overwrite the top-level `workloads` key with the confirmed array + - Each entry MUST include: `workload_id`, `model_id`, `sdk_method`, `capability`, `capability_confidence`, `structured_output`, `call_sites`, `target_bedrock_model`, and the user's `priority`/`latency_tier` selections (use defaults `"balanced"`/`"standard"` for high-confidence rows that were auto-accepted) + - Dropped rows are excluded from `workloads[]` — they do not appear + - Write the file atomically (write to `.tmp`, then rename) to prevent partial writes on failure + - If write fails: STOP. Output: "Failed to persist workloads to preferences.json — do not proceed to Design." + + **Single-workload case:** If only 1 workload exists (confirmation table skipped), persist it to `workloads[]` after Q16–Q22 completes, using the same schema. Design always reads `workloads[]` regardless of count. + + **Zero-workload case:** If no AI workloads detected and user doesn't report any, write `"workloads": []` to preferences.json. Design emits empty `design_blocks[]` for this case. + +5. **Question budget:** 4 global questions (Q14, Q15, framework, spend) + at most 2 per medium/low workload. For an app with 3 high-confidence workloads: 4 questions total, 0 per-workload. For an app with 2 high + 1 medium: 4 + 2 = 6 questions max. + +**Single-workload fallback:** If `workloads[]` has exactly 1 entry or is empty, skip the confirmation table and proceed with the existing Q16–Q22 flow below. + +**Known limitations (v1):** + +- Gateway calls (LiteLLM, OpenRouter) and custom HTTP calls to AI endpoints are not yet detected as separate workloads — they may be miscategorized or missed. Planned for v2. +- Merge and split actions are not supported in v1. Users who need to combine or split workloads should edit individual rows. + +--- + +## Q14 — What AI framework or orchestration layer are you using? (select all that apply) **Auto-detect signals** — scan IaC and application code before asking: @@ -42,7 +118,7 @@ Before presenting questions, show the AI detection context: - `mcp.server` / `mcp.client` imports, MCP config JSON files → F - A2A protocol config or SDK imports → F - Vapi, Bland.ai, Retell SDK imports → G -- Nova Sonic or Whisper integration in code → G +- Nova Sonic / Nova 2 Sonic or Whisper integration in code → G _Skip when:_ Auto-detection fully resolves the framework(s). Use detected value(s) with `chosen_by: "extracted"`. @@ -58,98 +134,59 @@ _Skip when:_ Auto-detection fully resolves the framework(s). Use detected value( > > _(Multiple selections allowed)_ -| Answer | Recommendation Impact | Migration Effort | Timeline | -| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | -------------------------------------------------- | -| A) No framework — direct API calls | Swap OpenAI/Gemini SDK calls to Bedrock SDK; recommend evaluating Bedrock Agents if planning agentic capabilities | Low | 1–3 weeks depending on call sites | -| B) LLM router/gateway | Add Bedrock as provider in gateway config, swap model IDs; no app code changes; verify gateway supports AWS SigV4 auth | Minimal | Hours to 1–3 days | -| C) LangChain / LangGraph | Provider swap to Bedrock via `ChatBedrock` class; existing chains/graphs/tools preserved; validate tool schemas with Bedrock tool-use format | Low | 1–3 days; 1 week if complex agent graphs | -| D) Multi-agent framework | Two paths: 1) Keep framework, swap to Bedrock as LLM provider — lower effort, 2) Migrate to Bedrock multi-agent orchestration — higher effort, deeper AWS integration; recommend path 1 unless managed infrastructure wanted | Medium | Path 1: 3–5 days; Path 2: 2–4 weeks | -| E) OpenAI Agents SDK / custom agent loop | Highest effort; OpenAI Agents SDK is tightly coupled to OpenAI API with no provider swap; recommend Bedrock Agents as replacement or LangGraph as portable intermediate step; tool-calling schema translation required | High | 2–4 weeks | -| F) MCP servers or A2A protocol | Bedrock Agents supports MCP tool use natively; A2A interop available; recommend Bedrock Agents as orchestration layer to preserve MCP/A2A investments | Low–Medium | 3–5 days for MCP; 1–2 weeks if A2A refactoring | -| G) Voice/conversational agent platform | Check if platform supports Bedrock natively — if yes, config change only; if no, evaluate Nova Sonic as replacement for voice layer | Minimal to Medium | Hours if native; 2–3 weeks if Nova Sonic migration | +| Answer | Recommendation Impact | Migration Effort | Timeline | +| ---------------------------------- | --------------------------------------------------------------------------------------------------------- | ----------------- | ---------------------------------------------------- | +| A) No framework — direct API calls | Swap SDK calls to Bedrock SDK; evaluate Bedrock Agents if planning agentic | Low | 1–3 weeks depending on call sites | +| B) LLM router/gateway | Add Bedrock as provider in gateway config; no app code changes; verify SigV4 auth | Minimal | Hours to 1–3 days | +| C) LangChain / LangGraph | Provider swap via `ChatBedrock`; chains/graphs/tools preserved; validate tool schemas | Low | 1–3 days; 1 week if complex graphs | +| D) Multi-agent framework | Path 1: Keep framework, swap LLM provider (lower effort). Path 2: Migrate to Bedrock multi-agent (deeper) | Medium | Path 1: 3–5 days; Path 2: 2–4 weeks | +| E) OpenAI Agents SDK | Highest effort; tightly coupled to OpenAI API; recommend Bedrock Agents or LangGraph as portable step | High | 2–4 weeks | +| F) MCP / A2A | Bedrock Agents supports MCP natively; A2A interop available; recommend Bedrock Agents as orchestration | Low–Medium | 3–5 days MCP; 1–2 weeks A2A | +| G) Voice platform | If platform supports Bedrock natively → config change; otherwise evaluate Nova 2 Sonic | Minimal to Medium | Hours if native; 2–3 weeks if Nova 2 Sonic migration | ### Combination Logic -| Combination | Approach | -| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- | -| A only | Simplest path — direct SDK migration | -| B only | Quick win — gateway config change, skip SDK migration steps | -| B + any other | Gateway swap is the quick win; assess framework migration as separate workstream | -| C + A | Two workstreams: LangChain provider swap (fast) + direct call migration (slower) | -| D + F | Complex — multi-agent with MCP tooling; recommend Bedrock Agents to unify orchestration and tool access | -| E + anything | E is the long pole; plan timeline around the Agents SDK migration; other layers may be quick wins | -| Multiple frameworks (C+D, C+E, etc.) | Assess each independently; prioritize by traffic volume or business criticality; recommend consolidating to one framework post-migration | +| Combination | Approach | +| ------------------------------------ | ------------------------------------------------------------------------------------------------------ | +| A only | Simplest path — direct SDK migration | +| B only | Quick win — gateway config change, skip SDK migration steps | +| B + any other | Gateway swap is the quick win; assess framework migration as separate workstream | +| C + A | Two workstreams: LangChain provider swap (fast) + direct call migration (slower) | +| D + F | Complex — multi-agent with MCP tooling; recommend Bedrock Agents to unify orchestration and tools | +| E + anything | E is the long pole; plan timeline around Agents SDK migration; other layers may be quick wins | +| Multiple frameworks (C+D, C+E, etc.) | Assess independently; prioritize by traffic volume or business criticality; consolidate post-migration | -**Key principle:** When a framework is detected, lead with "here's how your orchestration layer maps to AWS" rather than "here's which model replaces yours." Model recommendation from subsequent questions becomes a sub-decision within the framework migration. - -_Note: If answer includes B and no other selections, skip or abbreviate SDK migration steps. If answer is A only, proceed with standard model migration flow._ - -Interpret: - -``` -A -> ai_framework: ["direct"] — Full SDK migration required -B -> ai_framework: ["llm-router"] — Config change only (1-3 days) -C -> ai_framework: ["langchain"] — Provider swap via ChatBedrock -D -> ai_framework: ["multi-agent"] — Two migration paths available -E -> ai_framework: ["openai-agents"] — Highest effort; Bedrock Agents recommended -F -> ai_framework: ["mcp-a2a"] — Bedrock Agents for orchestration -G -> ai_framework: ["voice-platform"] — Check native Bedrock support -Multiple -> ai_framework: [array of all selected values] -``` +If answer includes B and no other selections, skip or abbreviate SDK migration steps. If answer is A only, proceed with standard model migration flow. -Default: _(auto-detect from code)_ — fall back to `["direct"]` if detection fails. +Interpret → `ai_framework` array (multiple selections → array of all selected values). Default: auto-detect from code, fallback `["direct"]`. --- ## Q15 — Approximately how much are you spending on OpenAI or Gemini per month? -**Rationale:** AI spend drives the IW Migrate credits calculation at the 35% rate (vs 25% for infrastructure-only). Also provides the cost engine with a baseline for Bedrock cost comparison when detailed billing CSV is not uploaded. - -**Pricing context:** GPT-5 ($1.25/$10 per 1M tokens) is actually cheaper than GPT-4o ($2.50/$10) and significantly cheaper than GPT-4 Turbo ($10/$30). Users on GPT-4 Turbo have the strongest cost savings case for migrating to Bedrock — Claude Sonnet 4.6 at $3/$15 is 70% cheaper on input tokens. Users on GPT-5 will see more modest savings since GPT-5 is already competitively priced. - -> AI spend helps me calculate your migration credits eligibility (35% rate for AI workloads) and establish a cost baseline for Bedrock comparison. -> > A) < $500/month > B) $500–$2,000/month > C) $2,000–$10,000/month > D) > $10,000/month > E) I don't know -| Answer | Recommendation Impact | -| -------------------- | ------------------------------------------------------------------------------------------------- | -| < $500/month | AWS Activate or low-tier IW Migrate credits; Bedrock cost comparison shows modest savings | -| $500–$2,000/month | IW Migrate credits at 35% of ARR; Bedrock cost comparison highlighted | -| $2,000–$10,000/month | Significant IW Migrate credits; Bedrock cost savings prominently featured; Savings Plans analysis | -| > $10,000/month | MAP eligibility likely; dedicated AI migration support; Bedrock provisioned throughput analysis | - -Interpret: +| Answer | Recommendation Impact | +| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| < $500/month | **AWS Activate Founders** (up to $5,000 credits, self-service, no VC needed — apply at aws.amazon.com/startups/credits); Bedrock free tier covers initial testing; Bedrock cost comparison shows modest savings | +| $500–$2,000/month | **AWS Activate Portfolio** (up to $200,000 credits for VC/accelerator-backed startups — requires Activate Provider Org ID); Bedrock cost comparison highlighted; credits apply to Bedrock third-party models including Claude | +| $2,000–$10,000/month | **AWS Activate Portfolio** (up to $200,000); Bedrock cost savings prominently featured; Savings Plans analysis; if agentic workload detected → flag **AWS Generative AI Accelerator** (up to $1M credits, cohort-based, adjacent to the credits-hub funnel) | +| > $10,000/month | **AWS Credits for AI Startups** ($200,000+, invite-only for startups ready to scale post-Activate-Portfolio — contact your AWS Account Manager; see aws.amazon.com/startups/credits); dedicated AI migration support; Bedrock provisioned throughput analysis; if agentic workload detected → also flag **AWS Generative AI Accelerator** (up to $1M credits, 8-week cohort — aws.amazon.com/startups/generative-ai/accelerator) | -``` -A -> ai_monthly_spend: "<$500" — Activate or low-tier IW Migrate -B -> ai_monthly_spend: "$500-$2K" — IW Migrate at 35% of ARR -C -> ai_monthly_spend: "$2K-$10K" — Significant IW Migrate; Savings Plans analysis -D -> ai_monthly_spend: ">$10K" — MAP eligibility; provisioned throughput analysis -E -> same as default (B) -``` +**Activate eligibility (Founders & Portfolio):** Pre-Series B, founded in the last 10 years, AWS Account on Paid Tier Plan, and either new to Activate Credits or requesting more credits than previously received. -Default: B — `ai_monthly_spend: "$500-$2K"`. +Interpret → `ai_monthly_spend`. Default: B → `"$500-$2K"`. --- ## Q16 — What matters most for your AI workloads? -**Rationale:** The primary priority is the top-level filter for Bedrock model selection. Quality, speed, and cost point to different model families. - -**Context for user:** When asking, help the user think about their actual priority rather than defaulting to "best quality": +Present with concrete anchors: Quality = legal analysis/code gen; Speed = autocomplete/live chat; Cost = classification/tagging at scale; Specialized = specific feature (→ Q17); Balanced = all-rounder. -- **Best quality/reasoning** — accuracy and depth matter most; you'd pay more or wait longer for better answers (e.g., legal analysis, complex code generation, medical summarization) -- **Fastest speed** — response time is the primary constraint; users are waiting in real-time (e.g., chat UI, autocomplete, live suggestions) -- **Lowest cost** — volume is high and budget is tight; good-enough quality at scale (e.g., classification, tagging, bulk summarization) -- **Specialized capability** — you rely on a specific feature like vision, function calling, or extended thinking (covered in Q17) -- **Balanced** — no single dimension dominates; you want a solid all-rounder - -> This determines our model selection strategy and optimization approach. -> > A) Best quality/reasoning — accuracy matters most, willing to pay more > B) Fastest speed — response time is the primary constraint > C) Lowest cost — high volume, budget tight, good-enough quality at scale @@ -157,35 +194,20 @@ Default: B — `ai_monthly_spend: "$500-$2K"`. > E) Balanced — no single dimension dominates > F) I don't know -| Answer | Recommendation Impact | -| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| Best quality/reasoning | Claude Sonnet 4.6 (latest, highest reasoning in Sonnet family) — primary; Claude Opus 4.6 for most demanding reasoning tasks | -| Fastest speed | Claude Haiku 4.5 — lowest latency in Claude family; also consider Amazon Nova Micro/Lite for cost-optimized speed | -| Lowest cost | Claude Haiku 4.5 or Amazon Nova Micro — lowest cost per token | -| Specialized capability | Deferred to Q17 to determine which model | -| Balanced | Claude Sonnet 4.6 as default balanced recommendation | +| Answer | Recommendation Impact | +| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Best quality/reasoning | Claude Sonnet 4.6 (latest, highest reasoning in Sonnet family) — primary; Claude Opus 4.7 for the most demanding reasoning tasks (same headline on-demand $5/$25 as Opus 4.6 on standard Bedrock pricing); Claude Opus 4.6 remains a valid alternative | +| Fastest speed | Claude Haiku 4.5 — lowest latency in Claude family; also consider Amazon Nova Micro/Lite for cost-optimized speed | +| Lowest cost | Claude Haiku 4.5 or Amazon Nova Micro — lowest cost per token | +| Specialized capability | Deferred to Q17 to determine which model | +| Balanced | Claude Sonnet 4.6 as default balanced recommendation | -Interpret: - -``` -A -> ai_priority: "quality" — Claude Sonnet 4.6; Opus 4.6 for hardest tasks -B -> ai_priority: "speed" — Haiku 4.5 or Nova Micro -C -> ai_priority: "cost" — Haiku 4.5 or Nova Micro -D -> ai_priority: "specialized" — deferred to Q17 -E -> ai_priority: "balanced" — Claude Sonnet 4.6 -F -> same as default (E) -``` - -Default: E — `ai_priority: "balanced"`. +Interpret → `ai_priority`. Default: E → `"balanced"`. --- ## Q17 — What is your MOST CRITICAL specialized AI feature? -**Rationale:** Specialized features can override the priority-based model selection from Q16. Some features are only available in specific models. - -> Some features are only available in specific models. This helps me ensure the recommended model supports your critical requirement. -> > A) Function calling / Tool use > B) Ultra-long context (> 300K tokens) > C) Extended thinking / Chain-of-thought @@ -197,44 +219,25 @@ Default: E — `ai_priority: "balanced"`. > I) Real-time conversational speech > J) None — standard features are sufficient -| Answer | Recommendation Impact | -| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | -| Function calling / Tool use | Claude Sonnet 4.6 — best-in-class tool use on Bedrock via structured JSON tool schemas; supports parallel tool calls and multi-turn tool use | -| Ultra-long context (> 300K tokens) | Claude Sonnet 4.6 — supports 1M token context window (beta); no chunking strategy required for most use cases | -| Extended thinking / Chain-of-thought | Claude Sonnet 4.6 with extended thinking mode; Claude Opus 4.6 for most complex reasoning | -| Prompt caching | Claude Sonnet 4.6 with prompt caching enabled; cost savings analysis included | -| RAG optimization | Amazon Bedrock Knowledge Bases recommended alongside model; Titan Embeddings for vector store | -| Agentic workflows | Claude Sonnet 4.6 with Bedrock Agents; multi-agent orchestration guidance included | -| Real-time speed (< 500ms) | Claude Haiku 4.5 or Nova Micro; streaming response guidance included | -| Multimodal with image generation | Claude Sonnet 4.6 (vision) + Amazon Nova Canvas or Titan Image Generator for generation | -| Real-time conversational speech | Amazon Nova Sonic recommended for speech-to-speech; latency guidance included | -| None | Default recommendation from Q16 priority stands | - -Interpret: - -``` -A -> ai_critical_feature: "function-calling" — Claude Sonnet 4.6 -B -> ai_critical_feature: "long-context" — Claude Sonnet 4.6 (1M context) -C -> ai_critical_feature: "extended-thinking" — Claude Sonnet 4.6 extended thinking; Opus 4.6 -D -> ai_critical_feature: "prompt-caching" — Claude Sonnet 4.6 with caching -E -> ai_critical_feature: "rag" — Bedrock Knowledge Bases + Titan Embeddings -F -> ai_critical_feature: "agentic" — Claude Sonnet 4.6 + Bedrock Agents -G -> ai_critical_feature: "real-time-speed" — Haiku 4.5 or Nova Micro -H -> ai_critical_feature: "multimodal-generation" — Sonnet 4.6 + Nova Canvas/Titan Image -I -> ai_critical_feature: "speech" — Nova Sonic (hard override — Claude has no speech) -J -> (no constraint written — Q16 priority stands) -``` - -Default: J — no additional override. +| Answer | Recommendation Impact | +| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Function calling / Tool use | Claude Sonnet 4.6 — best-in-class tool use on Bedrock via structured JSON tool schemas; supports parallel tool calls and multi-turn tool use | +| Ultra-long context (> 300K tokens) | Claude Sonnet/Opus 4.6 long-context SKUs where available (standard on-demand for 4.6 matches base tier in US East N. Virginia per [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/)); or Llama 4 Scout (10M), Llama 4 Maverick (1M), Nova 2 Pro/Lite (1M) for very large native context windows | +| Extended thinking / Chain-of-thought | Claude Sonnet 4.6 with extended thinking mode; Claude Opus 4.6 for most complex reasoning | +| Prompt caching | Claude Sonnet 4.6 with prompt caching enabled; cost savings analysis included. **Caveat:** caching only helps for long, repeated context (system prompts, documents). Per-model minimum token thresholds (~1K–4K tokens) and TTL apply — short prompts won't cache. Verify current minimums at docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html before recommending. | +| RAG optimization | Amazon Bedrock Knowledge Bases recommended alongside model; Titan Embeddings for vector store | +| Agentic workflows | Claude Sonnet 4.6 with Bedrock Agents; multi-agent orchestration guidance included | +| Real-time speed (< 500ms) | Claude Haiku 4.5 or Nova Micro; streaming response guidance included | +| Multimodal with image generation | Claude Sonnet 4.6 (vision) + Amazon Nova Canvas for generation | +| Real-time conversational speech | Amazon Nova 2 Sonic recommended for speech-to-speech; latency guidance included | +| None | Default recommendation from Q16 priority stands | + +Interpret → `ai_critical_feature`. Default: J → no override. --- ## Q18 — What's your AI usage volume and cost tolerance? -**Rationale:** Volume at scale changes the economics significantly. High-volume workloads should use provisioned throughput or cheaper models even if quality is slightly lower. - -> Volume determines whether on-demand or provisioned throughput is more cost-effective, and whether we should optimize for cost over quality. -> > A) Low volume + quality priority — small-scale, quality matters most > B) Medium volume + balanced — moderate production use, balanced approach > C) High volume + cost critical — high scale, budget is tight, need cost control @@ -245,118 +248,100 @@ Default: J — no additional override. | Medium volume + balanced | On-demand Claude Sonnet or Haiku depending on Q16; Savings Plans analysis | | High volume + cost critical | **Provisioned throughput strongly recommended**; Claude Haiku or Nova Micro; prompt caching analysis included | -Interpret: - -``` -A -> ai_volume_cost: "low-quality" — On-demand; quality model -B -> ai_volume_cost: "medium-balanced" — On-demand; Savings Plans analysis -C -> ai_volume_cost: "high-cost-critical" — Provisioned throughput; cheaper models; prompt caching -``` - -Default: A — `ai_volume_cost: "low-quality"`. +Interpret → `ai_token_volume`: A → `"low"`, B → `"medium"`, C → `"high"`. Default: A → `"low"`. --- ## Q19 — Which Gemini or OpenAI model are you currently using? -**Rationale:** The source model establishes the baseline Bedrock recommendation — a like-for-like capability match. This is a starting point only; answers to Q16 (priority), Q17 (features), Q18 (volume), Q21 (latency), and Q22 (complexity) can all override this baseline. - -**Override hierarchy:** - -1. Q17 special features — hard overrides (e.g., speech-to-speech forces Nova Sonic regardless of source model) -2. Q16 priority — adjusts up or down within the Claude family -3. Q18/Q21 volume and latency — may further adjust toward provisioned throughput or faster models -4. Q19 source model — baseline only, used when no overrides apply +**Auto-detect signal:** If `ai-workload-profile.json` exists and `models[0].model_id` is set with detection confidence ≥ 0.8, map to the matching Q19 answer and **skip Q19**. Set `ai_model_baseline` with `chosen_by: "extracted"`. If multiple models detected with similar confidence, ask Q19. + +_Skip when:_ Primary model fully resolved from discovery. Use detected value with `chosen_by: "extracted"`. + +Establishes baseline Bedrock recommendation. **Override hierarchy:** Q17 special features (hard override) > Q16 priority > Q18/Q21 volume and latency > Q19 source model (baseline only). + +> A) Gemini 3.5 Flash (GA — current flagship Flash model) +> B) Gemini 3.5 Flash Thinking (thinking budget enabled) +> C) Gemini 3.1 Pro +> D) Gemini 3.1 Flash-Lite (high-volume, low-cost) +> E) Gemini 2.5 Flash (standard, no thinking budget) +> F) Gemini 2.5 Flash Thinking (thinking budget enabled — variable output pricing) +> G) Gemini 2.5 Pro +> H) Gemini 3 Pro / 3.1 Pro Preview +> I) Gemini Flash (2.0 Flash) or Gemini Flash 1.5 _(EOL Sep 2025 — flag for source model upgrade)_ +> J) Gemini Pro 1.5 _(EOL Sep 2025 — flag for source model upgrade)_ +> K) GPT-3.5 Turbo +> L) GPT-4 / GPT-4 Turbo +> M) GPT-4o +> N) GPT-5.4 / GPT-5.4 Mini / GPT-5.4 Nano +> O) GPT-5 / GPT-5.x (older) +> P) GPT-5.5 / GPT-5.5 Pro +> Q) o-series (o1, o3) +> R) Other / Multiple models +> S) I don't know + +| Source Model | Baseline Bedrock Recommendation | Pricing Context | +| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Gemini 3.5 Flash (GA) | Nova Lite ($0.06/$0.24) — 94% cheaper | Gemini 3.5 Flash is $1.50/$9.00 — 5x more expensive than old 2.5 Flash; Nova Lite is the cost-equivalent; very strong migration case | +| Gemini 3.5 Flash Thinking | Claude Sonnet 4.6 with extended thinking ($3/$15) | At $1.50/$9.00 base + thinking tokens, Sonnet 4.6 is comparable or cheaper at full thinking; profile actual thinking usage before committing | +| Gemini 3.1 Flash-Lite | Nova Micro ($0.035/$0.14) — 88% cheaper; or Nova Lite ($0.06/$0.24) — 76% cheaper | Gemini 3.1 Flash-Lite is $0.25/$1.50; strong Bedrock cost case | +| Gemini 2.5 Flash (standard) | Nova Lite ($0.06/$0.24) — 88% cheaper | Gemini 2.5 Flash is $0.30/$2.50; Nova Lite is the cost-equivalent; strong migration case | +| Gemini 2.5 Flash Thinking | Claude Sonnet 4.6 with extended thinking ($3/$15) | Thinking output pricing on Gemini 2.5 Flash ranges $0.60–$3.50/M depending on thinking budget; at full thinking budget Sonnet 4.6 is comparable or cheaper; flag that thinking token costs vary and user should profile their actual thinking budget usage before committing | +| Gemini 2.5 Pro | Nova 2 Pro ($1.38/$11) — 9% cheaper; or Nova Pro ($0.80/$3.20) — 62% cheaper | Gemini 2.5 Pro is $1.25/$10; migration case is cost + AWS consolidation | +| Gemini 3 Pro / 3.1 Pro | Claude Sonnet 4.6 ($3/$15) — agentic reliability; or Nova 2 Pro ($1.38/$11) — cost | Gemini 3.1 Pro is $2/$12 — cheaper than Sonnet 4.6; migration case is agentic reliability and AWS ecosystem, NOT cost. Be honest: Gemini 3.1 Pro leads on general benchmarks. | +| Gemini Flash 1.5 / 2.0 (older) | Nova Lite ($0.06/$0.24) or Nova Micro ($0.035/$0.14) — **flag Gemini 1.5 Flash as EOL (Sep 2025); recommend upgrading source model to 3.5 Flash before or alongside migration** | Strong Bedrock cost savings; 1.5 Flash is past EOL so migration is doubly urgent | +| Gemini Pro 1.5 (older) | Claude Sonnet 4.6 ($3/$15) — **flag Gemini 1.5 Pro as EOL (Sep 2025); recommend upgrading source model to 3.1 Pro before or alongside migration** | 1.5 Pro is past EOL; migration to Bedrock and source model upgrade should be planned together | +| GPT-3.5 Turbo | Claude Haiku 4.5 ($1/$5) — cost-equivalent | Haiku is faster and cheaper | +| GPT-4 / GPT-4 Turbo | Claude Sonnet 4.6 ($3/$15) — quality equivalent | Major savings: GPT-4 Turbo is $10/$30 vs Sonnet $3/$15 | +| GPT-4o | Claude Sonnet 4.6 ($3/$15) — performance equivalent | Modest savings on output; input slightly higher on Bedrock | +| GPT-5.4 | Claude Sonnet 4.6 ($3/$15) — near price parity | GPT-5.4 is $2.50/$15 — ~5% cheaper; migration case is AWS consolidation, not cost | +| GPT-5.4 Mini | Nova Lite ($0.06/$0.24) — massive cost savings | 94% cheaper on Bedrock; strong migration case | +| GPT-5.4 Nano | Nova Micro ($0.035/$0.14) — massive cost savings | 87% cheaper on Bedrock; strong migration case | +| GPT-5.4 Pro | Nova 2 Pro ($1.38/$11) — flagship reasoning on AWS | 94% cheaper on Bedrock; strongest migration case | +| GPT-5 / GPT-5.x (older) | Claude Sonnet 4.6 ($3/$15) — performance equivalent | GPT-5 is $1.25/$10 — savings story is quality/features, not cost | +| GPT-5 (flagship use case) | Claude Opus 4.6 ($5/$25) — flagship-to-flagship | Opus still cheaper than GPT-5 Pro ($15/$120) | +| GPT-5.5 | Claude Opus 4.6 ($5/$25) — flagship-to-flagship | Bedrock 17% cheaper on output ($25 vs $30); same input price | +| GPT-5.5 (cost-sensitive) | Claude Sonnet 4.6 ($3/$15) — 53% cheaper | Strong cost case; Sonnet leads on agentic reliability | +| GPT-5.5 Pro | Nova 2 Pro ($1.38/$11) — flagship reasoning on AWS | 95% cheaper on Bedrock; strongest migration case | +| o-series (o1, o3) | Claude Sonnet 4.6 with extended thinking; Opus 4.6 for most demanding | o1 is $15/$60 — significant savings with Sonnet 4.6 at $3/$15 | + +**Override examples:** GPT-4 + Q16=cost → Haiku; Flash + Q17=extended thinking → Sonnet; GPT-4o + Q17=speech → Nova 2 Sonic; GPT-3.5 + Q22=complex → Sonnet; GPT-5 + Q16=balanced → Sonnet; GPT-5.5 + Q16=cost → Sonnet 4.6; Gemini 2.5 Flash Thinking + Q16=cost → Nova Lite (if thinking budget is low) or Sonnet 4.6 (if full thinking mode). + +Interpret → `ai_model_baseline`. Default: auto-detect from code, fallback Q16 priority-based. -> The model you're currently using helps me recommend the closest Bedrock equivalent as a starting point. -> -> A) Gemini Flash (1.5/2.0/2.5 Flash) -> B) Gemini Pro (1.5/2.5/3 Pro) -> C) GPT-3.5 Turbo -> D) GPT-4 / GPT-4 Turbo -> E) GPT-4o -> F) GPT-5 / GPT-5.x -> G) o-series (o1, o3) -> H) Other / Multiple models -> I) I don't know - -| Source Model | Baseline Bedrock Recommendation | Pricing Context | -| ------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------- | -| Gemini Flash variants | Claude Haiku 4.5 ($1/$5) — speed and cost optimized | Strong savings vs Gemini Flash pricing | -| Gemini Pro variants | Claude Sonnet 4.6 ($3/$15) — quality match | Comparable pricing tier | -| GPT-3.5 Turbo | Claude Haiku 4.5 ($1/$5) — cost-equivalent | Haiku is faster and cheaper | -| GPT-4 / GPT-4 Turbo | Claude Sonnet 4.6 ($3/$15) — quality equivalent | Major savings: GPT-4 Turbo is $10/$30 vs Sonnet $3/$15 | -| GPT-4o | Claude Sonnet 4.6 ($3/$15) — performance equivalent | Modest savings on output; input slightly higher on Bedrock | -| GPT-5 / GPT-5.x | Claude Sonnet 4.6 ($3/$15) — performance equivalent | GPT-5 is $1.25/$10 — savings story is quality/features, not cost | -| GPT-5 (flagship use case) | Claude Opus 4.6 ($5/$25) — flagship-to-flagship | Opus still cheaper than GPT-5 Pro ($15/$120) | -| o-series (o1, o3) | Claude Sonnet 4.6 with extended thinking; Opus 4.6 for most demanding | o1 is $15/$60 — significant savings with Sonnet 4.6 at $3/$15 | - -**Example overrides:** - -- GPT-4 user (baseline: Sonnet 4.6) + Q16=lowest cost → **Haiku 4.5** -- Gemini Flash user (baseline: Haiku 4.5) + Q17=extended thinking → **Sonnet 4.6 with extended thinking** -- GPT-4o user (baseline: Sonnet 4.6) + Q17=real-time speech → **Nova Sonic** (Claude has no speech) -- GPT-3.5 user (baseline: Haiku 4.5) + Q22=complex reasoning → **Sonnet 4.6** (complexity overrides cost mapping) -- GPT-5 user (baseline: Opus 4.6) + Q16=balanced → **Sonnet 4.6** (priority overrides flagship mapping) - -Interpret: +--- -``` -A -> ai_model_baseline: "claude-haiku-4-5" — speed/cost optimized -B -> ai_model_baseline: "claude-sonnet-4-6" — quality match -C -> ai_model_baseline: "claude-haiku-4-5" — cost equivalent -D -> ai_model_baseline: "claude-sonnet-4-6" — quality equivalent; major savings -E -> ai_model_baseline: "claude-sonnet-4-6" — performance equivalent -F -> ai_model_baseline: "claude-sonnet-4-6" — default; "claude-opus-4-6" for flagship use cases -G -> ai_model_baseline: "claude-sonnet-4-6-extended-thinking" — reasoning equivalent -H -> ai_model_baseline: "claude-sonnet-4-6" — safe default for multiple models -I -> same as default (use Q16 priority to determine) -``` +## Q20 — What input types must the model accept: text only, images (vision), or audio/video? -Default: _(auto-detect from code)_ — fall back to Q16 priority-based selection. +**Auto-detect signal:** Read `integration.capabilities_summary`: ---- +| Signal | Extract | Skip Q20? | +| ------------------------------------------------ | -------------------------------------------------------------------------- | ------------------------------ | +| `vision: true` | `ai_vision: "vision-required"` | Yes — `chosen_by: "extracted"` | +| `speech_to_text: true` or `text_to_speech: true` | `ai_vision: "audio-video"` | Yes | +| all false / text only | `ai_vision: "text-only"` | Yes | +| `image_generation: true` and `vision: false` | note in `ai_capabilities_required`; skip Q20 (image output ≠ vision input) | Yes | -## Q20 — Do you need vision (image understanding) or just text? +_Skip when:_ Modalities fully resolved from `capabilities_summary`. Use detected value with `chosen_by: "extracted"`. -**Rationale:** Vision capability narrows the model selection to multimodal-capable models only. - -> Vision capability limits which models are available. This ensures the recommendation supports your input types. -> > A) Text only > B) Vision required — model must process images > C) Audio/Video inputs needed -| Answer | Recommendation Impact | -| ------------------ | ------------------------------------------------------------------------------------- | -| Text only | Full model catalog available; cheapest/fastest text model per Q16 priority | -| Vision required | Claude Sonnet family (multimodal) required; Haiku excluded for vision tasks | -| Audio/Video inputs | Amazon Nova Reel (video) or Nova Sonic (audio); Claude excluded for audio/video input | +| Answer | Recommendation Impact | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------- | +| Text only | Full model catalog available; cheapest/fastest text model per Q16 priority | +| Vision required | Claude Sonnet or Haiku (both support multimodal vision); Nova Micro excluded (text-only) | +| Audio/Video inputs | Amazon Nova 2 Sonic (audio); Nova Reel v1 for video (Legacy — EOL Sep 30, 2026); Claude excluded for audio/video input | -Interpret: - -``` -A -> (no constraint written — full model catalog) -B -> ai_vision: "required" — Claude Sonnet family required; Haiku excluded for vision -C -> ai_vision: "audio-video" — Nova Reel (video) or Nova Sonic (audio); Claude excluded -``` - -Default: A — no constraint (text only). +Interpret → `ai_vision`. Default: A → no constraint. --- ## Q21 — How important is AI response speed? -**Rationale:** Latency requirements can override cost and quality preferences from Q16. - -**Context for user:** When asking, anchor each option in a real scenario: +Present with concrete anchors: Critical = autocomplete/live chat/real-time transcription; Important = chat assistant/search augmentation; Flexible = report generation/batch analysis. -- **Critical (< 500ms)** — users are staring at a loading spinner; every millisecond matters (e.g., autocomplete, live chat, real-time transcription) -- **Important (< 2s)** — users expect a quick response but a brief pause is acceptable (e.g., chat assistant, search augmentation, inline suggestions) -- **Flexible (2–10s)** — users submit a request and can wait; background or async is fine (e.g., report generation, batch analysis, email drafting) - -> Latency requirements can override cost and quality preferences from your earlier answers. -> > A) Critical (< 500ms) — users staring at a loading spinner > B) Important (< 2s) — quick response expected, brief pause acceptable > C) Flexible (2–10s) — users can wait, background/async acceptable @@ -367,46 +352,201 @@ Default: A — no constraint (text only). | Important (< 2s) | Claude Sonnet 4.6 with streaming; standard on-demand acceptable | | Flexible (2–10s) | Any model; batch inference considered for cost savings at high volume | -Interpret: +Interpret → `ai_latency`. Default: B → `"important"`. -``` -A -> ai_latency: "critical" — Haiku 4.5 or Nova Micro; streaming; provisioned throughput -B -> ai_latency: "important" — Sonnet 4.6 with streaming; on-demand -C -> ai_latency: "flexible" — Any model; batch inference for cost savings -``` +--- + +## Q22 — How complex are your AI tasks? -Default: B — `ai_latency: "important"`. +Present with concrete examples: Simple = classify/extract/summarize; Moderate = analyze+JSON/few-shot; Complex = multi-turn reasoning/tool use/agentic. + +> A) Simple (classification, short summaries, extraction) +> B) Moderate (analysis, structured content, few-shot) +> C) Complex (multi-step reasoning, tool use, agentic workflows) + +| Answer | Recommendation Impact | +| -------- | ------------------------------------------------------------------------------------------------- | +| Simple | Claude Haiku 4.5 or Nova Micro sufficient; significant cost savings vs larger models | +| Moderate | Claude Sonnet 4.6 recommended; Haiku may suffice with prompt engineering | +| Complex | Claude Sonnet 4.6 required; extended thinking considered; Claude Opus 4.7 / 4.6 for hardest tasks | + +Interpret → `ai_complexity`. Default: B → `"moderate"`. --- -## Q22 — How complex are your AI tasks? +## Category G — Agentic Workflows (If `agentic_profile` exists in `ai-workload-profile.json`) -**Rationale:** Task complexity determines whether a cheaper/faster model can handle the workload or whether a more capable model is required. +_Fire when:_ `ai-workload-profile.json` contains `agentic_profile` with `is_agentic: true`. -**Context for user:** When asking, give concrete examples so the user doesn't over- or under-estimate: +_Skip entirely when:_ `agentic_profile` is absent from `ai-workload-profile.json`. + +--- -- **Simple** — single-step tasks with short prompts: classify this text, extract these fields, summarize this paragraph -- **Moderate** — multi-step prompts with examples or structured output: analyze this document and return JSON, generate content following a template, few-shot classification -- **Complex** — multi-turn reasoning, tool use, or long chain-of-thought: agentic workflows, code generation with debugging loops, research tasks that require planning and iteration +## Agentic Context Summary -> Task complexity determines whether a cheaper/faster model can handle the workload or whether a more capable model is required. +Before presenting Category G questions, show: + +> **Agentic Context Summary:** +> **Framework:** [from `agentic_profile.framework`] +> **Agents detected:** [from `agentic_profile.agent_count`] ([list `agents[].agent_id`]) +> **Orchestration pattern:** [from `agentic_profile.orchestration_pattern`] +> **Tools:** [from `agentic_profile.tool_count`] tools detected +> **Memory:** [from `agentic_profile.has_memory`; if true, backend: `agentic_profile.memory_backend`] +> **Human-in-the-loop:** [from `agentic_profile.has_human_in_loop`] + +--- + +## Q23 — How do you want to migrate your agent system? + +**Auto-detect signals** — recommend default based on `agentic_profile.framework`: + +- `gateway_type` is `"llm_router"` (LiteLLM or OpenRouter detected) → Default to **A (retarget)**. These users are already abstracted from the model provider — migration is a config change (swap model IDs), not a code rewrite. Set `migration_approach: "retarget"` automatically and skip Q23 unless the user explicitly asks to evaluate Harness or Strands. +- `langgraph`, `crewai`, `autogen` → Default to A (retarget). These frameworks support Bedrock as a model provider with minimal code changes. +- `openai_agents` → Surface all options. OpenAI Agents SDK is tightly coupled to OpenAI API; retarget is harder. Note partial retarget (HTTP-compatible routing to Bedrock) as a bridge. +- `strands` → Already AWS-native. Recommend B (Harness) for managed deployment or note "already on target framework." +- `custom` → Surface all options. Custom loops vary widely in complexity. + +_Skip when:_ Auto-detection fully resolves AND user has no preference signal. Use detected default with `chosen_by: "extracted"`. + +> Your agent system can migrate to AWS in different ways, each with different effort and risk: > -> A) Simple (classification, short summaries, extraction) -> B) Moderate (analysis, structured content, few-shot) -> C) Complex (multi-step reasoning, tool use, agentic workflows) +> A) **Retarget** — Keep your current framework ([framework name]), swap the model layer to Bedrock. Fastest path, lowest risk. Your orchestration code stays the same. +> B) **AgentCore Harness** — Declare your agent as configuration (model + tools + prompt). Get managed runtime, memory, identity, and observability. Good for simpler agents or incremental migration. _(Preview — 4 regions: us-east-1, us-west-2, eu-central-1, ap-southeast-2)_ +> C) **Strands native** — Rewrite orchestration using AWS Strands SDK on AgentCore. Most AWS-integrated, highest effort. Best for teams wanting full AWS-native multi-agent capabilities. +> D) **I'm not sure** — Help me decide based on my workload. -| Answer | Recommendation Impact | -| -------- | ------------------------------------------------------------------------------------------- | -| Simple | Claude Haiku 4.5 or Nova Micro sufficient; significant cost savings vs larger models | -| Moderate | Claude Sonnet 4.6 recommended; Haiku may suffice with prompt engineering | -| Complex | Claude Sonnet 4.6 required; extended thinking considered; Claude Opus 4.6 for hardest tasks | +| Answer | When it fits | Effort range | Risk | +| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------- | ------------------------------ | +| A) Retarget | Working system, team knows the framework, need to ship fast. LangGraph/CrewAI/AutoGen with Bedrock model provider support. | 1–3 weeks depending on agent count, tool count, test coverage | Low — orchestration unchanged | +| B) AgentCore Harness | Simple single-agent, OpenAI Assistants migration, want managed runtime, or incremental migration (run existing models on AWS infra first). | 3–10 days depending on tool complexity and memory requirements | Low — config-based, reversible | +| C) Strands native | OpenAI Agents SDK or custom loops where retarget doesn't work well, multi-agent systems, team willing to refactor for AWS-native benefits. | 2–6 weeks depending on agent count, graph complexity, tool count | Medium — orchestration rewrite | +| D) Undecided | — | — | — | -Interpret: +**For OpenAI Agents SDK users:** Note that a partial retarget (HTTP-compatible routing to Bedrock while keeping OpenAI SDK orchestration) is a valid short-lived bridge before committing to B or C. This is not a fourth path — it's a Phase 0 step within B or C. +**If answer is D:** Recommend A (retarget) as default for LangGraph/CrewAI/AutoGen users. Recommend B (Harness) for OpenAI Assistants or simple single-agent patterns. Recommend C (Strands) only if user explicitly wants AWS-native multi-agent and accepts refactor cost. + +Interpret → `ai_constraints.agentic.migration_approach`: A → `"retarget"`, B → `"harness"`, C → `"strands"`, D → `"undecided"` (treated as `"retarget"` in Design unless overridden). Default: auto-detect based on framework. + +--- + +## Q24 — Do your agents need to remember context across sessions? + +> A) No — each request is independent, no memory needed +> B) Within a session — conversation history during a single interaction, but fresh start each time +> C) Across sessions — remember user preferences, past interactions, accumulated knowledge between separate conversations + +| Answer | Recommendation Impact | +| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| No memory | Standard stateless invocation. No AgentCore Memory needed. | +| Within session | AgentCore Harness sessions are stateful by default (microVM per session). No additional config needed for Harness path. For retarget path: existing framework memory (e.g., LangGraph checkpointer) continues to work. | +| Across sessions | AgentCore Memory service recommended. Persists knowledge, user preferences, and interaction history across sessions. For retarget path: evaluate existing memory backend migration (Redis → ElastiCache, Postgres → RDS, vector store → OpenSearch Serverless). | + +Interpret → `ai_constraints.agentic.memory_requirement`: A → `"none"`, B → `"session"`, C → `"cross_session"`. Default: B → `"session"`. + +--- + +## Q25 — How long do your agent tasks typically run? + +> A) Quick (< 30 seconds) — simple tool calls, single-turn responses +> B) Medium (30 seconds – 5 minutes) — multi-step reasoning, several tool calls +> C) Long (5 minutes – 1 hour) — complex research, multi-agent collaboration, iterative refinement +> D) Very long (1+ hours) — extended autonomous work, large-scale data processing + +| Answer | Recommendation Impact | +| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Quick | Standard invocation. Any deployment model works. | +| Medium | AgentCore Runtime recommended for managed scaling. Harness sessions handle this natively. | +| Long | AgentCore Runtime strongly recommended (supports up to 8-hour sessions). Serverless alternatives (Lambda) will timeout. | +| Very long | AgentCore Runtime required (8-hour max session). If tasks exceed 8 hours: recommend breaking into sub-tasks with session chaining, or evaluate custom compute (ECS/EKS). | + +Interpret → `ai_constraints.agentic.task_duration`: A → `"quick"`, B → `"medium"`, C → `"long"`, D → `"very_long"`. Default: B → `"medium"`. + +--- + +## Q26 — Do you want to migrate incrementally? + +> A) Yes — run my existing models (OpenAI/Gemini) on AWS infrastructure first, then swap to Bedrock models later when I'm confident +> B) No — do a full model swap to Bedrock in one go +> C) I'm not sure + +| Answer | Recommendation Impact | +| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Yes (incremental) | AgentCore Harness multi-model switching: deploy on Harness with existing OpenAI/Gemini model (API key in AgentCore Identity), then override `--model-id` per invocation to A/B test Bedrock. Swap default when confident. Works for both Harness and retarget paths. | +| No (full swap) | Standard migration: swap model layer directly to Bedrock. Faster to complete but higher risk per deployment. | +| Not sure | Default to incremental if using Harness path (it's free — multi-model switching is built in). Default to full swap if retarget path with LangChain/LangGraph (simpler to test with framework's built-in model switching). | + +Interpret → `ai_constraints.agentic.incremental_migration`: A → `true`, B → `false`, C → auto-select based on `migration_approach`. Default: `true` for Harness path, `false` for retarget path. + +--- + +## Category G Combination Logic + +| Combination | Design Impact | +| --------------------------------------- | -------------------------------------------------------------------------------- | +| A (retarget) + C (cross-session memory) | Retarget model layer + migrate memory backend to AWS (Redis → ElastiCache, etc.) | +| B (harness) + A (no memory) | Simplest Harness config — model + tools + prompt, no memory setup | +| B (harness) + C (cross-session memory) | Harness + AgentCore Memory service | +| B (harness) + D (very long tasks) | Flag: 8-hour session limit. Recommend task decomposition or session chaining. | +| C (strands) + C (cross-session memory) | Strands SessionManager + AgentCore Memory | +| Any + A (incremental) | Include incremental migration script in Generate artifacts | + +--- + +## Category H — Startup Programs (Always fires when Category F fires) + +_Fire when:_ `ai-workload-profile.json` exists (same trigger as Category F). Ask once, after Q26 if agentic, or after Q22 if non-agentic. + +--- + +## Q27 — Have you applied for AWS Activate credits? + +**Rationale:** AWS Activate credits apply directly to Bedrock usage (including Claude, Llama, Nova, and other third-party models). Surfacing eligibility at the migration decision moment helps startups reduce the cost of the migration itself. This takes 30 seconds to answer and can unlock $5K–$200K in credits. + +**Activate eligibility:** Pre-Series B, founded in the last 10 years, AWS Account on Paid Tier Plan, and either new to Activate Credits or requesting more credits than previously received. + +> AWS Activate credits offset Bedrock costs during and after migration — including Claude, Llama, and Nova models. Eligible startups can get $5K–$200K depending on funding stage. +> +> A) Yes — already have AWS Activate credits +> B) No — haven't applied yet (self-funded or pre-VC) +> C) No — VC/accelerator-backed but haven't applied +> D) I don't know + +| Answer | Recommendation Impact | +| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Already have credits | Note credit balance in migration plan; flag Bedrock usage as credit-eligible | +| No — self-funded | Flag **AWS Activate Founders** (up to $5,000, self-service): aws.amazon.com/startups/credits — apply before starting migration to offset Bedrock testing costs | +| No — VC/accelerator-backed | Flag **AWS Activate Portfolio** (up to $200,000): requires Activate Provider Org ID from your VC/accelerator — contact them for the Org ID before applying | +| Don't know | Surface both tiers; recommend checking with investors/accelerator for Org ID | + +If `ai_monthly_spend` is `">$10K"`: also flag **AWS Credits for AI Startups** ($200,000+, invite-only for startups ready to scale post-Activate-Portfolio — contact your AWS Account Manager; aws.amazon.com/startups/credits). + +If `ai_monthly_spend` is `"$2K-$10K"` or `">$10K"` AND `agentic_profile.is_agentic == true`: also flag **AWS Generative AI Accelerator** (up to $1M credits, 8-week cohort — adjacent cohort program, distinct from the credits-hub funnel): aws.amazon.com/startups/generative-ai/accelerator + +Interpret → `startup_program_status`: A → `"has_credits"`, B → `"eligible_founders"`, C → `"eligible_portfolio"`, D → `"unknown"`. Default: D → `"unknown"`. + +--- + +## Preferences Output — `ai_constraints.agentic` + +Category G answers are stored in `preferences.json` → `ai_constraints.agentic`: + +```json +{ + "ai_constraints": { + "agentic": { + "migration_approach": "retarget|harness|strands|undecided", + "memory_requirement": "none|session|cross_session", + "task_duration": "quick|medium|long|very_long", + "incremental_migration": true + } + } +} ``` -A -> ai_complexity: "simple" — Haiku 4.5 or Nova Micro sufficient -B -> ai_complexity: "moderate" — Sonnet 4.6 recommended; Haiku may suffice -C -> ai_complexity: "complex" — Sonnet 4.6 required; Opus 4.6 for hardest tasks -``` -Default: B — `ai_complexity: "moderate"`. +**Field contract (consumed by Design phase):** + +- `migration_approach` — Routes Design to the correct path: `"retarget"` uses existing model-swap flow, `"harness"` loads `design-ref-harness.md`, `"strands"` loads `design-ref-agentic-to-agentcore.md` +- `memory_requirement` — Determines whether AgentCore Memory is included in design +- `task_duration` — Determines AgentCore Runtime recommendation and session limit warnings +- `incremental_migration` — Determines whether incremental migration artifacts are generated diff --git a/migration-to-aws/steering/clarify-compute.md b/migration-to-aws/steering/clarify-compute.md index 6187f3a9..69b0a4dc 100644 --- a/migration-to-aws/steering/clarify-compute.md +++ b/migration-to-aws/steering/clarify-compute.md @@ -7,20 +7,22 @@ This file covers two related categories: --- -## Category B — Configuration Gaps (Billing-Source Inventories Only) +## Category B — Configuration Gaps (Billing-Only Mode) -_Fire when:_ `gcp-resource-inventory.json` exists with `metadata.source == "billing"` AND at least one resource has `config_confidence == "assumed"`. -_Skip when:_ `metadata.source == "terraform"`. +_Fire when:_ `billing-profile.json` exists AND `gcp-resource-inventory.json` does NOT exist (billing-only mode). +_Skip when:_ `gcp-resource-inventory.json` exists (Terraform/IaC provides configuration directly). -These fill factual gaps in the inferred inventory. Answers update the inventory understanding — they do not produce design constraints directly. +These fill factual gaps that billing data alone cannot answer. Answers update the inventory understanding — they do not produce design constraints directly. -- **Cloud SQL HA**: Single-zone or high-availability? _(ask if SKU says Zonal)_ +Each question fires only when the matching `gcp_service_type` appears in `billing-profile.json → services[]`: + +- **Cloud SQL HA**: Single-zone or high-availability? _(fire if `google_sql_database_instance` in billing services)_ > Default: assume Zonal is intentional. -- **Cloud Run service count**: How many distinct services? _(ask if Cloud Run billing is present)_ +- **Cloud Run service count**: How many distinct services? _(fire if `google_cloud_run_service` in billing services)_ > Default: assume 1 service. -- **Memorystore memory size**: How much memory (GB)? _(ask if cannot be derived from usage units)_ +- **Memorystore memory size**: How much memory (GB)? _(fire if `google_redis_instance` in billing services)_ > Default: estimate from usage amount. -- **Cloud Functions generation**: Gen 1 or Gen 2? _(ask if SKU does not specify)_ +- **Cloud Functions generation**: Gen 1 or Gen 2? _(fire if `google_cloudfunctions_function` in billing services)_ > Default: assume Gen 1. Record Category B answers in `metadata.inventory_clarifications`. @@ -68,10 +70,16 @@ A -> kubernetes: "eks-managed" — EKS recommended, preserves K8s investment B -> kubernetes: "eks-or-ecs" — EKS with managed node groups to reduce operational burden C -> kubernetes: "ecs-fargate" — Strong ECS Fargate recommendation, eliminates K8s management D -> (no constraint written — no K8s workloads) -E -> same as default (B) — assume neutral, evaluate both EKS and ECS +E -> same as default — see IaC-signal default rule below ``` -Default: B — `kubernetes: "eks-or-ecs"`. +**Default (IaC-signal driven):** + +- If `gcp-resource-inventory.json` contains `google_container_cluster` resources → Default **C** (`kubernetes: "ecs-fargate"`). Teams that answer "I don't know" are better served by Fargate's lower operational overhead; EKS remains available via explicit answers A and B. +- If no `google_container_cluster` in inventory (Cloud Run, Cloud Functions, or billing-only) → Default **C** (`kubernetes: "ecs-fargate"`). No Kubernetes signal; Fargate is the lower-ops starting point. +- If inventory is absent (billing-only mode) → Default **C** (`kubernetes: "ecs-fargate"`). + +**Rationale:** Teams that answer E ("I don't know") have not expressed a Kubernetes preference. Defaulting to Fargate gives them a simpler, lower-ops starting point regardless of what discovery found. Teams who actively want EKS will answer A or B explicitly. EKS remains fully available via explicit answers A and B. --- @@ -79,7 +87,9 @@ Default: B — `kubernetes: "eks-or-ecs"`. _Fire when:_ Compute resources present AND WebSocket usage cannot be determined from inventory. -**Rationale:** WebSocket support affects load balancer configuration. App Runner is now on KTLO (keep the lights on) and is no longer recommended for any workload — this question confirms whether ALB WebSocket configuration is needed in templates. +**Auto-extract signal:** Only when application code was analyzed (see Clarify Step 2 item 14). If code was scanned and no WebSocket patterns found, extract `websocket: false` and skip. **If no code was analyzed** (Terraform-only), always ask Q9 — do not infer absence of WebSockets. + +**Rationale:** WebSocket support affects load balancer configuration. > WebSocket support affects load balancer configuration. This confirms whether ALB WebSocket configuration is needed in the migration templates. > @@ -108,7 +118,9 @@ Default: B — no constraint. _Fire when:_ Cloud Run present in inventory. Skip when: no Cloud Run. -**Rationale:** Cloud Run's scale-to-zero is its primary cost advantage. If traffic is constant, that advantage disappears and AWS becomes cost-competitive or cheaper. This drives whether we recommend migrating Cloud Run at all. +**Auto-extract signal:** When Cloud Run `min_instance_count` / `min_instances` > 0 in Terraform config, extract `cloud_run_traffic_pattern: "constant-24-7"` with `chosen_by: "extracted"` and **skip Q10**. + +**Rationale:** Cloud Run's scale-to-zero is its primary cost advantage. > Cloud Run's scale-to-zero is its primary cost advantage. Understanding your traffic pattern helps me determine whether migrating Cloud Run to AWS makes financial sense. > diff --git a/migration-to-aws/steering/clarify-database.md b/migration-to-aws/steering/clarify-database.md index 35db650e..ff448ffe 100644 --- a/migration-to-aws/steering/clarify-database.md +++ b/migration-to-aws/steering/clarify-database.md @@ -2,7 +2,7 @@ _Fire when:_ Database resources present (Cloud SQL, Spanner, Memorystore). -Traffic pattern and I/O intensity determine the Aurora configuration — standard vs I/O-Optimized, read replicas, Serverless v2, or DSQL. +**Q6 (`availability`) selects RDS vs Aurora** for Cloud SQL PostgreSQL/MySQL — see `clarify-global.md` Q6. Q12–Q13 tune sizing and storage **within** the family Q6 chose. **Q12–Q13 never override Q6.** --- @@ -12,35 +12,37 @@ Before asking questions, detect the database engine from IaC (`database_version` > "I see Cloud SQL for PostgreSQL (or MySQL) in your Terraform." -Handle non-Aurora-compatible engines: - -| Detected Engine | Migration Target | Notes | +| Detected Engine | Migration Target family | Notes | | ------------------------ | ----------------------------------------------------------------- | ------------------------------------ | -| Cloud SQL for PostgreSQL | Aurora PostgreSQL | Direct migration path | -| Cloud SQL for MySQL | Aurora MySQL | Direct migration path | +| Cloud SQL for PostgreSQL | **RDS PostgreSQL** or **Aurora PostgreSQL** (per Q6) | Q6 selects family; engine from IaC | +| Cloud SQL for MySQL | **RDS MySQL** or **Aurora MySQL** (per Q6) | Q6 selects family; engine from IaC | | Cloud SQL for SQL Server | **RDS for SQL Server** | Aurora doesn't support SQL Server | | Spanner | Aurora DSQL (global distributed) or DynamoDB (key-value patterns) | Migration path differs significantly | | Firestore | DynamoDB | NoSQL migration | | AlloyDB | Aurora PostgreSQL | Closest equivalent | -If the engine is not PostgreSQL or MySQL, note that Aurora doesn't support it and flag the appropriate RDS or DynamoDB target. Ask the user to confirm if detection is ambiguous. +If the engine is not PostgreSQL or MySQL, note the appropriate RDS or DynamoDB target. Ask the user to confirm if detection is ambiguous. + +**Skip Q12/Q13/Q13b** when Cloud SQL (PostgreSQL or MySQL) is not present in inventory — auto-detect from IaC; no separate screening question required. --- ## Q12 — What does your database traffic pattern look like? -_Fire when:_ Cloud SQL present in inventory. Skip when: no Cloud SQL. +_Fire when:_ Cloud SQL (PostgreSQL or MySQL) present in inventory. Skip when: no Cloud SQL. + +**Auto-extract signal (dev-tier):** When **all** Cloud SQL instances match dev pattern (`db-f1-micro`, `db-g1-small`, or `tier` contains `micro`/`small` with `availability_type: ZONAL`), extract `database_traffic: "steady"` with `chosen_by: "extracted"` and **skip Q12**. When instances mix dev and prod tiers, ask Q12. -**Rationale:** Database traffic pattern determines whether standard Aurora is sufficient or whether more specialized options (read replicas, DSQL, Serverless v2) are needed. Asking about the pattern rather than whether they have a problem avoids leading the answer. +**Rationale:** Traffic pattern informs capacity planning on the target **already chosen by Q6**. -**Context for user:** When asking, give concrete examples so the user can pattern-match to their situation: +**Context for user:** Give concrete examples so the user can pattern-match: - **Steady, predictable load** — consistent query volume day-to-day, no major spikes (e.g., internal CRUD app, content CMS) - **Read-heavy with occasional write spikes** — mostly reads with bursts of writes at certain times (e.g., reporting dashboards, catalog browsing with periodic bulk imports) - **Write-heavy or globally distributed writes** — high write throughput or writes coming from multiple regions (e.g., IoT ingestion, multi-region user-generated content, event logging) - **Rapidly growing** — traffic is noticeably increasing month over month, doubling every few months (e.g., post-launch growth, viral product) -> Understanding your database traffic pattern helps me recommend the right Aurora configuration — standard vs I/O-Optimized, read replicas, or Serverless v2. +> Understanding your database traffic pattern helps me recommend the right sizing on AWS — within the RDS or Aurora family Q6 already selected. > > A) Steady, predictable load — consistent volume, no major spikes > B) Read-heavy with occasional write spikes — mostly reads, periodic write bursts @@ -49,22 +51,22 @@ _Fire when:_ Cloud SQL present in inventory. Skip when: no Cloud SQL. > E) N/A — We don't use Cloud SQL > F) I don't know -| Answer | Recommendation Impact | -| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------- | -| Steady, predictable | Standard Aurora Multi-AZ; straightforward sizing from current Cloud SQL config | -| Read-heavy with write spikes | Aurora with read replicas; auto-scaling read capacity | -| Write-heavy or globally distributed | **Aurora DSQL considered** for global active-active; architecture review flagged | -| Rapidly growing | Aurora with headroom planning; **Aurora Serverless v2** for elastic scaling; revisit sizing at 6-month intervals | +| Answer | When Q6 = Inconvenient or Significant Issue (**RDS** path) | When Q6 = Mission-Critical or Catastrophic (**Aurora** path) | +| ----------------------------------- | ----------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | +| Steady, predictable | Size **RDS** from Cloud SQL config; single-AZ or Multi-AZ per Q6 | **Aurora** standard Multi-AZ; size from Cloud SQL config | +| Read-heavy with write spikes | **RDS** with read replicas where justified; size writer for spikes | **Aurora** with read replicas; auto-scaling read capacity where supported | +| Write-heavy or globally distributed | Size **RDS** writer; flag architecture review if multi-region writes required | **Aurora DSQL** considered for global active-active; architecture review flagged | +| Rapidly growing | **RDS** with headroom on instance class; plan capacity reviews | **Aurora** with headroom; **Aurora Serverless v2** for elastic scaling | Interpret: ``` -A -> database_traffic: "steady" — Standard Aurora Multi-AZ -B -> database_traffic: "read-heavy" — Aurora with read replicas; auto-scaling read capacity -C -> database_traffic: "write-heavy-global" — Aurora DSQL considered; architecture review flagged -D -> database_traffic: "rapidly-growing" — Aurora Serverless v2 for elastic scaling +A -> database_traffic: "steady" +B -> database_traffic: "read-heavy" +C -> database_traffic: "write-heavy-global" +D -> database_traffic: "rapidly-growing" E -> (no constraint written) -F -> same as default (A) — assume steady traffic +F -> same as default (A) ``` Default: A — `database_traffic: "steady"`. @@ -73,32 +75,79 @@ Default: A — `database_traffic: "steady"`. ## Q13 — What's your typical database I/O workload? -_Fire when:_ Cloud SQL present in inventory. Skip when: no Cloud SQL. +_Fire when:_ Cloud SQL (PostgreSQL or MySQL) present in inventory. Skip when: no Cloud SQL. -**Rationale:** Aurora has two pricing modes — standard and I/O-Optimized. The right choice depends on actual I/O intensity. Choosing wrong can mean paying 40% more than necessary. +**Auto-extract signal (dev-tier):** Same dev-tier pattern as Q12 — extract `db_io_workload: "low"` with `chosen_by: "extracted"` and **skip Q13** only when **all** instances are dev-tier. When instances mix dev and prod tiers, ask Q13. + +**Rationale:** On AWS, storage and I/O billing differ between RDS and Aurora. This captures how disk-heavy the workload is. **Q6 still governs RDS vs Aurora** — this question only selects storage/I/O options within that family. -> Aurora has two pricing modes — standard and I/O-Optimized. Choosing wrong can mean paying 40% more than necessary. -> > A) Low (< 1,000 IOPS) — Mostly reads, infrequent writes > B) Medium (1,000–10,000 IOPS) — Balanced workload > C) High (> 10,000 IOPS) — Write-heavy, high transactions > D) N/A — We don't use Cloud SQL > E) I don't know -| Answer | Recommendation Impact | -| -------------------------- | ----------------------------------------------------------------------------------------- | -| Low (< 1,000 IOPS) | Aurora standard pricing recommended | -| Medium (1,000–10,000 IOPS) | Aurora standard pricing; flag I/O-Optimized as option if workload grows | -| High (> 10,000 IOPS) | **Aurora I/O-Optimized recommended** — can save up to 40% vs standard at high I/O volumes | +| Answer | When Q6 = Inconvenient or Significant Issue (**RDS** path) | When Q6 = Mission-Critical or Catastrophic (**Aurora** path) | +| -------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------ | +| Low (< 1,000 IOPS) | **gp3** (default RDS storage) | **Aurora Standard** storage/I/O billing | +| Medium (1,000–10,000 IOPS) | **gp3**; note optional Provisioned IOPS if I/O grows | **Aurora Standard**; note optional switch to I/O-Optimized | +| High (> 10,000 IOPS) | **io2** or Provisioned IOPS on RDS | **Aurora I/O-Optimized** | Interpret: ``` -A -> db_io_workload: "low" — Aurora standard pricing -B -> db_io_workload: "medium" — Aurora standard; flag I/O-Optimized as option if workload grows -C -> db_io_workload: "high" — Aurora I/O-Optimized recommended (up to 40% savings at high I/O) +A -> db_io_workload: "low" +B -> db_io_workload: "medium" +C -> db_io_workload: "high" D -> (no constraint written) -E -> same as default (B) — assume medium I/O +E -> same as default (B) ``` Default: B — `db_io_workload: "medium"`. + +--- + +## Q13b — Approximately how large is your primary database? + +_Fire when:_ Cloud SQL (PostgreSQL or MySQL) present in inventory. Skip when: no Cloud SQL, or engine is SQL Server (DMS is always recommended for SQL Server regardless of size). + +**Rationale:** Database size is the primary driver of migration tooling selection. pg_dump/pg_restore is sufficient for small databases but becomes impractically slow above ~10GB within a typical maintenance window. pgcopydb's parallel copy cuts migration time by 3–5x for medium databases. Very large databases (>500GB) require DMS for continuous replication regardless of whether a maintenance window exists. + +**Auto-detect signal:** Read disk size from `google_sql_database_instance`: `config.disk_size`, `config.disk_size_gb`, or `gcp_config.disk_size_gb`. Map to Q13b band and **skip Q13b** when unambiguous: + +| Disk size (GB) | `db_size` value | Skip Q13b? | +| -------------- | --------------- | ------------------------------ | +| < 10 | `"<10GB"` | Yes — `chosen_by: "extracted"` | +| 10 – 99 | `"10-100GB"` | Yes — `chosen_by: "extracted"` | +| 100 – 499 | `"100-500GB"` | Yes — `chosen_by: "extracted"` | +| ≥ 500 | `">500GB"` | Yes — `chosen_by: "extracted"` | + +If multiple instances disagree, ask Q13b. Record raw GB in `metadata.inventory_clarifications.db_size_gb` when extracted. + +> Database size determines which migration tool we recommend — this directly affects your migration window length and risk. +> +> A) < 10 GB — small, pg_dump/pg_restore completes in minutes +> B) 10 GB – 100 GB — medium, pgcopydb recommended for parallel copy +> C) 100 GB – 500 GB — large, pgcopydb required; plan for multi-hour window +> D) > 500 GB — very large, AWS DMS strongly recommended regardless of window +> E) I don't know + +| Answer | Tooling Recommendation | +| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| < 10 GB | **pg_dump/pg_restore** — fast, simple, no extra tooling required | +| 10 GB – 100 GB | **pgcopydb** — parallel table copy + index rebuild; 3–5x faster than pg_dump; requires `wal_level=logical` on Cloud SQL if CDC mode used | +| 100 GB – 500 GB | **pgcopydb** required — pg_dump at this size risks exceeding maintenance window; plan for 4–12 hour window depending on table count and index complexity | +| > 500 GB | **AWS DMS strongly recommended** — single-pass export/import at this scale is high-risk; DMS provides continuous replication with minimal cutover window (minutes, not hours) | +| I don't know | Default to pgcopydb (safer than pg_dump at unknown scale); flag for user to verify before migration | + +Interpret: + +``` +A -> design_constraints.db_size: { value: "<10GB", chosen_by: "user" } — pg_dump/pg_restore recommended +B -> design_constraints.db_size: { value: "10-100GB", chosen_by: "user" } — pgcopydb recommended +C -> design_constraints.db_size: { value: "100-500GB", chosen_by: "user" } — pgcopydb required; flag extended window +D -> design_constraints.db_size: { value: ">500GB", chosen_by: "user" } — AWS DMS strongly recommended; flag in migration plan +E -> design_constraints.db_size: { value: "unknown", chosen_by: "default" } — default to pgcopydb; flag for verification +``` + +Default: E — `design_constraints.db_size: { value: "unknown", chosen_by: "default" }` (default to pgcopydb; safer than pg_dump at unknown scale). diff --git a/migration-to-aws/steering/clarify-global.md b/migration-to-aws/steering/clarify-global.md index 0f0fffea..7c99c65b 100644 --- a/migration-to-aws/steering/clarify-global.md +++ b/migration-to-aws/steering/clarify-global.md @@ -8,7 +8,9 @@ Present questions with a conversational tone and brief context explaining why ea ## Q1 — Where are your users located? -**Rationale:** Geography drives AWS region selection and CDN strategy. It does NOT by itself justify multi-region architecture or Aurora Global Database — those decisions require understanding write patterns and RTO/RPO requirements from Q6 (uptime) and Q7 (maintenance window). Recommending multi-region based on geography alone would over-engineer most architectures and significantly increase cost. +**Auto-extract signal:** When `gcp-resource-inventory.json` shows a **single** GCP region among PRIMARY compute/database resources, map to the closest AWS region and **skip Q1** with `target_region` `chosen_by: "extracted"`. When multiple regions are present, suggest the closest AWS region as default but still ask Q1. + +**Rationale:** Geography drives AWS region selection and CDN strategy. > I need to understand your user base to recommend the right AWS region and CDN strategy. > @@ -48,18 +50,20 @@ Default: A — single region, closest AWS region to GCP region in inventory. > D) HIPAA — Healthcare data > E) FedRAMP / Government — Federal compliance > F) GDPR / Data residency — EU data sovereignty requirements -> G) I don't know +> G) CCPA / CPRA — California Consumer Privacy Act / California Privacy Rights Act +> H) I don't know > > _(Multiple selections allowed)_ -| Answer | Recommendation Impact | -| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | -| None | Full service catalog available, any region | -| SOC 2 / ISO 27001 | CloudTrail, Config, Security Hub enabled by default; encryption at rest required | -| PCI DSS | Dedicated VPC with strict segmentation, WAF required, no shared tenancy for cardholder data, specific RDS encryption config | -| HIPAA | BAA-eligible services only, encryption in transit and at rest mandatory, specific logging requirements, us-east-1/us-west-2 preferred | -| FedRAMP | GovCloud regions required (us-gov-east-1, us-gov-west-1), GovCloud-specific service endpoints, limited service catalog | -| GDPR | EU regions required (eu-west-1, eu-central-1), data residency constraints, no cross-region replication outside EU without explicit consent | +| Answer | Recommendation Impact | +| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| None | Full service catalog available, any region | +| SOC 2 / ISO 27001 | CloudTrail, Config, Security Hub enabled by default; encryption at rest required | +| PCI DSS | CloudTrail, Config, Security Hub + PCI DSS standard enabled by default; dedicated VPC with strict segmentation; WAF required; no shared tenancy for cardholder data; specific RDS encryption config | +| HIPAA | CloudTrail, Config, Security Hub (FSBP only — Security Hub does not provide a HIPAA-specific standard) enabled by default; BAA-eligible services only; encryption in transit and at rest mandatory; specific logging requirements; us-east-1/us-west-2 preferred; engage a qualified HIPAA auditor for end-to-end posture validation | +| FedRAMP | CloudTrail, Config, Security Hub (FSBP only — NIST 800-53 is the target control set but is not directly subscribable in Security Hub the way PCI DSS is; engage your AWS account team for agency-level attestation) enabled by default; GovCloud regions required (us-gov-east-1, us-gov-west-1); GovCloud-specific service endpoints; limited service catalog | +| GDPR | EU regions required (eu-west-1, eu-central-1), data residency constraints, no cross-region replication outside EU without explicit consent | +| CCPA / CPRA | Consumer privacy posture: data inventory, access/deletion workflows, opt-out of sale/sharing where applicable, retention minimization, encryption and audit logging (CloudTrail); prefer documenting data flows and subprocessors — confirm target regions with legal/compliance (often US) | Interpret: @@ -70,7 +74,8 @@ C -> compliance: ["pci"] — Dedicated VPC, WAF required, strict segmentation D -> compliance: ["hipaa"] — BAA-eligible services only, encryption mandatory, us-east-1/us-west-2 preferred E -> compliance: ["fedramp"] — GovCloud regions required (us-gov-east-1, us-gov-west-1) F -> compliance: ["gdpr"] — EU regions required (eu-west-1, eu-central-1), data residency constraints -G -> same as default (A) — no constraint assumed; verify with compliance team before production +G -> compliance: ["ccpa"] — CCPA/CPRA: logging, retention, consumer-request readiness; document data flows; align region/subprocessor choices with legal review +H -> same as default (A) — no constraint assumed; verify with compliance team before production ``` Default: A — no constraint. @@ -79,6 +84,16 @@ Default: A — no constraint. ## Q3 — Approximately how much are you spending on GCP per month in total? +**Auto-extract signal:** If `billing-profile.json` exists, map `summary.total_monthly_spend` to the spend band below and **skip Q3** when unambiguous (`chosen_by: "extracted"`). If billing is absent or ambiguous, ask Q3. + +| Monthly USD | `gcp_monthly_spend` | +| ------------- | ------------------- | +| < 1,000 | `"<$1K"` | +| 1,000–4,999 | `"$1K-$5K"` | +| 5,000–19,999 | `"$5K-$20K"` | +| 20,000–99,999 | `"$20K-$100K"` | +| ≥ 100,000 | `">$100K"` | + **Rationale:** Total GCP spend is the primary input for ARR estimation, which determines credits eligibility tier. Also provides a sanity check for cost estimates when billing data is not uploaded. > Total GCP spend helps me estimate AWS credits eligibility and provides a cost baseline for the migration plan. @@ -90,30 +105,71 @@ Default: A — no constraint. > E) > $100,000/month > F) I don't know -**Billing enrichment:** If `billing-profile.json` exists, show: +**Billing enrichment (when Q3 is not skipped):** If `billing-profile.json` exists but extraction was skipped due to ambiguity, show: > Your billing data shows ~$[total_monthly_spend]/month. Does this match your expectation? -| Answer | Recommendation Impact | -| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| < $1,000/month | AWS Activate credits eligibility (~$5K–$25K); cost estimates use conservative ranges | -| $1,000–$5,000/month | IW Migrate credits at 25% of ARR (~$3K–$15K/yr); mid-range estimates | -| $5,000–$20,000/month | IW Migrate credits at 25% of ARR (~$15K–$60K/yr); Reserved Instance recommendations included | -| $20,000–$100,000/month | IW Migrate credits at 25% of ARR (~$60K–$300K/yr); Savings Plans analysis; AWS Specialist consultation eligible (>=$60K ARR) | -| > $100,000/month | MAP eligibility (>$500K ARR); Enterprise support tier; dedicated migration team engagement | +| Answer | Recommendation Impact | +| ---------------------- | -------------------------------------------------------------------------------------------------- | +| < $1,000/month | Entry-tier migration funding programs may apply; cost estimates use conservative ranges | +| $1,000–$5,000/month | Migration funding review may apply; cost estimates use mid-range assumptions | +| $5,000–$20,000/month | Migration funding review may apply; reserved pricing options are evaluated in cost recommendations | +| $20,000–$100,000/month | Migration funding and support program review may apply; savings commitment options are evaluated | +| > $100,000/month | Enterprise migration program review may apply; dedicated migration support path may be recommended | + +Interpret: + +``` +A -> gcp_monthly_spend: "<$1K" — entry-tier funding review; conservative cost assumptions +B -> gcp_monthly_spend: "$1K-$5K" — funding review; mid-range cost assumptions +C -> gcp_monthly_spend: "$5K-$20K" — funding review; reserved pricing recommendations +D -> gcp_monthly_spend: "$20K-$100K" — funding/support review; savings commitment analysis +E -> gcp_monthly_spend: ">$100K" — enterprise program/support review +F -> same as default (B) +``` + +Default: B — `gcp_monthly_spend: "$1K-$5K"`. + +--- + +## Q3.5 — Do you have active GCP Committed Use Discounts (CUDs)? + +**Rationale:** Active CUDs affect migration timing and cost comparison accuracy. If a customer has unexpired CUDs, they'll continue paying commitment fees even after migrating — this is a sunk cost that affects the migration ROI timeline. Also determines whether to compare against GCP list price or committed rate. + +**Conditional:** Only ask if `billing-profile.json` exists AND `commitments.has_active_cuds == true`. If billing data shows active CUDs, present the detected information and ask for confirmation/details. -Interpret (write constraint only — do NOT surface the downstream notes to the user): +> Your billing data shows active Committed Use Discounts (~[effective_discount_percent]% effective discount). CUD timing affects migration ROI — commitment fees continue regardless of usage until the term expires. +> +> A) Yes, and they expire within 6 months +> B) Yes, and they expire in 6–12 months +> C) Yes, and they have more than 12 months remaining +> D) Yes, but I'm not sure when they expire +> E) No active CUDs / I don't know +> F) I plan to let them expire and not renew + +| Answer | Recommendation Impact | +| ----------------------------- | --------------------------------------------------------------------------------------------------------------------- | +| Expire within 6 months | Migration timing favorable — plan migration to coincide with CUD expiration for clean cost transition | +| Expire in 6–12 months | Consider phased migration starting now; some overlap cost is acceptable for operational benefits | +| More than 12 months remaining | Factor CUD overlap cost into ROI analysis; migration still viable if operational benefits justify dual-payment period | +| Not sure when they expire | Recommend customer check GCP console (Billing → Commitments) before finalizing migration timeline | +| No active CUDs | No commitment overlap concern; migrate on any timeline | +| Plan to let them expire | Align migration completion with CUD expiration date for optimal cost transition | + +Interpret: ``` -A -> gcp_monthly_spend: "<$1K" -B -> gcp_monthly_spend: "$1K-$5K" -C -> gcp_monthly_spend: "$5K-$20K" -D -> gcp_monthly_spend: "$20K-$100K" -E -> gcp_monthly_spend: ">$100K" -F -> same as default (billing-informed bucket if billing data exists, otherwise B) +A -> cud_status: "expiring_soon" — Align migration with CUD expiration +B -> cud_status: "expiring_medium" — Phased migration acceptable; some overlap cost +C -> cud_status: "long_remaining" — Factor overlap into ROI; justify with operational benefits +D -> cud_status: "unknown_expiry" — Recommend checking GCP console +E -> cud_status: "none" — No constraint +F -> cud_status: "not_renewing" — Align migration completion with expiration ``` -Default: If `billing-profile.json` exists, use the billing-informed bucket from Step 2 extraction. Otherwise B — `gcp_monthly_spend: "$1K-$5K"`. +Default: E — `cud_status: "none"` (no constraint on migration timing). + +If `billing-profile.json` does not exist or `commitments.has_active_cuds == false`, **skip this question entirely**. --- @@ -154,8 +210,19 @@ Default: B — no constraint, evaluate full compute options. ## Q6 — If your application went down unexpectedly right now, what would happen? +**Auto-extract signal (Cloud SQL PostgreSQL/MySQL only):** Read `availability_type` from `google_sql_database_instance` (`config.availability_type` or top-level). When unambiguous: + +| GCP value | `availability` extracted | Skip Q6? | +| ---------- | ------------------------ | ------------------------------ | +| `ZONAL` | `"single-az"` | Yes — `chosen_by: "extracted"` | +| `REGIONAL` | `"multi-az"` | Yes — `chosen_by: "extracted"` | + +**Never auto-extract:** `multi-az-ha` and `multi-region` require Q6 user answers (Mission-Critical / Catastrophic) — IaC cannot infer these. Cloud SQL `REGIONAL` is RDS Multi-AZ (`multi-az`), not Aurora (`multi-az-ha`). Skip Q6 only when **all** instances agree. When instances disagree or `availability_type` is missing on any instance, ask Q6. + **Rationale:** Availability requirements drive database engine selection, deployment topology, and whether multi-AZ is mandatory. Aurora Global Database and multi-region compute are only recommended when Catastrophic is selected AND Q1 confirms global users — both signals are required. +**Cloud SQL PostgreSQL / MySQL → RDS vs Aurora (decision order):** For customers on Cloud SQL (PostgreSQL or MySQL), **Q6 is the only question that selects the AWS product family** — **RDS** (PostgreSQL or MySQL, matching the Cloud SQL engine) vs **Aurora** (Aurora PostgreSQL or Aurora MySQL). **Q12–Q13 never override Q6**; they tune sizing, replicas, storage/I/O billing, and Aurora variants **after** Q6 has chosen RDS or Aurora. When Cloud SQL is detected, you may add: _"For dev/staging or workloads where brief outage is tolerable, RDS PostgreSQL is usually simpler and cheaper; Aurora is for mission-critical HA needs."_ + **Context for user:** When asking, include these descriptions so the user can self-select accurately: - **Inconvenient** — users can wait, no revenue impact (e.g., internal tool, dev/staging environment, hobby project) @@ -199,8 +266,10 @@ Default: B — `availability: "multi-az"`. **Database migration tooling notes:** -- For PostgreSQL databases <10GB: **pg_dump/pg_restore** is sufficient. -- For larger PostgreSQL databases (>10GB): **pgcopydb** offers parallel table copying and index rebuilding, significantly reducing migration time within the same maintenance window. +- Read `preferences.json` → `design_constraints.db_size.value` (set by Q13b in `clarify-database.md`) to select the right tool. If absent, fall back to the size thresholds below. +- For PostgreSQL databases `db_size: "<10GB"` or unknown-small: **pg_dump/pg_restore** is sufficient. +- For PostgreSQL databases `db_size: "10-100GB"` or `"100-500GB"`: **pgcopydb** offers parallel table copying and index rebuilding, significantly reducing migration time within the same maintenance window. +- For PostgreSQL databases `db_size: ">500GB"`: **AWS DMS strongly recommended** regardless of maintenance window — single-pass export/import at this scale is high-risk. - pgcopydb's CDC mode requires `wal_level=logical` on Cloud SQL, which must be enabled explicitly. > The maintenance window determines your migration cutover strategy and which database migration tooling we recommend. Zero-downtime migrations require significantly more complex infrastructure. @@ -211,12 +280,12 @@ Default: B — `availability: "multi-az"`. > D) Flexible — we can schedule one if needed > E) I don't know -| Answer | Recommendation Impact | -| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Weekly window | Standard cutover with DNS switchover during window; **pg_dump/pg_restore** for PostgreSQL <10GB; **pgcopydb** for larger databases — parallel copying cuts migration time significantly; no DMS licensing, no replication lag risk | -| Monthly window | Cutover timed to monthly window; pg_dump/pg_restore or **pgcopydb** depending on DB size; blue/green for application layer | -| Zero downtime | **AWS DMS required** for live database replication; blue/green deployment for application layer; Aurora blue/green deployments; Route 53 weighted routing for traffic shifting | -| Flexible | Recommend scheduling a weekly window to enable pg_dump/pgcopydb approach; falls back to DMS if window cannot be arranged | +| Answer | Recommendation Impact | +| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Weekly window | Standard cutover with DNS switchover during window; **pg_dump/pg_restore** for PostgreSQL <10GB; **pgcopydb** for larger databases — parallel copying cuts migration time significantly; no DMS licensing, no replication lag risk | +| Monthly window | Cutover timed to monthly window; pg_dump/pg_restore or **pgcopydb** depending on DB size; blue/green for application layer | +| Zero downtime | **AWS DMS required** for live database replication; blue/green deployment for application layer; **RDS blue/green deployments** (RDS path per Q6) or **Aurora blue/green deployments** (Aurora path per Q6); Route 53 weighted routing for traffic shifting | +| Flexible | Recommend scheduling a weekly window to enable pg_dump/pgcopydb approach; falls back to DMS if window cannot be arranged | Interpret: diff --git a/migration-to-aws/steering/clarify.md b/migration-to-aws/steering/clarify.md index f1a85a49..24a3c602 100644 --- a/migration-to-aws/steering/clarify.md +++ b/migration-to-aws/steering/clarify.md @@ -1,10 +1,12 @@ # Phase 2: Clarify Requirements -**Phase 2 of 5** — Ask adaptive questions before design begins, then interpret answers into ready-to-apply design constraints. +**Phase 2 of 6** — Ask adaptive questions before design begins, then interpret answers into ready-to-apply design constraints. + +> **HARD GATE — Clarify before Design:** Do not load `steering/design.md` (or any later phase) until this phase finishes **and** `$MIGRATION_DIR/.phase-status.json` records `phases.clarify` as `"completed"`. Writing `preferences.json` without updating phase status is a protocol violation. If the user asks to skip questions, use documented defaults and still complete this phase (including phase status). The output — `preferences.json` — is consumed directly by Design and Estimate without any further interpretation. -Questions are organized into **six named categories (A–F)** with documented firing rules. Up to 22 questions across categories, depending on which discovery artifacts exist and which GCP services are detected. A standalone **AI-Only** flow exists for migrations that only move AI/LLM calls to Bedrock. +Questions are organized into **six named categories (A–F)** with documented firing rules. Up to 22 questions across categories, depending on which discovery artifacts exist and which GCP services are detected. Questions are presented in **progressive batches** (up to 3 batches) with intermediate saves between each — partial answers persist across sessions. A standalone **AI-Only** flow exists for migrations that only move AI/LLM calls to Bedrock. ## Category Reference Files @@ -12,23 +14,37 @@ Questions are organized into **six named categories (A–F)** with documented fi | --------------------- | ---------------------------- | --------- | ----------------------------------------------- | | `clarify-global.md` | A — Global/Strategic | Q1–Q7 | Always | | `clarify-compute.md` | B — Config Gaps, C — Compute | Q8–Q11 | Compute or billing-source resources present | -| `clarify-database.md` | D — Database | Q12–Q13 | Database resources present | -| `clarify-ai.md` | F — AI/Bedrock | Q14–Q22 | `ai-workload-profile.json` exists | +| `clarify-database.md` | D — Database | Q12–Q13b | Database resources present | +| `clarify-ai.md` | F — AI/Bedrock | Q14–Q26 | `ai-workload-profile.json` exists | | `clarify-ai-only.md` | _(standalone)_ | Q1–Q10 | AI-only migration (no infrastructure artifacts) | --- ## Step 0: Prior Run Check -If `$MIGRATION_DIR/preferences.json` already exists: +Check `$MIGRATION_DIR/` for existing state: + +**Case 1 — Completed preferences exist** (`preferences.json` present): > "I found existing migration preferences from a previous run. Would you like to:" > > A) Re-use these preferences and skip questions > B) Start fresh and re-answer all questions -- If A: skip to Validation Checklist, proceed with existing file. -- If B: continue to Step 1. +- If A: Run Step 2 item 6 only (BigQuery detection) on current discovery artifacts. If `bigquery_present` is **true**, output the Step 4 **BigQuery / deferred analytics** advisory block once (even though questions are skipped), then skip to Validation Checklist with the existing `preferences.json`. +- If B: delete `preferences.json`, continue to Step 1. + +**Case 2 — Draft preferences exist** (`preferences-draft.json` present, no `preferences.json`): + +> "I found a partial set of answers from a previous session ([N] of [total] batches completed). Would you like to:" +> +> A) Resume from where you left off — I'll pick up the remaining questions +> B) Start fresh and re-answer all questions + +- If A: load the draft, read `metadata.batches_completed` to determine which batches are done, skip completed batches when entering Step 4. +- If B: delete `preferences-draft.json`, continue to Step 1. + +**Case 3 — No prior state**: Continue to Step 1. --- @@ -47,9 +63,9 @@ At least one discovery artifact must exist to proceed. - **Full migration**: `gcp-resource-inventory.json` or `billing-profile.json` exists (may also have `ai-workload-profile.json`) - **AI-only migration**: ONLY `ai-workload-profile.json` exists (no infrastructure or billing artifacts) -**If AI-only**: Read `steering/clarify-ai-only.md` NOW and follow that flow. Skip all remaining steps below. +**If AI-only**: Read `clarify-ai-only.md` NOW and follow that flow. Skip all remaining steps below. -> **HARD GATE — AI-Only Path:** You MUST read `steering/clarify-ai-only.md` before presenting any questions. The question text, answer options, and interpretation rules are ONLY in that file — they are NOT in this file. Do NOT fabricate questions from the summaries above. +> **HARD GATE — AI-Only Path:** You MUST read `clarify-ai-only.md` before presenting any questions. The question text, answer options, and interpretation rules are ONLY in that file — they are NOT in this file. Do NOT fabricate questions from the summaries above. ### Discovery Summary @@ -73,6 +89,66 @@ Present a discovery summary: --- +## Step 1.5: Fast-Path Gate (Simple Stacks) + +**After presenting the Discovery Summary**, check `$MIGRATION_DIR/migration-preview.json` for fast-path eligibility: + +``` +IF migration-preview.json exists + AND eligible_for_clarify_fast_path == true +THEN offer infra fast-path (3 questions) +ELSE IF eligible_for_clarify_simple_path == true +THEN offer simple hybrid path (~6 questions) +ELSE skip to Step 2 (full Clarify) +``` + +### Infra fast-path (no AI) + +**If `eligible_for_clarify_fast_path`**, present this offer before any questions: + +> "Your stack looks straightforward — [primary_resource_count] resource(s), no database, no AI detected. +> +> Want to use smart defaults and answer just 3 questions instead of up to 22? +> +> **[Yes — 3 questions]** / **[No — ask me everything]**" + +**If user chooses Yes:** + +1. Ask only: **Q1** (region), **Q2** (compliance), **Q7** (maintenance window) — from `clarify-global.md`. +2. Apply documented defaults for ALL other questions. Record each in `metadata.questions_defaulted`. +3. Still run the BigQuery advisory if `bigquery_present` is true. +4. Write `preferences.json` with `metadata.clarify_mode: "fast_path"`. Skip Steps 2–4. + +### Simple hybrid path (simple infra + lightweight AI) + +**If `eligible_for_clarify_simple_path`**, present: + +> "Your stack looks straightforward with lightweight AI ([model IDs from profile]) — no agentic framework detected. +> +> Want a short question set (~6 questions) instead of the full flow? I'll use discovery for region, database sizing, and model detection. +> +> **[Yes — short path]** / **[No — ask me everything]**" + +**If user chooses Yes:** + +1. Run **Step 2 extraction** (mandatory — do not skip). +2. Run **Step 2.5 Detected Settings Confirmation** (mandatory — wait for user response). +3. Ask only questions **not** resolved by extraction (after any user corrections): + - **Q2** (compliance) — always ask + - **Q7** (maintenance window) — always ask + - **Q16** (AI priority) — from `clarify-ai.md` + - **Q21** (AI latency) — from `clarify-ai.md` + - **Q3** (GCP spend) — only if billing did not extract it + - **Q1** (region) — only if region extraction ambiguous (multiple GCP regions) +4. Apply documented defaults for all other unanswered questions. Record in `metadata.questions_defaulted`. +5. Write `preferences.json` with `metadata.clarify_mode: "simple_hybrid"`. Skip Step 4 batch loop — go to Category E opt-in (if applicable) then Step 5. + +**Agentic hard block:** If `agentic_profile.is_agentic == true`, **never offer** infra fast-path or simple hybrid path. Agentic workloads require Q23–Q26. + +**If user chooses No, or neither path is eligible:** Continue to Step 2 (full Clarify). + +--- + ## Step 2: Extract Known Information Before generating questions, scan the inventory to extract values that are already known: @@ -80,11 +156,154 @@ Before generating questions, scan the inventory to extract values that are alrea 1. **GCP regions** — Extract all GCP regions from the inventory. Map to the closest AWS region as a suggested default for Q1. 2. **Resource types present** — Build a set of resource types: compute (Cloud Run, Cloud Functions, GKE, GCE), database (Cloud SQL, Spanner, Memorystore), storage (Cloud Storage), messaging (Pub/Sub). 3. **Billing SKUs** — If `billing-profile.json` exists, check if any SKU reveals storage class, HA configuration, or other answerable questions. -4. **GCP spend baseline** — If `billing-profile.json` exists, read `summary.total_monthly_spend` and map it to the Q3 bucket (`<1000` → A, `1000-5000` → B, `5000-20000` → C, `20000-100000` → D, `>100000` → E). Use this as the **billing-informed default for Q3** instead of the hardcoded B. Still ask Q3 (the user may know about spend beyond this billing export), but if they answer "I don't know" or "use all defaults", apply the billing-derived bucket. -5. **Config confidence** — If inventory `metadata.source = "billing"`, identify resources with `config_confidence = "assumed"` for Category B questions. -6. **AI framework detection** — If `ai-workload-profile.json` exists, check `integration.gateway_type` and `integration.frameworks` for auto-detection of Q14 answer. +4. **Billing-only mode** — If `billing-profile.json` exists and `gcp-resource-inventory.json` does NOT exist, check `billing-profile.json → services[]` for Category B question matching. +5. **AI framework detection** — If `ai-workload-profile.json` exists, check `integration.gateway_type` and `integration.frameworks` for auto-detection of Q14 answer. +6. **BigQuery / analytics warehouse** — Set `bigquery_present` to **true** if **any** of: (a) a resource in `gcp-resource-inventory.json` has `gcp_type` (or equivalent type field) starting with `google_bigquery_`; (b) `billing-profile.json` lists a service/SKU that clearly indicates **BigQuery** (e.g., service name or SKU contains `BigQuery`). Otherwise `bigquery_present` is **false**. +7. **Database size auto-detect (Q13b)** — For each `google_sql_database_instance`, read `config.disk_size`, `config.disk_size_gb`, or `gcp_config.disk_size_gb`. Map to Q13b band and **skip Q13b** when unambiguous: + +| Disk size (GB) | `db_size` value | Skip Q13b? | +| -------------- | --------------- | ------------------------------ | +| < 10 | `"<10GB"` | Yes — `chosen_by: "extracted"` | +| 10 – 99 | `"10-100GB"` | Yes — `chosen_by: "extracted"` | +| 100 – 499 | `"100-500GB"` | Yes — `chosen_by: "extracted"` | +| ≥ 500 | `">500GB"` | Yes — `chosen_by: "extracted"` | -Record extracted values. Questions whose answers are fully determined by extraction will be skipped and the extracted value used directly with `chosen_by: "extracted"`. +If multiple instances disagree, ask Q13b. Record in `metadata.inventory_clarifications.db_size_gb` when extracted. + +1. **Q6 from Cloud SQL HA** — For each `google_sql_database_instance`, read `availability_type` (or `config.availability_type`): + +| GCP value | `availability` extracted | +| ---------- | ------------------------ | +| `ZONAL` | `"single-az"` | +| `REGIONAL` | `"multi-az"` | + +Skip Q6 only when **all** Cloud SQL PostgreSQL/MySQL instances agree on the same mapped value. **`multi-az-ha` and `multi-region` are never auto-extracted** — those require Q6 user answers (Mission-Critical / Catastrophic). Cloud SQL `REGIONAL` maps to `multi-az` (RDS Multi-AZ), not `multi-az-ha` (Aurora). Record in `metadata.inventory_clarifications.cloud_sql_ha`. When `availability_type` is missing on any instance, or instances disagree, ask Q6. + +1. **Q12/Q13 dev-tier defaults** — When **all** Cloud SQL instances match dev pattern (`db-f1-micro`, `db-g1-small`, or `tier` contains `micro`/`small` with `availability_type: ZONAL`), extract and **skip Q12 and Q13**. When instances mix dev and prod tiers, do not extract — ask Q12 and Q13. + +``` +database_traffic: "steady" — chosen_by: "extracted" +db_io_workload: "low" — chosen_by: "extracted" +``` + +1. **Q3 GCP spend from billing** — If `billing-profile.json` exists, map `summary.total_monthly_spend` to spend band and **skip Q3** when unambiguous: + +| Monthly USD | `gcp_monthly_spend` | +| ------------- | ------------------- | +| < 1,000 | `"<$1K"` | +| 1,000–4,999 | `"$1K-$5K"` | +| 5,000–19,999 | `"$5K-$20K"` | +| 20,000–99,999 | `"$20K-$100K"` | +| ≥ 100,000 | `">$100K"` | + +1. **Q1 region extraction** — When inventory has a **single** GCP region among PRIMARY compute/database resources, map to closest AWS region and **skip Q1** with `target_region` `chosen_by: "extracted"`. When multiple regions, suggest default but still ask Q1. + +1. **Q19 primary model** — If `ai-workload-profile.json` exists and `models[0].model_id` is set with confidence ≥ 0.8, map to Q19 answer and **skip Q19**. Set `ai_model_baseline` with `chosen_by: "extracted"`. + +1. **Q20 input modalities** — If `integration.capabilities_summary` exists: + +| Signal | Extract | Skip Q20? | +| ------------------------------------ | --------------------------------------------------------------------------------- | -------------------------------------------------------- | +| `vision: true` | `ai_vision: "vision-required"` | Yes | +| `image_generation: true` (no vision) | note in `ai_capabilities_required`; Q20 may still ask unless text-only path clear | Partial — skip if only text + image gen via separate API | +| all false / text only | `ai_vision: "text-only"` | Yes | + +When `image_generation: true` and `vision: false`, set `ai_capabilities_required` derived from profile and skip Q20 (image output is not vision _input_). + +1. **Q9 WebSocket scan** — Only when application code was **actually analyzed**. Treat code as analyzed when **any** of: (a) `discover-app-code.md` ran and found source files; (b) `ai-workload-profile.json` → `metadata.sources_analyzed.application_code == true`; (c) a companion app directory was scanned. Scan for WebSocket usage: `websocket`, `WebSocket`, `socket.io`, `@nestjs/websockets`, FastAPI WebSocket, `ws` package imports. If code was analyzed and **no matches**, extract `websocket: false` and **skip Q9**. If matches found, ask Q9 to confirm. + **If no application code was available** (Terraform-only workspace, no code discovery), do **NOT** extract Q9 — leave Q9 in the question flow. Absence of a code scan is not evidence of no WebSockets. + +1. **Q10 Cloud Run traffic** — If Cloud Run `min_instance_count` / `min_instances` > 0 in Terraform config, extract `cloud_run_traffic_pattern: "constant-24-7"` and skip Q10. Otherwise ask Q10. + +1. **Multi-instance Cloud SQL conflicts** — When multiple `google_sql_database_instance` resources **disagree** on values used for Q6, Q12/Q13, or Q13b (e.g. one ZONAL and one REGIONAL; mixed dev/prod tiers; different disk sizes): + - Do **not** extract a single global value or skip the affected question(s) + - Record per-instance values in `metadata.inventory_clarifications.cloud_sql_instances[]` (address, `availability_type`, `tier`, `disk_size_gb`) + - In Step 2.5, show a **per-instance breakdown** (see below) instead of a single summary row + - Ask the affected question(s), or let the user pick a global posture during Step 2.5 confirmation + +Record all extracted values in `metadata.inventory_clarifications` where applicable. Questions fully resolved by extraction are **skipped** (not asked) with `chosen_by: "extracted"` and listed in `metadata.questions_skipped_extracted`. + +**After Step 2 completes, proceed to Step 2.5 before Step 3 or any questions.** + +--- + +## Step 2.5: Confirm Detected Settings (Mandatory Gate) + +**When to run:** After Step 2 whenever any setting was extracted (i.e., `questions_skipped_extracted` would be non-empty, or any constraint was populated with `chosen_by: "extracted"` in working memory). + +**Skip Step 2.5 only when** Step 2 produced zero extractions — nothing inferred from IaC, billing, or code. Proceed directly to Step 3. + +**HARD GATE — do NOT present question batches or ask individual questions until the user responds to this summary.** + +Present a structured table (omit rows for settings not extracted): + +``` +### What we detected from your Terraform, billing, and code + +| Setting | Detected value | Source | Question skipped | +| ------- | -------------- | ------ | ---------------- | +| Region | us-west-2 (GCP us-west1) | gcp-resource-inventory.json | Q1 | +| GCP monthly spend | $1K–$5K (~$2,400/mo) | billing-profile.json | Q3 | +| Database availability | Single-AZ (Cloud SQL `ZONAL`) | Terraform `availability_type` | Q6 | +| Database size | 10–100 GB (allocated disk: 10 GB) | Terraform `disk_size` | Q13b | +| DB traffic / I/O | Steady / Low (dev-tier `db-f1-micro`) | Terraform tier + ZONAL | Q12, Q13 | +| Cloud Run traffic | Constant 24/7 (`min_instances > 0`) | Terraform | Q10 | +| WebSockets | None detected (code scanned) | application code scan | Q9 | +| AI framework | Direct SDK (no gateway) | ai-workload-profile.json | Q14 | +| AI model | gemini-2.5-flash | ai-workload-profile.json | Q19 | +| Input modalities | Text only | ai-workload-profile.json | Q20 | + +Does this look correct? + +- Reply **"looks good"** or **"correct"** to proceed — I'll only ask about what we couldn't infer. +- To fix something, name the setting and the correct value, e.g. **"availability: mission-critical"**, **"db size: 100-500GB"**, **"region: eu-central-1"**, **"model: gpt-4o"**. I'll update that setting; if the correction is ambiguous I'll ask the full question for that item. +- Reply **"ask me everything"** to discard all extractions and run the full question flow (clear `questions_skipped_extracted`; set all previously extracted constraints to pending). +``` + +**Multi-instance Cloud SQL conflicts:** When instances disagree, replace the single-row summary with a per-instance table and do **not** skip the conflicting question until resolved: + +``` +| Instance | availability_type | tier | disk_size (GB) | +| -------- | ----------------- | ---- | -------------- | +| google_sql_database_instance.main | ZONAL | db-f1-micro | 10 | +| google_sql_database_instance.analytics | REGIONAL | db-n1-standard-4 | 100 | + +These instances disagree on availability. Which posture should we use for the migration design? +A) Most conservative (highest HA) | B) Use [instance name] as primary | C) Ask me the full Q6 question +``` + +**Override handling** — when the user corrects a detected value: + +| User correction (examples) | Update constraint | Re-ask? | +| ------------------------------------------------ | -------------------------------------------------- | ------------------------- | +| `availability: mission-critical` / `multi-az-ha` | `availability: "multi-az-ha"`, `chosen_by: "user"` | No — value is explicit | +| `availability: significant` / `multi-az` | `availability: "multi-az"`, `chosen_by: "user"` | No | +| `availability: dev` / `single-az` | `availability: "single-az"`, `chosen_by: "user"` | No | +| `db size: <10GB` / `10-100GB` / etc. | Set `db_size` to stated band, `chosen_by: "user"` | No if band is explicit | +| `region: [AWS region]` | Set `target_region`, `chosen_by: "user"` | No | +| `model: [model name]` | Set `ai_model_baseline`, `chosen_by: "user"` | No if maps cleanly to Q19 | +| `websockets: yes` | Set `websocket: "required"`, `chosen_by: "user"` | No | +| `spend: $5K-$20K` | Set `gcp_monthly_spend`, `chosen_by: "user"` | No if band is explicit | +| Vague correction ("that's wrong") | Remove that item from skipped list | Yes — ask full question | + +For each override: remove the associated question ID(s) from `metadata.questions_skipped_extracted`, set `chosen_by: "user"`, and record in `metadata.detected_settings` with `"confirmed": false` and `"corrected_by_user": true`. + +When user confirms: mark all rows `"confirmed": true` in `metadata.detected_settings`. + +**`metadata.detected_settings` schema** (write to `preferences.json` at Step 5): + +```json +"detected_settings": [ + { + "key": "availability", + "value": "single-az", + "source": "terraform:availability_type=ZONAL", + "questions_skipped": ["Q6"], + "confirmed": true, + "corrected_by_user": false + } +] +``` --- @@ -92,22 +311,22 @@ Record extracted values. Questions whose answers are fully determined by extract ### Category Definitions and Firing Rules -| Category | Name | Firing Rule | Reference File | Questions | -| -------- | ------------------ | ----------------------------------------------------------------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------- | -| **A** | Global/Strategic | **Always fires** | `clarify-global.md` | Q1 (location), Q2 (compliance), Q3 (GCP spend), Q4 (funding stage), Q5 (multi-cloud), Q6 (uptime), Q7 (maintenance) | -| **B** | Configuration Gaps | Inventory with `metadata.source == "billing"` AND at least one `config_confidence == "assumed"` | `clarify-compute.md` | Cloud SQL HA, Cloud Run count, Memorystore memory, Functions gen | -| **C** | Compute Model | Compute resources present (Cloud Run, Cloud Functions, GKE, GCE) | `clarify-compute.md` | Q8 (K8s sentiment), Q9 (WebSocket), Q10 (Cloud Run traffic), Q11 (Cloud Run spend) | -| **D** | Database Model | Database resources present (Cloud SQL, Spanner, Memorystore) | `clarify-database.md` | Q12 (DB traffic pattern), Q13 (DB I/O) | -| **E** | Migration Posture | **Disabled by default** — requires explicit user opt-in | _(inline below)_ | HA upgrades, right-sizing | -| **F** | AI/Bedrock | `ai-workload-profile.json` exists | `clarify-ai.md` | Q14–Q22 | +| Category | Name | Firing Rule | Reference File | Questions | +| -------- | ------------------ | ------------------------------------------------------------------------------ | --------------------- | ------------------------------------------------------------------------------------------------------------------- | +| **A** | Global/Strategic | **Always fires** | `clarify-global.md` | Q1 (location), Q2 (compliance), Q3 (GCP spend), Q4 (funding stage), Q5 (multi-cloud), Q6 (uptime), Q7 (maintenance) | +| **B** | Configuration Gaps | `billing-profile.json` exists AND `gcp-resource-inventory.json` does NOT exist | `clarify-compute.md` | Cloud SQL HA, Cloud Run count, Memorystore memory, Functions gen | +| **C** | Compute Model | Compute resources present (Cloud Run, Cloud Functions, GKE, GCE) | `clarify-compute.md` | Q8 (K8s sentiment), Q9 (WebSocket), Q10 (Cloud Run traffic), Q11 (Cloud Run spend) | +| **D** | Database Model | Database resources present (Cloud SQL, Spanner, Memorystore) | `clarify-database.md` | Q12 (DB traffic pattern), Q13 (DB I/O), Q13b (DB size) | +| **E** | Migration Posture | **Disabled by default** — requires explicit user opt-in | _(inline below)_ | HA upgrades, right-sizing | +| **F** | AI/Bedrock | `ai-workload-profile.json` exists | `clarify-ai.md` | Q14–Q26 (Q14–Q22 always; Q23–Q26 only when `agentic_profile.is_agentic == true`) | **Apply firing rules to determine which categories are active:** 1. Category A is always active. -2. Check inventory `metadata.source` — if `"billing"` with assumed configs, Category B is active. +2. Check for billing-only mode — if `billing-profile.json` exists and `gcp-resource-inventory.json` does NOT, Category B is active. 3. Check for compute resources — if present, Category C is active. Within C, skip Q8 if no GKE present. Skip Q10/Q11 if no Cloud Run present. 4. Check for database resources — if present, Category D is active. -5. Category E is disabled by default. Do not activate unless user opts in. +5. Category E is disabled by default. Offered after the last batch completes in Step 4 (see **Category E Opt-In** in Step 4). If user declines or does not respond, apply Category E defaults (no HA upgrades, no right-sizing). 6. Check for `ai-workload-profile.json` — if present, Category F is active. **If no IaC, billing data, or code is available** (empty discovery): only Category A is active. All service-specific categories are skipped. @@ -120,12 +339,12 @@ Record extracted values. Questions whose answers are fully determined by extract > > **Read these files based on which categories are active:** > -> | Active Category | File to Read | -> | --------------- | ------------------------------- | -> | A (always) | `steering/clarify-global.md` | -> | B or C | `steering/clarify-compute.md` | -> | D | `steering/clarify-database.md` | -> | F | `steering/clarify-ai.md` | +> | Active Category | File to Read | +> | --------------- | --------------------- | +> | A (always) | `clarify-global.md` | +> | B or C | `clarify-compute.md` | +> | D | `clarify-database.md` | +> | F | `clarify-ai.md` | > > **Do NOT proceed to Step 4 until you have read every applicable file above.** @@ -134,9 +353,38 @@ Record extracted values. Questions whose answers are fully determined by extract Apply these before presenting questions: - **Q5 = "Yes, multi-cloud required"** — Immediately record `compute: "eks"`. Skip Q8 (Kubernetes sentiment) — all container workloads resolve to EKS. +- **Q6 extracted** — When Step 2 mapped Cloud SQL `availability_type` → skip Q6. - **Q10/Q11 N/A** — Cloud Run not present, auto-skip. -- **Q12/Q13 N/A** — Cloud SQL not present, auto-skip. -- **Q14 auto-detected** — If `integration.gateway_type` and `integration.frameworks` fully resolve the framework, skip Q14 and use extracted value. +- **Q10 extracted** — When min_instances > 0, skip Q10. +- **Q12/Q13 N/A** — Cloud SQL (PostgreSQL or MySQL) not present in inventory, auto-skip. +- **Q12/Q13 extracted** — Dev-tier Cloud SQL (Step 2 item 9), skip Q12 and Q13. +- **Q13b extracted** — Unambiguous disk size from inventory (Step 2 item 7), skip Q13b. +- **Q3 extracted** — Billing band mapped (Step 2 item 10), skip Q3. +- **Q1 extracted** — Single-region inventory (Step 2 item 11), skip Q1. +- **Q9 extracted** — No WebSocket signals in a completed code scan (Step 2 item 14), skip Q9. **Do not extract** when no code was analyzed. +- **Q14 auto-detected** — If `integration.gateway_type` is non-null OR `integration.frameworks` is non-empty in `ai-workload-profile.json`, skip Q14. Set `ai_framework` with `chosen_by: "extracted"`. +- **Q19 auto-detected** — Primary model from `ai-workload-profile.json` (Step 2 item 12), skip Q19. +- **Q20 auto-detected** — Modalities from `capabilities_summary` (Step 2 item 13), skip Q20. + +### Batch Planning + +After determining active categories, organize questions into **up to three batches** presented sequentially with intermediate saves: + +| Batch | Name | Categories | Questions | Fires When | +| ----- | ---------------------- | ------------------------------------------ | --------------------------------- | ----------------------------------------- | +| **1** | Strategic Requirements | A (Global/Strategic) | Q1–Q7 (minus Q4) | Always | +| **2** | Infrastructure | B (Config Gaps), C (Compute), D (Database) | Q8–Q13b + Category B prompts | Any compute or database resources present | +| **3** | AI Workloads | F (AI/Bedrock) | Q14–Q26 (Q23–Q26 only if agentic) | `ai-workload-profile.json` exists | + +**Determine active batches:** + +1. Batch 1 is always active. +2. Batch 2 is active if Category B, C, or D fired. +3. Batch 3 is active if Category F fired. + +Record the ordered list of active batches and count the questions per batch (after extraction and early-exit filtering). These counts are used for per-batch progress messaging — not shown as a grand total upfront. + +**Category E** (Migration Posture) is offered after the last substantive batch completes, before writing final `preferences.json`. --- @@ -147,156 +395,194 @@ _Default behavior when disabled:_ Apply conservative defaults — no HA upgrades If the user opts in, present after all other categories: -- **HA upgrade preference**: Should we recommend upgrading Single-AZ to Multi-AZ where possible? - > Default: No — keep current topology. -- **Right-sizing from billing**: Should we use billing utilization data to right-size instance types? - > Default: No — match current capacity. +### Q-E1 — Should we recommend upgrading Single-AZ to Multi-AZ where possible? + +> A) Yes — upgrade to Multi-AZ for higher availability | B) No — keep current topology + +Interpret → `ha_upgrade`: A → `true`, B → `false`. Default: B → `false`. + +### Q-E2 — Should we use billing utilization data to right-size instance types? + +> A) Yes — right-size based on utilization | B) No — match current capacity + +Interpret → `right_sizing`: A → `true`, B → `false`. Default: B → `false`. --- -## Step 4: Present Questions Interactively +## Step 4: Present Questions in Progressive Batches -Present questions **one at a time** in conversational order. Wait for the user's response to each question before presenting the next one. This creates a natural dialogue rather than an overwhelming form. +**Prerequisite:** Step 2.5 confirmation must be complete (user said "looks good" or finished correcting) before presenting Batch 1. Do not re-show the full detected-settings table here unless the user asks for a recap. -**Formatting rule:** When presenting answer options, put each option on its own line with a blank line between options so they are visually separated and easy to scan. Never run options together on consecutive lines without spacing. +**BigQuery / deferred analytics (mandatory callout):** If Step 2 set `bigquery_present` to **true**, output this block **once**, **before** any questions (same turn as Batch 1), then continue with the question flow: -### Flow +> **BigQuery / analytics warehouse:** Your discovery inputs include BigQuery. This skill **does not** select an AWS analytics or data-warehouse target (no Athena, Redshift, Glue, or EMR recommendation from the plugin). **Before** warehouse, data lake, SQL analytics, or BI cutover planning, engage your **AWS account team** and/or a **data analytics migration partner** to assess query patterns, data volumes, ETL/ELT, and downstream consumers. Design will mark these resources as **`Deferred — specialist engagement`**. -1. **Calculate question set**: Before starting, determine which questions will be asked. Remove all **pre-excluded** questions first — these are never counted in M and never mentioned to the user: - - Category firing rules (which categories are active based on inventory artifacts) - - N/A skips (Cloud Run not present → remove Q10/Q11; Cloud SQL not present → remove Q12/Q13; no GKE → remove Q8) - - Auto-detection skips (Q14 framework auto-detected from code) - - IDE/CLI mode skips (Q4 in non-web mode) +Questions are presented in sequential batches with a save after each. After each batch the user can skip individual questions (defaults applied), say **"use defaults for the rest"** to apply defaults for all remaining batches and proceed immediately, or answer normally. - Pre-excluded questions do not exist from the user's perspective — do NOT announce them as "skipped" during the flow. +### Batch Loop - Calculate `M` = total questions remaining after removing pre-excluded questions. **Do NOT factor in answer-dependent early-exit skips** (e.g., Q5→Q8) — those are the only skips communicated to the user mid-flow. +For each active batch (determined in Batch Planning above), execute steps 4a–4d: - **Double-check:** List out the specific question numbers that will be asked. Verify the count matches M. +#### 4a. Present Batch -2. **Opening message**: Present a brief introduction before the first question: +Use a conversational tone with brief context explaining why each question matters. Number questions within each batch starting from 1. - > Before mapping your infrastructure to AWS, I have **M questions** to tailor the migration plan. I'll ask them one at a time. - > - > You can answer each, or say **"use all defaults"** to accept defaults for all remaining questions. +**Batch 1 — Strategic Requirements (always first):** -3. **Ask one question at a time**: For each question in order: - - Show the progress indicator: **"Question N of M"** - - When entering a new section for the first time, show the section header - - Present the question with its context and answer options - - **Wait for the user's response before presenting the next question** - - Apply the interpret rule immediately after receiving the answer - - Check inline early-exit rules: if Q5 = "Yes, multi-cloud required", record `compute: "eks"`, remove Q8 from remaining questions, add Q8 to `questions_skipped_early_exit`, and reduce `M` by the number of questions skipped. +``` +Before mapping your infrastructure to AWS, I have a few sections of questions +to tailor the migration plan. You can answer each, skip individual ones +(I'll use sensible defaults), or say "use defaults for the rest" at any point. - **Communicating skipped questions:** When an answer causes later questions to be skipped, reduce `M` by the number of skipped questions and show a one-line note before the next question: - > *(Skipping [count] question(s) based on your previous answer — [M] questions total now.)* +Let's start with your strategic requirements. - Then continue the progress indicator using the updated `M`. For example, if the original M was 8 and 1 question is skipped after Question 4, M becomes 7. The next question shown is "Question 5 of 7", then "Question 6 of 7", then "Question 7 of 7". +--- Strategic Requirements --- - Keep it short and general. Never reference internal question IDs (Q8, Q14, etc.) or technical constraint details (compute: "eks") in user-facing messages. The user doesn't need to know which specific question was removed or why — just that the total changed. +Question 1: [Q1 text with context] +Question 2: [Q2 text with context] +... +Question [N]: [Q7 text with context] +``` -4. **Handle user shortcuts at any point during the conversation**: - - **"use all defaults"** or **"defaults for the rest"** → Before applying defaults, present a plain-language summary of what the defaults mean for the remaining unanswered questions so the customer knows exactly what they're getting. Use a bulleted list where each item is a self-contained sentence — no option codes, no table columns. Example: +**Batch 2 — Infrastructure (if active):** - > Here's what I'll assume for the remaining questions: - > - > - **Uptime**: Your app needs high availability — we'll deploy across multiple availability zones (Multi-AZ) - > - **Maintenance window**: Flexible — we can schedule a maintenance window when you're ready for cutover - > - **Kubernetes**: Neutral stance — we'll evaluate both EKS and ECS and recommend the best fit - > - **Database I/O**: Medium workload — Aurora standard pricing (we can switch to I/O-Optimized later if needed) - > - *(... list all remaining unanswered questions ...)* - > - > These are conservative, middle-of-the-road choices. Does this work for you, or would you prefer to go through them one by one? +After Batch 1 answers are saved, present: - - If the customer **accepts** → Apply all defaults, skip to Step 5 - - If the customer **rejects** or wants to go through them → Continue presenting questions one at a time as normal from where they left off +``` +Got it — your strategic preferences are saved. -5. **Handle "I don't know" or "idk" answers**: When the user selects "I don't know" (or says "idk", "not sure", etc.) for any question, do NOT silently apply the default. Instead, explain the default and confirm: +Next up: [N] questions about your compute and database setup. +You can answer each, skip individual ones, or say "use defaults for the rest." - > The default for this question is **[default option label]** — [plain-language explanation of what this means for their migration]. - > - > Is that acceptable, or would you like to reconsider? - > - > **Exception — Q3 (GCP spend):** Do not explain downstream implications (credits, pricing tiers, savings programs). Just state the default bucket and, if billing data exists, note it matches their billing data. Nothing more. +--- Infrastructure --- - - If the customer **accepts** → Apply the default, record `chosen_by: "default"`, move to the next question - - If the customer **wants to reconsider** → Re-present the same question's options (excluding "I don't know") and let them choose +Question 1: [first active question text with context] +... +``` -6. **Section transitions**: When moving between categories, briefly introduce the new section: - > *Now let's talk about your infrastructure...* - > - > *(or)* *Now let's discuss your AI workloads...* +**Batch 3 — AI Workloads (if active):** -7. **Completion**: After the last question is answered, confirm: - > That's all my questions. Let me generate your migration preferences. +After prior batch answers are saved, present. Adapt the intro based on whether this is the second or third batch: - **If questions were skipped**, acknowledge it briefly so the user isn't surprised by the gap between the announced total and the number actually asked: - > That's all my questions — the rest didn't apply to your setup. Let me generate your migration preferences. +``` +[Infrastructure preferences saved. / Strategic preferences saved.] -### Question Order +Last section — [N] questions about your AI workloads, then we're ready to design. +You can answer each, skip individual ones, or say "use defaults for the rest." -Present in this order (skipping any that don't apply): +--- AI Workloads --- +Question 1: [first active question text with context] +... ``` ---- Section 1: About Your Users & Requirements --- -Q1 → Q2 → Q3 → Q4 → Q5 → Q6 → Q7 ---- Section 2: Your Infrastructure --- -[Category B questions, if applicable] -Q8 → Q9 → Q10 → Q11 → Q12 → Q13 +If Batch 3 is the second batch (Batch 2 was skipped because no infra resources), use "Next up" instead of "Last section" if appropriate. + +**Single-batch shortcut:** If only Batch 1 is active (no infrastructure or AI categories fired), skip the multi-batch framing. Present Batch 1 questions with a simpler intro and proceed directly to Category E opt-in then Step 5 after answers — no draft file needed. + +#### 4b. Wait for Response + +Wait for the user's response to the current batch. Do NOT present the next batch or proceed to Design without a response or an explicit "use defaults for the rest." + +**"Use defaults for the rest" handling:** If the user says this at any point: ---- Section 3: AI Workloads --- -Q14 → Q15 → Q16 → Q17 → Q18 → Q19 → Q20 → Q21 → Q22 +1. Apply documented defaults for all unanswered questions in the current batch. +2. Apply documented defaults for all questions in remaining batches. +3. Skip directly to Category E opt-in, then Step 5 (write final `preferences.json`). + +#### 4c. Interpret Batch Answers + +Apply the interpret rule (from the category reference files) for every answered question in the batch. For skipped questions within the batch, apply the documented default. + +Apply early-exit rules triggered by this batch's answers. For example, if Batch 1 includes Q5 = "Yes, multi-cloud required", record `compute: "eks"` and mark Q8 as skipped (early-exit) for Batch 2. + +#### 4d. Save Draft + +**If more batches remain** after this one: Write (or update) `$MIGRATION_DIR/preferences-draft.json` with all answers collected so far. Use the same schema as `preferences.json` with these additional `metadata` fields: + +```json +{ + "metadata": { + "draft": true, + "batches_completed": ["strategic"], + "batches_remaining": ["infrastructure", "ai"], + "migration_type": "full", + "timestamp": "", + ... + }, + "design_constraints": { ... }, + "ai_constraints": { ... } +} ``` -Wait for the user's response to EVERY question. Do NOT batch questions. Do NOT proceed to Design without completing all questions or receiving an explicit "use all defaults". +Batch name values: `"strategic"`, `"infrastructure"`, `"ai"`. ---- +Return to **4a** for the next batch. -## Answer Combination Triggers +**If this was the last active batch**: Do not write a draft — proceed to **Category E opt-in** then **Step 5**. + +### Category E Opt-In + +After the last substantive batch is answered (but before writing final `preferences.json`), offer Category E if `billing-profile.json` exists: -| Scenario | Key Answers | Recommendation | -| ---------------------------- | -------------------------------------------- | --------------------------------------------------------- | -| Early-stage credits | Q4 = Pre-seed/Seed | AWS Activate Founders or Portfolio credits | -| Growth-stage credits | Q4 = Series B+ and Q3 spend | IW Migrate or MAP credits based on ARR | -| Must stay portable | Q5 = Yes multi-cloud | EKS only, no ECS Fargate | -| Kubernetes-averse | Q5 = No + Q8 = Frustrated | ECS Fargate strongly recommended | -| WebSocket app | Q9 = Yes | ALB WebSocket config required | -| Low-traffic Cloud Run | Q10 = Business hours + Q11 < $100 | Recommend staying on Cloud Run | -| High I/O database | Q13 = High IOPS | Aurora I/O-Optimized | -| Write-heavy global DB | Q6 = Catastrophic + Q12 = Write-heavy/global | Aurora DSQL | -| Rapidly growing DB | Q12 = Rapidly growing | Aurora Serverless v2 | -| Zero downtime required | Q7 = No downtime | Blue/green + AWS DMS required | -| HIPAA compliance | Q2 = HIPAA | BAA services only, specific regions | -| FedRAMP required | Q2 = FedRAMP | GovCloud regions only | -| Gateway-only AI | Q14 = B only (LLM router/gateway) | Config change only; skip SDK migration | -| LangChain/LangGraph AI | Q14 includes C | Provider swap via ChatBedrock; 1–3 days | -| OpenAI Agents SDK | Q14 includes E | Highest AI effort; Bedrock Agents; 2–4 weeks | -| Multi-agent + MCP | Q14 = D + F | Bedrock Agents to unify orchestration + MCP | -| Voice platform AI | Q14 includes G | Check native Bedrock support; Nova Sonic if needed | -| GPT-4 Turbo migration | Q19 = GPT-4 Turbo | Claude Sonnet 4.6 — 70% cheaper on input | -| o-series migration | Q19 = o-series | Claude Sonnet 4.6 with extended thinking | -| High-volume cost-critical AI | Q18 = High + cost critical | Nova Micro or Haiku 4.5 + provisioned throughput | -| Reasoning/agent workload | Q17 = Extended thinking | Claude Sonnet 4.6 extended thinking; Opus 4.6 for hardest | -| Speech-to-speech AI | Q17 = Real-time speech | Nova Sonic | -| RAG workload | Q17 = RAG optimization | Bedrock Knowledge Bases + Titan Embeddings | -| Vision workload | Q20 = Vision required | Claude Sonnet 4.6 (multimodal) | -| Latency-critical AI | Q21 = Critical | Haiku 4.5 or Nova Micro + streaming | -| Complex reasoning tasks | Q22 = Complex | Claude Sonnet 4.6; Opus 4.6 for hardest | +> "Would you also like HA upgrade and right-sizing recommendations based on your billing data? If not, I'll use conservative defaults (no upgrades, match current capacity)." + +If user opts in, present Q-E1–Q-E2 (defined in **Category E — Migration Posture** above). Otherwise, apply Category E defaults (`ha_upgrade: false`, `right_sizing: false`). --- -## Step 5: Interpret and Write preferences.json +## Answer Combination Triggers + +| Scenario | Key Answers | Recommendation | +| ---------------------------------------- | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | +| Early-stage funding path | Q3 = lower spend band | Entry-tier migration funding program review | +| Growth-stage funding path | Q3 = higher spend band | Migration funding/support program review based on spend profile | +| Must stay portable | Q5 = Yes multi-cloud | EKS only, no ECS Fargate | +| Kubernetes-averse | Q5 = No + Q8 = Frustrated | ECS Fargate strongly recommended | +| WebSocket app | Q9 = Yes | ALB WebSocket config required | +| Low-traffic Cloud Run | Q10 = Business hours + Q11 < $100 | Recommend staying on Cloud Run | +| Cloud SQL Postgres — dev/low HA | Q6 = Inconvenient + Cloud SQL in inventory | **RDS PostgreSQL** single-AZ | +| Cloud SQL Postgres — prod HA (RDS) | Q6 = Significant Issue + Cloud SQL in inventory | **RDS PostgreSQL** Multi-AZ | +| Cloud SQL Postgres — mission-critical | Q6 = Mission-Critical + Cloud SQL in inventory | **Aurora PostgreSQL** Multi-AZ; apply Q12/Q13 | +| Cloud SQL Postgres — global catastrophic | Q6 = Catastrophic + Q1 = Global + Cloud SQL in inventory | **Aurora PostgreSQL Global Database** | +| High I/O database (RDS path) | Q6 = Inconvenient/Significant + Q13 = High | **RDS** io2 or Provisioned IOPS | +| High I/O database (Aurora path) | Q6 = Mission-Critical/Catastrophic + Q13 = High | Aurora I/O-Optimized | +| Write-heavy global DB | Q6 = Mission-Critical/Catastrophic + Q12 = Write-heavy/global | Aurora DSQL architecture review (RDS path only: size writer; flag review) | +| Rapidly growing DB (RDS path) | Q6 = Inconvenient/Significant + Q12 = Rapidly growing | RDS with headroom on instance class | +| Rapidly growing DB (Aurora path) | Q6 = Mission-Critical/Catastrophic + Q12 = Rapidly growing | Aurora Serverless v2 | +| Zero downtime required | Q7 = No downtime | Blue/green + AWS DMS required (RDS or Aurora blue/green per Q6) | +| HIPAA compliance | Q2 = HIPAA | BAA services only, specific regions | +| FedRAMP required | Q2 = FedRAMP | GovCloud regions only | +| CCPA / CPRA | Q2 = G (CCPA / CPRA) | Consumer privacy, logging/retention, data-inventory posture; confirm regions with legal review | +| Gateway-only AI | Q14 = B only (LLM router/gateway) | Config change only; skip SDK migration | +| LangChain/LangGraph AI | Q14 includes C | Provider swap via ChatBedrock; 1–3 days | +| OpenAI Agents SDK | Q14 includes E | Highest AI effort; Bedrock Agents; 2–4 weeks | +| Multi-agent + MCP | Q14 = D + F | Bedrock Agents to unify orchestration + MCP | +| Voice platform AI | Q14 includes G | Check native Bedrock support; Nova 2 Sonic if needed | +| GPT-5.5 migration | Q19 = GPT-5.5 | Claude Opus 4.6 — Bedrock 17% cheaper on output; or Sonnet 4.6 for 53% savings | +| GPT-5.5 Pro migration | Q19 = GPT-5.5 Pro | Nova 2 Pro — 95% cheaper on Bedrock | +| GPT-5.4 migration | Q19 = GPT-5.4 | Claude Sonnet 4.6 — near price parity; AWS consolidation | +| GPT-5.4 Mini/Nano migration | Q19 = GPT-5.4 Mini or Nano | Nova Lite/Micro — 87-94% cheaper on Bedrock | +| GPT-4 Turbo migration | Q19 = GPT-4 Turbo | Claude Sonnet 4.6 — 70% cheaper on input | +| o-series migration | Q19 = o-series | Claude Sonnet 4.6 with extended thinking | +| High-volume cost-critical AI | Q18 = High + cost critical | Nova Micro or Haiku 4.5 + provisioned throughput | +| Reasoning/agent workload | Q17 = Extended thinking | Claude Sonnet 4.6 extended thinking; Opus 4.6 for hardest | +| Speech-to-speech AI | Q17 = Real-time speech | Nova 2 Sonic | +| RAG workload | Q17 = RAG optimization | Bedrock Knowledge Bases + Titan Embeddings | +| Vision workload | Q20 = Vision required | Claude Sonnet 4.6 (multimodal) | +| Latency-critical AI | Q21 = Critical | Haiku 4.5 or Nova Micro + streaming | +| Complex reasoning tasks | Q22 = Complex | Claude Sonnet 4.6; Opus 4.6 for hardest | -Apply the interpret rule for every answered question (defined in each category file). For skipped questions, apply the documented default. +--- -**Before writing the file**, verify that every question Q1–Q22 appears in exactly one of: `questions_asked`, `questions_defaulted`, `questions_skipped_extracted`, `questions_skipped_early_exit`, or `questions_skipped_not_applicable`. If any question is missing, fix it before writing the file. +## Step 5: Assemble and Write preferences.json -Write `$MIGRATION_DIR/preferences.json`: +Assemble all interpreted answers from the completed batches into the final `$MIGRATION_DIR/preferences.json`. If `preferences-draft.json` exists, use it as the base — merge in the final batch's answers, remove the draft-specific metadata fields (`draft`, `batches_completed`, `batches_remaining`), and set `metadata.timestamp` to the current time. Write `$MIGRATION_DIR/preferences.json`: ```json { "metadata": { + "migration_type": "full", "timestamp": "", "discovery_artifacts": ["gcp-resource-inventory.json", "ai-workload-profile.json"], "questions_asked": [ @@ -306,7 +592,6 @@ Write `$MIGRATION_DIR/preferences.json`: "Q5", "Q6", "Q7", - "Q14", "Q16", "Q17", "Q19", @@ -316,8 +601,19 @@ Write `$MIGRATION_DIR/preferences.json`: "questions_defaulted": ["Q9"], "questions_skipped_extracted": ["Q14"], "questions_skipped_early_exit": ["Q8"], - "questions_skipped_not_applicable": ["Q4", "Q10", "Q11", "Q12", "Q13"], + "questions_skipped_not_applicable": ["Q4", "Q10", "Q11", "Q12", "Q13", "Q13b"], + "detected_settings": [ + { + "key": "availability", + "value": "multi-az", + "source": "terraform:availability_type=REGIONAL", + "questions_skipped": ["Q6"], + "confirmed": true, + "corrected_by_user": false + } + ], "category_e_enabled": false, + "clarify_mode": "full", "inventory_clarifications": {} }, "design_constraints": { @@ -329,27 +625,23 @@ Write `$MIGRATION_DIR/preferences.json`: "cutover_strategy": { "value": "maintenance-window-weekly", "chosen_by": "user" }, "kubernetes": { "value": "eks-or-ecs", "chosen_by": "user" }, "database_traffic": { "value": "steady", "chosen_by": "user" }, - "db_io_workload": { "value": "medium", "chosen_by": "user" } + "db_io_workload": { "value": "medium", "chosen_by": "user" }, + "db_size": { "value": "10-100GB", "chosen_by": "user" } }, "ai_constraints": { "ai_framework": { "value": ["direct"], "chosen_by": "extracted" }, "ai_monthly_spend": { "value": "$500-$2K", "chosen_by": "user" }, "ai_priority": { "value": "balanced", "chosen_by": "user" }, "ai_critical_feature": { "value": "function-calling", "chosen_by": "user" }, - "ai_volume_cost": { "value": "low-quality", "chosen_by": "user" }, - "ai_model_baseline": { "value": "claude-sonnet-4-6", "chosen_by": "derived" }, + "ai_token_volume": { "value": "low", "chosen_by": "user" }, + "ai_model_baseline": { "value": "claude-sonnet-4-6", "chosen_by": "user" }, "ai_vision": { "value": "text-only", "chosen_by": "user" }, "ai_latency": { "value": "important", "chosen_by": "user" }, "ai_complexity": { "value": "moderate", "chosen_by": "user" }, "ai_capabilities_required": { "value": ["text_generation", "streaming", "function_calling"], - "chosen_by": "derived" + "chosen_by": "extracted" } - }, - "question_details": { - "Q1": { "question": "Where are your users located?", "answer": "A) Single region", "interpreted_as": { "target_region": "us-east-1" } }, - "Q8": { "question": "How does your team feel about managing Kubernetes?", "answer": null, "skipped_reason": "early_exit", "note": "Q5=multi-cloud → EKS only" }, - "Q9": { "question": "Do any services need WebSocket support?", "answer": "I don't know", "interpreted_as": {}, "note": "Default: no constraint" } } } ``` @@ -363,11 +655,13 @@ Write `$MIGRATION_DIR/preferences.json`: 5. For billing-source inventories, `metadata.inventory_clarifications` records Category B answers. 6. `metadata.questions_skipped_early_exit` records questions skipped due to early-exit logic (e.g., Q8 skipped because Q5=multi-cloud). 7. `metadata.questions_skipped_extracted` records questions skipped because inventory already provided the answer. -8. `metadata.questions_skipped_not_applicable` records questions skipped because the relevant service wasn't in the inventory. -9. `ai_constraints` section is present ONLY if Category F fired. Omit entirely if no AI artifacts exist. -10. `ai_constraints.ai_capabilities_required` is the UNION of detected capabilities from `ai-workload-profile.json` + critical feature from Q17 + vision from Q20. `chosen_by` is `"derived"`. -11. `ai_constraints.ai_framework` is an array (Q14 is select-all-that-apply). If auto-detected, `chosen_by` is `"extracted"`. -12. `question_details` — audit log of Q&A. One entry per question in `questions_asked` + `questions_defaulted` + `questions_skipped_early_exit`. Fields: `question`, `answer` (verbatim or `null` if skipped), `interpreted_as`, optional `skipped_reason`/`note`. Omit `questions_skipped_not_applicable`. Informational only — downstream phases ignore it. +8. `metadata.detected_settings` records each auto-detected setting with source, confirmation status, and whether the user corrected it in Step 2.5. +9. `metadata.questions_skipped_not_applicable` records questions skipped because the relevant service wasn't in the inventory. +10. `ai_constraints` section is present ONLY if Category F fired. Omit entirely if no AI artifacts exist. +11. `ai_constraints.ai_capabilities_required` is the UNION of detected capabilities from `ai-workload-profile.json` + critical feature from Q17 + vision from Q20. `chosen_by` is `"derived"`. +12. `ai_constraints.ai_framework` is an array (Q14 is select-all-that-apply). If auto-detected, `chosen_by` is `"extracted"`. + +After writing `preferences.json`, delete `$MIGRATION_DIR/preferences-draft.json` if it exists. --- @@ -377,7 +671,7 @@ Write `$MIGRATION_DIR/preferences.json`: | ----------------------- | -------------------- | ------------------------------------------------- | | Q1 — Location | A (single region) | `target_region`: closest AWS region to GCP region | | Q2 — Compliance | A (none) | no constraint | -| Q3 — GCP spend | Billing-informed bucket if `billing-profile.json` exists, otherwise B ($1K–$5K) | `gcp_monthly_spend` mapped from billing data or `"$1K-$5K"` | +| Q3 — GCP spend | B ($1K–$5K) | `gcp_monthly_spend: "$1K-$5K"` | | Q4 — Funding stage | _(skip in IDE mode)_ | no constraint | | Q5 — Multi-cloud | B (AWS-only) | no constraint | | Q6 — Uptime | B (significant) | `availability: "multi-az"` | @@ -388,13 +682,14 @@ Write `$MIGRATION_DIR/preferences.json`: | Q11 — Cloud Run spend | B ($100–$500) | `cloud_run_monthly_spend: "$100-$500"` | | Q12 — DB traffic | A (steady) | `database_traffic: "steady"` | | Q13 — DB I/O | B (medium) | `db_io_workload: "medium"` | +| Q13b — DB size | E (unknown) | `db_size: "unknown"` → default to pgcopydb | | Q14 — AI framework | _(auto-detect)_ | `ai_framework` from code detection | | Q15 — AI spend | B ($500–$2K) | `ai_monthly_spend: "$500-$2K"` | | Q16 — AI priority | E (balanced) | `ai_priority: "balanced"` | | Q17 — Critical feature | J (none) | no additional override | -| Q18 — Volume + cost | A (low + quality) | `ai_volume_cost: "low-quality"` | +| Q18 — Volume + cost | A (low + quality) | `ai_token_volume: "low"` | | Q19 — Current model | _(auto-detect)_ | `ai_model_baseline` from code detection | -| Q20 — Vision | A (text only) | no constraint | +| Q20 — Input types | A (text only) | no constraint | | Q21 — AI latency | B (important) | `ai_latency: "important"` | | Q22 — Task complexity | B (moderate) | `ai_complexity: "moderate"` | @@ -404,9 +699,12 @@ Write `$MIGRATION_DIR/preferences.json`: Before handing off to Design: +- [ ] If extractions were made, Step 2.5 detected-settings confirmation was shown and user responded before questions +- [ ] If extractions were made, `metadata.detected_settings` records each inferred value with `confirmed` status +- [ ] If `bigquery_present` was **true**, the Step 4 BigQuery specialist advisory was shown before questions — **or**, if Step 0 option A (reuse preferences), the same advisory was shown after BigQuery detection - [ ] `preferences.json` written to `$MIGRATION_DIR/` - [ ] `design_constraints.target_region` is populated with `value` and `chosen_by` -- [ ] `design_constraints.availability` is populated (if Q6 was asked or defaulted) +- [ ] `design_constraints.availability` is populated when Cloud SQL PostgreSQL/MySQL is in inventory (asked, extracted, or defaulted — Design must not run with absent/null availability) - [ ] Only keys with non-null values are present in `design_constraints` - [ ] Every entry in `design_constraints` and `ai_constraints` has `value` and `chosen_by` fields - [ ] Config gap answers recorded in `metadata.inventory_clarifications` (billing mode only) @@ -415,17 +713,31 @@ Before handing off to Design: - [ ] If Category F fired, `ai_constraints.ai_framework` is populated (from detection or Q14) - [ ] If Category F fired, `ai_capabilities_required` is derived from detection + Q17 + Q20 - [ ] `ai_constraints.ai_framework` is an array (Q14 is multi-select) -- [ ] `question_details` includes an entry for every question in `questions_asked`, `questions_defaulted`, and `questions_skipped_early_exit` - [ ] Output is valid JSON +- [ ] `preferences-draft.json` has been deleted (if it existed) +- [ ] `metadata.clarify_mode` is set to `"fast_path"`, `"simple_hybrid"`, or `"full"` --- -## Step 6: Update Phase Status +## Completion Handoff Gate (Fail Closed) -Update `$MIGRATION_DIR/.phase-status.json`: +Load `steering/handoff-gates.md`. **Re-read from disk** before checking. + +**Re-entry guard:** If `aws-design.json` (or `aws-design-ai.json` / `aws-design-billing.json`) exists and `phases.design` is `"completed"`: STOP unless the user explicitly confirms re-running Clarify. Emit `GATE_FAIL | phase=clarify | field=aws-design.json | reason=stale_downstream`. + +**Checks (all must PASS):** + +1. `preferences.json` exists and parses as JSON. +2. Step 5 validation checklist items all pass (including `metadata.clarify_mode`). +3. If `gcp-resource-inventory.json` contains `google_sql_database_instance` → `design_constraints.availability.value` is set (non-null, non-empty). + +**On any FAIL:** Emit `GATE_FAIL | phase=clarify | field= | reason=missing`. **Do NOT modify artifacts to pass the gate.** **Do NOT update `.phase-status.json`.** Tell the user to answer the missing question or re-run Clarify. + +**On PASS:** Emit `HANDOFF_OK | phase=clarify | artifacts=preferences.json`. + +## Step 6: Update Phase Status -- Set `phases.clarify` to `"completed"` -- Update `last_updated` to current timestamp +Only after `HANDOFF_OK`. In the **same turn** as the output message below, use the Phase Status Update Protocol (Write tool) to write `.phase-status.json` with `phases.clarify` set to `"completed"`. Output to user: "Clarification complete. Proceeding to Phase 3: Design AWS Architecture." diff --git a/migration-to-aws/steering/clustering-classification-rules.md b/migration-to-aws/steering/clustering-classification-rules.md index 1bd311b3..f03ddace 100644 --- a/migration-to-aws/steering/clustering-classification-rules.md +++ b/migration-to-aws/steering/clustering-classification-rules.md @@ -4,6 +4,19 @@ Hardcoded lists for classifying GCP resources as PRIMARY or SECONDARY. Each PRIMARY resource is assigned a `tier` indicating its infrastructure layer. +## Priority 0: Excluded Resources (Skip Entirely) + +These resource types are **excluded from classification, clustering, and migration**. Do not classify them as PRIMARY or SECONDARY. Do not create clusters for them. Do not include them in `gcp-resource-inventory.json`. + +### Authentication Providers + +Third-party and GCP-adjacent authentication resources. Users should keep their existing auth provider — do not recommend migrating to AWS Cognito or any AWS auth service. + +- `google_identity_platform_*` — GCP Identity Platform (all variants: config, tenant, default_supported_idp_config, inbound_saml_config, oauth_idp_config) +- `google_firebase_auth_*` — Firebase Authentication (all variants) + +If encountered: log as "Auth provider detected — excluded from migration scope. Keep your existing auth solution." and skip. + ## Priority 1: PRIMARY Resources (Workload-Bearing) These resource types are always PRIMARY: diff --git a/migration-to-aws/steering/design-ai.md b/migration-to-aws/steering/design-ai.md index 03e265ee..9338c808 100644 --- a/migration-to-aws/steering/design-ai.md +++ b/migration-to-aws/steering/design-ai.md @@ -10,7 +10,7 @@ Read `$MIGRATION_DIR/ai-workload-profile.json`: -- `summary.ai_source` — `"gemini"`, `"openai"`, `"both"`, `"other"` +- `summary.ai_source` — `"gemini"`, `"openai"`, `"anthropic"`, `"both"`, `"other"` - `models[]` — Detected AI models with service, capabilities, evidence - `integration` — SDK, frameworks, languages, gateway type, capability summary - `infrastructure[]` — Terraform resources related to AI (may be empty) @@ -18,19 +18,117 @@ Read `$MIGRATION_DIR/ai-workload-profile.json`: Read `$MIGRATION_DIR/preferences.json` → `ai_constraints` (if present). If absent: use defaults (prefer managed Bedrock, no latency constraint, no budget cap). +**Region selection for AI workloads:** If `design_constraints.target_region` was derived from GCP region proximity (not explicitly chosen by the user), verify the selected Bedrock models are available in that region. Use the AWS Documentation MCP server to check model availability. If the target region lacks the selected model, prefer the geographically closest AWS region where it is available. + **Load source-specific design reference based on `ai_source`:** - `"gemini"` → load `steering/design-ref-ai-gemini-to-bedrock.md` - `"openai"` → load `steering/design-ref-ai-openai-to-bedrock.md` -- `"both"` → load both files -- `"other"` or absent → load `steering/design-ref-ai.md` (traditional ML rubric) +- `"anthropic"` → load `steering/design-ref-ai-anthropic-to-bedrock.md` (Anthropic SDK → Bedrock Converse API client swap; do NOT use design-ref-ai-openai-to-bedrock.md for Anthropic SDK users) +- `"both"` → load both `design-ref-ai-gemini-to-bedrock.md` and `design-ref-ai-openai-to-bedrock.md` +- `"other"` or absent → load `steering/design-ref-ai.md` (traditional ML rubric — Vision API, Speech API, Document AI, custom models only; do NOT use for Anthropic SDK users) + +--- + +## Step 0.5: Regional Availability Validation + +Read target region from `preferences.json` → `design_constraints.target_region` (default: `us-east-1`). + +Call `get_regional_availability` from the `awsknowledge` MCP server for: + +1. Each Bedrock model ID being considered (from the loaded model mapping tables) +2. If `agentic_profile.is_agentic == true`: check `bedrock-agentcore` (Runtime) +3. If `agentic_profile.is_agentic == true` AND `ai_constraints.agentic.migration_approach == "harness"`: check `bedrock-agentcore` harness capability + +**If any recommended service is unavailable in target region:** + +- Add to `regional_warnings[]` in output: `{"service": "...", "target_region": "...", "nearest_available": "...", "impact": "..."}` +- Note in user summary with alternative region suggestion +- Do NOT block the design — proceed with the recommendation and flag the constraint + +**If MCP call fails after 3 attempts:** Use the static table in `steering/ai-migration-guardrails.md` as fallback. Add `"regional_validation": "fallback_static"` to output metadata. + +--- + +## Step 0.6: Agentic Design Routing + +**Skip this step if `agentic_profile` is absent from `ai-workload-profile.json`.** + +If `agentic_profile.is_agentic == true`: + +1. Load `steering/ai-migration-guardrails.md` (shared warnings — load once, do not reload in sub-files) +2. Read `preferences.json` → `ai_constraints.agentic.migration_approach` +3. Route based on approach: + +| `migration_approach` | Action | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `"retarget"` | Continue with standard model-swap design below (Parts 1–6). The existing framework stays; only the model layer changes. Load `steering/retarget-gotchas.md` for framework-specific migration pitfalls to include in the code migration plan (Part 5). | +| `"harness"` | Load `steering/design-ref-harness.md`. If file does not exist: continue with standard model-swap design, add note to user summary: "AgentCore Harness design reference not yet available. Proceeding with model-layer migration only. For Harness guidance, see https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/harness.html" | +| `"strands"` | Load `steering/design-ref-agentic-to-agentcore.md`. | +| `"undecided"` | Treat as `"retarget"` (safest default). Note in user summary: "No migration approach selected — defaulting to retarget (keep framework, swap model layer). Re-run Clarify to select a different approach." | + +**Regardless of approach:** Continue with Parts 1–6 below for model selection and mapping. The agentic design ref (Harness/Strands) adds agent infrastructure on top of the model-layer design — it does not replace it. --- ## Part 1: Bedrock Model Selection +**Multi-workload iteration (when `workloads[]` is present):** + +If `preferences.json` contains a non-empty `workloads[]` array (written by Clarify after user confirmation), iterate per workload instead of per model. **Design MUST read workloads from `preferences.json` (not `ai-workload-profile.json`)** because Clarify may have edited, dropped, or re-confirmed rows. + +Fallback: if `preferences.json` has no `workloads[]` field but `ai-workload-profile.json` does, use the profile's `workloads[]` directly (no user edits were made). + +For each `workloads[]` entry: + +1. **Use the workload's `capability` to select the Bedrock target class:** + + | Capability | Target Class | Default Model | + | ------------------- | ------------------------------------------------------ | ------------------------------ | + | `text_generation` | Text/reasoning | Apply override hierarchy below | + | `structured_output` | Text/reasoning (same models support structured output) | Apply override hierarchy below | + | `image_generation` | Image generation | Amazon Nova Canvas | + | `embedding` | Embedding | Amazon Titan Embed Text v2 | + | `speech_to_text` | Speech-to-text | Amazon Transcribe | + | `text_to_speech` | Text-to-speech | Amazon Polly | + | `unknown` | Text/reasoning (default) | Apply override hierarchy below | + +2. **For text/reasoning capabilities:** Apply the existing override hierarchy from `ai_constraints`: + - Q17 special features (hard override) > Q16 priority > Q18/Q21 volume and latency > source model baseline + - This ensures single-workload sophistication is preserved per workload + +3. **Emit one `design_block` per workload** in `aws-design-ai.json`: + + ```json + "design_blocks": [ + { + "workload_id": "wl_3a1f2c", + "model_id": "gemini-2.5-flash", + "target_bedrock_model": "amazon.nova-lite-v1:0", + "capability": "text_generation", + "capability_confidence": "medium", + "rationale": "text_generation + medium confidence + balanced priority → Nova Lite", + "confidence_warning": null + } + ] + ``` + +4. **Confidence warning:** Set `confidence_warning` to a non-null string (identifying the workload and noting manual review required) when `capability_confidence == "low"`. Null for `high` and `medium`. + +5. **Preserve input order:** `design_blocks[]` order matches `workloads[]` order. + +6. **Empty workloads:** If `workloads[]` is empty, emit `aws-design-ai.json` with `"design_blocks": []` and proceed with the existing `models[]` path below as fallback. + +**Fallback (no `workloads[]` or single entry):** If `workloads[]` is absent or has exactly one entry, fall through to the existing per-model logic below (backward compatible). + +--- + For each model in `models[]`, select the best-fit Bedrock model using the loaded design reference mapping tables. Do NOT use a hardcoded mapping — the design-ref files contain tier-organized tables with pricing and competitive analysis. +Treat model mapping as compatibility-guided, not 1:1 parity. Before cutover, require validation of prompts, tool-calling behavior, and eval metrics for the selected Bedrock model. + +**If `models[]` is empty:** Skip per-model rows; output a short **placeholder strategy** (one representative Bedrock model family per `ai_source` rubric) and dependency on Clarify answers — do not fabricate `models[]` entries. + **Apply user preference overrides from `ai_constraints`:** | Preference | Override | @@ -51,6 +149,19 @@ Overall assessment = weakest across all models. If any `"recommend_stay"`, flag **Model comparison table** (include in output and user summary): Model, Provider, Max Context, Input/Output Price per 1M, Price Comparison, Streaming, Function Calling, Assessment. +**Quota risk assessment** (per `steering/bedrock-quotas.md`): + +After selecting models, assess quota risk based on `ai_token_volume` from `preferences.json`: + +| `ai_token_volume` | Selected Model Family | `quota_risk` | Action | +| ------------------------- | ---------------------------------- | ------------ | --------------------------------------------------------------------------------- | +| `"high"` or `"very_high"` | Any | `"high"` | Flag: "Request Bedrock quota increase before migration (allow 1–5 business days)" | +| `"medium"` | Claude (5× burndown) | `"medium"` | Flag: "Monitor TPM usage; quota increase may be needed at peak" | +| `"medium"` | Nova / Llama / other (1× burndown) | `"low"` | No action | +| `"low"` | Any | `"low"` | No action | + +Include `quota_risk` in `aws-design-ai.json` → `ai_architecture` alongside `honest_assessment`. + --- ## Part 1B: Volume-Based Strategy @@ -65,6 +176,44 @@ If `ai_token_volume` is `"high"`, generate a `tiered_strategy`: Set `tiered_strategy: null` for low/medium volume. +**Intelligent Prompt Routing — automated alternative to manual tiering:** +If `ai_token_volume` is `"high"` AND the selected models are within the same family +(e.g., Claude Haiku + Claude Sonnet, or Nova Lite + Nova Pro), note Bedrock Intelligent +Prompt Routing as an option. It automatically routes each request to the cheapest model +that can handle it at adequate quality — the AWS-native automation of the tiered strategy above. + +> Intelligent Prompt Routing only routes within a single model family. It does NOT replace +> cross-provider routing (e.g., Claude ↔ GPT-4o). If the startup was using OpenRouter or +> LiteLLM to route across providers, they still need app-level routing for cross-family calls. +> One-line caveat: adds a routing-prediction latency hop; verify model support at +> docs.aws.amazon.com/bedrock/latest/userguide/prompt-routing.html before recommending. + +--- + +## Part 1C: Multi-Model Coordination Warnings + +If `models[]` contains more than one model, check for coordination patterns and generate warnings. These help the user understand that migrating multiple models requires coordinated testing, not independent swaps. + +**Check and warn:** + +1. **Embeddings + generation model detected** — If `models[]` contains both an embeddings model (capabilities_used includes `"embeddings"`) AND a text generation model: + > ⚠️ "Migrating the embedding model (e.g., text-embedding-3-small → Titan Embeddings v2) requires re-embedding all documents in your vector store. Plan for re-indexing time and temporary storage. Test retrieval quality with the new embeddings before switching generation model." + > ⚠️ "Verify vector index dimension compatibility before cutover. OpenAI text-embedding-3-small outputs 1536 dimensions; Titan Embeddings v2 is configurable (256, 512, or 1024). A mismatch will cause insert failures at the vector store layer — your index must be rebuilt at the target dimension before any data is written." + +2. **Models at different price tiers** — If `models[]` contains both a mini/nano/lite model AND a flagship model (infer from model_id naming: `*-mini`, `*-nano`, `*-lite` vs flagship): + > ⚠️ "These models appear to work as a cascade or routing pattern (cheap model for classification/filtering, expensive model for generation). Test the Bedrock replacement pair together — validate that the cheaper model's classification accuracy is preserved with its Bedrock equivalent before testing the expensive model." + +3. **More than 3 models** — If `models[]` count > 3: + > ⚠️ "Multiple models detected ([count]). Recommend a tiered migration strategy: migrate and validate one model at a time, starting with the lowest-risk (highest-volume, simplest task). See Part 1B for tiered routing recommendations." + +4. **Text generation + image generation** — If `models[]` contains both text generation AND image generation capabilities: + > ⚠️ "Image generation migration (e.g., DALL-E/gpt-image → Nova Canvas) requires separate evaluation. Image quality is subjective — plan for human evaluation alongside automated metrics." + +5. **Speech models** — If `models[]` contains speech-to-text or text-to-speech capabilities: + > ⚠️ "Speech model migration targets different AWS services (Whisper → Amazon Transcribe, TTS → Amazon Polly or Nova Sonic) with different pricing models and APIs. These are not Bedrock model swaps — they require separate integration work." + +Record all triggered warnings in `aws-design-ai.json` → `multi_model_warnings[]`. Each warning: `{"type": "embeddings_reindex|cascade_pair|multi_model_tiered|image_separate|speech_separate", "message": "..."}`. + --- ## Part 2: Feature Parity Validation @@ -94,12 +243,13 @@ For each model in `models[]`, record: - **Workload type**: text generation, embeddings, vision, code generation, custom model - **Integration pattern mapping**: -| GCP Pattern | AWS Pattern | Effort | -| ------------ | ------------------------------ | ------ | -| `direct_sdk` | Bedrock SDK (boto3 / AWS SDK) | Medium | -| `framework` | LangChain/LlamaIndex + Bedrock | Low | -| `rest_api` | Bedrock REST API | Medium | -| `mixed` | Match per-model | Varies | +| GCP Pattern | AWS Pattern | Effort | +| ------------ | ------------------------------------------------ | ------- | +| `direct_sdk` | Mantle OpenAI-compat (if OpenAI source + region) | Minimal | +| `direct_sdk` | Bedrock SDK (boto3 / AWS SDK) | Medium | +| `framework` | LangChain/LlamaIndex + Bedrock | Low | +| `rest_api` | Bedrock REST API | Medium | +| `mixed` | Match per-model | Varies | - **Migration complexity**: Low / Medium / High @@ -127,18 +277,44 @@ For each detected `integration.pattern` and `ai_source`, generate before/after m **Patterns to include (matched to detected language and source):** -| Pattern | Source | Target | Key Change | -| -------------------- | ------------------------- | ------------------- | ------------------------------------- | -| Direct SDK | Vertex AI | boto3 Converse API | `generate_content()` → `converse()` | -| Direct SDK | OpenAI | boto3 Converse API | `completions.create()` → `converse()` | -| LangChain | ChatVertexAI / ChatOpenAI | ChatBedrock | Swap import and model_id | -| LlamaIndex | Vertex / OpenAI LLM | BedrockConverse | Swap import | -| LLM Router (LiteLLM) | Any | Config change | `model="bedrock/"` (1 line) | -| Embeddings | TextEmbeddingModel | Titan Embeddings v2 | `invoke_model` with JSON body | -| Streaming | `stream=True` | `converse_stream` | Event loop over `contentBlockDelta` | +| Pattern | Source | Target | Key Change | +| -------------------- | ------------------------- | ---------------------- | ---------------------------------------------------------------------------------------------------- | +| Direct SDK (OpenAI) | OpenAI | Mantle (OpenAI-compat) | Change `OPENAI_BASE_URL` + `OPENAI_API_KEY` + model string (zero code changes) | +| Direct SDK | Vertex AI | boto3 Converse API | `generate_content()` → `converse()` | +| Direct SDK | OpenAI | boto3 Converse API | `completions.create()` → `converse()` (use if Mantle region unavailable or Converse features needed) | +| Direct SDK | Anthropic | boto3 Converse API | `messages.create()` → `converse()` with Claude model ID on Bedrock | +| LangChain | ChatVertexAI / ChatOpenAI | ChatBedrock | Swap import and model_id | +| LlamaIndex | Vertex / OpenAI LLM | BedrockConverse | Swap import | +| LLM Router (LiteLLM) | Any | Config change | `model="bedrock/"` (1 line) | +| Embeddings | TextEmbeddingModel | Titan Embeddings v2 | `invoke_model` with JSON body | +| Streaming | `stream=True` | `converse_stream` | Event loop over `contentBlockDelta` | + +**Mantle (OpenAI-compatible endpoints):** If `ai_source = "openai"` and `integration.pattern = "direct_sdk"`, prefer the Mantle path as the primary migration option. Mantle provides OpenAI-compatible Chat Completions and Responses APIs on Bedrock — the existing OpenAI SDK code works with zero changes, only environment variable updates. Check [Mantle regional availability](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) — if the target region does not have Mantle, fall back to the boto3 Converse API path. Record `migration_path: "mantle"` or `migration_path: "converse"` in `aws-design-ai.json` → `ai_architecture.code_migration`. + +**Mantle throughput caveat (medium/high volume):** Mantle runs on a shared 10,000 RPM account limit. For workloads with `ai_token_volume = "medium"` or `"high"`, add a note in the design summary: "Mantle is subject to a shared 10K RPM account limit. At medium/high volume, monitor for 429s and consider migrating to `bedrock-runtime` (Converse API) for dedicated throughput." See `steering/ai-migration-guardrails.md` for the full risk table. + +**gpt-oss migration path:** If `ai_source = "openai"` and the user wants to preserve OpenAI model architecture while consolidating on AWS, offer `gpt-oss` on Bedrock as a fourth migration path alongside Mantle, Converse API, and framework swap. Record `migration_path: "gpt-oss"` in `aws-design-ai.json` → `ai_architecture.code_migration`. The gpt-oss path uses the Converse API with the gpt-oss Bedrock model ID — it is not an OpenAI-compatible endpoint. Note the Claude 4.7+ output TPM cap (2M) if the user is migrating from a high-output OpenAI workload. Generate concrete code examples using actual model IDs from the selected Bedrock models. Only include patterns matching the detected integration. +**OpenRouter-specific guidance** (if `gateway_type == "llm_router"` AND `detection_signals` contains OpenRouter evidence): + +OpenRouter is a hosted routing service (not self-hosted like LiteLLM). It adds a margin on top of provider pricing. Present three options to the user: + +> **If the startup was using OpenRouter primarily for cost-based routing within one model family** +> (e.g., routing between Claude Haiku and Claude Sonnet, or Nova Lite and Nova Pro), +> Bedrock Intelligent Prompt Routing is the AWS-native replacement — no routing infrastructure +> needed. If they routed across providers (e.g., Claude ↔ GPT-4o), they still need +> app-level or LiteLLM routing after migration. + +| Option | Action | Effort | Trade-off | +| ------------------------------- | --------------------------------------------------------- | --------- | -------------------------------------------------------------------------- | +| A) Direct Bedrock (recommended) | Remove OpenRouter, call Bedrock API directly | 1–2 weeks | Removes middleman + margin; requires SDK changes | +| B) Self-hosted LiteLLM | Replace OpenRouter with LiteLLM proxy pointing to Bedrock | 1–3 days | Preserves router pattern; removes OpenRouter dependency; adds self-hosting | +| C) Keep OpenRouter | Use OpenRouter with `amazon/` prefixed Bedrock models | Hours | Lowest effort; retains OpenRouter dependency and margin | + +Record user's choice (or recommend A if not asked) in `aws-design-ai.json` → `code_migration.openrouter_path`: `"direct"` / `"litellm"` / `"keep_openrouter"`. + --- ## Part 6: Generate Output @@ -157,6 +333,9 @@ Write `aws-design-ai.json` to `$MIGRATION_DIR/`. | `ai_architecture.code_migration` | object | `primary_pattern`, `framework`, `files_to_modify[]`, `dependency_changes` | | `ai_architecture.infrastructure` | array | GCP resource → AWS equivalent mappings with confidence | | `ai_architecture.services_to_migrate` | array | GCP service → AWS service with effort and notes | +| `regional_warnings` | array | Per-service: `service`, `target_region`, `nearest_available`, `impact` (empty array if all services available) | +| `multi_model_warnings` | array | Per-warning: `type`, `message` (empty array if single model or no coordination issues) | +| `agentic_design` | object/null | Present only when `agentic_profile.is_agentic == true`. Contains `migration_approach`, path-specific config (e.g., `harness_config`). Null or absent for non-agentic workloads. | ## Validation Checklist @@ -165,8 +344,21 @@ Write `aws-design-ai.json` to `$MIGRATION_DIR/`. - [ ] Every `bedrock_models[]` entry has pricing (`source_provider_price`, `bedrock_price`, `price_comparison`) - [ ] `capability_mapping` covers every `true` capability from `capabilities_summary` - [ ] `code_migration.primary_pattern` matches `integration.pattern` -- [ ] All model IDs use current Bedrock identifiers +- [ ] All model IDs use current Bedrock identifiers (Active status per `steering/ai-model-lifecycle.md`) +- [ ] No Legacy model is used as `bedrock_models[].aws_model_id` unless no Active alternative exists (with EOL date noted) - [ ] `honest_assessment` logic is consistent (weakest model drives overall) +- [ ] `regional_warnings` is present (empty array `[]` if no issues; populated if any service unavailable in target region) +- [ ] `multi_model_warnings` is present (empty array `[]` if single model or no coordination issues) +- [ ] If `agentic_profile.is_agentic == true`: `agentic_design` object is present with `migration_approach` matching `preferences.json` +- [ ] If `agentic_profile.is_agentic == false` or absent: `agentic_design` is null or absent + +## Completion Handoff Gate (Fail Closed) + +Before returning control to `design.md`, require: + +- `aws-design-ai.json` exists and passes the Validation Checklist above. + +If this gate fails: STOP and output: "design-ai did not produce a valid `aws-design-ai.json`; do not complete Phase 3." ## Present Summary diff --git a/migration-to-aws/steering/design-billing.md b/migration-to-aws/steering/design-billing.md index a74f6cec..9f53f020 100644 --- a/migration-to-aws/steering/design-billing.md +++ b/migration-to-aws/steering/design-billing.md @@ -41,8 +41,8 @@ For each billing service, attempt lookup in order: 1. Look up `gcp_service_type` in `steering/design-ref-fast-path.md` → Direct Mappings table 2. If found: assign AWS service 3. Enrich with SKU hints: - - If `top_skus` mention "PostgreSQL" → specify "RDS Aurora PostgreSQL" - - If `top_skus` mention "MySQL" → specify "RDS Aurora MySQL" + - If `top_skus` mention "PostgreSQL" → **RDS PostgreSQL** or **Aurora PostgreSQL** per `design_constraints.availability` in `preferences.json` (same Q6 gate as `design-infra.md` step 7b). Default to **RDS PostgreSQL single-AZ** when availability unknown (billing-only path has no HA signal — do not assume Multi-AZ). + - If `top_skus` mention "MySQL" → **RDS MySQL** or **Aurora MySQL** per Q6 gate. Default to **RDS MySQL single-AZ** when availability unknown. - If `top_skus` mention "CPU Allocation" → indicates compute (Fargate) - If `top_skus` mention "Storage" → check if object storage (S3) or block storage (EBS) @@ -50,22 +50,26 @@ For each billing service, attempt lookup in order: Look up `gcp_service_type` in the table below. These are default mappings for common GCP services when no configuration data is available. The IaC path uses the full rubric in category files and may select a different AWS target based on actual configuration. -| `gcp_service_type` | Billing Name | Default AWS Target | Alternatives (chosen by IaC path) | -| -------------------------------- | -------------------- | ------------------ | --------------------------------- | -| `google_cloud_run_service` | Cloud Run | Fargate | Lambda, EC2 | -| `google_cloudfunctions_function` | Cloud Functions | Lambda | Fargate | -| `google_compute_instance` | Compute Engine | EC2 | Fargate, ASG | -| `google_container_cluster` | GKE | EKS | ECS, Fargate | -| `google_app_engine_application` | App Engine | Fargate | Amplify, Lambda | -| `google_firestore_database` | Firestore | DynamoDB | — | -| `google_bigquery_dataset` | BigQuery | Athena | Redshift | -| `google_compute_forwarding_rule` | Cloud Load Balancing | ALB | NLB | -| `google_compute_backend_service` | Cloud Load Balancing | ALB Target Groups | NLB | -| `google_pubsub_topic` | Pub/Sub | SNS | SQS, SNS FIFO | -| `google_pubsub_subscription` | Pub/Sub | SQS | SNS Subscription | -| `google_cloud_tasks_queue` | Cloud Tasks | SQS | EventBridge | - -If found: assign the Default AWS Target. Set rationale to: "Billing heuristic: [GCP service] → [AWS service]. Provide Terraform files for configuration-aware mapping." +| `gcp_service_type` | Billing Name | Default AWS Target | Alternatives (chosen by IaC path) | +| -------------------------------- | -------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `google_cloud_run_service` | Cloud Run | Fargate | Lambda, EC2 | +| `google_cloudfunctions_function` | Cloud Functions | Lambda | Fargate | +| `google_compute_instance` | Compute Engine | EC2 | Fargate, ASG | +| `google_container_cluster` | GKE | EKS | ECS, Fargate | +| `google_app_engine_application` | App Engine | Fargate | Amplify, Lambda | +| `google_firestore_database` | Firestore | DynamoDB | — | +| `google_bigquery_dataset` | BigQuery | **`Deferred — specialist engagement`** | **No** Athena/Redshift/Glue in automated output. **`human_expertise_required: true`**. User must engage **AWS account team** and/or **data analytics migration partner**. | +| `google_compute_forwarding_rule` | Cloud Load Balancing | ALB | NLB | +| `google_compute_backend_service` | Cloud Load Balancing | ALB Target Groups | NLB | +| `google_pubsub_topic` | Pub/Sub | SNS | SQS, SNS FIFO | +| `google_pubsub_subscription` | Pub/Sub | SQS | SNS Subscription | +| `google_cloud_tasks_queue` | Cloud Tasks | SQS | EventBridge | + +If found: assign the Default AWS Target. Set rationale to: "Billing heuristic: [GCP service] → [AWS service]. Provide Terraform files for configuration-aware mapping." **Exception:** For BigQuery, use: "Billing indicates BigQuery spend — **no automated AWS analytics target**; engage AWS account team / data analytics migration partner (`Deferred — specialist engagement`)." + +**Set `human_expertise_required`**: If `gcp_service_type` is `google_bigquery_dataset` (or billing rows clearly represent BigQuery analytics), set `human_expertise_required: true` and `aws_service` to **`Deferred — specialist engagement`** (same rules as `design-infra.md` BigQuery gate). For all other services, set `human_expertise_required: false`. This field is REQUIRED on every service in the output. + +**Preferred AWS target check**: **Skip** when `aws_service` is **`Deferred — specialist engagement`**. Otherwise verify the assigned `aws_service` aligns with the Preferred AWS Target Services table in `steering/design-ref-fast-path.md`. If a non-preferred service is selected (e.g., App Runner for containerized workloads), substitute the preferred alternative (e.g., Fargate). Add a note to the rationale: "Preferred target: [alternative] selected for stronger ecosystem integration." **2c. If not found in either table:** proceed to Step 3. @@ -124,21 +128,23 @@ Write to `$MIGRATION_DIR/aws-design-billing.json`: }, "monthly_cost": 450.00, "confidence": "billing_inferred", + "human_expertise_required": false, "rationale": "Fast-path: Cloud Run → Fargate. SKU hints: CPU + Memory allocation.", "sku_hints": ["CPU Allocation Time", "Memory Allocation Time"] }, { "gcp_service": "Cloud SQL", "gcp_service_type": "google_sql_database_instance", - "aws_service": "RDS Aurora PostgreSQL", + "aws_service": "RDS PostgreSQL", "aws_config": { "region": "us-east-1", "high_availability": false, + "multi_az": false, "inventory_clarifications_applied": true }, "monthly_cost": 800.00, "confidence": "billing_inferred", - "rationale": "Fast-path: Cloud SQL → RDS Aurora. SKU hints: PostgreSQL engine. User confirmed single-zone (Category B).", + "rationale": "Cloud SQL PostgreSQL → RDS PostgreSQL (single-AZ). SKU hints: PostgreSQL engine. User confirmed single-zone (Category B).", "sku_hints": ["DB custom CORE", "DB custom RAM"] } ], @@ -160,18 +166,29 @@ Write to `$MIGRATION_DIR/aws-design-billing.json`: - `metadata.total_services` equals `mapped_services` + `unmapped_services` - Every service from `billing-profile.json` appears in either `services[]` or `unknowns[]` - All `confidence` values are `"billing_inferred"` +- Every `services[]` entry has `human_expertise_required` (boolean) — `true` for BigQuery; `false` for all others +- BigQuery entries must have `aws_service` exactly **`Deferred — specialist engagement`** (not Athena/Redshift/Glue) - Every `services[]` entry has `gcp_service`, `aws_service`, `monthly_cost`, `rationale` - Every `unknowns[]` entry has `gcp_service`, `monthly_cost`, `reason`, `suggestion` - Output is valid JSON +## Completion Handoff Gate (Fail Closed) + +Before returning control to `design.md`, require: + +- `aws-design-billing.json` exists and passes the Output Validation Checklist above. + +If this gate fails: STOP and output: "design-billing did not produce a valid `aws-design-billing.json`; do not complete Phase 3." + ## Present Summary After writing `aws-design-billing.json`, present a concise summary to the user: 1. Mapped X of Y GCP billing services to AWS equivalents -2. Accuracy notice: billing-inferred confidence, provide .tf files for higher accuracy -3. Per-service table: GCP service → AWS service (with monthly GCP cost) +2. Accuracy notice: every mapping here is **Estimated from billing only** (JSON: `billing_inferred`) — suggest providing Terraform for a tighter mapping +3. Per-service table: GCP service → AWS service (with monthly GCP cost); label recommendation type as **Estimated from billing only** unless you also have IaC-backed design 4. Unmapped services list with suggestions 5. Total monthly GCP spend +6. If any service has **`Deferred — specialist engagement`**: state **prominently** that **no AWS analytics target was chosen**; direct the user to **AWS account team** and/or **data analytics migration partner**. Do **not** recommend Athena, Redshift, or Glue in the summary. Keep it under 20 lines. The user can ask for details or re-read `aws-design-billing.json` at any time. diff --git a/migration-to-aws/steering/design-infra.md b/migration-to-aws/steering/design-infra.md index 93dbc596..7173df0d 100644 --- a/migration-to-aws/steering/design-infra.md +++ b/migration-to-aws/steering/design-infra.md @@ -18,20 +18,29 @@ Sort clusters by `creation_order_depth` (lowest first, representing foundational For each cluster, process `primary_resources` first, then `secondary_resources` (as classified during discover phase — see `gcp-resource-clusters.json`). -### Pass 1: Fast-Path Lookup +### Pass 1: Fast-Path Lookup (Direct Mappings table only) For each PRIMARY resource in the cluster: 1. Extract GCP type (e.g., `google_sql_database_instance`) -2. Look up in `steering/design-ref-fast-path.md` → Direct Mappings table -3. If found (deterministic 1:1 match): assign AWS service with confidence = `deterministic` -4. If not found: proceed to Pass 2 +2. Look up in `steering/design-ref-fast-path.md` → **Direct Mappings** table (not the Preferred Target table — that applies later in Pass 2). +3. If found and conditions match: assign AWS service with confidence = **`deterministic`**. Set `human_expertise_required: false` (no Direct Mapping row requires it). +4. If `gcp_type` is `google_sql_database_instance` with PostgreSQL or MySQL engine: **always proceed to Pass 2** (Cloud SQL is not in Direct Mappings — see `design-ref-fast-path.md`). Confidence = **`inferred`** after rubric. +5. If not found: proceed to Pass 2 (confidence will be **`inferred`** after rubric, or **`billing_inferred`** on the billing-only path). + +**Definitions:** See the top of `steering/design-ref-fast-path.md` for **`deterministic` vs `inferred` vs `billing_inferred`** and the note that **design-ref-index.md “Typical AWS target” ≠ deterministic**. ### Pass 2: Rubric-Based Selection For resources not covered by fast-path: -1. Determine service category (via `steering/design-ref-index.md`): +**0. BigQuery specialist gate (mandatory — before rubric):** If `gcp_type` **starts with** `google_bigquery_` (e.g. `google_bigquery_dataset`, `google_bigquery_table`, `google_bigquery_routine`, `google_bigquery_data_transfer_config`, `google_bigquery_job`, `google_bigquery_ml_*`): + +1. **Do not** recommend a specific AWS analytics or warehouse service (Athena, Redshift, Glue, EMR, Lake Formation, or a prescribed “data lake on S3” architecture). +2. Set `aws_service` to **`Deferred — specialist engagement`**, `human_expertise_required` to **`true`**, `confidence` to **`inferred`**, and `aws_config` to include `specialist_engagement` (text: engage **AWS account team** and/or **data analytics migration partner** before choosing any AWS target) and `no_automated_aws_target`: `true`. Set `rubric_applied` to `["BigQuery specialist gate — no automated AWS service target"]`. +3. **Skip** rubric steps 1–6 and the Preferred AWS target check for this resource. + +4. Determine service category (via `steering/design-ref-index.md`): - `google_compute_instance` → compute - `google_cloudfunctions_function` → compute - `google_sql_database_instance` → database @@ -42,11 +51,11 @@ For resources not covered by fast-path: **Catch-all for unknown types**: If resource type not found in `design-ref-index.md`: - Check resource name pattern (e.g., "scheduler" → orchestration, "log" → monitoring, "metric" → monitoring) - If pattern match: use that category - - If no pattern match: **STOP**. Output: "Unknown GCP resource type: [type]. Not in fast-path.md or index.md. Cannot auto-map. Please file an issue with this resource type." + - If no pattern match: **STOP**. Output: "Unknown GCP resource type: [type]. Not in design-ref-fast-path.md or design-ref-index.md. Cannot auto-map. Please file an issue with this resource type." -2. Load rubric from corresponding `steering/design-ref-*.md` file (e.g., `design-ref-compute.md`, `design-ref-database.md`) +5. Load rubric from corresponding `steering/design-ref-*.md` file (e.g., `design-ref-compute.md`, `design-ref-database.md`) -3. Evaluate 6 criteria (1-sentence each): +6. Evaluate 6 criteria (1-sentence each): - **Eliminators**: Feature incompatibility (hard blocker) - **Operational Model**: Managed vs self-hosted fit - **User Preference**: From `preferences.json` design_constraints @@ -54,7 +63,25 @@ For resources not covered by fast-path: - **Cluster Context**: Affinity with other resources in this cluster - **Simplicity**: Prefer fewer resources / less config -4. Select best-fit AWS service. Confidence = `inferred` +7. Select best-fit AWS service. Confidence = `inferred` + +7b. **Cloud SQL Q6 gate (mandatory — after rubric):** For `google_sql_database_instance` (PostgreSQL or MySQL), read `preferences.json` → `design_constraints.availability` and **enforce**: + +| `availability` | Required `aws_service` | +| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `single-az` | `RDS PostgreSQL` or `RDS MySQL` (engine match) | +| `multi-az` | `RDS PostgreSQL` or `RDS MySQL` + `multi_az: true` | +| `multi-az-ha` | `Aurora PostgreSQL` or `Aurora MySQL` | +| `multi-region` | `Aurora PostgreSQL` or `Aurora MySQL` Global Database | +| absent / null / missing | **Do not proceed** — Cloud SQL PostgreSQL/MySQL is present but Q6 was not answered. Return to Clarify to ask Q6 (or apply the documented Q6 default) before assigning RDS/Aurora topology. Do not infer Aurora from the rubric alone. | + +**IaC extraction note:** Only `single-az` and `multi-az` can be auto-extracted from Terraform (`ZONAL` / `REGIONAL`). **`multi-az-ha` and `multi-region` are never inferred from IaC** — they require explicit user intent via Q6 (Mission-Critical / Catastrophic). Cloud SQL `REGIONAL` maps to `multi-az` (RDS Multi-AZ), not `multi-az-ha` (Aurora). + +If rubric or fast-path would select Aurora when `availability` is `single-az` or `multi-az`, **replace with RDS**. If rubric would select RDS when `availability` is `multi-az-ha` or `multi-region`, **replace with Aurora**. Add `"User Preference: availability="` to `rubric_applied`. Q12/Q13 must not override this gate. + +1. **Set `human_expertise_required`**: If the BigQuery specialist gate applied, already `true`. Otherwise set `false` unless another rubric explicitly requires it. This field is REQUIRED on every resource in the output. + +1. **Preferred AWS target check**: **Skip** if `aws_service` is **`Deferred — specialist engagement`**. **Skip Aurora substitution** for Cloud SQL when Q6 availability is `single-az` or `multi-az` (RDS is correct). Otherwise verify the selected `aws_service` aligns with the Preferred AWS Target Services table in `steering/design-ref-fast-path.md`. If a non-preferred service is selected (e.g., App Runner for containerized workloads), substitute the preferred alternative (e.g., Fargate). Add a note to the rationale: "Preferred target: [alternative] selected for stronger ecosystem integration." ## Step 3: Handle Secondary Resources @@ -62,10 +89,12 @@ For each SECONDARY resource: 1. Use `steering/design-ref-index.md` for category 2. Apply fast-path (most secondaries have deterministic mappings) -3. If rubric needed: apply same 6-criteria approach +3. If rubric needed: apply the **BigQuery specialist gate** (Pass 2 step 0) first when `gcp_type` starts with `google_bigquery_`; otherwise apply the same 6-criteria approach as Pass 2 ## Step 3.5: Validate AWS Architecture (using awsknowledge) +If `aws_service` is **`Deferred — specialist engagement`**, **do not** validate against concrete AWS analytics SKUs; add a `warnings[]` entry that specialist engagement is required. + **Validation checks** (if awsknowledge available): For each mapped AWS service, verify: @@ -120,8 +149,9 @@ For each mapped AWS service, verify: "memory": "1024", "region": "us-east-1" }, - "confidence": "deterministic", - "rationale": "1:1 compute mapping with Cold Start considerations", + "confidence": "inferred", + "human_expertise_required": false, + "rationale": "Rubric: Compute Engine → Fargate (example — not a Direct Mapping row; Cloud Run/Compute Engine use Pass 2)", "rubric_applied": [ "Eliminators: PASS", "Operational Model: Managed Fargate", @@ -146,18 +176,30 @@ For each mapped AWS service, verify: - Every cluster has `cluster_id` matching a cluster from `gcp-resource-clusters.json` - Every cluster has `gcp_region` and `aws_region` - Every resource has `gcp_address`, `gcp_type`, `gcp_config`, `aws_service`, `aws_config` +- Every resource has `human_expertise_required` (boolean) — `true` for all `google_bigquery_*` resources (specialist gate); `false` for others unless a rubric explicitly requires it +- Every `google_bigquery_*` resource has `aws_service` exactly **`Deferred — specialist engagement`** (not Athena, Redshift, Glue, etc.) +- Every `google_sql_database_instance` resource has `aws_service` ∈ {`RDS PostgreSQL`, `RDS MySQL`, `Aurora PostgreSQL`, `Aurora MySQL`} with non-empty `rationale` citing Q6 availability value. If `availability` is `single-az` or `multi-az`, `aws_service` MUST be RDS (not Aurora). If `multi-az-ha` or `multi-region`, MUST be Aurora. - All `confidence` values are either `"deterministic"` or `"inferred"` - All `rationale` fields are non-empty - Every resource from every evaluated cluster appears in the output - No duplicate `gcp_address` values across clusters - Output is valid JSON +## Completion Handoff Gate (Fail Closed) + +Before returning control to `design.md`, require: + +- `aws-design.json` exists and passes the Output Validation Checklist above. + +If this gate fails: STOP and output: "design-infra did not produce a valid `aws-design.json`; do not complete Phase 3." + ## Present Summary After writing `aws-design.json`, present a concise summary to the user: 1. Total resources mapped and cluster count -2. Per-cluster table: GCP resource → AWS service (one line each, include confidence) -3. Any warnings (regional fallbacks, inferred mappings with low confidence) +2. Per-cluster table: GCP resource → AWS service (one line each). For how each mapping was chosen, use **plain English** from `steering/design-ref-fast-path.md` → **User-facing vocabulary** — **Standard pairing** (`deterministic`), **Tailored to your setup** (`inferred`), or **Estimated from billing only** (`billing_inferred`). Lead with the bold phrase; include the JSON value in parentheses only if the user is technical. +3. Any warnings (regional fallbacks; call out **Tailored to your setup** rows that deserve extra review) +4. If any resource has **`Deferred — specialist engagement`**: state **prominently** that **no AWS analytics target was chosen**. Direct the user to **their AWS account team and/or a data analytics migration partner**. Do **not** recommend Athena, Redshift, Glue, or EMR in the chat summary. Keep it under 20 lines. The user can ask for details or re-read `aws-design.json` at any time. diff --git a/migration-to-aws/steering/design-ref-agentic-to-agentcore.md b/migration-to-aws/steering/design-ref-agentic-to-agentcore.md new file mode 100644 index 00000000..720f3420 --- /dev/null +++ b/migration-to-aws/steering/design-ref-agentic-to-agentcore.md @@ -0,0 +1,253 @@ +# Strands Agents + AgentCore Runtime Design Reference + +> Loaded by `design-ai.md` Step 0.6 when `agentic_profile.is_agentic == true` AND `ai_constraints.agentic.migration_approach == "strands"`. + +**Prerequisites:** `steering/ai-migration-guardrails.md` must already be loaded (Step 0.6 loads it before this file). Do NOT duplicate regional caveats, pricing rules, or effort estimation rules here. + +--- + +## What is Strands Agents + +Strands Agents is an open-source SDK from AWS (open-sourced May 2025, 1.0 released July 2025) that takes a model-driven approach to building AI agents. It powers production features inside AWS services (Amazon Q Developer, AWS Glue, VPC Reachability Analyzer) and is the framework underlying the AgentCore Harness. + +**Key differentiators vs other frameworks:** + +- **Model-driven:** The LLM drives tool selection and planning autonomously — no hardcoded task flows +- **Multi-agent primitives:** Agents-as-Tools (hierarchical), Swarms (collaborative), Graphs (deterministic), A2A protocol (cross-organization) +- **AWS-native deployment:** First-class deployment on AgentCore Runtime with microVM isolation, 8-hour sessions, auto-scaling +- **Multi-model:** Supports Bedrock, OpenAI, Anthropic, and any OpenAI-compatible endpoint. Different models per agent in the same system. +- **Session management:** Built-in durable session persistence (S3, file-based, or custom DAO) +- **Async-native:** Full async support with streaming, concurrent agent evaluation, cancellation + +**When to recommend Strands (this path):** + +- Startup is on OpenAI Agents SDK or custom agent loops where retarget doesn't work well (tightly coupled to OpenAI API) +- Multi-agent system that would benefit from structured primitives (Graphs, Swarms, Agents-as-Tools) +- Team wants AWS-native agent infrastructure (AgentCore Runtime, Memory, Gateway, Identity, Observability) +- Team is willing to refactor orchestration code (accepts 2-6 week effort) +- Startup doesn't know Strands exists — this is the plugin surfacing an option they wouldn't discover from a base LLM + +**When NOT to recommend Strands:** + +- Working LangGraph/CrewAI/AutoGen system where retarget (model swap) is sufficient +- Team needs to ship in < 2 weeks (retarget or Harness is faster) +- Simple single-agent pattern (Harness is simpler — config vs code) +- Team has no Python/TypeScript expertise (Strands SDK is Python and TypeScript) + +--- + +## Framework-to-Strands Mapping + +Map the detected `agentic_profile.framework` and `orchestration_pattern` to Strands primitives: + +### OpenAI Agents SDK → Strands Agent + +| OpenAI Agents SDK | Strands Equivalent | Notes | +| ---------------------------------------- | ------------------------------------------------------------ | -------------------------------------------------------- | +| `Assistant` / `Agent` definition | `Agent(name=..., model=..., system_prompt=..., tools=[...])` | Direct mapping. System prompt, tools, model all map 1:1. | +| `Runner.run()` / `Runner.run_streamed()` | `agent("prompt")` or `agent.stream_async("prompt")` | Strands agent is callable directly. | +| `function` tool type | `@tool` decorated function | Same concept, different decorator syntax. | +| `handoff` to another agent | `@tool` wrapping another agent (Agents-as-Tools pattern) | Strands uses agents-as-tools for delegation. | +| Thread / conversation state | `SessionManager` with file or S3 backend | Strands persists full conversation automatically. | +| `response.output` | `result = agent("prompt"); str(result)` | Agent returns result object. | + +**Bridge option (Phase 0):** Before full Strands rewrite, startups on OpenAI Agents SDK can do a partial retarget — route model calls through Bedrock using OpenAI-compatible endpoint format while keeping the Agents SDK orchestration intact. This buys time on AWS infrastructure without rewriting. Then migrate to Strands when ready. + +### LangGraph → Strands Graphs + +| LangGraph | Strands Equivalent | Notes | +| --------------------------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------- | +| `StateGraph(state_schema)` | `GraphBuilder()` | Strands graphs don't require explicit state schema — state flows via agent context. | +| `graph.add_node("name", function)` | `builder.add_node(agent, "name")` | Strands nodes are agents, not arbitrary functions. Wrap functions as single-purpose agents. | +| `graph.add_edge("a", "b")` | `builder.add_edge("a", "b")` | Direct mapping. | +| `graph.add_conditional_edges("a", router_fn)` | `builder.add_edge("a", "b", condition=fn)` | Condition function receives state, returns bool. | +| `graph.set_entry_point("start")` | `builder.set_entry_point("start")` | Direct mapping. | +| `graph.compile()` | `builder.build()` | Returns executable graph. | +| `MemorySaver` / checkpointing | `SessionManager` with S3 or file backend | Different API but same concept — durable state across invocations. | + +**Key difference:** LangGraph nodes are arbitrary functions; Strands graph nodes are Agents. For non-agent nodes (pure data transformation), wrap in a minimal Agent with a focused system prompt and no tools. + +### CrewAI → Strands Swarms or Agents-as-Tools + +| CrewAI | Strands Equivalent | Notes | +| ------------------------------------------------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------------------- | +| `Agent(role=..., goal=..., backstory=..., tools=[...])` | `Agent(name=..., system_prompt=..., tools=[...])` | Map `role`+`goal`+`backstory` into `system_prompt`. | +| `Task(description=..., agent=...)` | Task is implicit — the orchestrator agent decides what to delegate. | Strands is model-driven; tasks aren't pre-defined. | +| `Crew(agents=[...], process=Process.sequential)` | Sequential: chain agents with output piping, or use Graph with linear edges. | | +| `Crew(agents=[...], process=Process.hierarchical)` | `Swarm([agent1, agent2, agent3])` or Agents-as-Tools with manager agent. | Swarm for collaborative; Agents-as-Tools for explicit delegation. | +| `crew.kickoff()` | `swarm("task description")` or `manager_agent("task description")` | | + +**Key difference:** CrewAI pre-defines tasks and assigns them to agents. Strands is model-driven — the orchestrator agent (or swarm) decides dynamically which specialist to consult. This is more flexible but requires good system prompts. + +### AutoGen → Strands Agents-as-Tools or Swarms + +| AutoGen | Strands Equivalent | Notes | +| ---------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------ | +| `AssistantAgent(name=..., system_message=...)` | `Agent(name=..., system_prompt=...)` | Direct mapping. | +| `UserProxyAgent` | `handoff_to_user` tool from `strands_tools` | Built-in human-in-the-loop. | +| `GroupChat(agents=[...])` | `Swarm([agent1, agent2, ...])` | Swarm provides collaborative multi-agent without fixed turn order. | +| `GroupChatManager` | Implicit in Swarm coordination, or explicit manager Agent with Agents-as-Tools. | | +| `initiate_chat()` | `swarm("initial message")` or `agent("initial message")` | | + +### Custom Agent Loops → Strands Agent + +Custom `while` loops with LLM call + tool dispatch map most directly to a single Strands Agent: + +```python +# Before (custom loop): +while not done: + response = openai.chat.completions.create(model="gpt-4o", messages=messages, tools=tool_schemas) + if response.tool_calls: + result = execute_tool(response.tool_calls[0]) + messages.append(tool_result) + else: + done = True + +# After (Strands): +from strands import Agent +from strands.models import BedrockModel + +agent = Agent( + model=BedrockModel(model_id="us.anthropic.claude-sonnet-4-6-20250514-v1:0"), + tools=[web_search, calculator, file_read], # your existing tool functions with @tool decorator + system_prompt="Your existing system prompt here" +) +result = agent("Your task here") +``` + +The Strands Agent handles the loop internally — model calls, tool dispatch, result parsing, context management. Your tool functions stay the same; just add the `@tool` decorator. + +--- + +## AgentCore Runtime Deployment + +Strands agents deploy on AgentCore Runtime for production: + +**What AgentCore Runtime provides:** + +- Serverless microVM isolation per session (no shared state between users) +- Auto-scaling from zero to thousands of sessions +- Up to 8-hour session duration for long-running agent tasks +- Built-in observability via OpenTelemetry (traces to CloudWatch, Datadog, etc.) +- VPC support and PrivateLink for network isolation + +**Deployment model:** + +| `task_duration` (from Clarify Q25) | Deployment recommendation | +| ---------------------------------- | -------------------------------------------------------------------------------------- | +| `quick` (< 30s) | AgentCore Runtime standard. Consider Lambda for simple single-turn if no state needed. | +| `medium` (30s – 5min) | AgentCore Runtime standard. Sessions handle this natively. | +| `long` (5min – 1hr) | AgentCore Runtime required. Lambda will timeout. | +| `very_long` (1hr+) | AgentCore Runtime with session chaining. Break into sub-tasks if > 8 hours. | + +**Memory integration:** + +| `memory_requirement` (from Clarify Q24) | Strands + AgentCore config | +| --------------------------------------- | ------------------------------------------------------------------------------------------- | +| `none` | No SessionManager needed. Stateless invocations. | +| `session` | `SessionManager` with in-session state. AgentCore Runtime sessions are stateful by default. | +| `cross_session` | `SessionManager` with S3 backend + AgentCore Memory service for long-term knowledge. | + +--- + +## Output: `agentic_design` in `aws-design-ai.json` + +When Strands path is selected, write this to `aws-design-ai.json`: + +```json +{ + "agentic_design": { + "migration_approach": "strands", + "strands_config": { + "agents": [ + { + "agent_id": "from agentic_profile.agents[].agent_id", + "strands_primitive": "Agent|Graph|Swarm", + "model_id": "from bedrock_models[].aws_model_id", + "system_prompt": "extracted or placeholder", + "tools": ["from tool_manifest, mapped to @tool functions"], + "role_in_system": "orchestrator|specialist|worker" + } + ], + "orchestration_primitive": "single_agent|agents_as_tools|swarm|graph", + "session_manager": "none|file|s3", + "memory_service": false, + "deployment_target": "agentcore_runtime", + "bridge_phase": true, + "source_framework": "from agentic_profile.framework" + }, + "regional_fit": "available|preview|unavailable", + "warnings": [] + } +} +``` + +**Mapping `orchestration_pattern` → `orchestration_primitive`:** + +| Detected pattern | Strands primitive | Rationale | +| ---------------- | ----------------- | ---------------------------------------------------------------------------------- | +| `single` | `single_agent` | One Agent with tools. Simplest. | +| `hierarchical` | `agents_as_tools` | Manager agent delegates to specialist agents wrapped as tools. | +| `swarm` | `swarm` | Multiple agents collaborate via shared memory. | +| `graph` | `graph` | Explicit node/edge workflow with conditional routing. | +| `sequential` | `graph` (linear) | Graph with linear edges, no branching. Simpler than full graph but same primitive. | +| `unknown` | `agents_as_tools` | Safe default — hierarchical delegation is the most common multi-agent pattern. | + +**`bridge_phase`:** Set to `true` if `source_framework == "openai_agents"` — indicates the partial retarget bridge (Phase 0) should be included in generated artifacts. + +--- + +## AgentCore Agent Performance Loop (Public Preview, May 2026) + +AgentCore launched an observe-evaluate-optimize-deploy loop in **public preview** (May 2026). APIs may change before GA; CloudTrail audit logging is not yet supported for these features. Surface this as an optional post-migration capability for teams that care about production evals, regression testing, prompt/tool optimization, and A/B rollout — not as an unconditional migration advantage. + +**When to surface:** `agentic_profile.is_agentic == true` AND the design targets AgentCore Runtime, Evaluations, or Gateway. Not gated on `migration_approach == "strands"` alone — the capability is tied to AgentCore, not the Strands SDK specifically. + +**Capabilities (all preview):** + +| Capability | What it does | Prerequisite | Cost note | +| -------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | ------------------------------------------------------------ | +| **Recommendations** | Analyzes production traces + evaluator outputs → recommends targeted updates to system prompts and tool descriptions | AgentCore traces + evaluations must be active | No separate charge; underlying AgentCore service costs apply | +| **A/B Testing** | Validates prompt/tool changes via controlled rollout before full deployment | AgentCore Gateway | No separate charge | +| **Batch Evaluation** | Replays curated or historical sessions to compare pre/post scores; catches regressions before changes reach end users | AgentCore Evaluations | No separate charge | +| **User Simulation** | Generates realistic multi-turn conversations using LLM-backed actors to reveal behaviors beyond scripted test cases | AgentCore Evaluations | Incurs Bedrock model invocation costs per simulated turn | + +**Caveats to surface explicitly:** + +- All capabilities are **public preview** — APIs may change before GA +- **CloudTrail not yet supported** — do not recommend for workloads requiring complete audit coverage +- User simulation incurs model invocation costs; estimate before enabling at scale +- Requires AgentCore traces/evaluations to be active before recommendations are useful + +**Output addition to `aws-design-ai.json`:** + +Add to `agentic_design` (optional — only when AgentCore Runtime/Evaluations/Gateway is in the design): + +```json +"performance_loop": { + "status": "preview", + "capabilities": ["recommendations", "batch_evaluations", "user_simulation", "ab_testing"], + "recommended_when": ["production eval requirements", "regression testing needed", "prompt/tool optimization desired", "A/B rollout required"], + "prerequisites": ["AgentCore traces and evaluations active", "AgentCore Gateway for A/B testing"], + "caveats": ["preview APIs — may change before GA", "CloudTrail not supported yet", "user simulation incurs model invocation costs"] +} +``` + +--- + +## Present Summary (Strands-specific additions) + +After the standard model comparison summary from `design-ai.md`, add: + +> **Agentic Migration: Strands Agents + AgentCore Runtime** +> +> - Source framework: [detected framework] +> - Strands primitive: [orchestration_primitive] (mapped from [detected orchestration_pattern]) +> - Agents to convert: [count] ([list agent_ids]) +> - Tools to migrate: [count] (existing functions get `@tool` decorator) +> - Deployment: AgentCore Runtime ([task_duration] sessions) +> - Memory: [session_manager] + [AgentCore Memory if cross_session] +> - Bridge phase: [yes/no — for OpenAI Agents SDK users] +> - Estimated effort: [range] depending on [drivers from guardrails] +> - **Performance loop (preview):** Because this design targets AgentCore Runtime, you can optionally add AgentCore's preview performance loop for evaluation, simulation, prompt/tool recommendations, and A/B validation. Note: CloudTrail not yet supported; user simulation incurs model costs. +> - **Note:** Strands Agents is an open-source AWS framework (strandsagents.com) that powers AgentCore internally. It provides multi-agent primitives (Graphs, Swarms, Agents-as-Tools, A2A) with native AgentCore deployment. diff --git a/migration-to-aws/steering/design-ref-ai-anthropic-to-bedrock.md b/migration-to-aws/steering/design-ref-ai-anthropic-to-bedrock.md new file mode 100644 index 00000000..e7025898 --- /dev/null +++ b/migration-to-aws/steering/design-ref-ai-anthropic-to-bedrock.md @@ -0,0 +1,81 @@ +# Anthropic SDK → Amazon Bedrock Migration + +> Loaded by `design-ai.md` when `ai_source == "anthropic"`. +> The user is already on Claude via the Anthropic SDK. Migration is a client swap only. +> No model change, no prompt rewriting, no retraining required. + +--- + +## Step 1: Map model IDs to Bedrock + +| Anthropic SDK model | Bedrock model ID | Tier | Input/Output per 1M | +| ------------------- | ------------------------------------------ | -------- | ------------------- | +| `claude-opus-4-*` | `anthropic.claude-opus-4-6-v1` | Premium | $5 / $25 | +| `claude-sonnet-4-*` | `anthropic.claude-sonnet-4-6` | Flagship | $3 / $15 | +| `claude-haiku-4-*` | `anthropic.claude-haiku-4-5-20251001-v1:0` | Fast | $1 / $5 | + +Older Claude models — Claude 3.5 Haiku, Claude 3 Sonnet, Claude 3.5 Sonnet (v1/v2), Claude 3 Haiku, and Claude 3.7 Sonnet — are past EOL or within the 90-day exclusion window. Do **not** recommend them as migration targets. See `steering/ai-model-lifecycle.md` for authoritative status (recomputed each run). + +**Recommendation:** Migrate to Claude 4.x directly. Converse API call shape is identical across generations. + +--- + +## Step 2: Rewrite client + +Before: `anthropic.Anthropic(api_key=...)` +After: `boto3.client("bedrock-runtime", region_name="us-east-1")` + +Remove ANTHROPIC_API_KEY. Use IAM role with bedrock:InvokeModel. + +--- + +## Step 3: Rewrite API calls + +Before: `client.messages.create(model=..., max_tokens=1024, messages=[{"role": "user", "content": "Hello"}])` + +After: `client.converse(modelId="anthropic.claude-sonnet-4-6", messages=[{"role": "user", "content": [{"text": "Hello"}]}], inferenceConfig={"maxTokens": 1024})` + +Key differences: + +- content is typed blocks [{"text": "..."}] not a plain string +- max_tokens moves to inferenceConfig.maxTokens +- response text at `response["output"]["message"]["content"][0]["text"]` + +--- + +## Step 4: System prompts + +Before: `system="You are helpful"` as a string param +After: `system=[{"text": "You are helpful"}]` as a list of blocks + +--- + +## Step 5: Streaming + +Before: `client.messages.stream()` context manager +After: `client.converse_stream()` — iterate `response["stream"]` for `contentBlockDelta` events + +--- + +## Step 6: IAM permissions + +Add bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream on arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-* + +--- + +## Step 7: Request Bedrock quota + +Request TPM increases via Service Quotas. Cross-region inference profiles (us.* prefix) have higher shared limits. + +--- + +## Validation checklist + +- [ ] anthropic.Anthropic() replaced with boto3.client("bedrock-runtime") +- [ ] client.messages.create() replaced with client.converse() +- [ ] content blocks use typed format [{"text": "..."}] +- [ ] max_tokens moved to inferenceConfig.maxTokens +- [ ] Streaming uses converse_stream with contentBlockDelta events +- [ ] ANTHROPIC_API_KEY removed from environment/secrets +- [ ] IAM role has bedrock:InvokeModel permission +- [ ] Model IDs updated to Bedrock format (Claude 4.x recommended) diff --git a/migration-to-aws/steering/design-ref-ai-gemini-to-bedrock.md b/migration-to-aws/steering/design-ref-ai-gemini-to-bedrock.md index 7108e182..78dfe875 100644 --- a/migration-to-aws/steering/design-ref-ai-gemini-to-bedrock.md +++ b/migration-to-aws/steering/design-ref-ai-gemini-to-bedrock.md @@ -6,49 +6,52 @@ This file is loaded by `design-ai.md` when `ai-workload-profile.json` has `summa Verify all pricing via AWS Pricing MCP or `steering/cached-prices.md`. +**Model lifecycle:** Before recommending any Bedrock model, check `steering/ai-model-lifecycle.md`. Do not recommend Legacy models as primary selections for new migrations. Legacy models are annotated below where they appear. + --- -## Competitive Reality (March 2026) +## Competitive Reality (May 2026) -Gemini 3.1 Pro Preview (Feb 19, 2026) has shifted the landscape. Be honest with users: +Gemini 3.5 Flash is now GA (May 2026) — the current flagship Flash model. Gemini 3.1 Pro is the current Pro tier. Be honest with users: - Gemini 3.1 Pro leads 13/16 Google-reported benchmarks and 6/10 on the Artificial Analysis Intelligence Index -- ARC-AGI-2: 77.1% (2.5x jump over Gemini 3 Pro), SWE-Bench: 80.6% (tied with Opus 4.6 at 80.8%) -- Costs $2/$12 per 1M tokens — less than half of Opus 4.6 ($5/$25), cheaper than Sonnet 4.6 ($3/$15) -- ~119 tokens/sec — faster than any Bedrock model at this quality tier +- ARC-AGI-2: 77.1%, SWE-Bench: 80.6% (tied with Opus 4.6 at 80.8%) +- Gemini 3.1 Pro costs $2/$12 per 1M tokens — less than half of Opus 4.6 ($5/$25), cheaper than Sonnet 4.6 ($3/$15) +- Gemini 3.5 Flash at $1.50/$9.00 is 5x more expensive than the old Gemini 2.5 Flash ($0.30/$2.50) — the Bedrock cost savings case is now much stronger for Flash-tier users **Where Bedrock still wins:** -- Claude Sonnet/Opus 4.6 lead on real-world agentic tasks (GDPval evaluation) — the gap between benchmarks and production agent reliability is real -- Claude prompt caching (90% savings on repeated content) has no Gemini equivalent at Preview tier +- Claude Sonnet 4.6 / Opus 4.x lead on real-world agentic tasks (GDPval evaluation) — the gap between benchmarks and production agent reliability is real +- Claude prompt caching (90% savings on repeated content) has no Gemini equivalent - Claude function calling remains best-in-class for complex multi-turn tool use - AWS ecosystem integration (Bedrock Agents, Knowledge Bases, Guardrails) has no Gemini equivalent **Migration case by tier:** -- Gemini Pro → Bedrock: driven by AWS consolidation, agentic reliability, or ecosystem — NOT cost or general benchmarks -- Gemini Flash/Lite → Nova Lite/Micro: still 64-88% cheaper, strong cost case -- Gemini 2.5 Pro → Bedrock: moderate case (older model, higher price than 3.1 Pro) +- Gemini 3.5 Flash → Bedrock: **strong cost case** — Nova Lite is 94% cheaper; even Claude Sonnet 4.6 is comparable at $3/$15 vs $1.50/$9.00 +- Gemini 3.1 Pro → Bedrock: driven by AWS consolidation, agentic reliability, or ecosystem — NOT cost or general benchmarks +- Gemini 3.1 Flash-Lite → Nova Lite/Micro: still 76-88% cheaper, strong cost case +- Gemini 2.5 Pro → Bedrock: moderate case (older model) --- ## Bedrock Model Portfolio -| Model | Best For | Complexity | Speed | Context | -| ----------------- | ---------------------------- | ---------- | ------ | ------- | -| Claude Sonnet 4.6 | Agentic tasks, tool use | High | High | 200K | -| Claude Opus 4.6 | Maximum reasoning | High | Medium | 200K | -| Claude Haiku 4.5 | Simple + fast | Medium | High | 200K | -| Llama 4 Maverick | Cost-effective + multimodal | Medium | High | 1M | -| Llama 4 Scout | Ultra-long context, cheapest | Medium | Medium | 10M | -| Nova 2 Pro | AWS flagship, multimodal | High | High | 1M | -| Nova 2 Lite | AWS mid-tier, long context | Medium | High | 1M | -| Nova Pro | AWS balanced | Medium | High | 300K | -| Nova Lite | AWS fast + cheapest | Medium | High | 300K | -| Nova Micro | AWS fastest, text-only | Low | High | 128K | -| Nova Premier | Complex reasoning | High | Medium | 1M | -| DeepSeek-R1 | Chain-of-thought reasoning | High | Medium | 128K | -| Mistral Large 3 | EU/Multilingual | High | Medium | 256K | +| Model | Best For | Complexity | Speed | Context | +| ----------------- | ------------------------------------------------------------------- | ---------- | ------ | ------- | +| Claude Sonnet 4.6 | Agentic tasks, tool use | High | High | 200K | +| Claude Opus 4.6 | Maximum reasoning | High | Medium | 200K | +| Claude Haiku 4.5 | Simple + fast | Medium | High | 200K | +| Llama 4 Maverick | Cost-effective + multimodal | Medium | High | 1M | +| Llama 4 Scout | Ultra-long context, cheapest | Medium | Medium | 10M | +| Nova 2 Pro | AWS flagship, multimodal | High | High | 1M | +| Nova 2 Lite | AWS mid-tier, long context | Medium | High | 1M | +| Nova Pro | AWS balanced | Medium | High | 300K | +| Nova Lite | AWS fast + cheapest | Medium | High | 300K | +| Nova Micro | AWS fastest, text-only | Low | High | 128K | +| Nova Premier | Complex reasoning — **Legacy** (EOL Sep 14, 2026) prefer Nova 2 Pro | High | Medium | 1M | +| DeepSeek-R1 | Chain-of-thought reasoning | High | Medium | 128K | +| Mistral Large 3 | EU/Multilingual | High | Medium | 256K | --- @@ -56,33 +59,41 @@ Gemini 3.1 Pro Preview (Feb 19, 2026) has shifted the landscape. Be honest with ### Gemini Pro Tier -| Gemini Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | -| ---------------------- | --------------------- | ------------------ | -------------- | ------------------- | -| Gemini 3.1 Pro Preview | $2.00 / $12.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | Gemini 24% cheaper | -| Gemini 3.1 Pro Preview | $2.00 / $12.00 | Claude Opus 4.6 | $5.00 / $25.00 | Gemini 54% cheaper | -| Gemini 3 Pro | $0.50 / $3.00 | Llama 4 Maverick | $0.24 / $0.97 | Bedrock 64% cheaper | -| Gemini 3 Pro | $0.50 / $3.00 | Llama 4 Scout | $0.17 / $0.66 | Bedrock 75% cheaper | -| Gemini 3 Pro | $0.50 / $3.00 | Nova Pro | $0.80 / $3.20 | Gemini 17% cheaper | -| Gemini 2.5 Pro | $1.25 / $10.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | Gemini 40% cheaper | -| Gemini 2.5 Pro | $1.25 / $10.00 | Nova Pro | $0.80 / $3.20 | Bedrock 62% cheaper | -| Gemini 3.1 Pro Preview | $2.00 / $12.00 | Nova 2 Pro | $1.38 / $11.00 | Bedrock 14% cheaper | -| Gemini 2.5 Pro | $1.25 / $10.00 | Nova 2 Pro | $1.38 / $11.00 | Gemini 9% cheaper | +| Gemini Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | +| -------------- | --------------------- | --------------------- | -------------- | ------------------- | +| Gemini 3.1 Pro | $2.00 / $12.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | Gemini 24% cheaper | +| Gemini 3.1 Pro | $2.00 / $12.00 | Claude Opus 4.7 / 4.6 | $5.00 / $25.00 | Gemini 54% cheaper | +| Gemini 3.1 Pro | $2.00 / $12.00 | Nova 2 Pro | $1.38 / $11.00 | Bedrock 14% cheaper | +| Gemini 3 Pro | $0.50 / $3.00 | Llama 4 Maverick | $0.24 / $0.97 | Bedrock 64% cheaper | +| Gemini 3 Pro | $0.50 / $3.00 | Llama 4 Scout | $0.17 / $0.66 | Bedrock 75% cheaper | +| Gemini 3 Pro | $0.50 / $3.00 | Nova Pro | $0.80 / $3.20 | Gemini 17% cheaper | +| Gemini 2.5 Pro | $1.25 / $10.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | Gemini 40% cheaper | +| Gemini 2.5 Pro | $1.25 / $10.00 | Nova Pro | $0.80 / $3.20 | Bedrock 62% cheaper | +| Gemini 2.5 Pro | $1.25 / $10.00 | Nova 2 Pro | $1.38 / $11.00 | Gemini 9% cheaper | + +> **Gemini 3.1 Pro breakpoint:** $4.00/$18.00 per 1M for prompts >200k tokens. Table above uses ≤200k rates. ### Gemini Flash/Lite Tier -| Gemini Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | -| ---------------- | --------------------- | ------------------ | -------------- | ------------------- | -| Gemini 2.5 Flash | $0.30 / $2.50 | Nova Lite | $0.06 / $0.24 | Bedrock 88% cheaper | -| Gemini 2.0 Flash | $0.10 / $0.40 | Nova Micro | $0.035 / $0.14 | Bedrock 65% cheaper | -| Gemini Flash | $0.075 / $0.30 | Nova Micro | $0.035 / $0.14 | Bedrock 53% cheaper | +| Gemini Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | +| ------------------------- | ----------------------------------------------------------- | ---------------------------------------- | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Gemini 3.5 Flash (GA) | $1.50 / $9.00 | Nova Lite | $0.06 / $0.24 | Bedrock 94% cheaper — strong migration case; 3.5 Flash is 5x more expensive than old 2.5 Flash | +| Gemini 3.5 Flash (GA) | $1.50 / $9.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | Gemini 33% cheaper — but Sonnet leads on agentic reliability | +| Gemini 3.1 Flash-Lite | $0.25 / $1.50 | Nova Lite | $0.06 / $0.24 | Bedrock 76% cheaper | +| Gemini 3.1 Flash-Lite | $0.25 / $1.50 | Nova Micro | $0.035 / $0.14 | Bedrock 88% cheaper | +| Gemini 2.5 Flash | $0.30 / $2.50 | Nova Lite | $0.06 / $0.24 | Bedrock 88% cheaper | +| Gemini 2.5 Flash Thinking | $0.30 / $0.60–$3.50 (varies by thinking budget) | Claude Sonnet 4.6 with extended thinking | $3.00 / $15.00 | Depends on thinking budget — at full thinking ($3.50/M output) Sonnet 4.6 is comparable; at low thinking ($0.60/M output) Gemini is cheaper. Profile actual thinking token usage before recommending. | +| Gemini 2.0 Flash | $0.10 / $0.40 | Nova Micro | $0.035 / $0.14 | Bedrock 65% cheaper | +| Gemini Flash 1.5 | **Legacy — EOL Sep 24, 2025. Migrate to Gemini 3.5 Flash.** | Nova Lite | $0.06 / $0.24 | If still in use, migrate source model first; strong Bedrock cost case once on 3.5 Flash | ### Legacy/Specialized -| Gemini Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | -| ----------------------- | --------------------- | --------------------- | ---------------- | ---------------------------------- | -| text-bison / chat-bison | Legacy | Llama 4 Scout | $0.17 / $0.66 | Bedrock (better quality + cheaper) | -| text-embedding-004 | $0.025 / N/A | Titan Embeddings V2 | $0.02 / N/A | Bedrock 20% cheaper | -| imagen-* | Varies | Titan Image Generator | $0.008-$0.04/img | Varies | +| Gemini Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | +| ----------------------- | -------------------------------------------------------------------- | ------------------- | --------------- | ---------------------------------------------------------------- | +| Gemini 1.5 Pro | **Legacy — EOL Sep 24, 2025. Migrate to Gemini 2.5 Pro or 3.x Pro.** | Claude Sonnet 4.6 | $3.00 / $15.00 | If still in use, migrate source model first | +| text-bison / chat-bison | Legacy | Llama 4 Scout | $0.17 / $0.66 | Bedrock (better quality + cheaper) | +| text-embedding-004 | $0.025 / N/A | Titan Embeddings V2 | $0.02 / N/A | Bedrock 20% cheaper | +| imagen-* | Varies | Nova Canvas | $0.04-$0.08/img | Titan Image Gen v2 is Legacy (EOL Jun 30, 2026); use Nova Canvas | _Percentages are blended savings using a 2:1 input-to-output token ratio. Actual savings depend on your input/output ratio._ @@ -96,7 +107,7 @@ Gemini 3.1 Pro Preview matches or beats Opus 4.6 on most reasoning benchmarks at - If user needs **general reasoning/coding quality** → Gemini 3.1 Pro is competitive or better. Migration case is weak unless driven by AWS consolidation. - If user needs **agentic reliability** (real-world multi-step tasks) → **Claude Sonnet 4.6** still leads on GDPval. This is the honest differentiator. -- If user needs **maximum reasoning on hardest problems** → Claude Opus 4.6 ($5/$25) — tied with Gemini 3.1 Pro on SWE-Bench. +- If user needs **maximum reasoning on hardest problems** → **Claude Opus 4.7** ($5/$25 headline on-demand, same tier as Opus 4.6) — use the latest [Claude on Bedrock](https://aws.amazon.com/bedrock/pricing/) model card for benchmark deltas vs Gemini; Opus 4.6 remains a same-price alternative where batch or regional availability matters. ### Speed-First @@ -144,7 +155,7 @@ Gemini Flash → **Nova Micro** (<200ms, text-only, cheapest), **Haiku 4.5** (<4 | Gemini Model | Monthly | Best Bedrock Match | Monthly | Difference | | ------------------------------- | ------- | ------------------------------ | ------- | ---------- | | Gemini 3.1 Pro Preview ($2/$12) | $1,200 | Claude Sonnet 4.6 ($3/$15) | $1,575 | +24% | -| Gemini 3.1 Pro Preview ($2/$12) | $1,200 | Claude Opus 4.6 ($5/$25) | $2,625 | +54% | +| Gemini 3.1 Pro Preview ($2/$12) | $1,200 | Claude Opus 4.7 / 4.6 ($5/$25) | $2,625 | +54% | | Gemini 3.1 Pro Preview ($2/$12) | $1,200 | Nova 2 Pro ($1.38/$11.00) | $1,032 | -14% | | Gemini 3 Pro ($0.50/$3.00) | $300 | Llama 4 Maverick ($0.24/$0.97) | $109 | -64% | | Gemini 3 Pro ($0.50/$3.00) | $300 | Llama 4 Scout ($0.17/$0.66) | $75 | -75% | @@ -167,12 +178,12 @@ Not available on other Bedrock models. This is a significant Claude advantage fo ## Feature Migration Notes -| Gemini Feature | Bedrock Equivalent | Notes | -| ---------------------- | ----------------------------------------------------------- | ------------------------------ | -| Function calling | Claude tools (excellent), Mistral (good) | Minimal changes | -| Structured output/JSON | Claude (excellent), Nova Pro (good) | Most models via prompt | -| Streaming | All major models | Same SSE pattern | -| Vision | Claude Sonnet/Haiku, Llama 4 Maverick | Multimodal parity | -| Context caching | Claude prompt caching | 90% savings on cached portions | -| Audio/video input | Nova Sonic (speech), Transcribe/Rekognition (preprocessing) | Different architecture | -| Embeddings | Amazon Titan Embeddings ($0.02/1M, 1536 dims) | Must re-embed all docs | +| Gemini Feature | Bedrock Equivalent | Notes | +| ---------------------- | ------------------------------------------------------------- | ----------------------------------------- | +| Function calling | Claude tools (excellent), Mistral (good) | Minimal changes | +| Structured output/JSON | Claude (excellent), Nova Pro (good) | Most models via prompt | +| Streaming | All major models | Same SSE pattern | +| Vision | Claude Sonnet/Haiku, Llama 4 Maverick | Multimodal parity | +| Context caching | Claude prompt caching | 90% savings on cached portions | +| Audio/video input | Nova 2 Sonic (speech), Transcribe/Rekognition (preprocessing) | Nova Sonic v1 is Legacy; use Nova 2 Sonic | +| Embeddings | Amazon Titan Embeddings ($0.02/1M, 1536 dims) | Must re-embed all docs | diff --git a/migration-to-aws/steering/design-ref-ai-openai-to-bedrock.md b/migration-to-aws/steering/design-ref-ai-openai-to-bedrock.md index d799514a..73d67754 100644 --- a/migration-to-aws/steering/design-ref-ai-openai-to-bedrock.md +++ b/migration-to-aws/steering/design-ref-ai-openai-to-bedrock.md @@ -8,36 +8,67 @@ Many GCP-hosted applications use OpenAI's API rather than Vertex AI. This guide Verify all pricing via AWS Pricing MCP or `steering/cached-prices.md`. Uses OpenAI Standard tier pricing. +**Model lifecycle:** Before recommending any Bedrock model, check `steering/ai-model-lifecycle.md`. Do not recommend Legacy models as primary selections for new migrations. Legacy models are annotated below where they appear. + --- -## Key Insight: The Landscape Has Changed (March 2026) +## Key Insight: The Landscape Has Changed (April 2026) **It is no longer "Bedrock is always cheaper."** It depends on the model. -- **OpenAI cheaper:** GPT-5.2 (50%), GPT-5.1/5 (40%), GPT-4.1 (43%), GPT-4o (29%), o4-mini/o3-mini/o1-mini (69%) -- **Bedrock cheaper:** Nova Lite vs Mini models (85-86%), Nova Micro vs Nano (58-65%), Nova Premier vs Pro models (88-92%), DeepSeek-R1 vs o3 (32%) +- **Bedrock cheaper:** GPT-5.5 flagship (17% cheaper output via Opus 4.6), Nova Lite vs Mini models (85-94%), Nova Micro vs Nano (65-87%), Nova 2 Pro vs Pro models (90-95%), DeepSeek-R1 vs o3 (32%) +- **OpenAI cheaper:** GPT-5.4 (5%), GPT-5.2 (50%), GPT-5.1/5 (40%), GPT-4.1 (43%), GPT-4o (29%), o4-mini/o3-mini/o1-mini (69%) + +> **GPT-5.5 note (April 23, 2026):** GPT-5.5 doubled pricing to $5/$30 per MTok vs GPT-5.4's $2.50/$15. Claude Opus 4.6 at $5/$25 now matches on input and is **17% cheaper on output**. This reverses the GPT-5.4 dynamic where OpenAI was cheaper — at the GPT-5.5 tier, Bedrock wins on cost. GPT-5.5 uses 40% fewer output tokens on coding tasks (per OpenAI), partially offsetting the price hike for Codex-style workloads. --- ## Model Mapping Tables -### Flagship (GPT-5 Series) +### GPT-5.5 Series (Latest — April 23, 2026) + +GPT-5.5 is the first fully retrained base model since GPT-4.5. Natively omnimodal (text + image + audio + video), 88.7% SWE-Bench Verified, 256K context in ChatGPT / 1M in API. Two variants: standard and Pro. No Mini/Nano variants at launch (expected Q3 2026). Percentages below are blended savings using a 2:1 input-to-output token ratio. + +| OpenAI Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | +| ------------ | --------------------- | -------------------- | -------------- | ------------------- | +| GPT-5.5 | $5.00 / $30.00 | Claude Opus 4.6 | $5.00 / $25.00 | Bedrock 17% cheaper | +| GPT-5.5 | $5.00 / $30.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | Bedrock 53% cheaper | +| GPT-5.5 Pro | $30.00 / $180.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 95% cheaper | + +> **Token efficiency caveat:** OpenAI reports GPT-5.5 uses ~40% fewer output tokens on Codex-style tasks vs GPT-5.4. Effective cost increase over GPT-5.4 is ~50% (not 100%) for coding workloads. For non-coding workloads, the full 2× price applies. + +### GPT-5.4 Series + +Percentages below are blended savings using a 2:1 input-to-output token ratio. GPT-5.4 uses breakpoint pricing at 272K input tokens; rates below assume <272K context. + +| OpenAI Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | +| ------------ | --------------------- | -------------------- | -------------- | ------------------- | +| GPT-5.4 | $2.50 / $15.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | OpenAI 5% cheaper | +| GPT-5.4 Mini | $0.75 / $4.50 | Nova Lite | $0.06 / $0.24 | Bedrock 94% cheaper | +| GPT-5.4 Nano | $0.20 / $1.25 | Nova Micro | $0.035 / $0.14 | Bedrock 87% cheaper | +| GPT-5.4 Pro | $30.00 / $180.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 94% cheaper | + +### Flagship (GPT-5/5.2 Series) Percentages below are blended savings using a 2:1 input-to-output token ratio. -| OpenAI Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | -| --------------- | --------------------- | ------------------ | -------------- | ------------------- | -| GPT-5.2 | $1.75 / $14.00 | Claude Opus 4.6 | $5.00 / $25.00 | OpenAI 50% cheaper | -| GPT-5.1 / GPT-5 | $1.25 / $10.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | OpenAI 40% cheaper | -| GPT-5 Mini | $0.25 / $2.00 | Nova Lite | $0.06 / $0.24 | Bedrock 86% cheaper | -| GPT-5 Nano | $0.05 / $0.40 | Nova Micro | $0.035 / $0.14 | Bedrock 58% cheaper | +| OpenAI Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | +| --------------- | --------------------- | --------------------- | -------------- | ------------------- | +| GPT-5.2 | $1.75 / $14.00 | Claude Opus 4.7 / 4.6 | $5.00 / $25.00 | OpenAI 50% cheaper | +| GPT-5.1 / GPT-5 | $1.25 / $10.00 | Claude Sonnet 4.6 | $3.00 / $15.00 | OpenAI 40% cheaper | +| GPT-5 Mini | $0.25 / $2.00 | Nova Lite | $0.06 / $0.24 | Bedrock 86% cheaper | +| GPT-5 Nano | $0.05 / $0.40 | Nova Micro | $0.035 / $0.14 | Bedrock 58% cheaper | ### Pro Models (Extended Reasoning) -| OpenAI Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | -| ------------ | --------------------- | ------------------ | -------------- | ------------------- | -| GPT-5.2 Pro | $21.00 / $168.00 | Nova Premier | $2.50 / $12.50 | Bedrock 92% cheaper | -| GPT-5 Pro | $15.00 / $120.00 | Nova Premier | $2.50 / $12.50 | Bedrock 88% cheaper | +> **Lifecycle note:** Nova Premier v1 is **Legacy** (EOL Sep 14, 2026). Nova 2 Pro (Preview) is the Active successor for reasoning-heavy workloads. Pricing differs — see `cached-prices.md`. + +| OpenAI Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | +| ------------ | --------------------- | -------------------- | -------------- | ------------------- | +| GPT-5.5 Pro | $30.00 / $180.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 95% cheaper | +| GPT-5.4 Pro | $30.00 / $180.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 94% cheaper | +| GPT-5.2 Pro | $21.00 / $168.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 93% cheaper | +| GPT-5 Pro | $15.00 / $120.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 90% cheaper | ### GPT-4.1 Series @@ -56,13 +87,15 @@ Percentages below are blended savings using a 2:1 input-to-output token ratio. ### Reasoning Models (o-series) -| OpenAI Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | -| --------------------------- | --------------------- | ------------------ | -------------- | ------------------- | -| o1-pro | $150.00 / $600.00 | Nova Premier | $2.50 / $12.50 | Bedrock 98% cheaper | -| o3-pro | $20.00 / $80.00 | Nova Premier | $2.50 / $12.50 | Bedrock 85% cheaper | -| o1 | $15.00 / $60.00 | Nova Premier | $2.50 / $12.50 | Bedrock 81% cheaper | -| o3 | $2.00 / $8.00 | DeepSeek-R1 | $1.35 / $5.40 | Bedrock 32% cheaper | -| o4-mini / o3-mini / o1-mini | $1.10 / $4.40 | Claude Sonnet 4.6 | $3.00 / $15.00 | OpenAI 69% cheaper | +> **Lifecycle note:** Nova Premier v1 is **Legacy** (EOL Sep 14, 2026). Table below uses Nova 2 Pro (Preview) as the Active replacement. + +| OpenAI Model | Price (in/out per 1M) | Best Bedrock Match | Bedrock Price | Winner | +| --------------------------- | --------------------- | -------------------- | -------------- | ------------------- | +| o1-pro | $150.00 / $600.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 98% cheaper | +| o3-pro | $20.00 / $80.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 87% cheaper | +| o1 | $15.00 / $60.00 | Nova 2 Pro (Preview) | $1.38 / $11.00 | Bedrock 83% cheaper | +| o3 | $2.00 / $8.00 | DeepSeek-R1 | $1.35 / $5.40 | Bedrock 32% cheaper | +| o4-mini / o3-mini / o1-mini | $1.10 / $4.40 | Claude Sonnet 4.6 | $3.00 / $15.00 | OpenAI 69% cheaper | ### Legacy Models @@ -91,8 +124,9 @@ _Percentages are blended savings using a 2:1 input-to-output token ratio. Actual **Migrate to Bedrock if:** -- Using Pro/expensive models (o1-pro, GPT-5.2 Pro) → 85-98% savings -- Using Mini/Nano models at high volume → 58-86% savings +- Using GPT-5.5 flagship → Bedrock 17% cheaper on output via Opus 4.6 ($5/$25 vs $5/$30); Sonnet 4.6 is 53% cheaper +- Using Pro/expensive models (GPT-5.5 Pro, GPT-5.4 Pro, o1-pro) → 87-98% savings via Nova 2 Pro +- Using Mini/Nano models at high volume → 87-94% savings via Nova Lite/Micro - Using legacy GPT-4/3.5 → 42-82% savings - Need AWS infrastructure integration - Need prompt caching (Claude only, 90% savings on cached content) @@ -101,9 +135,11 @@ _Percentages are blended savings using a 2:1 input-to-output token ratio. Actual **Consider staying on OpenAI if:** +- Using GPT-5.5 for omnimodal (audio/video) → Claude is text+image only; GPT-5.5 has native audio/video +- Using GPT-5.4 flagship → only 5% cheaper than Sonnet 4.6; marginal either way - Using mid-tier flagships (GPT-5, GPT-4.1, o3, o4-mini) → OpenAI 29-69% cheaper - Low volume (<$500/mo) where absolute savings are small -- Heavily integrated with OpenAI ecosystem (Assistants API, DALL-E, Whisper, Realtime) +- Heavily integrated with OpenAI ecosystem (Assistants API, gpt-image, Whisper, Realtime) - Need Realtime API (no Bedrock equivalent) **Analyze carefully:** Calculate actual token usage x model-specific pricing. Small % differences matter at scale. @@ -112,23 +148,71 @@ _Percentages are blended savings using a 2:1 input-to-output token ratio. Actual ## Feature Migration -| OpenAI Feature | Bedrock Equivalent | Notes | -| -------------------- | --------------------------------------------------------- | ----------------------------------- | -| Function calling | Claude tools (excellent, similar format) | Minimal changes | -| Streaming | All major models | Verify gateway format | -| Vision (GPT-4V) | Claude Sonnet/Haiku, Llama 4 Maverick | 70-95% cheaper | -| Embeddings (ada-002) | Titan Embeddings ($0.02/1M, 1536 dims) | Must re-embed all docs | -| DALL-E | Titan Image Generator ($0.008-$0.04/img) | 50-70% cheaper | -| Whisper (STT) | Amazon Transcribe ($0.024/min) | 4x more expensive but more features | -| TTS | Amazon Polly | Different pricing model | -| Assistants API | Bedrock Agents (sessions, action groups, knowledge bases) | 2-4 week migration | -| JSON mode | Claude (excellent), Nova Pro (good) | Most models via prompt | -| Realtime API | No equivalent | Stay on OpenAI for this | +| OpenAI Feature | Bedrock Equivalent | Notes | +| -------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| OpenAI SDK (direct) | Mantle OpenAI-compat endpoints | Zero code changes — set `OPENAI_BASE_URL` + API key + model ID | +| Function calling | Claude tools (excellent, similar format) | Minimal changes (works via Mantle or Converse API) | +| Streaming | All major models | Verify gateway format | +| Vision (GPT-4V) | Claude Sonnet/Haiku, Llama 4 Maverick | 70-95% cheaper | +| Embeddings (ada-002) | Titan Embeddings ($0.02/1M, 1536 dims) | Must re-embed all docs | +| DALL-E / gpt-image | Nova Canvas ($0.04-$0.08/img) | DALL-E EOL May 12, 2026; OpenAI replacement is gpt-image-1.5; Titan Image Gen v2 is Legacy (EOL Jun 30, 2026); use Nova Canvas | +| Whisper (STT) | Amazon Transcribe ($0.024/min) | 4x more expensive but more features | +| TTS | Amazon Polly | Different pricing model | +| Assistants API | See Assistants API decision tree below | Path depends on which Assistants features are used — see decision tree | +| JSON mode | Claude (excellent), Nova Pro (good) | Most models via prompt | +| Realtime API | No equivalent | Stay on OpenAI for this | --- ## Common Migration Paths +### OpenAI SDK → Mantle (minimal code changes) + +If the application uses the OpenAI Python/JS SDK directly (`from openai import OpenAI` / `new OpenAI()`), Bedrock's [Mantle OpenAI-compatible endpoints](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) allow migration with minimal code changes — primarily environment variables plus a model string swap: + +1. Set `OPENAI_BASE_URL=https://bedrock-mantle.{region}.api.aws/v1` +2. Set `OPENAI_API_KEY=` — use a Bedrock API key, **not** your existing OpenAI API key +3. Change model string (e.g., `gpt-5.4` → `anthropic.claude-sonnet-4-6` or `openai.gpt-oss-120b`) + +**Hard gates before recommending Mantle:** + +- **Model compatibility:** Verify the selected Bedrock model supports the Responses API — [check API compatibility](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html). Not all models do. Do not recommend Mantle Responses API unless the target model is confirmed compatible. +- **Region availability:** Mantle is available in 13 regions (us-east-1, us-east-2, us-west-2, ap-northeast-1, ap-south-1, ap-southeast-2, ap-southeast-3, eu-central-1, eu-west-1, eu-west-2, eu-south-1, eu-north-1, sa-east-1). If the target region is outside this list, do not recommend Mantle — use the boto3 Converse API path instead. + +Supports Chat Completions API, Responses API, streaming, and stateful conversations. + +**Responses API capabilities (when stateful conversations matter):** + +- **Stateful conversation management** — Bedrock rebuilds context automatically; no need to pass full conversation history on each request +- **Async / long-running inference** — background processing for workloads that exceed typical request timeouts (useful for complex agentic tasks) +- **Streaming + non-streaming** — both modes supported via the same endpoint + +### Assistants API → Migration Decision Tree + +Assistants API and Responses API are different surfaces. Do not treat all Assistants API usage as an env-var-only migration. Apply this decision tree: + +**1. App already uses OpenAI Responses API** (`responses.create`) +→ Mantle is the cleanest path. Env var swap + model string change. Minimal code changes. + +**2. App uses Assistants API only for stateful multi-turn conversation** (no hosted tools, no file search, no code interpreter, no persistent Assistant objects, no complex run lifecycle) +→ Mantle Responses API is viable. Requires migrating from `threads`/`runs` calls to `responses.create` — this is a small API migration (days), not a full redesign. Not a zero-code-change swap. + +**3. App uses Assistants API with simple hosted tools** (function calling only, no file search or code interpreter) +→ Mantle Responses API with tool use is viable. Moderate code migration (1-2 weeks) to adapt tool definitions and run lifecycle. + +**4. App uses Assistants API with file search, vector stores, code interpreter, persistent Assistant objects, or complex run lifecycle management** +→ Do not recommend Mantle. Evaluate: Bedrock Agents (sessions, action groups, knowledge bases) for full agentic replacement (2-4 week migration), or app-managed orchestration if the team prefers to own state. + +**When to prefer Converse API over Mantle:** If you need Bedrock-specific features (Guardrails, Knowledge Bases, prompt caching, Bedrock Agents integration) or your target region doesn't have Mantle. Mantle is the fastest path; Converse API is the most feature-complete path. + +### GPT-5.4 → Claude Sonnet 4.6 + +Near price parity (~5% difference). Migration case is driven by AWS consolidation, agentic reliability, or prompt caching — not cost. Both have ~200K+ context. Low risk. + +### GPT-5.4 Mini/Nano → Nova Lite/Micro + +87-94% savings. Strong cost case at any volume. Nova Lite (300K context) covers most GPT-5.4 Mini use cases. + ### GPT-4/4 Turbo → Claude Sonnet 4.6 70-90% savings, similar or better quality, longer context (200K vs 128K). Low risk. @@ -141,9 +225,9 @@ Similar cost, dramatically better quality, 1M context (vs 16K). Tier by complexity: simple → Nova Micro/Llama 4 Scout (60%), moderate → Llama 4 Maverick/Nova Pro (30%), complex → Claude Sonnet (10%). 85-95% savings. -### Pro models → Nova Premier +### Pro models → Nova 2 Pro -80-98% savings. Strong migration case at any volume. +83-98% savings. Strong migration case at any volume. (Nova Premier v1 is Legacy — use Nova 2 Pro instead.) --- diff --git a/migration-to-aws/steering/design-ref-ai.md b/migration-to-aws/steering/design-ref-ai.md index 3585ce1a..0a8d6ae3 100644 --- a/migration-to-aws/steering/design-ref-ai.md +++ b/migration-to-aws/steering/design-ref-ai.md @@ -6,8 +6,8 @@ If the detected AI workload is LLM-based (generative models), load the source-specific design reference instead of this file: -- If `ai-workload-profile.json` → `summary.ai_source` = `"gemini"`: load `steering/design-ref-ai-gemini-to-bedrock.md` -- If `ai-workload-profile.json` → `summary.ai_source` = `"openai"`: load `steering/design-ref-ai-openai-to-bedrock.md` +- If `ai-workload-profile.json` → `summary.ai_source` = `"gemini"`: load `design-ref-ai-gemini-to-bedrock.md` +- If `ai-workload-profile.json` → `summary.ai_source` = `"openai"`: load `design-ref-ai-openai-to-bedrock.md` - If `ai-workload-profile.json` → `summary.ai_source` = `"both"`: load both files - If `ai-workload-profile.json` → `summary.ai_source` = `"other"` or absent, OR if the workload is traditional ML (custom models, Vision API, Speech API): use the SageMaker/Rekognition/Textract rubric below. @@ -26,7 +26,7 @@ If the detected AI workload is LLM-based (generative models), load the source-sp - **Image classification, OCR** → AWS Rekognition (images) or Textract (OCR) - **Document understanding** → AWS Textract (more powerful for docs) -### Cloud ML Engine (deprecated — now part of Vertex AI) +### Cloud ML Engine (deprecated — legacy Terraform configs only) - **Model training** → SageMaker (managed training jobs) - **AutoML** → SageMaker Autopilot / Canvas diff --git a/migration-to-aws/steering/design-ref-compute.md b/migration-to-aws/steering/design-ref-compute.md index 9ab04325..53ff7dc3 100644 --- a/migration-to-aws/steering/design-ref-compute.md +++ b/migration-to-aws/steering/design-ref-compute.md @@ -1,308 +1,132 @@ -# Compute Mappings +# Compute Services Design Rubric -## google_cloud_run_service / google_cloud_run_v2_service +**Applies to:** Cloud Run (v1/v2), Cloud Functions (Gen 1/Gen 2), Compute Engine, GKE, App Engine -**Purpose:** Serverless container execution platform +**Table lookup first:** Check `design-ref-fast-path.md` **Direct Mappings** for this Terraform type. -**Default:** aws_ecs_service (ECS Fargate) +- `google_cloud_run_service`, `google_cloud_run_v2_service`, `google_cloudfunctions_function`, and `google_cloudfunctions2_function` are currently in Direct Mappings and usually resolve with `confidence: "deterministic"` when row conditions are met. +- `google_compute_instance`, `google_container_cluster`, and `google_app_engine_application` are not direct-mapped in `design-ref-fast-path.md`; use the rubric below (typically `confidence: "inferred"`). +- If a resource is not eligible for Direct Mappings (or row conditions are not met), use the rubric below. -**Rationale for default:** Fargate is the recommended AWS equivalent to Cloud Run — fully managed containers, auto-scaling, better dev/prod parity than App Runner, broader feature set (service discovery, load balancer integration, capacity providers). +## Eliminators (Hard Blockers) -**Candidates:** -- aws_ecs_service (ECS Fargate) -- aws_apprunner_service (App Runner) -- aws_lambda_function (Lambda) - -**Signals:** - -| Config Field | Value | Favors | Weight | Reason | -|---|---|---|---|---| -| timeout | > 900s | aws_ecs_service | strong | Exceeds Lambda maximum, App Runner max is 120s for request timeout | -| timeout | <= 120s | aws_apprunner_service | moderate | Within App Runner and Lambda limits, App Runner is simplest | -| timeout | 121s - 900s | aws_ecs_service | moderate | Exceeds App Runner, within Lambda, but long timeout suggests long-running work | -| memory | > 10240MB | aws_ecs_service | strong | Exceeds Lambda maximum (10GB) | -| memory | <= 3008MB | aws_lambda_function | weak | Within Lambda limits, but not a strong signal alone | -| min_instances | > 0 | aws_ecs_service | moderate | Always-on workload, Fargate more cost-effective than Lambda provisioned concurrency | -| min_instances | 0 or absent | aws_apprunner_service | weak | Scale-to-zero, all three support this | -| max_instances | > 100 | aws_ecs_service | moderate | High scale needs, Fargate has more predictable scaling | -| concurrency | 1 | aws_lambda_function | moderate | Single-request processing aligns with Lambda model | -| concurrency | > 80 | aws_ecs_service | moderate | High concurrency per instance suggests long-lived container | -| gpu | required | aws_ecs_service | strong | Only Fargate supports GPU tasks | -| allow_public | false | aws_ecs_service | moderate | Internal-only services are simpler on Fargate with private ALB | - -**Eliminators:** -- timeout > 900s → eliminates aws_lambda_function (Lambda max 900s) -- timeout > 120s → eliminates aws_apprunner_service (App Runner request timeout max 120s) -- memory > 10240MB → eliminates aws_lambda_function (Lambda max 10GB) -- memory > 30720MB → eliminates aws_apprunner_service (App Runner max 30GB) -- gpu required → eliminates aws_lambda_function and aws_apprunner_service - -**Peek at secondaries:** No - -**1:Many Expansion:** - -If App Runner: -- aws_apprunner_service — primary -- aws_apprunner_auto_scaling_configuration_version — if custom scaling needed -- aws_apprunner_vpc_connector — if VPC access needed -- aws_ecr_repository — for container image storage - -If Lambda: -- aws_lambda_function — primary -- aws_lambda_function_url — if public HTTP endpoint needed (replaces Cloud Run URL) -- aws_cloudwatch_log_group — Lambda logging -- aws_ecr_repository — if using container image (vs zip deployment) - -If ECS Fargate: -- aws_ecs_service — primary -- aws_ecs_task_definition — container configuration -- aws_ecs_cluster — if no cluster exists yet (shared across services) -- aws_lb — Application Load Balancer for HTTP traffic -- aws_lb_target_group — routing to ECS tasks -- aws_lb_listener — HTTP/HTTPS listener -- aws_ecr_repository — for container image storage -- aws_cloudwatch_log_group — ECS logging - -**Source Config to Carry Forward:** -- container_image — determines image source (GCR/AR → ECR migration needed) -- memory — determines memory allocation -- cpu — determines CPU allocation -- timeout — determines timeout configuration -- min_instances — determines min capacity / desired count -- max_instances — determines max capacity / auto-scaling limits -- concurrency — determines target tracking scaling metric -- env_vars — determines environment variables (references may need updating) -- secret_env_vars — determines secrets references (Secret Manager ARNs) -- vpc_connector — determines VPC configuration -- allow_public — determines load balancer scheme (internet-facing vs internal) -- service_account — determines IAM role (resolved during secondary mapping) - ---- - -## google_container_cluster - -**Purpose:** Kubernetes container orchestration - -**Default:** aws_ecs_service (ECS Fargate) - -**Rationale for default:** ECS Fargate is the standard AWS migration target for GKE workloads — managed container execution, no node management, integrates natively with VPC, ALB, and IAM. - -**Candidates:** -- ECS Fargate (via `aws_ecs_service` + `aws_ecs_task_definition`) -- ECS EC2 (same resources, different launch type) -- EKS (`aws_eks_cluster`) - -**Preference shortcuts (check preferences.json first):** -- `compute: "eks"` → always EKS. Skip all other criteria. -- `kubernetes: "eks-managed"` → EKS. -- `kubernetes: "ecs-fargate"` → ECS Fargate. -- `kubernetes: "eks-or-ecs"` → continue rubric evaluation. - -**Eliminators:** -- GPU node pools (any `google_container_node_pool` with accelerators configured) → eliminates ECS Fargate (limited GPU support), favor ECS EC2 -- 10+ node pools of different machine families → eliminates ECS Fargate and ECS EC2, favor EKS (complex topology requires Kubernetes orchestration) - -**Signals:** - -| Signal | Favors | Weight | Reason | -|---|---|---|---| -| Single node pool, standard machine type (e2-standard, n1-standard) | ECS Fargate | strong | Straightforward workload, no need for node management | -| Multiple node pools | EKS | moderate | Complex topology is harder to replicate on ECS | -| Cluster description mentions "web service", "API", "backend" | ECS Fargate | moderate | Typical ECS Fargate workload | -| High node count (initial or max > 20) | EKS | moderate | Large fleets benefit from Kubernetes orchestration | -| Workload Insights or Dataplane V2 features used | EKS | weak | Advanced networking features map better to EKS | - -**1:Many Expansion — ECS Fargate (most common case):** - -**MANDATORY — always add ALL of the following to `new_aws_resources` when mapping a GKE cluster to ECS:** - -| Resource | Always required? | Condition | -|---|---|---| -| `aws_ecs_cluster` | Yes, unless already created by a prior cluster | First ECS cluster in the migration | -| `aws_ecs_service` | Yes | One per node pool | -| `aws_ecs_task_definition` | Yes | One per node pool | -| `aws_ecr_repository` | **Always** | Container images must be re-pushed from GCR/Artifact Registry to ECR. Number of repos = number of distinct workloads. Cannot run on ECS without ECR. | -| `aws_lb` | **Always** (unless cluster is explicitly documented as internal-only with no external traffic) | GKE has implicit ingress. ECS has no external entry point without ALB. | -| `aws_lb_target_group` | Yes, with ALB | One per service family | -| `aws_lb_listener` | Yes, with ALB | HTTP port 80 and HTTPS port 443 | -| `aws_appautoscaling_target` | When node pool has `autoscaling` block | Registers ECS service as a scalable target | -| `aws_appautoscaling_policy` | When `aws_appautoscaling_target` is added | Defines scaling behavior (target tracking on CPU/memory) | - -**PROHIBITED omissions:** -- Never omit `aws_ecr_repository`. Container images cannot be pulled from GCR on ECS. -- Never omit `aws_lb` unless the cluster is explicitly documented as internal-only. "I don't see an ingress resource" is not a sufficient reason to omit the ALB — GKE clusters with external workloads rely on the cluster's default ingress. -- Never omit `aws_appautoscaling_target` + `aws_appautoscaling_policy` when node pool autoscaling is configured. - -**1:Many Expansion — EKS:** -- `aws_eks_cluster` — primary -- `aws_eks_node_group` — one per `google_container_node_pool` -- `aws_iam_role` — EKS cluster role -- `aws_iam_role` — EKS node group role -- `aws_ecr_repository` — always required (same reason as ECS) - -**Source Config to Carry Forward:** -- name — cluster name -- location / region — determines AWS region and AZ placement -- network — determines VPC placement (resolved from google_compute_network mapping) -- subnetwork — determines subnet placement -- logging_service / monitoring_service — determines CloudWatch logging/monitoring config -- min_master_version — Kubernetes version reference -- workload_identity_config — determines IAM trust policy configuration (absorbed into task role) - ---- - -## google_container_node_pool - -**This is a secondary resource. Its target depends on what `google_container_cluster` mapped to.** - -**CRITICAL: `google_container_node_pool` is NOT eligible for Pass 1. It must be evaluated in Pass 2 AFTER its parent cluster is resolved.** - -**When cluster → ECS Fargate or ECS EC2:** -- **Target: `absorbed`** -- Node pool parameters translate into ECS task definition config and service scaling config — there are no nodes to manage on Fargate. -- Node pool does NOT become a 1:1 AWS resource. -- `aws_ecs_service` and `aws_ecs_task_definition` appear in `new_aws_resources` of the cluster, not as direct mapping targets of the node pool. - -Carry forward these fields into the reason (they inform task definition and service config): -- `machine_type` — translates to vCPU/memory allocation for the task definition -- `disk_size_gb` — informs ephemeral storage config on the task -- `min_node_count` — becomes minimum task count / desired count baseline -- `max_node_count` — becomes maximum task count for auto-scaling policy -- `initial_node_count` — use as desired count if no billing data is available -- `autoscaling` block present → trigger `aws_appautoscaling_target` + `aws_appautoscaling_policy` in `new_aws_resources` - -**When cluster → EKS:** -- **Target: `aws_eks_node_group`** -- One node pool → one EKS managed node group. -- Carry forward: `machine_type`, `min_node_count`, `max_node_count`, `disk_size_gb`, `node_count` - ---- - -## google_compute_instance - -**Purpose:** Virtual machines - -**Default:** aws_instance (EC2) - -**Candidates:** -- aws_instance (EC2) - -**Signals:** None — direct 1:1 mapping. EC2 is the only candidate. - -**Eliminators:** None - -**Peek at secondaries:** No - -**1:Many Expansion:** -- aws_instance — primary -- aws_ebs_volume — if additional disks attached (beyond root) -- aws_eip — if static external IP assigned - -**Source Config to Carry Forward:** -- machine_type — determines instance_type (translation needed by Pillar 4) -- zone — determines availability_zone -- boot_disk.image — determines ami -- boot_disk.size — determines root volume size -- network_interface.network — determines VPC/subnet placement -- metadata.startup_script — determines user_data -- tags — determines instance tags -- service_account — determines IAM instance profile (resolved during secondary mapping) - ---- - -## google_cloudfunctions_function / google_cloudfunctions2_function - -**Purpose:** Serverless functions - -**Default:** aws_lambda_function (Lambda) - -**Candidates:** -- aws_lambda_function (Lambda) - -**Signals:** None — direct 1:1 mapping. Cloud Functions and Lambda are equivalent models. - -**Eliminators:** -- timeout > 900s (Cloud Functions 2nd gen supports up to 3600s) → Lambda cannot handle this, flag for user. Recommend ECS Fargate as alternative. -- memory > 10240MB → Lambda cannot handle this, flag for user. - -**Peek at secondaries:** No - -**1:Many Expansion:** -- aws_lambda_function — primary -- aws_cloudwatch_log_group — logging -- aws_lambda_event_source_mapping — if triggered by Pub/Sub (maps to SQS trigger) -- aws_lambda_permission — if triggered by HTTP (API Gateway or function URL) - -**Source Config to Carry Forward:** -- runtime — determines runtime -- entry_point — determines handler -- memory_mb — determines memory_size -- timeout — determines timeout -- environment_variables — determines environment variables -- vpc_connector — determines VPC configuration -- source_archive_bucket / source_archive_object — determines deployment package source -- trigger_http — determines function URL or API Gateway need -- event_trigger — determines event source mapping - ---- - -## Secondary: google_service_account - -**Behavior:** Service accounts are identity secondaries that provide IAM credentials to compute resources. They map to AWS IAM roles based on what the primary compute resource became. - -**Mapping Behavior:** - -When google_cloud_run_service maps to: -- aws_apprunner_service → service account maps to aws_iam_role (execution role for App Runner) -- aws_lambda_function → service account maps to aws_iam_role (execution role for Lambda) -- aws_ecs_service → service account maps to aws_iam_role (task execution role for ECS) - -When google_cloudfunctions_function maps to: -- aws_lambda_function → service account maps to aws_iam_role (execution role for Lambda) - -When google_container_cluster maps to: -- aws_eks_cluster → service account maps to aws_iam_role (IRSA - IAM Roles for Service Accounts via OpenID Connect provider) -- aws_ecs_cluster → service account maps to aws_iam_role (cluster execution role) - -When google_compute_instance maps to: -- aws_ec2_instance → service account maps to aws_iam_role (instance profile attached to EC2) - -**Implementation Notes:** -1. Service account IAM bindings (google_project_iam_member, google_*_iam_member) become policy statements on the AWS role -2. Service account email becomes role assume role policy (who can assume this role) -3. Service account keys should be migrated to AWS Secrets Manager -4. For Kubernetes (EKS), uses IRSA to bind service accounts to IAM roles via OpenID Connect - -**Skip Condition:** -If the primary compute resource is skipped or not mapped to AWS, skip this service account. - ---- - -## google_app_engine_application - -**Purpose:** Fully managed application hosting platform - -**Default:** aws_ecs_service (ECS Fargate) - -**Rationale for default:** App Engine applications are typically web apps or APIs. Fargate provides similar managed container hosting with auto-scaling, no server management, and better integration with the broader AWS ecosystem. - -**Candidates:** -- aws_ecs_service (ECS Fargate) — general-purpose, best for most App Engine apps -- aws_amplify_app (Amplify Hosting) — for static/JAMstack apps with backend APIs -- aws_apprunner_service (App Runner) — simpler setup, fewer configuration options - -**Signals:** - -| Signal | Favors | Weight | Reason | -|---|---|---|---| -| Standard environment (Python, Java, Node.js, Go, PHP, Ruby) | Fargate | strong | Container-based deployment matches standard env | -| Flexible environment (custom runtime/Dockerfile) | Fargate | strong | Custom containers are Fargate's strength | -| Static site with API backend | Amplify | moderate | Amplify handles static hosting + API routes | -| Simple web service, minimal config | App Runner | weak | App Runner is simpler but less configurable | - -**1:Many Expansion (Fargate):** -- aws_ecs_service -- aws_ecs_task_definition -- aws_lb (Application Load Balancer) -- aws_lb_target_group -- aws_lb_listener +| GCP Service | AWS | Blocker | +| --------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Cloud Run | Lambda | Execution time >15 min → use Fargate | +| Cloud Run | Fargate | GPU workload or >16 vCPU or >120 GB memory → use EC2 | +| Cloud Functions | Lambda | Python version not supported (e.g., Python 2.7) → use custom runtime on Fargate | +| GKE | EKS | Custom CRI incompatible → manual workaround or ECS | +| Any | App Runner | **Closed to new customers (April 30 2026).** Do not target App Runner for new migrations. Use Fargate (default), Lambda (event-driven), or EKS (K8s required). | + +## Signals (Decision Criteria) + +### Cloud Run / App Engine + +- **Always-on** or **cold-start sensitive** → Fargate (not Lambda) +- **Stateless microservice** + **<15 min execution** → Lambda +- **HTTP-only** + **container-native** → Fargate preferred (better dev/prod parity) + +### Cloud Functions + +- **Event-driven** + **<15 min** + **Python/Node/Go** → Lambda +- **Always-on or long** → run as Container on Fargate or ECS + +### Compute Engine (VMs) + +- **Always-on workload** → EC2 (reserved or on-demand based on cost sensitivity) +- **Batch/periodic jobs** → EC2 with Auto Scaling (scale to 0 in dev) +- **Windows-only workload** → EC2 (Lambda/Fargate support limited) + +### GKE + +- **Kubernetes orchestration explicitly required** (`kubernetes = "eks-managed"` or `"eks-or-ecs"` in `preferences.json`) → EKS +- **Default / no explicit K8s preference** (`kubernetes = "ecs-fargate"` or absent): + - → **Fargate** (absent kubernetes preference resolves to Fargate, not EKS — teams that want EKS answer A or B in Clarify) + +## 6-Criteria Rubric + +Apply in order; first match wins: + +1. **Eliminators**: Does GCP config violate AWS constraints? If yes: switch to alternative +2. **Operational Model**: Managed (Lambda, Fargate) vs Self-Hosted (EC2, EKS)? + - Prefer managed unless: Always-on + high baseline cost → EC2 +3. **User Preference**: From `preferences.json`: `design_constraints.kubernetes`, `design_constraints.cost_sensitivity`? + - If `kubernetes = "eks-managed"` → EKS (preserves K8s investment) + - If `kubernetes = "eks-or-ecs"` → EKS with managed node groups (user is competent with K8s) + - If `kubernetes = "ecs-fargate"` → Fargate (simpler managed containers) + - If `kubernetes` is **absent** → Fargate (treat same as `"ecs-fargate"` — do not default to EKS) + - If `cost_sensitivity` present and high → prefer Fargate (lower operational cost) +4. **Feature Parity**: Does GCP config require AWS-unsupported features? + - Example: GCP auto-scaling to zero + cold-start-sensitive → Fargate (not Lambda) +5. **Cluster Context**: Are other resources in this cluster using EKS/EC2/Fargate? + - Prefer same platform (affinity) +6. **Simplicity**: Fewer resources = higher score + - Fargate (1 service) > EC2 (N services for ASG + monitoring) + +## Examples + +### Example 1: Cloud Run (stateless API) + +- GCP: `google_cloud_run_service` (memory=512MB, timeout=60s, min_instances=1) +- Signals: HTTP, stateless, always-on +- Criterion 1 (Eliminators): PASS (60s < 15min doesn't apply; stateless OK) +- Criterion 2 (Operational Model): FARGATE preferred +- → **AWS: Fargate (0.5 CPU, 1 GB memory)** +- Confidence: `inferred` (rubric-based — Cloud Run is not in fast-path) + +### Example 2a: Cloud Functions (event processor, short-running) + +- GCP: `google_cloudfunctions_function` (runtime=python39, timeout=540s) +- Signals: Event-driven, 540s = 9 minutes (< 15min limit) +- Criterion 1 (Eliminators): PASS on timeout (540s < 900s) +- Criterion 2 (Operational Model): Lambda preferred for event-driven + short-running +- → **AWS: Lambda with EventBridge trigger** +- Confidence: `inferred` + +### Example 2b: Cloud Functions (long-running batch processor) + +- GCP: `google_cloudfunctions_function` (runtime=python39, timeout=1200s) +- Signals: Event-driven but 1200s = 20 minutes (> 15min limit) +- Criterion 1 (Eliminators): FAIL on timeout (1200s > 900s) → **cannot use Lambda** +- Criterion 2 (Operational Model): Fargate (managed + can handle longer execution) +- → **AWS: Fargate (0.5 CPU, 1 GB memory) with EventBridge trigger** +- Confidence: `inferred` + +### Example 3: Compute Engine (background job) + +- GCP: `google_compute_instance` (machine_type=e2-medium, region=us-central1, startup_script=...) +- Signals: Periodic batch job (inferred from startup script), always-on +- Criterion 1 (Eliminators): PASS +- Criterion 2 (Operational Model): EC2 (explicit compute control) +- Criterion 3 (User Preference): If `design_constraints.gcp_monthly_spend` indicates cost sensitivity, prefer auto-scaling → EC2 + ASG (scale to 0) +- → **AWS: EC2 t3.medium + Auto Scaling Group (min=0 in dev)** +- Confidence: `inferred` + +## Output Schema + +```json +{ + "gcp_type": "google_cloud_run_service", + "gcp_address": "example-service", + "gcp_config": { + "memory_mb": 512, + "timeout_seconds": 60 + }, + "aws_service": "Fargate", + "aws_config": { + "cpu": "0.5", + "memory_mb": 1024, + "region": "us-east-1" + }, + "confidence": "inferred", + "rationale": "Rubric: Cloud Run (stateless, <15min) → Fargate (always-on, managed)", + "rubric_applied": [ + "Eliminators: PASS", + "Operational Model: Managed preferred", + "User Preference: N/A", + "Feature Parity: Full", + "Cluster Context: Fargate affinity", + "Simplicity: Fargate (1 service)" + ] +} +``` diff --git a/migration-to-aws/steering/design-ref-database.md b/migration-to-aws/steering/design-ref-database.md index 8614e823..234a2dcc 100644 --- a/migration-to-aws/steering/design-ref-database.md +++ b/migration-to-aws/steering/design-ref-database.md @@ -1,240 +1,152 @@ -# Database Mappings +# Database Services Design Rubric -## google_sql_database_instance - -**Purpose:** Managed relational database (MySQL, PostgreSQL) - -**Default:** aws_db_instance (RDS) or aws_rds_cluster (Aurora) - -**Rationale for default:** AWS RDS is the closest equivalent to Cloud SQL — fully managed, multi-AZ support, automated backups. Aurora is preferred for higher scale requirements. +**Applies to:** Cloud SQL, Firestore, BigQuery, Memorystore (Redis), Cloud Spanner -**Candidates:** -- aws_db_instance (RDS MySQL or RDS PostgreSQL) -- aws_rds_cluster (Aurora MySQL or Aurora PostgreSQL) +**Cloud SQL PostgreSQL/MySQL:** Always use this rubric (confidence = `inferred`). **Q6 (`design_constraints.availability`) selects RDS vs Aurora** — see mapping table below. Q12/Q13 tune sizing within that family; they never override Q6. -**Signals:** +## Q6 → RDS vs Aurora (mandatory) -| Config Field | Value | Favors | Weight | Reason | -|---|---|---|---|---| -| database_version | >= MySQL 8.0 or PostgreSQL 13+ | aws_rds_cluster | moderate | Aurora supports latest versions better | -| backup_configuration.enabled | true | aws_db_instance or aws_rds_cluster | weak | Both support backups equally | -| backup_configuration.point_in_time_recovery_enabled | true | aws_rds_cluster | moderate | Aurora has superior PITR capabilities | -| settings.backup_config.transaction_log_retention_days | > 7 | aws_rds_cluster | moderate | Aurora supports longer retention | -| settings.ip_configuration.require_ssl | true | aws_db_instance or aws_rds_cluster | weak | Both enforce SSL | -| tier | Scaling for high IOPS | aws_rds_cluster | moderate | Aurora better for scaling to high I/O | -| scale (inferred from current load) | Large (> 100 GB, > 10K QPS) | aws_rds_cluster | strong | Aurora handles scale better | -| scale (inferred from current load) | Standard (< 100 GB, < 5K QPS) | aws_db_instance | moderate | RDS sufficient for smaller workloads | +Read `preferences.json` → `design_constraints.availability` **before** selecting `aws_service`: -**Eliminators:** -- Custom engine features not in AWS → flag for user, may need workaround +| `availability` | PostgreSQL target | MySQL target | `multi_az` in aws_config | +| -------------- | --------------------------------- | ---------------------------- | ------------------------ | +| `single-az` | RDS PostgreSQL | RDS MySQL | `false` | +| `multi-az` | RDS PostgreSQL | RDS MySQL | `true` | +| `multi-az-ha` | Aurora PostgreSQL | Aurora MySQL | `true` | +| `multi-region` | Aurora PostgreSQL Global Database | Aurora MySQL Global Database | global cluster | -**Peek at secondaries:** Yes — check for google_sql_database resources and backup configurations +Engine (PostgreSQL vs MySQL) comes from GCP `database_version`, not from Q12/Q13. -**1:Many Expansion (RDS):** -- aws_db_instance — primary -- aws_db_parameter_group — if custom parameters needed -- aws_db_option_group — if DB options needed -- aws_db_subnet_group — for multi-AZ placement -- aws_security_group — for database access control +## Eliminators (Hard Blockers) -**1:Many Expansion (Aurora):** -- aws_rds_cluster — cluster definition -- aws_rds_cluster_instance — one or more instances (2+ for HA) -- aws_rds_cluster_parameter_group — custom cluster parameters -- aws_db_subnet_group — for multi-AZ placement -- aws_security_group — for database access control -- aws_rds_cluster_backup — if backing up to specific window needed +| GCP Service | AWS | Blocker | +| ---------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Firestore | DynamoDB | ACID transactions spanning >100 items required → use RDS (DynamoDB limit: 100 items/transaction) | +| BigQuery | _(no auto-target)_ | **Plugin does not prescribe Athena/Redshift/Glue** — use `Deferred — specialist engagement` in design output; OLTP latency needs → Aurora or DynamoDB per workload review with specialists | +| Cloud SQL (PostgreSQL) | RDS or Aurora | PostGIS extension → supported on both RDS PostgreSQL and Aurora PostgreSQL | -**Source Config to Carry Forward:** -- database_version — determines engine/engine_version -- tier — determines instance_class or cluster instance type -- region — determines AWS region -- availability_type — determines multi_az (HA requirement) -- backup_configuration — determines backup retention and window -- settings.ip_configuration — determines db_subnet_group and security groups -- name — determines database name -- user credentials — determines master username/password (migrate to Secrets Manager) - ---- - -## google_firestore_database - -**Purpose:** NoSQL document database with real-time capabilities +## Signals (Decision Criteria) -**Default:** aws_dynamodb_table (DynamoDB) +### Cloud SQL -**Rationale for default:** DynamoDB is the closest AWS equivalent to Firestore — serverless NoSQL, auto-scaling, document-oriented data model. Similar pricing model (read/write units). +- **Product family** → **Q6 only** (see table above). Do not default to Aurora. +- **RDS path** (`single-az`, `multi-az`): Size from current Cloud SQL tier; `database_traffic` and `db_io_workload` select instance class, optional read replicas, and storage type (**gp3** default; **io2** / Provisioned IOPS when `db_io_workload = "high"`). +- **Aurora path** (`multi-az-ha`, `multi-region`): Apply Q12 traffic and Q13 I/O for Aurora Standard vs I/O-Optimized, read replicas, Serverless v2, or DSQL review. +- **Dev/low spend** → RDS `db.t4g.micro` or `db.t4g.small` single-AZ is appropriate when Q6 = Inconvenient — do not upsell to Aurora Serverless v2. +- **Migration tooling** — Read `preferences.json` → `design_constraints.db_size.value` (set by Q13b in Clarify) to select the right tool: `"<10GB"` → pg_dump/pg_restore; `"10-100GB"` or `"100-500GB"` → pgcopydb; `">500GB"` → AWS DMS strongly recommended; `"unknown"` → default to pgcopydb -**Candidates:** -- aws_dynamodb_table (DynamoDB) -- aws_rds_cluster (Aurora PostgreSQL — if relational features critical) +### Firestore -**Signals:** +- **Flexible schema** + **NoSQL** → DynamoDB +- **Strong consistency required** → DynamoDB supports strongly consistent reads via `ConsistentRead` parameter +- **Real-time sync** + **offline support** → DynamoDB Streams + Amplify (app-level) -| Config Field | Value | Favors | Weight | Reason | -|---|---|---|---|---| -| Data model | Document/JSON-like | aws_dynamodb_table | strong | Firestore's natural equivalent | -| Data model | Highly relational (lots of joins) | aws_rds_cluster | strong | Aurora handles complex relationships better | -| Query patterns | Mostly key-value reads | aws_dynamodb_table | strong | DynamoDB's strength | -| Query patterns | Range queries, filters | aws_dynamodb_table | moderate | DynamoDB GSI handles this, but more complex | -| Query patterns | Complex queries, aggregations | aws_rds_cluster | strong | Aurora PostgreSQL better for analytics | -| Real-time listeners | Required | aws_dynamodb_table + aws_kinesis | strong | DynamoDB Streams + Lambda for real-time | -| Real-time listeners | Not required | aws_dynamodb_table | moderate | Standard DynamoDB sufficient | -| Indexes | Multiple custom indexes | aws_dynamodb_table | moderate | DynamoDB supports up to 10 GSI | -| Indexes | Complex multi-column indexes | aws_rds_cluster | moderate | Aurora handles complex indexes better | +### BigQuery -**Eliminators:** -- ACID transactions across unrelated documents → DynamoDB transactions are limited (up to 25 items), consider Aurora instead +**Do not use this rubric to pick an AWS product.** For any `google_bigquery_*` resource, follow **`design-infra.md` → BigQuery specialist gate** only: set `aws_service` to **`Deferred — specialist engagement`**, `human_expertise_required: true`, and direct the customer to **their AWS account team and/or a data analytics migration partner**. Do **not** output Athena, Redshift, Glue, EMR, or similar as the automated mapping in `aws-design.json`. -**Peek at secondaries:** No +The sections below are **background for humans** after engagement — not for the agent to select automatically: -**1:Many Expansion (DynamoDB):** -- aws_dynamodb_table — primary -- aws_dynamodb_global_table — if multi-region replication needed -- aws_kinesis_stream — if real-time processing needed (for Streams) -- aws_lambda_function — for Streams processing (real-time listeners) - -**1:Many Expansion (Aurora PostgreSQL):** -- aws_rds_cluster — primary -- aws_rds_cluster_instance — 2+ instances for HA -- aws_db_subnet_group — for placement -- aws_security_group — for access control - -**Source Config to Carry Forward:** -- Collection structure — informs table design and GSI strategy -- Document structure — informs item schema -- Indexes — informs DynamoDB GSI configuration -- Real-time listeners — informs whether Streams and Lambda are needed - ---- - -## google_bigquery_dataset - -**Purpose:** Data warehouse and analytics - -**Default:** aws_athena_workgroup (Athena) or aws_redshift_cluster (Redshift) - -**Rationale for default:** Athena is the closest BigQuery equivalent for ad-hoc queries on data in S3 (similar pay-per-query model). Redshift is preferred for frequent/complex interactive queries requiring consistent performance. - -**Candidates:** -- aws_athena_workgroup (Athena — query data in S3) -- aws_redshift_cluster (Redshift — data warehouse cluster) - -**Signals:** - -| Config Field | Value | Favors | Weight | Reason | -|---|---|---|---|---| -| Query pattern | Infrequent, ad-hoc analytics | aws_athena_workgroup | strong | Athena's strength; similar to BigQuery pay-per-query | -| Query pattern | Frequent interactive queries (dashboards) | aws_redshift_cluster | strong | Redshift's consistency and performance | -| Data volume | Large (> 100 GB) | aws_athena_workgroup or aws_redshift_cluster | weak | Both handle large data; choice depends on query pattern | -| Data volume | Massive (> 1 TB, > 10K queries/month) | aws_redshift_cluster | moderate | Redshift more cost-effective at scale | -| Query complexity | Simple aggregations | aws_athena_workgroup | moderate | Athena sufficient | -| Query complexity | Complex, multi-table joins, UDFs | aws_redshift_cluster | moderate | Redshift better for complex queries | -| SLA requirements | Consistent sub-second performance | aws_redshift_cluster | strong | Redshift cluster SLA > Athena variable performance | -| Cost sensitivity | Budget-conscious, sporadic usage | aws_athena_workgroup | strong | Pay only for data scanned | -| Cost sensitivity | Predictable workload | aws_redshift_cluster | moderate | Cluster cost predictable once sized | - -**Eliminators:** -- Real-time data ingestion patterns (streaming) → Neither Athena nor Redshift ideal; consider Kinesis + DynamoDB instead - -**Peek at secondaries:** Yes — check for linked datasets, scheduled queries, or Pub/Sub triggers - -**1:Many Expansion (Athena):** -- aws_athena_workgroup — query execution workgroup -- aws_s3_bucket — data storage (linked from data_source) -- aws_glue_catalog_database — metadata catalog -- aws_glue_catalog_table — table definitions -- aws_athena_named_query — saved queries - -**1:Many Expansion (Redshift):** -- aws_redshift_cluster — primary data warehouse -- aws_redshift_subnet_group — cluster placement -- aws_security_group — access control -- aws_redshift_parameter_group — cluster parameters -- aws_s3_bucket — for external table data (Redshift Spectrum) - -**Source Config to Carry Forward:** -- Dataset name and description — determines Athena workgroup name or Redshift database name -- Table schemas — informs Glue catalog design (Athena) or Redshift table definitions -- Scheduled queries — informs Lambda/EventBridge triggers (Athena) or Redshift scheduled queries -- Linked sources — informs S3 bucket configuration (Athena) or external table setup (Redshift) - ---- - -## Secondary: google_sql_database - -**Behavior:** SQL databases are configuration secondaries that define which databases are created within a Cloud SQL instance. They are absorbed into the primary database resource configuration. - -**Mapping Behavior:** - -When google_sql_database_instance maps to: -- aws_db_instance (RDS) → Each google_sql_database is absorbed as `db_name` during RDS initialization -- aws_rds_cluster (Aurora) → Each google_sql_database is absorbed as `database_name` during Aurora initialization - -**Implementation:** -1. Database names and character sets are carried forward to AWS resource configuration -2. Database creation SQL (`CREATE DATABASE ...`) runs during RDS/Aurora initialization -3. No separate aws_db resource is created -4. Multiple databases on same instance each become their own initialization - -**Skip Condition:** -If the primary google_sql_database_instance is skipped or not mapped to AWS, skip these secondaries. - ---- - -## Secondary: google_sql_user - -**Behavior:** SQL users are configuration secondaries that define database authentication credentials. They map to AWS Secrets Manager for secure credential storage. - -**Mapping Behavior:** - -When google_sql_database_instance maps to: -- aws_db_instance (RDS) → Service account credentials become `master_username` / `master_password` stored in aws_secretsmanager_secret -- aws_rds_cluster (Aurora) → Service account credentials become `master_username` / `master_password` stored in aws_secretsmanager_secret - -**Implementation:** -1. GCP service account email/password becomes AWS master user credentials -2. Credentials must be stored in AWS Secrets Manager (not in code/config) -3. Connection strings must be updated with new AWS credentials -4. Application code must retrieve credentials from Secrets Manager - -**Security Note:** -Never commit database passwords. Use Secrets Manager for credential management and rotation. - -**Skip Condition:** -If the primary google_sql_database_instance is skipped, skip these secondaries. - ---- - -## google_redis_instance (Memorystore) - -**Purpose:** In-memory cache and data store - -**Default:** aws_elasticache_replication_group (ElastiCache Redis) - -**Rationale for default:** ElastiCache Redis is the direct 1:1 equivalent to Memorystore Redis. Same Redis engine, managed service with multi-AZ failover. - -**Candidates:** -- aws_elasticache_replication_group (ElastiCache Redis) — always the target - -**This is a fast-path (deterministic) mapping.** No rubric evaluation needed. - -**Signals:** - -| Signal | Favors | Weight | Reason | -|---|---|---|---| -| Cluster mode enabled | ElastiCache Redis with cluster mode | strong | Preserve sharding topology | -| High availability required | ElastiCache Redis Multi-AZ with auto-failover | strong | Match HA configuration | -| Single node, dev/test | ElastiCache Redis (single node) | moderate | Cost optimization for dev | - -**1:Many Expansion:** -- aws_elasticache_replication_group -- aws_elasticache_subnet_group -- aws_elasticache_parameter_group (if custom parameters) - -**Source Config to Carry Forward:** -- tier → determines node type (e.g., BASIC → cache.t4g.micro, STANDARD_HA → cache.r7g.large) -- memory_size_gb → determines node size -- redis_version → determines engine version -- auth_enabled → determines transit encryption and AUTH token -- replica_count → determines number of replicas per shard +- Warehousing, SQL analytics, BI, and ML-on-data choices require assessment (e.g. query patterns, data volume, SLAs, cost model). +- **BigQuery ML** (`google_bigquery_ml_*`) uses the **same specialist gate** — no automated SageMaker/Redshift ML target from this plugin. + +### Memorystore (Redis) + +- **In-memory cache** → ElastiCache Redis (fast-path, 1:1 mapping) +- **Cluster mode enabled** → ElastiCache Redis with cluster mode +- **High availability required** → ElastiCache Redis Multi-AZ with auto-failover + +### Cloud Spanner + +- **Global strong consistency** → Aurora DSQL (distributed SQL with strong consistency across regions) +- **Single-region relational** → Aurora PostgreSQL (simpler, lower cost if global distribution not needed) +- **Key-value access patterns dominant** → DynamoDB Global Tables (if workload is mostly key-value lookups) + +## 6-Criteria Rubric + +Apply in order: + +1. **Eliminators**: Does GCP config require AWS-unsupported features? If yes: switch +2. **Operational Model**: Apply **Q6 availability mapping** first. + - `single-az` / `multi-az` → **RDS** (simpler, lower cost for dev and production with acceptable outage windows) + - `multi-az-ha` / `multi-region` → **Aurora** (faster failover, Global Database when required) +3. **User Preference**: From `preferences.json`: + - `design_constraints.database_traffic` — RDS: size writer/replicas; Aurora: replica count, Serverless v2, DSQL review when write-heavy-global **and** Q6 is Aurora tier + - `design_constraints.db_io_workload` — RDS: gp3 vs io2; Aurora: Standard vs I/O-Optimized (**only when Q6 is Aurora tier**) + - **Never** upgrade RDS → Aurora based on traffic or I/O alone +4. **Feature Parity**: Does GCP config need features unavailable in AWS? + - Example: Cloud SQL with binary log replication → Aurora (full support) when Q6 requires Aurora tier + - Example: Firestore with offline-first SDK → DynamoDB (plus app-level sync) +5. **Cluster Context**: Are other resources in cluster using RDS? Prefer same family +6. **Simplicity**: For Q6 Inconvenient/Significant Issue, prefer RDS over Aurora unless eliminators block + +## Examples + +### Example 1: Cloud SQL PostgreSQL (dev / low HA) + +- GCP: `google_sql_database_instance` (database_version=POSTGRES_15, tier=db-f1-micro) +- Q6: `availability: "single-az"` +- Criterion 2: RDS PostgreSQL single-AZ +- Criterion 3: `db_io_workload: "low"` → gp3 storage +- → **AWS: RDS PostgreSQL (`db.t4g.micro`, single-AZ, gp3)** +- Confidence: `inferred` + +### Example 2: Firestore (mobile app) + +- GCP: `google_firestore_document` (root_path=users, auto_id=true) +- Signals: NoSQL, real-time, offline-first (inferred from Firestore choice) +- Criterion 1 (Eliminators): PASS (DynamoDB supports eventual consistency) +- Criterion 2 (Operational Model): DynamoDB (managed NoSQL) +- Criterion 3 (User Preference): NoSQL type detected from GCP resource → DynamoDB confirmed +- → **AWS: DynamoDB (on-demand billing for dev)** +- Confidence: `inferred` + +### Example 3: BigQuery (analytics) + +- GCP: `google_bigquery_dataset` (location=us, schema=[large table]) +- **Agent output:** `aws_service`: **`Deferred — specialist engagement`**, `human_expertise_required`: **`true`**, `confidence`: **`inferred`**, `rubric_applied`: `["BigQuery specialist gate — no automated AWS service target"]` +- **User-facing:** Engage **AWS account team** and/or **data analytics migration partner** before choosing AWS analytics architecture. **Do not** state Athena vs Redshift vs Glue as the plugin’s recommendation. + +### Example 4: Cloud SQL PostgreSQL (production Multi-AZ RDS) + +- GCP: `google_sql_database_instance` (database_version=POSTGRES_15, tier=db-custom-2-7680) +- Q6: `availability: "multi-az"` +- Criterion 3: `database_traffic: "read-heavy"` → optional RDS read replica +- → **AWS: RDS PostgreSQL Multi-AZ + optional read replica** +- Confidence: `inferred` + +### Example 5: Cloud SQL PostgreSQL (mission-critical) + +- GCP: `google_sql_database_instance` (database_version=POSTGRES_15) +- Q6: `availability: "multi-az-ha"` +- Criterion 3: `db_io_workload: "high"` → Aurora I/O-Optimized +- → **AWS: Aurora PostgreSQL Multi-AZ, I/O-Optimized storage** +- Confidence: `inferred` + +## Output Schema + +```json +{ + "gcp_type": "google_sql_database_instance", + "gcp_address": "dev-postgres-db", + "gcp_config": { + "database_version": "POSTGRES_15", + "tier": "db-f1-micro" + }, + "aws_service": "RDS PostgreSQL", + "aws_config": { + "engine_version": "15", + "instance_class": "db.t4g.micro", + "multi_az": false, + "storage_type": "gp3" + }, + "confidence": "inferred", + "human_expertise_required": false, + "rationale": "Q6 single-az → RDS PostgreSQL; dev-tier sizing from Cloud SQL config; gp3 for low I/O", + "rubric_applied": [ + "User Preference: availability=single-az", + "Operational Model: RDS over Aurora for low-HA workload", + "Simplicity: RDS PostgreSQL single-AZ" + ] +} +``` diff --git a/migration-to-aws/steering/design-ref-fast-path.md b/migration-to-aws/steering/design-ref-fast-path.md index 5bfcbee3..52218418 100644 --- a/migration-to-aws/steering/design-ref-fast-path.md +++ b/migration-to-aws/steering/design-ref-fast-path.md @@ -1,300 +1,113 @@ -# Fast Path: Unconditional Mappings +# Fast-Path: Direct GCP→AWS Mappings -> This file contains deterministic resource mappings that skip the evaluation rubric entirely. Use as a lookup table in Pass 1 of Design phase. -> -> **CRITICAL:** If a resource type appears in any table below, use the mapping directly. Do NOT proceed to Pass 2 rubric evaluation. This is not optional guidance — it is an authoritative lookup. +**Confidence: `deterministic`** (1:1 mapping, no rubric evaluation needed) ---- - -## Purpose - -Fast-path mappings are 100% deterministic mappings with **zero viable alternatives** and **no config-dependent conditions**. They encode domain knowledge that doesn't change: - -- Single candidate (no alternatives to evaluate) -- No eliminator rules apply (service limits don't matter) -- Operational model is identical -- No conflicts possible - -Example: `google_compute_network` always maps to `aws_vpc`. There is no alternative VPC service on AWS. No rubric needed. - ---- - -## Direct Mappings (Primary) - -Unconditional mappings for primary resources. These resolve in Pass 1 without rubric evaluation. - -**Usage:** If resource type matches column 1, use mapping in column 2. Set `confidence: "deterministic"`. Add `new_aws_resources` from column 4. Do not proceed to Pass 2. - -| GCP Resource Type | AWS Target | Reason | new_aws_resources | -|---|---|---|---| -| `google_compute_network` | `aws_vpc` | Single candidate for VPC abstraction. No alternatives on AWS. Always 1:1. | `[aws_internet_gateway, aws_route_table, aws_route]` | -| `google_compute_subnetwork` | `aws_subnet` | Single candidate for subnet abstraction. Always 1:1. | `[]` | -| `google_storage_bucket` | `aws_s3_bucket` | Single candidate for object storage. Storage class affects S3 tier but destination is always S3. | `[]` | -| `google_compute_firewall` | `aws_security_group` | Single candidate for firewall rules. Always 1:1. | `[]` | -| `google_dns_record_set` | `aws_route53_record` | Single candidate for DNS records. Always 1:1. | `[]` | -| `google_dns_managed_zone` | `aws_route53_zone` | Single candidate for managed DNS zone. Always 1:1. | `[]` | -| `google_sql_database_instance` (PostgreSQL) | `aws_rds_cluster` (Aurora PostgreSQL) | 1:1 mapping; Serverless v2 for dev, Provisioned for prod. | `[aws_rds_cluster_instance]` | -| `google_sql_database_instance` (MySQL) | `aws_rds_cluster` (Aurora MySQL) | 1:1 mapping; Serverless v2 for dev, Provisioned for prod. | `[aws_rds_cluster_instance]` | -| `google_sql_database_instance` (SQL Server) | `aws_db_instance` (SQL Server) | 1:1 mapping; always provisioned (no serverless). | `[]` | -| `google_redis_instance` | `aws_elasticache_replication_group` | 1:1 mapping; preserve cluster mode and node type. | `[aws_elasticache_subnet_group]` | -| `google_service_account` | `aws_iam_role` | 1:1 mapping; map permissions directly, adjust service principals. | `[aws_iam_role_policy]` | - ---- - -## Skip Mappings - -Resources that should never be mapped to AWS resources. Record in "Skipped Resources" section of aws-design-report.md. +## What `deterministic` vs `inferred` means -**Usage:** If resource type matches below, skip it entirely. Do NOT map to AWS. Record reason in report. +Use these labels **only** as defined here — they describe _how the mapping was chosen_, not whether the AWS architecture is "obvious." -| GCP Resource Type | Reason | -|---|---| -| `google_project_service` | API enablement is implicit on AWS. Services are enabled by default. No AWS equivalent needed. | -| `data.google_service_account` | Data source only (read-only). No mapping needed. Use actual service account if created. | -| `null_resource` | Terraform orchestration artifact. Not needed on AWS. | -| `time_sleep` | Terraform orchestration artifact. Not needed on AWS. | +| Label | Meaning | +| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **`deterministic`** | The GCP **Terraform resource type** appears in the **Direct Mappings** table below, the row's **Conditions** are satisfied, and the AWS target is taken from that row. **No** 6-criteria rubric is run for that mapping. | +| **`inferred`** | The resource type is **not** in Direct Mappings (or BigQuery / specialist gate applies). The agent loads the category file from `steering/design-ref-index.md`, runs eliminators and the **6-criteria rubric** (and may apply **Preferred AWS Target Services**), then picks the AWS service. | +| **`billing_inferred`** | Billing-only design path: mappings from billing SKUs/service names — see `steering/design-billing.md`. | ---- - -## Secondary Behavior Lookups +### User-facing vocabulary (chat, MIGRATION_GUIDE, migration-report) -Determines how secondary resources map **based on what their primary became**. - -**Usage (Pass 1):** -1. Primary is resolved (fast-path or rubric) -2. Secondary's `serves` array is checked -3. For each primary the secondary serves, look up behavior in this table -4. If found: apply behavior. If not found: check "Direct Mappings (Secondary)" table - -**Format:** If secondary_role matches column 1 AND primary became the AWS service in column 2, apply column 3. - -| Secondary Role | Primary Becomes | Behavior | Target | Notes | -|---|---|---|---|---| -| `identity` | `aws_lambda_function` | maps | `aws_iam_role` | Lambda requires explicit IAM role | -| `identity` | `aws_ecs_task_definition` | maps | `aws_iam_role` | ECS task requires explicit IAM role | -| `identity` | `aws_apprunner_service` | maps | `aws_iam_role` | App Runner requires explicit IAM role | -| `identity` | `aws_ec2_instance` | absorbed | `aws_iam_instance_profile` | Instance profile is attached to instance config | -| `identity` | `aws_rds_cluster` | absorbed | (not separate) | RDS uses DB-level authentication | -| `identity` | skip | skip | (not needed) | If primary is skipped, identity not needed | -| `network_path` | `aws_vpc` | absorbed | (not separate) | Subnet, firewall rules absorbed into VPC config | -| `network_path` | `aws_security_group` | absorbed | (not separate) | Rules absorbed into security group config | -| `network_path` | skip | skip | (not needed) | If primary is skipped, network not needed | -| `access_control` | any | maps | `aws_iam_policy_statement` | IAM bindings become policy statements on roles | -| `access_control` | skip | skip | (not needed) | If primary is skipped, access control not needed | -| `configuration` | `aws_rds_db_instance` | absorbed | (db_name, db_user fields) | Database and user absorbed into RDS config | -| `configuration` | `aws_secretsmanager_secret` | absorbed | (secret fields) | Secret versions absorbed into secret config | -| `configuration` | `aws_route53_zone` | absorbed | (record config) | DNS records absorbed into zone config | -| `configuration` | skip | skip | (not needed) | If primary is skipped, configuration not needed | -| `encryption` | any | maps | `aws_kms_key_policy` | KMS permissions referenced in resource policy | -| `encryption` | skip | skip | (not needed) | If primary is skipped, encryption not needed | -| `orchestration` | any | skip | (not needed) | Terraform orchestration not needed on AWS | +JSON artifacts **must** keep the `confidence` string values above. When speaking or writing **for end users**, lead with plain English — do **not** use "deterministic," "inferred," or "rubric" as the primary label unless the user asks for technical detail. ---- +| JSON `confidence` | Say this to users | Optional one-line hint | +| ------------------ | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| `deterministic` | **Standard pairing** | Same AWS target for this GCP resource type whenever it matches our fixed list — quick to sanity-check. | +| `inferred` | **Tailored to your setup** | Based on your Terraform configuration, how the resource fits the rest of your stack, and your migration preferences — review again if those change. | +| `billing_inferred` | **Estimated from billing only** | From GCP spend line items without full infrastructure detail — add Terraform for a tighter mapping. | -## Direct Mappings (Secondary) +**BigQuery / specialist gate** rows still store `confidence: "inferred"` in JSON; in user-facing text you may say **Tailored to your setup** and emphasize **specialist engagement** (no automated AWS analytics target). -Mappings for secondary resources that don't depend on what their primary became. Use if not found in "Secondary Behavior Lookups" above. +**Canonical reference:** This subsection — other phase files should point here instead of redefining wording. -**Usage (Pass 1):** -1. Secondary not resolved by "Secondary Behavior Lookups" -2. Check this table -3. If secondary type matches column 1: apply mapping in column 2 +**Common confusion:** `steering/design-ref-index.md` lists a **typical AWS target** per GCP service. That is not automatically the same as **`deterministic`**. Confidence is `deterministic` only when the exact Terraform resource type appears in the Direct Mappings table above and its conditions are met; otherwise confidence is `inferred` via rubric evaluation. -| GCP Secondary Type | AWS Target | Reason | -|---|---|---| -| `google_kms_crypto_key` | `aws_kms_key` | 1:1 mapping. Always separate resource. | -| `google_kms_key_ring` | (absorbed) | Key ring is metadata on aws_kms_key. Not separate. | -| `google_compute_router_nat` | `aws_nat_gateway` | 1:1 mapping. NAT routing rules absorbed into route table. | -| `google_compute_address` | `aws_eip` (if address_type=EXTERNAL) or `aws_nat_gateway` (if purpose=NAT) | Determined by config (address_type, purpose) — rubric needed if ambiguous. | +**Add-ons (ALB, NAT, etc.):** A row may say "Fargate" while the architecture diagram also includes an **ALB** or **NAT Gateway** from **other** Terraform resources. Confidence is still per **resource row** — e.g. `google_cloud_run_service` = `inferred`; `google_compute_forwarding_rule` + backend = often `inferred` (see `design-ref-networking.md`). --- -## Examples +**Direct Mappings use confidence: `deterministic`** (fixed table lookup — no rubric for that resource) -### Example 1: Primary Fast-Path Resolution +## Direct Mappings Table -**Resource:** `google_storage_bucket.data_lake` +| GCP Service | AWS Service | Conditions | Notes | +| ------------------------------------------- | -------------------- | ---------- | ---------------------------------------------------- | +| `google_storage_bucket` | S3 | Always | 1:1 mapping; preserve ACL/versioning/lifecycle rules | +| `google_cloud_run_service` | Fargate | Always | Preferred container runtime target | +| `google_cloud_run_v2_service` | Fargate | Always | v2 API variant of Cloud Run | +| `google_cloudfunctions_function` | Lambda | Always | Gen 1 function mapping | +| `google_cloudfunctions2_function` | Lambda | Always | Gen 2 function mapping | +| `google_sql_database_instance` (SQL Server) | RDS SQL Server | Always | Always provisioned (no serverless) | +| `google_compute_network` | VPC | Always | 1:1; preserve CIDR ranges | +| `google_compute_firewall` | Security Group | Always | 1:1 rule mapping; adjust CIDR if needed | +| `google_dns_managed_zone` | Route 53 Hosted Zone | Always | Preserve zone name and records | +| `google_service_account` | IAM Role | Always | Map permissions directly; adjust service principals | +| `google_secret_manager_secret` | Secrets Manager | Always | Create secret metadata and IAM-scoped access | +| `google_secret_manager_secret_version` | Secrets Manager | Always | Carry current value or explicit migration TODO | +| `google_redis_instance` | ElastiCache Redis | Always | 1:1 mapping; preserve cluster mode and node type | -**Pass 1 Evaluation:** -``` -Check Direct Mappings (Primary) table: - google_storage_bucket → aws_s3_bucket ✓ found +### Cloud SQL PostgreSQL / MySQL — NOT in Direct Mappings -Result: - source: "google_storage_bucket.data_lake" - target: "aws_s3_bucket" - confidence: "deterministic" - reason: "Fast path: Object storage bucket → S3. Single candidate, no alternatives." - source_config: { "storage_class": "NEARLINE", "lifecycle_rule": "delete after 90 days" } - related: { "gcp": [], "aws": [] } - new_aws_resources: [] +**Do not** assign `google_sql_database_instance` (PostgreSQL or MySQL) from this table. Always use **Pass 2** + `design-ref-database.md` rubric with confidence = **`inferred`**. -Do NOT proceed to Pass 2 for this resource. -``` +**Q6 (`design_constraints.availability`) is the only selector for RDS vs Aurora:** ---- - -### Example 2: Secondary Fast-Path Resolution (After Primary) +| `availability` value | AWS target (match engine from GCP) | +| -------------------- | -------------------------------------------------- | +| `single-az` | **RDS PostgreSQL** or **RDS MySQL** | +| `multi-az` | **RDS PostgreSQL** or **RDS MySQL** Multi-AZ | +| `multi-az-ha` | **Aurora PostgreSQL** or **Aurora MySQL** Multi-AZ | +| `multi-region` | **Aurora Global Database** (engine per GCP) | -**Primary Resolution (from fast-path or rubric):** -``` -google_sql_database_instance → aws_rds_db_instance -``` +Q12 (traffic) and Q13 (I/O) tune sizing and storage **within** the family Q6 chose — they never upgrade Inconvenient/Significant Issue workloads to Aurora. -**Secondary:** `google_sql_database.main` (secondary_role: configuration, serves: [google_sql_database_instance.main]) +## Skip Mappings Table -**Pass 1 Evaluation:** -``` -Check Secondary Behavior Lookups: - secondary_role=configuration, primary_becomes=aws_rds_db_instance - → behavior: absorbed, target: (db_name field) ✓ found +These GCP resources do **not** require AWS equivalents in v1.0: -Result: - source: "google_sql_database.main" - target: "(absorbed into aws_rds_db_instance.database_name)" - confidence: "deterministic" - reason: "Fast path: Database configuration absorbed. Database name becomes db_name on RDS instance." - related: { "gcp": ["google_sql_database_instance.main"] } +| GCP Service | Reason | +| ---------------------------- | ------------------------------------------------------------------------------------- | +| `google_project` | AWS account structure (manual, not IaC) | +| `google_monitoring_*` | Fallback to CloudWatch (managed) | +| `google_logging_*` | Fallback to CloudWatch Logs (managed) | +| `google_compute_address` | Elastic IPs managed by ALB/NAT (not standalone) | +| `google_identity_platform_*` | Auth provider — keep existing solution, do not migrate to AWS Cognito or any AWS auth | +| `google_firebase_auth_*` | Auth provider — keep existing solution, do not migrate to AWS Cognito or any AWS auth | -Do NOT proceed to Pass 2 for this resource. -``` - ---- +## Preferred AWS Target Services -### Example 3: Secondary Deferred to Pass 2 +For the workload categories below, always steer toward the **preferred** AWS service. If rubric evaluation or LLM reasoning would select a non-preferred service, substitute the preferred alternative instead. -**Primary:** `google_cloud_run_service.api` (not in fast-path, needs rubric) - -**Secondary:** `google_service_account.app` (secondary_role: identity, serves: [google_cloud_run_service.api]) - -**Pass 1 Evaluation:** -``` -Check Secondary Behavior Lookups: - secondary_role=identity, primary_becomes=??? (not yet resolved) - -Primary not yet resolved. Cannot determine behavior. - -Result: Leave secondary for Pass 2. - -(After Pass 2 resolves primary to aws_apprunner_service, - return to secondary in Pass 2 Phase 2) -``` - ---- +| Workload Category | Preferred AWS Target | Rationale | +| ----------------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------ | +| Containerized workloads | Fargate (default), Lambda (event-driven), EKS (K8s required) | Deeper integration with VPC, ALB, IAM, and auto-scaling than lighter-weight alternatives | +| Third-party auth in use | Keep existing auth provider | Startups using Auth0, Firebase Auth, Clerk, etc. should preserve their existing investment | -### Example 4: Skip Mapping +**Enforcement:** Check every resource mapping against this table before writing to `aws-design.json` or `aws-design-billing.json`. If a mapping would select a non-preferred service, substitute the preferred alternative. -**Resource:** `google_project_service.run` +**Exception:** For Cloud SQL PostgreSQL/MySQL, **Q6 availability always overrides** any implicit Aurora preference. Do not substitute Aurora when `availability` is `single-az` or `multi-az`. -**Pass 1 Evaluation:** -``` -Check Skip Mappings table: - google_project_service → skip ✓ found - -Result: - source: "google_project_service.run" - target: "(skipped)" - confidence: "deterministic" - reason: "Fast path skip: API enablement not needed on AWS. Services are enabled by default." - related: { "gcp": [], "aws": [] } - -Note: Add to "Skipped Resources" section of aws-design-report.md - -Do NOT proceed to Pass 2 for this resource. -``` - ---- - -## Pass 1 Algorithm - -**For each resource in cluster (both primary and secondary):** - -### Primary Resource - -``` -1. Is resource_type in "Direct Mappings (Primary)" table? - YES → Record mapping. Set confidence=deterministic. Add new_aws_resources. - Mark as RESOLVED. Do not proceed to Pass 2. - NO → Continue to step 2. - -2. Is resource_type in "Skip Mappings" table? - YES → Skip resource. Record in skipped list. - Mark as RESOLVED. Do not proceed to Pass 2. - NO → Leave for Pass 2 (rubric evaluation) -``` - -### Secondary Resource - -``` -1. Are ALL primaries in secondary.serves[] already resolved? - NO → Leave for Pass 2 (wait for primaries) - YES → Continue to step 2. - -2. For each primary served: - Check "Secondary Behavior Lookups" for (secondary_role, primary_became): - FOUND → Record behavior. Mark secondary as RESOLVED. Do not proceed to Pass 2. - NOT FOUND → Continue to step 3. - -3. Is secondary_type in "Direct Mappings (Secondary)" table? - YES → Record mapping. Mark as RESOLVED. Do not proceed to Pass 2. - NO → Leave for Pass 2 (rubric evaluation needed) -``` - ---- - -## After Pass 1 - -**If ALL resources resolved by Pass 1:** -- Write cluster block to aws-design.json -- Add all new_aws_resources to cluster's new_aws_resources array -- Update metadata: `resolved_by_fast_path` += count -- Move to next cluster (skip Pass 2 entirely) - -**If ANY resources unresolved:** -- Record Pass 1 results (mapped, skipped, absorbed) -- Proceed to Pass 2 (rubric) for remaining resources -- Use Pass 1 results as context for Pass 2 evaluation - ---- - -## Maintenance Notes - -**Adding new fast-path mappings:** - -1. Ensure mapping is **100% unconditional** (no config-dependent logic) -2. Verify **no viable alternative** exists on AWS -3. Include **clear reasoning** for why there's no ambiguity -4. Test with sample resources before committing - -**Example (good candidate for fast-path):** -``` -google_compute_instance → aws_ec2_instance -Reason: Single candidate. No serverless alternative for fixed VMs. -No config field changes this decision. -``` - -**Example (NOT a fast-path candidate):** -``` -google_cloud_run_service → ??? (could be Lambda, App Runner, or ECS) -Reason: Timeout, memory, concurrency, GPU support, and scale behavior -determine candidate. Config-dependent. Needs rubric. -``` - ---- +## Secondary Behavior Lookups -## Related Files +For resources in the Skip Mappings table but present in inventory: -- `design.md` — Pass 1 (fast-path) and Pass 2 (rubric) algorithm -- `design-ref-index.md` — Lookup for which design-ref file contains resource -- `design-ref-compute.md` — Rubric rules for compute resources -- `design-ref-database.md` — Rubric rules for database resources -- `design-ref-networking.md` — Rubric rules for networking resources -- `design-ref-storage.md` — Rubric rules for storage resources -- `design-ref-messaging.md` — Rubric rules for messaging resources -- `design-ref-ai.md` — Rubric rules for AI/ML resources +1. Log as "secondary resource, no AWS equivalent needed" +2. Do not include in aws-design.json +3. Note in aws-design.json warnings array --- -**Last Updated:** 2026-02-24 -**Status:** Active (Production) +**Workflow:** +1. Extract GCP resource type +2. Look up in Direct Mappings table +3. If found and condition met: assign AWS service (confidence = deterministic) +4. If `google_sql_database_instance` (PostgreSQL/MySQL): skip Direct Mappings → apply `design-ref-database.md` rubric (confidence = inferred) +5. If found in Skip Mappings: skip it (confidence = n/a) +6. If not found: use `steering/design-ref-index.md` to determine category → apply rubric in that category's file diff --git a/migration-to-aws/steering/design-ref-harness.md b/migration-to-aws/steering/design-ref-harness.md new file mode 100644 index 00000000..aa2a0b43 --- /dev/null +++ b/migration-to-aws/steering/design-ref-harness.md @@ -0,0 +1,153 @@ +# AgentCore Harness Design Reference + +> Loaded by `design-ai.md` Step 0.6 when `agentic_profile.is_agentic == true` AND `ai_constraints.agentic.migration_approach == "harness"`. + +**Prerequisites:** `steering/ai-migration-guardrails.md` must already be loaded (Step 0.6 loads it before this file). Do NOT duplicate regional caveats, pricing rules, or effort estimation rules here. + +--- + +## When Harness Fits + +Recommend Harness when: + +- Single agent with tools (not complex multi-agent orchestration) +- OpenAI Assistants API migration (closest 1:1 mapping — Assistants → Harness declarations) +- Team wants managed runtime, memory, identity, and observability without building it +- Incremental migration: run existing OpenAI/Gemini models on AgentCore infrastructure, swap to Bedrock per-invocation +- Agent tasks run < 8 hours (Harness session limit) +- Team prefers config-first iteration over code-first + +## When Harness Does NOT Fit + +Do NOT recommend Harness as primary path when: + +- Complex multi-agent graphs with custom state management (recommend retarget or Strands) +- Team needs to stay in their current framework for velocity (recommend retarget) +- Agent count > 3 with inter-agent coordination (recommend retarget with AgentCore Runtime, or Strands) +- Custom runtime dependencies that can't be containerized (evaluate custom container image on Harness, or use AgentCore Runtime directly) + +For these cases: fall back to retarget path (standard model-swap design in Parts 1–6 of `design-ai.md`). + +--- + +## Harness Configuration Mapping + +Map discovered elements to Harness configuration: + +| Discovered element | Harness config | Notes | +| --------------------------------------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------- | +| `bedrock_models[0].aws_model_id` | `model` (default) | Primary Bedrock model from Part 1 model selection | +| `agents[0].role` or system prompt from code | `systemPrompt` | Extract from code if possible; placeholder if not | +| Tools with `transport: "mcp"` | `tools[]: {"type": "remote_mcp", "config": {"remoteMcp": {"url": "..."}}}` | Direct MCP server connection | +| Tools with `transport: "api"` | `tools[]: {"type": "remote_mcp"}` or `{"type": "agentcore_gateway"}` | Wrap API as MCP server, or use Gateway for centralized auth | +| Tools with `transport: "function"` (browser/web) | `tools[]: {"type": "agentcore_browser"}` | Built-in browser tool | +| Tools with `transport: "function"` (code execution) | `tools[]: {"type": "agentcore_code_interpreter"}` | Built-in code interpreter | +| Tools with `transport: "function"` (other) | `tools[]: {"type": "inline_function", "config": {...}}` | Client-side execution; Harness pauses and returns call to your code | +| `memory_requirement: "session"` | Default behavior | Harness sessions are stateful by default (microVM per session) | +| `memory_requirement: "cross_session"` | AgentCore Memory service | Configure memory persistence across sessions | +| `memory_requirement: "none"` | No memory config needed | Stateless invocations | + +**Tool mapping decision:** + +``` +For each tool in tool_manifest[]: +├── transport == "mcp" → remote_mcp (direct connection) +├── transport == "api" +│ ├── auth_hint == "oauth" or multiple tools share auth → agentcore_gateway (centralized auth) +│ └── auth_hint == "api_key" or "none" → remote_mcp (simpler, wrap as MCP) +├── transport == "function" +│ ├── tool does web browsing/scraping → agentcore_browser +│ ├── tool executes code/scripts → agentcore_code_interpreter +│ └── other local function → inline_function (client-side) +└── transport == "unknown" → inline_function (safest default; client controls execution) +``` + +--- + +## Incremental Migration via Multi-Model Switching + +If `ai_constraints.agentic.incremental_migration == true`: + +**Phase 0: Deploy on Harness with existing source provider model** + +1. Store source provider API key in AgentCore Identity token vault +2. Create Harness with source model as default (e.g., `--model-provider open_ai --model-id gpt-4o`) +3. Deploy and validate: existing behavior preserved on AWS infrastructure +4. Benefit: AWS observability, security, and scaling — without changing the model yet + +**Phase 1: A/B test with Bedrock model** + +1. Override `--model-id` per invocation with Bedrock model from design +2. Compare responses: quality, latency, tool-calling behavior +3. Run evaluation prompts from `test_comparison.py` against both models on same session + +**Phase 2: Switch default to Bedrock** + +1. Update Harness default model to Bedrock model ID +2. Keep source provider credentials as fallback +3. Monitor for 48 hours + +**Phase 3: Remove source provider** + +1. Delete API key from AgentCore Identity token vault +2. Remove source provider from Harness config +3. Migration complete + +--- + +## Output: `agentic_design` in `aws-design-ai.json` + +When Harness path is selected, write this to `aws-design-ai.json`: + +```json +{ + "agentic_design": { + "migration_approach": "harness", + "harness_config": { + "name": "from agentic_profile.agents[0].agent_id", + "model_id": "from bedrock_models[0].aws_model_id", + "system_prompt": "extracted from code or placeholder", + "tools": [ + { + "type": "remote_mcp|agentcore_browser|agentcore_code_interpreter|agentcore_gateway|inline_function", + "name": "tool name from tool_manifest", + "config": {} + } + ], + "memory_enabled": true, + "memory_type": "session|cross_session", + "incremental_migration": true, + "source_model_provider": "open_ai|google", + "source_model_id": "from models[0].model_id" + }, + "regional_fit": "available|preview|unavailable", + "deployment_regions": ["us-west-2", "us-east-1"], + "warnings": [] + } +} +``` + +**Field rules:** + +- `harness_config.name` — Derived from first agent's `agent_id`. Use kebab-case. +- `harness_config.model_id` — The Bedrock model ID selected in Part 1 (e.g., `us.anthropic.claude-sonnet-4-6-20250514-v1:0`) +- `harness_config.system_prompt` — Extracted from agent code if available; otherwise `"[TODO: Add system prompt from your agent definition]"` +- `harness_config.tools` — Mapped from `tool_manifest[]` using the tool mapping decision tree above +- `harness_config.source_model_provider` — `"open_ai"` or `"google"` based on `summary.ai_source` +- `harness_config.source_model_id` — Original model ID from `models[0].model_id` +- `regional_fit` — Result of Step 0.5 regional check for AgentCore Harness in target region + +--- + +## Present Summary (Harness-specific additions) + +After the standard model comparison summary from `design-ai.md`, add: + +> **Agentic Migration: AgentCore Harness** +> +> - Approach: Config-based agent deployment on AgentCore +> - Tools mapped: [count] tools → [types breakdown] +> - Memory: [session/cross-session/none] +> - Incremental migration: [yes/no] +> - Regional availability: [available/preview in target region] +> - Estimated effort: [range] depending on [drivers from guardrails] diff --git a/migration-to-aws/steering/design-ref-index.md b/migration-to-aws/steering/design-ref-index.md index f81374d3..062ec10d 100644 --- a/migration-to-aws/steering/design-ref-index.md +++ b/migration-to-aws/steering/design-ref-index.md @@ -1,70 +1,74 @@ # GCP Service → Design Reference Mapping +> **Column note:** **Typical AWS target** is the usual rubric outcome for that Terraform type. It is **not** the same as **`deterministic` confidence** in `aws-design.json`. Only resource types listed in **`design-ref-fast-path.md` → Direct Mappings** get `deterministic`; everything else in this table is mapped via rubric → `inferred` (unless `billing_inferred` on the billing-only path). + ## Compute Services -| GCP Service | Resource Type | Reference File | Fast-Path | +| GCP Service | Resource Type | Reference File | Typical AWS target | | ------------------- | -------------------------------- | -------------- | ------------------ | -| Cloud Run | `google_cloud_run_service` | `compute.md` | Fargate | -| Cloud Functions | `google_cloudfunctions_function` | `compute.md` | Lambda | -| Compute Engine (VM) | `google_compute_instance` | `compute.md` | EC2 or Fargate | -| GKE | `google_container_cluster` | `compute.md` | EKS | -| App Engine | `google_app_engine_application` | `compute.md` | Fargate or Amplify | +| Cloud Run | `google_cloud_run_service` | `design-ref-compute.md` | Fargate | +| Cloud Functions | `google_cloudfunctions_function` | `design-ref-compute.md` | Lambda | +| Compute Engine (VM) | `google_compute_instance` | `design-ref-compute.md` | EC2 or Fargate | +| GKE | `google_container_cluster` | `design-ref-compute.md` | EKS | +| App Engine | `google_app_engine_application` | `design-ref-compute.md` | Fargate or Amplify | ## Database Services -| GCP Service | Resource Type | Reference File | Fast-Path | -| ---------------------- | ------------------------------ | -------------- | --------------------- | -| Cloud SQL (PostgreSQL) | `google_sql_database_instance` | `database.md` | RDS Aurora PostgreSQL | -| Cloud SQL (MySQL) | `google_sql_database_instance` | `database.md` | RDS Aurora MySQL | -| Cloud SQL (SQL Server) | `google_sql_database_instance` | `database.md` | RDS SQL Server | -| Firestore (instance) | `google_firestore_database` | `database.md` | DynamoDB | -| Firestore (document) | `google_firestore_document` | `database.md` | DynamoDB | -| BigQuery | `google_bigquery_dataset` | `database.md` | Redshift or Athena | -| Memorystore (Redis) | `google_redis_instance` | `database.md` | ElastiCache Redis | -| Cloud Spanner | `google_spanner_instance` | `database.md` | Aurora DSQL | +| GCP Service | Resource Type | Reference File | Typical AWS target | +| ---------------------- | ------------------------------ | -------------- | --------------------------------------------------------------------------------- | +| Cloud SQL (PostgreSQL) | `google_sql_database_instance` | `design-ref-database.md` | RDS PostgreSQL or Aurora PostgreSQL (Q6 selects family) | +| Cloud SQL (MySQL) | `google_sql_database_instance` | `design-ref-database.md` | RDS MySQL or Aurora MySQL (Q6 selects family) | +| Cloud SQL (SQL Server) | `google_sql_database_instance` | `design-ref-database.md` | RDS SQL Server | +| Firestore (instance) | `google_firestore_database` | `design-ref-database.md` | DynamoDB | +| Firestore (document) | `google_firestore_document` | `design-ref-database.md` | DynamoDB | +| BigQuery | `google_bigquery_*` | `design-ref-database.md` | **`Deferred — specialist engagement`** only (see `design-infra.md` BigQuery gate) | +| Memorystore (Redis) | `google_redis_instance` | `design-ref-database.md` | ElastiCache Redis | +| Cloud Spanner | `google_spanner_instance` | `design-ref-database.md` | Aurora DSQL | ## Storage Services -| GCP Service | Resource Type | Reference File | Fast-Path | -| ------------------- | ----------------------- | -------------- | --------- | -| Cloud Storage (GCS) | `google_storage_bucket` | `storage.md` | S3 | -| Filestore | `google_filestore_instance` | `storage.md` | EFS | +| GCP Service | Resource Type | Reference File | Typical AWS target | +| ------------------- | --------------------------- | -------------- | ------------------ | +| Cloud Storage (GCS) | `google_storage_bucket` | `design-ref-storage.md` | S3 | +| Filestore | `google_filestore_instance` | `design-ref-storage.md` | EFS | ## Networking Services -| GCP Service | Resource Type | Reference File | Fast-Path | +| GCP Service | Resource Type | Reference File | Typical AWS target | | -------------------- | --------------------------------- | --------------- | ------------------ | -| VPC Network | `google_compute_network` | `networking.md` | VPC | -| Firewall Rules | `google_compute_firewall` | `networking.md` | Security Groups | -| Cloud Load Balancing | `google_compute_forwarding_rule` | `networking.md` | ALB/NLB | -| Cloud CDN | (part of compute_backend_service) | `networking.md` | CloudFront | -| Cloud DNS | `google_dns_managed_zone` | `networking.md` | Route 53 | -| Cloud Interconnect | (custom config) | `networking.md` | AWS Direct Connect | -| Cloud Armor | `google_compute_security_policy` | `networking.md` | AWS WAF | +| VPC Network | `google_compute_network` | `design-ref-networking.md` | VPC | +| Firewall Rules | `google_compute_firewall` | `design-ref-networking.md` | Security Groups | +| Cloud Load Balancing | `google_compute_forwarding_rule` | `design-ref-networking.md` | ALB/NLB | +| Cloud CDN | (part of compute_backend_service) | `design-ref-networking.md` | CloudFront | +| Cloud DNS | `google_dns_managed_zone` | `design-ref-networking.md` | Route 53 | +| Cloud Interconnect | (custom config) | `design-ref-networking.md` | AWS Direct Connect | +| Cloud Armor | `google_compute_security_policy` | `design-ref-networking.md` | AWS WAF | ## Messaging Services -| GCP Service | Resource Type | Reference File | Fast-Path | +| GCP Service | Resource Type | Reference File | Typical AWS target | | ----------- | -------------------------- | -------------- | ------------------ | -| Pub/Sub | `google_pubsub_topic` | `messaging.md` | SNS or SQS | -| Cloud Tasks | `google_cloud_tasks_queue` | `messaging.md` | SQS or EventBridge | +| Pub/Sub | `google_pubsub_topic` | `design-ref-messaging.md` | SNS or SQS | +| Cloud Tasks | `google_cloud_tasks_queue` | `design-ref-messaging.md` | SQS or EventBridge | ## AI/ML Services -| GCP Service | Resource Type | Reference File | Fast-Path | +| GCP Service | Resource Type | Reference File | Typical AWS target | | -------------------------- | ------------------- | ------------------------- | ----------------------- | -| Vertex AI (LLM/Gemini) | (generative models) | `ai-gemini-to-bedrock.md` | Bedrock | -| OpenAI (in GCP env) | (openai SDK) | `ai-openai-to-bedrock.md` | Bedrock | -| Vertex AI (traditional ML) | (custom endpoints) | `ai.md` | SageMaker | -| Vertex AI (pipelines) | (custom config) | `ai.md` | SageMaker Pipelines | -| Cloud Vision API | (managed API) | `ai.md` | Textract or Rekognition | +| Vertex AI (LLM/Gemini) | (generative models) | `design-ref-ai-gemini-to-bedrock.md` | Bedrock | +| OpenAI (in GCP env) | (openai SDK) | `design-ref-ai-openai-to-bedrock.md` | Bedrock | +| Vertex AI (traditional ML) | (custom endpoints) | `design-ref-ai.md` | SageMaker | +| Vertex AI (pipelines) | (custom config) | `design-ref-ai.md` | SageMaker Pipelines | +| Cloud Vision API | (managed API) | `design-ref-ai.md` | Textract or Rekognition | ## Secondary/Infrastructure Services -| GCP Service | Resource Type | Reference File | Fast-Path | -| ---------------- | ------------------------ | ----------------- | ---------- | -| Service Accounts | `google_service_account` | `networking.md` | IAM Roles | -| Cloud Monitoring | (managed) | Not in v1.0 scope | CloudWatch | +| GCP Service | Resource Type | Reference File | Typical AWS target | +| ------------------------ | -------------------------------------- | ----------------- | ------------------ | +| Service Accounts | `google_service_account` | `design-ref-networking.md` | IAM Roles | +| Secret Manager (secret) | `google_secret_manager_secret` | `design-ref-security.md` | Secrets Manager | +| Secret Manager (version) | `google_secret_manager_secret_version` | `design-ref-security.md` | Secrets Manager | +| Cloud Monitoring | (managed) | Not in v1.0 scope | CloudWatch | --- @@ -72,5 +76,7 @@ 1. Extract GCP resource type from Terraform 2. Find in table above -3. If resource found in `fast-path.md` Direct Mappings table: use that mapping (confidence = deterministic) +3. If resource found in `design-ref-fast-path.md` Direct Mappings table: use that mapping (confidence = deterministic) 4. Otherwise: load Reference File listed above and apply 6-criteria rubric (confidence = inferred) + +**User-facing labels** for chat and reports: see `design-ref-fast-path.md` → **User-facing vocabulary** (e.g. **Standard pairing** / **Tailored to your setup** / **Estimated from billing only**). diff --git a/migration-to-aws/steering/design-ref-messaging.md b/migration-to-aws/steering/design-ref-messaging.md index 3ac0d85b..14531566 100644 --- a/migration-to-aws/steering/design-ref-messaging.md +++ b/migration-to-aws/steering/design-ref-messaging.md @@ -1,133 +1,90 @@ -# Messaging Mappings - -## google_pubsub_topic - -**Purpose:** Pub/Sub message topic for publish-subscribe patterns - -**Default:** aws_sns_topic + aws_sqs_queue (SNS+SQS) - -**Rationale for default:** SNS+SQS is the most flexible AWS messaging pattern. It supports both fan-out and queuing, covering the full range of Pub/Sub usage patterns. - -**Candidates:** -- aws_sns_topic (SNS only) -- aws_sqs_queue (SQS only) -- aws_sns_topic + aws_sqs_queue (SNS+SQS combo) - -**Signals:** - -| Config Field | Value | Favors | Weight | Reason | -|---|---|---|---|---| -| All subscriptions are push | true | aws_sns_topic | strong | Push delivery aligns with SNS model | -| All subscriptions are pull | true | aws_sqs_queue | strong | Pull/polling aligns with SQS model | -| Mix of push and pull | true | aws_sns_topic + aws_sqs_queue | strong | SNS for fan-out, SQS for pull consumers | -| Multiple subscriptions | true | aws_sns_topic | moderate | Fan-out pattern is SNS's strength | -| Single subscription, pull | true | aws_sqs_queue | strong | No fan-out needed, SQS is simpler | -| Subscription has filter | present | aws_sns_topic | moderate | SNS supports subscription filter policies | - -**Eliminators:** None — all candidates work for any Pub/Sub config, they differ in fit. - -**Peek at secondaries:** Yes — must look at google_pubsub_subscription resources that serve this topic. - -**How to peek:** -1. Find all secondaries where `serves` includes this topic's address -2. Filter to type `google_pubsub_subscription` -3. Check each subscription's config for: push_endpoint (push) vs subscription_type: pull -4. Check for dead_letter_topic, filter, ack_deadline_seconds - -**Decision logic:** -1. If all subscriptions have push_endpoint → SNS only -2. If all subscriptions are pull with single subscriber → SQS only -3. If multiple subscribers or mix of push/pull → SNS + SQS -4. If no subscriptions found → default to SNS + SQS - -**1:Many Expansion:** - -If SNS only: -- aws_sns_topic — primary -- aws_sns_topic_subscription — one per push subscription - -If SQS only: -- aws_sqs_queue — primary -- aws_sqs_queue (DLQ) — if dead_letter_topic is configured on any subscription - -If SNS + SQS: -- aws_sns_topic — fan-out -- aws_sqs_queue — one per pull subscription -- aws_sns_topic_subscription — to connect SQS queues to SNS topic -- aws_sqs_queue (DLQ) — if dead_letter_topic is configured - -**Source Config to Carry Forward:** -- message_retention_duration — determines message retention (SQS: MessageRetentionPeriod, SNS: N/A) -- name — determines resource naming - -**Important:** SNS does not support message retention. Messages are delivered immediately and not stored. If message retention is required, use SQS (which supports 1-14 day retention via MessageRetentionPeriod). Do not set message_retention on SNS resources. - -**Subscription config to carry forward (from secondaries):** -- ack_deadline_seconds — determines SQS VisibilityTimeout -- message_retention_duration — determines SQS MessageRetentionPeriod -- dead_letter_topic — determines DLQ configuration and maxReceiveCount -- max_delivery_attempts — determines SQS maxReceiveCount on redrive policy -- push_endpoint — determines SNS subscription endpoint -- filter — determines SNS subscription filter policy -- retry_minimum_backoff / retry_maximum_backoff — determines SQS retry behavior - ---- - -## google_pubsub_topic (dead letter) - -**Behavior:** A Pub/Sub topic used as a dead letter destination maps differently from a regular topic. It becomes: -- aws_sqs_queue with purpose "dead-letter" if the parent subscription maps to SQS -- aws_sns_topic with purpose "dead-letter" if the parent subscription maps to SNS - -Detect by checking if this topic is referenced in any subscription's `dead_letter_topic` field. If so, it does not get its own independent mapping — it's created as part of the parent topic's 1:many expansion. - ---- - -## Secondary: google_pubsub_subscription - -**Note:** Subscriptions are classified as secondaries in Pillar 1 but play a special role in Pillar 2. Their config drives the primary topic's mapping decision (push vs pull). After the topic is mapped, subscriptions map as follows: - -- If topic mapped to SNS → push subscription becomes aws_sns_topic_subscription -- If topic mapped to SQS → pull subscription is absorbed into SQS config (SQS is inherently pull) -- If topic mapped to SNS+SQS → push subscriptions become SNS subscriptions, pull subscriptions become SQS queues subscribed to SNS - ---- - -## Secondary: google_service_account (pubsub_invoker) - -**Behavior:** The pubsub_invoker service account is a GCP-specific pattern for authenticating Pub/Sub push deliveries to Cloud Run. On AWS: -- If using SNS → push to HTTP endpoint, SNS handles delivery natively, no separate identity needed -- If target is Lambda → SNS invokes Lambda directly via aws_lambda_permission, no separate identity -- If target is ECS/App Runner → SNS pushes to HTTPS endpoint, authentication handled differently - -This service account typically maps to nothing on AWS. Skip with reason. - ---- - -## google_cloud_tasks_queue - -**Purpose:** Task queue for asynchronous HTTP callback execution with rate limiting and retry logic - -**Default:** SQS + Lambda (or EventBridge for scheduled tasks) - -**Rationale for default:** Cloud Tasks is a rate-limited, retryable task queue — SQS provides equivalent queueing with dead-letter support. EventBridge handles scheduling patterns that Cloud Tasks also supports. - -**Candidates:** -- SQS Standard + Lambda — for HTTP callback pattern with retry -- SQS FIFO + Lambda — if ordering matters -- EventBridge + SNS/SQS — for scheduled/delayed task execution - -**Signals:** - -| Signal | Favors | Weight | Reason | -|---|---|---|---| -| HTTP target tasks (push to URL) | SQS + Lambda | strong | Lambda invoked by SQS, calls target URL | -| Rate limiting configured | SQS + Lambda (with concurrency) | moderate | Lambda reserved concurrency acts as rate limiter | -| Scheduled task execution | EventBridge + SQS | strong | EventBridge handles cron/rate-based scheduling | -| Ordered delivery required | SQS FIFO | strong | FIFO guarantees ordering | - -**1:Many Expansion:** -- aws_sqs_queue -- aws_lambda_function (processor) -- aws_lambda_event_source_mapping -- aws_sqs_queue (dead-letter, if retry config present) +# Messaging Services Design Rubric + +**Applies to:** Pub/Sub, Cloud Tasks + +**Quick lookup (no rubric):** Check `design-ref-fast-path.md` first (Pub/Sub → SNS/SQS, etc.) + +## Eliminators (Hard Blockers) + +| GCP Service | AWS | Blocker | +| ----------- | --- | ------------------------------------------------------------------------------------------------------- | +| Pub/Sub | SNS | Exactly-once delivery required → SNS FIFO + SQS FIFO (SNS FIFO supports exactly-once via deduplication) | +| Pub/Sub | SQS | Multiple subscribers per topic → SNS (not SQS) | +| Cloud Tasks | SQS | Scheduled/delayed task execution → EventBridge + SNS/SQS | + +## Signals (Decision Criteria) + +### Pub/Sub + +- **Multiple subscribers, broadcast** → SNS (pub/sub pattern) +- **Single consumer, durability** → SQS (queue pattern) +- **Exactly-once delivery** → SNS FIFO + SQS FIFO (deduplication enabled) +- **Real-time, low latency** → SNS (vs SQS polling delay) + +### Cloud Tasks + +- **HTTP callback execution** → EventBridge + SNS/SQS (route to Lambda/Fargate) +- **Delayed/scheduled queue** → SQS + Lambda (ScheduledEvents) + +## 6-Criteria Rubric + +Apply in order: + +1. **Eliminators**: Does GCP config require AWS-unsupported features? If yes: switch +2. **Operational Model**: Managed (SNS, SQS, EventBridge) vs Custom queue? + - Prefer managed +3. **User Preference**: From `preferences.json`: `design_constraints.availability`? + - SNS and SQS are multi-AZ by default — no special config needed for HA + - If ordering or exactly-once delivery required → SQS FIFO (see Eliminators) +4. **Feature Parity**: Does GCP config need features unavailable in AWS? + - Example: Pub/Sub ordering guarantee → SQS FIFO (has ordering) +5. **Cluster Context**: Are other resources using SNS/SQS? Match if possible +6. **Simplicity**: SNS + SQS (coupled) vs separate services + +## Examples + +### Example 1: Pub/Sub Topic (broadcast) + +- GCP: `google_pubsub_topic` (name="user-events", message_retention_duration="7d") +- Signals: Broadcast events, multiple subscribers likely +- Criterion 1 (Eliminators): PASS (retention not critical for broadcast) +- Criterion 2 (Operational Model): SNS (pub/sub) +- → **AWS: SNS Topic (Standard)** +- Note: SNS does not support message retention like GCP Pub/Sub. If retention is critical, use SQS instead. +- Confidence: `inferred` + +### Example 2: Pub/Sub Topic (exactly-once) + +- GCP: `google_pubsub_topic` + `google_pubsub_subscription` (exactly_once_delivery=true) +- Signals: Exactly-once delivery required +- Criterion 1 (Eliminators): Exactly-once required → **use SNS FIFO + SQS FIFO** +- → **AWS: SNS FIFO Topic + SQS FIFO Queue (deduplication enabled)** +- Confidence: `inferred` + +### Example 3: Cloud Tasks Queue (scheduled) + +- GCP: `google_cloud_tasks_queue` (rate_limits=1000 msg/sec, retry_config=[max_retries=5]) +- Signals: Task scheduling, retry configuration +- Criterion 1 (Eliminators): PASS +- → **AWS: SQS (standard) + Lambda ScheduledEvents (for scheduling)** +- Confidence: `inferred` + +## Output Schema + +```json +{ + "gcp_type": "google_pubsub_topic", + "gcp_address": "user-events", + "gcp_config": { + "message_retention_duration": "604800s", + "subscribers": 3 + }, + "aws_service": "SNS", + "aws_config": { + "topic_name": "user-events", + "display_name": "User Events" + }, + "confidence": "inferred", + "rationale": "Pub/Sub with multiple subscribers → SNS (broadcast pattern)" +} +``` diff --git a/migration-to-aws/steering/design-ref-networking.md b/migration-to-aws/steering/design-ref-networking.md index 34bc8b2d..22ac4d70 100644 --- a/migration-to-aws/steering/design-ref-networking.md +++ b/migration-to-aws/steering/design-ref-networking.md @@ -1,236 +1,125 @@ -# Networking Mappings +# Networking Services Design Rubric -## google_compute_network +**Applies to:** VPC, Firewall, Load Balancing, DNS, Cloud Interconnect, Cloud Armor -**Purpose:** Virtual private cloud network +**Quick lookup (no rubric):** Check `design-ref-fast-path.md` first (VPC → VPC, Firewall → Security Groups, etc.) -**Default:** aws_vpc (VPC) +## Eliminators (Hard Blockers) -**Candidates:** -- aws_vpc +| GCP Service | AWS | Blocker | +| -------------------- | -------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| Cloud Interconnect | Direct Connect | Dedicated connection (4-12 weeks setup) → use VPN as temp | +| Cloud Load Balancing | ALB | SSL certificate passthrough → NLB (L4, pass-through) | +| Cloud Load Balancing | NLB | Host/path-based routing required → ALB (L7) | +| Firewall Rules | Security Group | Ingress on port 22, 3389, or 5900 with source `0.0.0.0/0` or `::/0` → **BLOCKED** — see Administrative Port Safety Rule below | -**Signals:** None — direct 1:1 mapping. +## Signals (Decision Criteria) -**Eliminators:** None +### VPC Network -**Peek at secondaries:** No +- Always → AWS VPC (1:1 deterministic) +- Preserve CIDR blocks, subnets, routing tables -**1:Many Expansion:** +### Firewall Rules -A GCP VPC always expands to multiple AWS resources: -- aws_vpc — primary -- aws_internet_gateway — GCP VPCs have implicit internet access, AWS requires explicit IGW -- aws_route_table — at least one for public subnets -- aws_route — default route to IGW +- Always → AWS Security Groups (1:1 deterministic) +- Convert direction (ingress/egress) and IP ranges -**Note on subnets:** GCP VPC subnets are separate resources (google_compute_subnetwork). On AWS, you typically need subnets in at least 2 AZs for services like RDS and ECS. The number of AWS subnets may exceed the number of GCP subnets. This is handled in the secondary mapping for google_compute_subnetwork. +### Cloud Load Balancing -**Source Config to Carry Forward:** -- name — determines VPC name (becomes VPC tag or resource identifier) -- auto_create_subnetworks — if true, GCP auto-creates subnets in every region. AWS doesn't have this concept; subnets must be explicit. -- routing_mode — REGIONAL is standard (AWS VPCs are regional by nature). GLOBAL has no AWS equivalent. +- **HTTP/HTTPS + hostname/path routing** → ALB (Layer 7) +- **TCP/UDP + high throughput** → NLB (Layer 4) +- **TLS passthrough** → NLB (Layer 4, no termination) +- For internet-facing ALB: terminate TLS on 443 and configure 80 as redirect-only to HTTPS (no direct HTTP forwarding). ---- +### Cloud DNS -## google_compute_subnetwork +- Always → Route 53 (1:1 deterministic) +- Preserve zone name, record types, TTLs -**Purpose:** Subnet within a VPC network +### Cloud Interconnect -**Default:** aws_subnet +- **Dedicated connection** → AWS Direct Connect +- **Temporary/dev connectivity** → AWS Site-to-Site VPN (quicker, lower cost) -**Candidates:** -- aws_subnet +### Cloud Armor -**Signals:** None — direct 1:1 mapping. +- **DDoS protection + WAF rules** → AWS WAF + AWS Shield Standard (Shield Standard is automatic, no extra cost) +- **Rate limiting** → AWS WAF rate-based rules +- **Bot management** → AWS WAF Bot Control +- **IP allowlist/denylist** → AWS WAF IP set rules -**Eliminators:** None +## 6-Criteria Rubric -**Peek at secondaries:** No +Apply in order: -**1:Many Expansion:** -- aws_subnet — primary -- aws_route_table — if custom routes defined -- aws_network_acl — if custom access control needed +1. **Eliminators**: Does GCP config require AWS-unsupported features? If yes: switch +2. **Operational Model**: Managed (ALB, Route 53) vs Custom (VPN, custom routing)? + - Prefer managed +3. **User Preference**: From `preferences.json`: `design_constraints.compliance`? + - **PCI or HIPAA:** Neither framework mandates Direct Connect. **Bias toward documented private connectivity** between sites and AWS (e.g. **AWS Direct Connect** or **Site-to-Site VPN** with encryption, monitoring, and change control) — choose with your **QSA / BAA / security team**; many compliant designs use VPN-only or no hybrid link when all workloads stay in AWS. + - **FedRAMP:** GovCloud and federal boundary requirements dominate; **private connectivity** is often part of the approved architecture — still **confirm with your authorizing official / security team**, not this advisor alone. + - If `compliance` includes `"ccpa"` (CCPA / CPRA) → VPN or Direct Connect both acceptable; prioritize **documented data paths**, retention controls, and logging for consumer privacy workflows — not a forced Direct Connect gate. + - If none of the above: VPN or public-internet paths are commonly acceptable when encrypted and documented. +4. **Feature Parity**: Does GCP config require AWS-unsupported features? + - Example: GCP policy-based routing → Custom route table rules (AWS does this) +5. **Cluster Context**: Are other resources in cluster using specific load balancers? Match +6. **Simplicity**: Fewer resources = higher score -**Source Config to Carry Forward:** -- name — determines subnet name -- ip_cidr_range — determines CIDR block -- region — determines AWS region (mapped from GCP region) -- network — determines VPC association (resolved from google_compute_network mapping) -- private_ip_google_access — determines DNS hostname type -- log_config — determines VPC Flow Logs configuration +## Examples ---- +### Example 1: VPC Network -## google_compute_address (Static IP) +- GCP: `google_compute_network` (auto_create_subnetworks=false, routing_mode=REGIONAL) +- Signals: Explicit subnets, regional routing +- Criterion 1 (Eliminators): PASS +- → **AWS: VPC (us-east-1 region)** +- Confidence: `deterministic` -**Purpose:** Static external IP address +### Example 2: Firewall Rules -**Default:** aws_eip (Elastic IP) or aws_internet_gateway (if network-level) +- GCP: `google_compute_firewall` (allow=[tcp:443], source_ranges=[0.0.0.0/0]) +- Signals: HTTPS ingress, public +- → **AWS: Security Group (ingress rule: 443/tcp from 0.0.0.0/0)** +- Confidence: `deterministic` -**Candidates:** -- aws_eip — for instance-level static IP -- aws_internet_gateway — for network NAT +### Example 3: Cloud Load Balancing (HTTP + path-based) -**Signals:** +- GCP: `google_compute_forwarding_rule` + `google_compute_backend_service` (path_matcher=["/api/*" → api-backend]) +- Signals: Path-based routing, HTTP/HTTPS +- Criterion 1 (Eliminators): PASS +- Criterion 2 (Operational Model): ALB (managed, L7) +- → **AWS: ALB with target groups + listener rules (path-based)** +- Confidence: `inferred` -| Config Field | Value | Favors | Weight | Reason | -|---|---|---|---|---| -| address_type | INTERNAL | aws_network_interface | moderate | Internal static IP | -| address_type | EXTERNAL | aws_eip | strong | External static IP | -| purpose | NAT | aws_nat_gateway | strong | NAT gateway for outbound NAT | -| purpose | GCE_ENDPOINT | aws_network_interface | moderate | Instance endpoint | -| network_tier | PREMIUM | aws_eip | moderate | Premium tier maps to standard EIP | -| network_tier | STANDARD | aws_eip | weak | Standard tier may have performance implications | +### Example 4: Cloud DNS Zone -**Eliminators:** None +- GCP: `google_dns_managed_zone` (dns_name="example.com.") +- Signals: Public DNS zone +- → **AWS: Route 53 Hosted Zone (example.com)** +- Confidence: `deterministic` -**Peek at secondaries:** No +## Output Schema -**1:Many Expansion (EIP):** -- aws_eip — primary -- aws_eip_association — if associated to instance - -**1:Many Expansion (NAT Gateway):** -- aws_nat_gateway — primary -- aws_eip — elastic IP for NAT gateway -- aws_route — for NAT routing - -**Source Config to Carry Forward:** -- address_type — determines EIP or NAT gateway type -- region — determines AWS region -- network — determines VPC (resolved from google_compute_network mapping) - ---- - -## google_compute_forwarding_rule (Load Balancer) - -**Purpose:** Load balancing and traffic forwarding - -**Default:** aws_lb (Load Balancer) - ALB or NLB depending on protocol - -**Candidates:** -- aws_lb with type = "application" (ALB for HTTP/HTTPS) -- aws_lb with type = "network" (NLB for TCP/UDP) - -**Signals:** - -| Config Field | Value | Favors | Weight | Reason | -|---|---|---|---|---| -| load_balancing_scheme | EXTERNAL | aws_lb | strong | Public load balancer | -| load_balancing_scheme | INTERNAL | aws_lb with internal=true | moderate | Internal load balancer | -| ip_protocol | TCP | aws_lb type network | strong | NLB for TCP | -| ip_protocol | UDP | aws_lb type network | strong | NLB for UDP | -| ip_protocol | HTTP | aws_lb type application | strong | ALB for HTTP | -| ip_protocol | HTTPS | aws_lb type application | strong | ALB for HTTPS | -| ports | 80, 443 | aws_lb type application | strong | ALB for web traffic | -| ports | 3306, 5432 | aws_lb type network | strong | NLB for database traffic | -| all_ports | true | aws_lb type network | strong | NLB for all ports (passthrough) | - -**Eliminators:** None - -**Peek at secondaries:** Yes — must check google_backend_service and google_target_pool resources - -**1:Many Expansion (ALB - Application Load Balancer):** -- aws_lb with type = "application" -- aws_lb_listener — one per port/protocol -- aws_lb_target_group — one per backend service -- aws_lb_listener_rule — for path/host-based routing -- aws_security_group — for ALB ingress rules - -**1:Many Expansion (NLB - Network Load Balancer):** -- aws_lb with type = "network" -- aws_lb_listener — one per port/protocol -- aws_lb_target_group — one per backend service -- aws_security_group — for NLB ingress rules (if using security groups) - -**Source Config to Carry Forward:** -- name — determines load balancer name -- ip_protocol — determines ALB vs NLB -- ports — determines listener ports -- load_balancing_scheme — determines internal vs external -- region — determines AWS region -- network / subnetwork — determines VPC and subnet placement - ---- - -## Secondary: google_compute_subnetwork - -**Behavior:** Subnets are network_path secondaries that define which subnets exist within a VPC. Each subnet maps to an AWS subnet. - -**Mapping Behavior:** - -When google_compute_network maps to: -- aws_vpc → Each google_compute_subnetwork becomes a separate aws_subnet - -**Implementation Notes:** -1. GCP subnet CIDR range carries forward to aws_subnet cidr_block -2. GCP subnet region determines AWS availability zone (may need adjustment for multi-AZ) -3. Each subnet gets a route table association -4. **Important:** AWS typically requires subnets in at least 2 AZs for HA services (RDS, ECS) - - If GCP has 1 subnet, create 2-3 AWS subnets across AZs - - This may result in MORE AWS subnets than GCP subnets -5. Private IP Google Access (GCP) → DNS hostname type on AWS subnet -6. VPC Flow Logs configuration → aws_flow_log resource - -**Skip Condition:** -If the primary google_compute_network is skipped, skip these secondaries. However, AWS requires at least one subnet, so plan accordingly. - ---- - -## Secondary: google_compute_firewall - -**Behavior:** Firewalls are network_path secondaries that define network access control rules. They map to AWS security groups. - -**Mapping Behavior:** - -When firewall serves compute resource: -- Firewall rules become aws_security_group with ingress/egress rules attached to compute resources - -When firewall serves network resource: -- Becomes aws_security_group that can be referenced by multiple compute resources - -**Rule Translation:** -- GCP firewall `allow` rules → aws_security_group ingress rules -- GCP firewall `deny` rules → aws_security_group deny rules (or use separate deny policy) -- GCP firewall `sourceRanges` → security group CIDR blocks -- GCP firewall `targetTags` → security group resource associations (EC2 tags) -- GCP firewall ports/protocols → security group port/protocol specifications - -**Implementation Notes:** -1. Firewall direction (ingress/egress) must be explicitly set in AWS security groups -2. GCP has implicit default allow; AWS is implicit deny -3. Stateful rules: GCP firewalls are stateful, AWS security groups are stateful -4. Named ranges and service accounts in rules must be expanded to actual IPs/IDs -5. Multiple rules may need to be combined into single security group - -**Skip Condition:** -If the primary compute resource is skipped, firewall rules can be skipped. Otherwise, create security group for access control. - ---- - -## Cloud Interconnect - -**Purpose:** Dedicated/partner connectivity between on-premises and cloud - -**Default:** AWS Direct Connect - -**Rationale for default:** Direct Connect is the 1:1 equivalent for dedicated enterprise connectivity. For development or temporary migration needs, Site-to-Site VPN is a faster, lower-cost alternative. - -**Candidates:** -- AWS Direct Connect — dedicated connection, lowest latency, highest bandwidth -- AWS Site-to-Site VPN — encrypted tunnel over public internet, quicker setup - -**Signals:** - -| Signal | Favors | Weight | Reason | -|---|---|---|---| -| Dedicated interconnect (10/100 Gbps) | Direct Connect | strong | Requires dedicated fiber; 6+ months setup | -| Partner interconnect | Direct Connect (hosted) | moderate | Partner-facilitated, faster setup | -| Development/temporary connectivity | Site-to-Site VPN | strong | Quicker, lower cost, sufficient for migration | -| Compliance (PCI, HIPAA, FedRAMP) | Direct Connect | strong | Explicit data path required | - -**Migration Notes:** -- Direct Connect requires physical provisioning (plan 2-6 months lead time) -- During migration, use Site-to-Site VPN as temporary bridge -- BGP configuration must be adapted from GCP Cloud Router to AWS VGW/TGW +```json +{ + "gcp_type": "google_compute_forwarding_rule", + "gcp_address": "global-https-lb", + "gcp_config": { + "load_balancing_scheme": "EXTERNAL", + "protocol": "HTTPS" + }, + "aws_service": "Application Load Balancer", + "aws_config": { + "load_balancer_type": "application", + "scheme": "internet-facing", + "listener": { + "protocol": "HTTPS", + "port": 443 + }, + "region": "us-east-1" + }, + "confidence": "inferred", + "rationale": "Rubric: GCP global HTTPS LB → AWS ALB (L7, host/path routing)" +} +``` diff --git a/migration-to-aws/steering/design-ref-security.md b/migration-to-aws/steering/design-ref-security.md new file mode 100644 index 00000000..4c7e40a8 --- /dev/null +++ b/migration-to-aws/steering/design-ref-security.md @@ -0,0 +1,71 @@ +# Security Services Design Rubric + +**Applies to:** Secret Manager and encryption/identity-adjacent security resources. + +## Account-Wide Security Baseline + +The plugin always emits `baseline.tf` with account-wide security controls — GuardDuty, CloudTrail (multi-region, log file validation), IMDSv2 enforcement, EBS encryption by default, S3 public access block, Access Analyzer, and budget alerts. For compliance stacks (soc2/pci/hipaa/fedramp), Config and Security Hub are added conditionally. + +**Do not duplicate these controls in `security.tf` or other domain files.** See `steering/generate-artifacts-infra.md` Step 1.5 for the full baseline specification. + +--- + +## Deterministic mappings + +| GCP Service Type | AWS Service | Notes | +| -------------------------------------- | --------------- | ------------------------------------------------------------------------------------------ | +| `google_secret_manager_secret` | Secrets Manager | Create one secret metadata resource. | +| `google_secret_manager_secret_version` | Secrets Manager | Represent current value as a secret version or migration TODO if plaintext is unavailable. | + +## Secret migration rules + +1. Never place cleartext secrets directly in generated Terraform variable defaults. +2. Generate Secrets Manager resources and reference them from compute/database resources via ARN, not plaintext environment values. +3. If source secret values are not available from discovery inputs, generate TODO placeholders with explicit migration steps. +4. Add least-privilege IAM access scoped to specific secret ARNs. + +--- + +## Security Group Patterns + +Apply these patterns when generating `vpc.tf`. Start with no access and add only what is needed. + +| Component | Default Inbound | Default Outbound | +| ------------ | ------------------------------- | ------------------ | +| ALB | 443 from 0.0.0.0/0; 80 redirect | Fargate SG only | +| Fargate | ALB SG only (on app port) | 443 (HTTPS), DB SG | +| EKS nodes | ALB SG only (on app port) | 443 (HTTPS), DB SG | +| RDS/Aurora | Fargate SG only (on DB port) | None | +| ElastiCache | Fargate SG only (on cache port) | None | +| Lambda (VPC) | None | 443, DB SG | + +**Hard rules:** + +- Never emit a security group ingress rule with `cidr_blocks = ["0.0.0.0/0"]` for ports 22 (SSH), 3389 (RDP), or 5900 (VNC). Emit a commented-out placeholder pointing to SSM Session Manager instead. +- ALB port 80 must redirect to 443 — never forward HTTP directly. +- All compute (Fargate, EKS nodes, EC2) must be in private subnets. ALB is the only public-facing component. + +--- + +## IaC Security Scanning + +After generating Terraform, recommend running a security scanner before `terraform apply`: + +| Tool | Purpose | Command | +| ----------- | -------------------------------------------- | ------------------------- | +| **checkov** | Multi-framework IaC scanner (Terraform, etc) | `checkov -d terraform/` | +| **tfsec** | Terraform-specific security scanner | `tfsec terraform/` | +| **trivy** | IaC + container image scanning | `trivy config terraform/` | + +Include this recommendation in the generated `terraform/README.md` under a "Before You Apply" section. Do not block artifact generation on scanner availability — recommend it as a post-generation step. + +--- + +## Costing handoff + +When `aws_service = "Secrets Manager"` is selected, estimate: + +- Per-secret monthly storage cost +- API call cost (per 10K requests) + +Use `steering/cached-prices.md` first; use `estimate-infra.md` MCP fallback recipes only if needed. diff --git a/migration-to-aws/steering/design-ref-storage.md b/migration-to-aws/steering/design-ref-storage.md index 8e5d176d..53cfdcbe 100644 --- a/migration-to-aws/steering/design-ref-storage.md +++ b/migration-to-aws/steering/design-ref-storage.md @@ -2,7 +2,7 @@ **Applies to:** Cloud Storage (GCS), Filestore -**Quick lookup (no rubric):** Check `fast-path.md` first (Cloud Storage → S3, deterministic) +**Quick lookup (no rubric):** Check `design-ref-fast-path.md` first (Cloud Storage → S3, deterministic) ## Deterministic Mapping @@ -14,21 +14,22 @@ Confidence: `deterministic` (always 1:1, no decision tree) - Bucket versioning → S3 versioning - Lifecycle rules → S3 Lifecycle policies -- Access control (UNIFORM vs FINE-GRAINED) → S3 ACLs + Bucket Policies +- Access control (UNIFORM) → S3 Object Ownership "BucketOwnerEnforced" (ACLs disabled) + Bucket Policies +- Access control (FINE-GRAINED) → S3 Bucket Policies (ACLs disabled; translate per-object grants to policy conditions) - Regional location → S3 region selection - Encryption (default or CSEK) → S3 encryption (default AES-256 or KMS) ## GCS → S3 Attribute Mapping -| GCS Attribute | S3 Equivalent | Notes | -| ----------------------------- | ------------------------------------------- | -------------------------------------------- | -| `location` (region) | `region` | Direct mapping; respect user's region choice | -| `versioning_enabled` | `versioning_enabled` | 1:1 copy | -| `lifecycle_rules` | `lifecycle_rule` | Adapt rule conditions | -| `uniform_bucket_level_access` | `block_public_acl` + policies | Convert UNIFORM to S3 ACL block | -| `encryption` (CSEK) | `sse_algorithm = "aws:kms"` | Use AWS KMS (customer-managed key) | -| `cors` | `cors_rule` | 1:1 copy | -| `retention_policy` | `object_lock_configuration` (if applicable) | Object Lock stricter than GCS retention | +| GCS Attribute | S3 Equivalent | Notes | +| ----------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------- | +| `location` (region) | `region` | Direct mapping; respect user's region choice | +| `versioning_enabled` | `versioning_enabled` | 1:1 copy | +| `lifecycle_rules` | `lifecycle_rule` | Adapt rule conditions | +| `uniform_bucket_level_access` | `object_ownership = "BucketOwnerEnforced"` + bucket policy | ACLs disabled (AWS default since Apr 2023); use bucket policies for access control | +| `encryption` (CSEK) | `sse_algorithm = "aws:kms"` | Use AWS KMS (customer-managed key) | +| `cors` | `cors_rule` | 1:1 copy | +| `retention_policy` | `object_lock_configuration` (if applicable) | Object Lock stricter than GCS retention | ## Output Schema @@ -68,68 +69,43 @@ Confidence: `deterministic` (always 1:1, no decision tree) **Filestore (`google_filestore_instance`) → EFS (`aws_efs_file_system`)** -Confidence: `deterministic` (both managed NFS, 1:1 mapping) +Confidence: `deterministic` (both are managed NFS file systems) -**Behavior preservation:** - -- Managed NFS file system → Managed NFS file system -- Performance tier selection → EFS throughput mode -- Network-attached storage → VPC mount targets -- Shared file access across instances → Shared file access across instances +**Attribute mapping:** -### Filestore → EFS Attribute Mapping +| Filestore Attribute | EFS Equivalent | Notes | +| ------------------- | --------------------------------- | ------------------------------------------ | +| `tier` (STANDARD) | `throughput_mode = "bursting"` | General purpose, burstable throughput | +| `tier` (PREMIUM) | `throughput_mode = "provisioned"` | High-performance, provisioned throughput | +| `capacity_gb` | No pre-provisioned size | EFS scales automatically (no capacity set) | +| `network` | `mount_target` subnet | Place mount targets in same VPC subnets | +| `file_shares.name` | Mount target path | Preserve share name | -| Filestore Attribute | EFS Equivalent | Notes | -| ------------------------- | ----------------------------------------- | -------------------------------------------------------- | -| `tier` (STANDARD) | `throughput_mode = "bursting"` | Standard performance maps to bursting throughput | -| `tier` (PREMIUM) | `throughput_mode = "provisioned"` | Premium performance maps to provisioned throughput | -| `tier` (ENTERPRISE) | `throughput_mode = "provisioned"` | Enterprise maps to provisioned with higher IOPS | -| `capacity_gb` | (no direct equivalent — EFS auto-scales) | EFS grows/shrinks automatically; no pre-provisioned size | -| `networks[].network` | `aws_efs_mount_target.subnet_id` | Mount target placed in mapped VPC subnet | -| `file_shares[].name` | Mount target path | Share name becomes mount path convention | -| `file_shares[].capacity` | (no direct equivalent) | EFS has no capacity limits per share | - -### Filestore → EFS Output Schema +**Output Schema:** ```json { "gcp_type": "google_filestore_instance", - "gcp_address": "my-app-nfs", + "gcp_address": "shared-data", "gcp_config": { "tier": "STANDARD", - "file_shares": [ - { - "name": "vol1", - "capacity_gb": 1024 - } - ], - "networks": [ - { - "network": "projects/my-project/global/networks/my-vpc", - "modes": ["MODE_IPV4"] - } - ] + "capacity_gb": 1024, + "network": "default" }, "aws_service": "EFS", "aws_config": { - "creation_token": "my-app-nfs", "throughput_mode": "bursting", "performance_mode": "generalPurpose", "encrypted": true, - "mount_targets": [ - { - "subnet_id": "subnet-xxx", - "security_groups": ["sg-xxx"] - } - ] + "region": "us-east-1" }, "confidence": "deterministic", - "rationale": "Filestore → EFS is 1:1 deterministic; both are managed NFS. EFS auto-scales (no capacity_gb equivalent)." + "rationale": "Filestore → EFS is 1:1 deterministic; both are managed NFS" } ``` ## Notes -Cloud Storage and Filestore have direct AWS equivalents with no decision tree required. All mappings are deterministic. +Cloud Storage has no AWS equivalent variations. All mappings are direct. For non-storage use cases (static site hosting, data lakes, etc.), the hosting compute service (Fargate, Amplify) determines architecture, not the bucket itself. diff --git a/migration-to-aws/steering/design.md b/migration-to-aws/steering/design.md index 11dc45ca..7990830e 100644 --- a/migration-to-aws/steering/design.md +++ b/migration-to-aws/steering/design.md @@ -4,7 +4,8 @@ ## Prerequisites -Read `$MIGRATION_DIR/preferences.json`. If missing: **STOP**. Output: "Phase 2 (Clarify) not completed. Run Phase 2 first." +1. Read `$MIGRATION_DIR/.phase-status.json`. If missing, invalid, or `phases.clarify` is not exactly `"completed"`: **STOP**. Output: "Phase 2 (Clarify) not completed or phase state is missing/invalid. Run `steering/clarify.md` until Clarify finishes and `.phase-status.json` shows `phases.clarify`: `completed`." +2. Read `$MIGRATION_DIR/preferences.json`. If missing: **STOP**. Output: "Phase 2 (Clarify) not completed. Run Phase 2 first." Check which discovery artifacts exist in `$MIGRATION_DIR/`: @@ -48,16 +49,42 @@ Produces: `aws-design-ai.json` ## Phase Completion -After all applicable sub-designs finish, use the Phase Status Update Protocol (Write tool) to write `.phase-status.json` with `phases.design` set to `"completed"` — **in the same turn** as the output message below. +Before marking Design complete, enforce route output gates (fail closed): + +1. Determine which design routes ran: + - IaC route: `gcp-resource-inventory.json` AND `gcp-resource-clusters.json` exist + - Billing-only route: `billing-profile.json` exists AND `gcp-resource-inventory.json` does NOT exist + - AI route: `ai-workload-profile.json` exists +2. Require at least one route to be active. If none active: STOP. +3. For each active route, require its expected artifact: + - IaC route -> `aws-design.json` + - Billing-only route -> `aws-design-billing.json` + - AI route -> `aws-design-ai.json` +4. If any active route is missing its expected output: STOP and output: "Design route [name] did not produce required artifact(s). Re-run the failed sub-design before completing Phase 3." + +## Completion Handoff Gate (Fail Closed) + +Load `steering/handoff-gates.md`. **Re-read from disk** each active route artifact before checking. + +**Re-entry guard:** If `estimation-infra.json` (or sibling estimate artifacts) exists and `phases.estimate` is `"completed"`: STOP unless the user explicitly confirms re-running Design. Emit `GATE_FAIL | phase=design | field=estimation-infra.json | reason=stale_downstream`. + +**On any route gate FAIL:** Emit `GATE_FAIL | phase=design | field= | reason=missing`. **Do NOT modify artifacts to pass the gate.** **Do NOT update `.phase-status.json`.** + +**On PASS:** Emit `HANDOFF_OK | phase=design | artifacts=`. + +After `HANDOFF_OK`, use the Phase Status Update Protocol (read-merge-write) to update `.phase-status.json` — **in the same turn** as the output message below: + +- Set `phases.design` to `"completed"` +- Set `current_phase` to `"estimate"` Output to user: "AWS Architecture designed. Proceeding to Phase 4: Estimate Costs." ## Reference Files -Sub-design files may reference rubrics in `steering/`: +Sub-design files may reference rubrics in the `steering/design-ref-*.md` files: - `steering/design-ref-index.md` — GCP type → rubric file lookup -- `steering/design-ref-fast-path.md` — Deterministic 1:1 GCP→AWS mappings +- `steering/design-ref-fast-path.md` — Direct (table) mappings vs rubric path; **User-facing vocabulary** for presenting `confidence` to users (**Standard pairing** / **Tailored to your setup** / **Estimated from billing only**) - `steering/design-ref-compute.md` — Compute service rubric - `steering/design-ref-database.md` — Database service rubric - `steering/design-ref-storage.md` — Storage service rubric diff --git a/migration-to-aws/steering/discover-app-code.md b/migration-to-aws/steering/discover-app-code.md index d2750082..89a4f9b0 100644 --- a/migration-to-aws/steering/discover-app-code.md +++ b/migration-to-aws/steering/discover-app-code.md @@ -30,8 +30,37 @@ Recursively scan the entire target directory tree for source code and dependency **Exit gate:** If NO source code files or dependency manifests are found, **exit cleanly**. Return no output artifacts. Other sub-discovery files may still produce artifacts. +**Secret file exclusion (HARD — no exceptions):** Before scanning any file, skip the following paths entirely — do NOT read, parse, or include their contents in any output artifact: + +- `.env*` (matches `.env`, `.env.local`, `.env.production`, `.env.staging`, and any other `.env` variant) +- `credentials.json`, `service-account.json`, `*-service-account.json` +- `*.pem`, `*.key`, `*.p12`, `*.pfx` +- `secrets.yaml`, `secrets.yml` + +If any of these files are encountered during the recursive scan, log: "Skipped [filename] — potential secret file excluded from discovery scope." Do NOT include them in source file counts or any output. + --- +## Step 0.5: Auth SDK Exclusion List + +Before scanning for GCP imports, check for third-party auth SDK imports. These are **recognized but excluded** from migration — they carry no AWS recommendation. + +| Import Pattern | Auth Provider | +| ------------------------------------------------------------------------ | ------------------ | +| `auth0` / `@auth0/` / `auth0-python` | Auth0 | +| `@supabase/supabase-js` / `supabase` (with `.auth`) | Supabase Auth | +| `firebase-admin` (with `.auth`) / `firebase/auth` / `@angular/fire/auth` | Firebase Auth | +| `@clerk/` / `clerk-sdk-python` | Clerk | +| `@okta/` / `okta-sdk-python` / `okta-jwt-verifier` | Okta | +| `keycloak` / `@keycloak/keycloak-admin-client` / `python-keycloak` | Keycloak | +| `next-auth` / `@auth/core` | NextAuth / Auth.js | + +If any auth SDK import is detected: + +1. Log: "Detected [provider] auth SDK in [file] — excluded from migration scope. Keep your existing auth solution." +2. Do **not** infer a GCP resource or recommend an AWS replacement +3. Do **not** include in the AI signal scan or any output artifact + ## Step 1: Detect GCP SDK Imports Scan source files for GCP service imports: @@ -70,6 +99,26 @@ Confidence for inferred resources: 0.60-0.80 (inferring existence, not reading i --- +## Step 2.5: WebSocket Signal Scan + +After Step 2 (Infer Resources from Code), scan source files for WebSocket / long-lived connection patterns: + +| Pattern | Evidence | +| --------------- | ------------------------------------------------------------------------------------------------------ | +| WebSocket API | `websocket`, `WebSocket`, `socket.io`, `@nestjs/websockets`, FastAPI `WebSocket`, `ws` package imports | +| SSE / long-poll | `EventSource`, `text/event-stream`, long-polling handlers | + +Record in discovery metadata (do not write a separate file): + +- `websocket_signals_found`: `true` if any pattern matches active (non-commented) code +- `websocket_signal_files`: array of file paths where matches were found + +If **no matches**, set `websocket_signals_found: false`. Clarify Step 2 uses this to skip Q9 with `websocket: false` extracted **only when this step ran** (source files were found). + +If **no source code files were found** (Step 0 exit gate), do **not** set `websocket_signals_found` — Clarify must ask Q9; absence of a scan is not evidence of no WebSockets. + +--- + ## Step 3: Flag AI Signals Scan source code files and dependency manifests for AI-relevant patterns. For each match, record the pattern, file location, and confidence score. @@ -99,6 +148,56 @@ Also check dependency manifests for AI/ML SDK dependencies: --- +## Step 3B: Agentic Framework Signals + +Scan source code and dependency manifests for agentic framework patterns. These are separate from AI model signals — they indicate agent orchestration, not just LLM usage. + +| Pattern | What to look for | Confidence | +| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | +| 3B.1 LangGraph | `from langgraph` imports, `StateGraph(`, `add_node(`, `add_edge(`, `.compile()` | 95% | +| 3B.2 CrewAI | `from crewai` imports, `Crew(`, `Agent(` with `role=`, `Task(` class definitions | 95% | +| 3B.3 AutoGen | `from autogen` imports, `AssistantAgent(`, `GroupChat(`, `ConversableAgent(` | 95% | +| 3B.4 OpenAI Agents SDK | `from openai.agents` or `from agents import`, `openai.beta.assistants`, `Runner(` | 95% | +| 3B.5 Strands Agents | `from strands` imports, `Agent(` with `tools=`, `Swarm(`, `GraphBuilder(` | 95% | +| 3B.5a Pydantic AI | `from pydantic_ai import Agent`, `from pydantic_ai.models` imports, `Agent(model=`, `.run_sync(`, `.run(` | 95% | +| 3B.5b Agno | `from agno.agent import Agent`, `from agno.models` imports, `Agent(model=`, `Team(`, `.print_response(` | 95% | +| 3B.6 Custom agent loop | `while` loop body containing BOTH an LLM call (completions/generate) AND tool execution (function dispatch from model output) AND result parsing back to model | 80% | +| 3B.7 Tool definitions | `@tool` decorators, `function_declarations=`, `tools=[{...schema...}]`, tool schema objects with `name`+`description`+`parameters` | 90% | +| 3B.8 MCP integration | `from mcp.server` or `from mcp.client`, `mcp.json` config files, `MCPClient(` | 90% | +| 3B.9 Agent memory | `ConversationBufferMemory(`, `ChatMessageHistory(`, `MemorySaver(`, vector store retrieval feeding agent context | 85% | + +Also check dependency manifests for agentic framework dependencies: + +- `langgraph` (LangGraph) +- `crewai` (CrewAI) +- `pyautogen` or `autogen` (AutoGen) +- `openai` with agents imports (OpenAI Agents SDK — same package, different import path) +- `strands-agents` (Strands Agents SDK) +- `pydantic-ai` (Pydantic AI) +- `agno` (Agno) +- `mcp` (Model Context Protocol SDK) + +--- + +## Step 3.5: Agentic Classification Gate + +Determine whether the codebase contains agentic workflows. Execute these rules in order: + +1. If ANY framework signal from 3B.1–3B.5b detected → set `is_agentic: true`, set `framework` to the detected framework name +2. If NO framework signal BUT 3B.6 (custom agent loop) detected → set `is_agentic: true`, set `framework: "custom"` +3. If 3B.7 (tool definitions) detected WITH an LLM call in a loop pattern → set `is_agentic: true`, set `framework: "custom"` +4. If ONLY 3B.8 (MCP integration) detected WITHOUT an agent loop → set `is_agentic: false` (MCP alone does not imply agent orchestration) +5. If ONLY 3B.9 (agent memory) detected WITHOUT an agent loop or framework → set `is_agentic: false` +6. If NO agentic signals detected → set `is_agentic: false` + +**If multiple frameworks detected:** Set `framework` to the one with the most agent definitions. Record others in `detection_signals`. + +**If `is_agentic: false`:** Skip Steps 5.5 and 6.5. Continue to Step 4. + +**If `is_agentic: true`:** Continue to Step 4, then execute Steps 5.5 and 6.5 after Step 5. + +--- + ## Step 4: AI Detection Gate Compute overall AI confidence from all signals found in Step 3: @@ -125,7 +224,10 @@ Before finalizing AI detection, verify signals are genuine: - **Vector database alone is not AI** — Require embeddings library imports (langchain, llama-index). A Firestore/Datastore by itself is a regular database. - **Dead/commented-out code excluded** — Only count active code. -**Exit gate:** If overall AI confidence < 70%, **exit cleanly**. Do not generate `ai-workload-profile.json`. Report to the parent orchestrator: signals found, confidence level, and reason for not generating the AI profile. The inferred resources from Steps 1-2 remain available for other sub-files (e.g., discover-iac.md may use them for evidence merge). If no other sub-discoverer produces artifacts, the parent orchestrator will inform the user to provide Terraform files or billing exports. +**Exit gate:** If overall AI confidence < 70%: + +- **If** `$MIGRATION_DIR/ai-workload-profile.json` **already exists** with `metadata.profile_source` = `"iac_vertex"` (from `discover-iac.md` Step 7d — strong Vertex Terraform): **exit cleanly** without deleting or modifying that file. Report to the parent orchestrator: signals found, confidence below 70%, and that the **IaC-inferred AI profile is retained** for Clarify. +- **Otherwise:** Do **not** generate `ai-workload-profile.json`. Report signals found, confidence level, and reason for not generating the AI profile. The inferred resources from Steps 1-2 remain available for other sub-files (e.g., discover-iac.md may use them for evidence merge). If no other sub-discoverer produces artifacts, the parent orchestrator will inform the user to provide Terraform files or billing exports. **If confidence >= 70%**, continue to Steps 5-8 below. @@ -145,16 +247,19 @@ Scan files that contained AI signals for specific model information: - `GenerativeModel("gemini-pro")` -> model_id: `"gemini-pro"` - `aiplatform.Model.list(filter='display_name="my-model"')` -> model_id: `"my-model"` - `TextEmbeddingModel.from_pretrained("text-embedding-004")` -> model_id: `"text-embedding-004"` + - Look for Gemini model string patterns: `gemini-pro`, `gemini-1.5-*`, `gemini-2.0-*`, `gemini-2.5-*`, `gemini-3-*`, `gemini-3.1-*` — including versioned variants like `gemini-2.5-flash-latest`, `gemini-2.5-flash-preview-*`, `gemini-2.5-flash-thinking`, `gemini-2.5-pro-preview-*`. Normalize all `gemini-2.5-flash-*` variants to `model_id: "gemini-2.5-flash"` (or `"gemini-2.5-flash-thinking"` if the thinking variant is explicit). Normalize all `gemini-2.5-pro-*` variants to `model_id: "gemini-2.5-pro"`. Record the raw string in `detection_signals` for reference. + - **Gemini 1.5 Legacy:** Normalize `gemini-1.5-flash*` → `model_id: "gemini-1.5-flash"` and `gemini-1.5-pro*` → `model_id: "gemini-1.5-pro"`. Set `lifecycle_status: "legacy"` and `eol_date: "2025-09-24"` on these model entries in `ai-workload-profile.json`. Surface in the AI Context Summary during Clarify: "⚠️ Gemini 1.5 Flash/Pro is past EOL (Sep 2025). Plan to upgrade the source model to 2.5 Flash/Pro as part of or before the AWS migration." **OpenAI patterns:** - `client.chat.completions.create(model="gpt-4o")` -> model_id: `"gpt-4o"` - `openai.ChatCompletion.create(model="gpt-4")` -> model_id: `"gpt-4"` (legacy API) - `client.embeddings.create(model="text-embedding-3-small")` -> model_id: `"text-embedding-3-small"` - Model strings in config files or environment variables: `OPENAI_MODEL`, `MODEL_NAME`, etc. - - Look for model string patterns: `gpt-*`, `o1*`, `o3*`, `o4*`, `text-embedding-*`, `dall-e-*`, `whisper-*`, `tts-*` + - Look for model string patterns: `gpt-*`, `o1*`, `o3*`, `o4*`, `text-embedding-*`, `dall-e-*`, `gpt-image-*`, `whisper-*`, `tts-*` - **Other provider patterns:** + **Anthropic patterns:** - `anthropic.Anthropic().messages.create(model="claude-*")` -> model_id: `"claude-*"` + - Look for model string patterns: `claude-3-*`, `claude-sonnet-*`, `claude-haiku-*`, `claude-opus-*` - **Capabilities used** — Determine from API calls and method signatures: - `text_generation`: `generate_content()`, `predict()`, `messages.create()`, `chat.completions.create()` @@ -164,7 +269,7 @@ Scan files that contained AI signals for specific model information: - `embeddings`: `TextEmbeddingModel`, `VertexAIEmbeddings`, `client.embeddings.create()`, embedding API calls - `batch_processing`: batch predict calls, bulk processing patterns - `json_mode`: `response_format={"type": "json_object"}` (OpenAI), structured output schemas - - `image_generation`: `client.images.generate()` (DALL-E), Imagen API calls + - `image_generation`: `client.images.generate()` (gpt-image / DALL-E legacy), Imagen API calls - `speech_to_text`: `client.audio.transcriptions.create()` (Whisper) - `text_to_speech`: `client.audio.speech.create()` (TTS) @@ -187,6 +292,134 @@ Scan files that contained AI signals for specific model information: --- +## Step 5B: Disambiguate AI Workloads by SDK Method + +After extracting model details (Step 5), split the detected AI usage into distinct **workloads** — one per unique combination of `(model_id, sdk_method, structured_output)`. This enables downstream phases to produce one Bedrock recommendation per workload instead of collapsing multiple capabilities into a single recommendation. + +**Load** `steering/sdk-capability-map.json` from the plugin source. If missing or malformed, halt with diagnostic: `"[Discover] Failed to load sdk-capability-map.json"`. + +**For each AI call site detected in Steps 3–5:** + +1. **Resolve the SDK method** — the fully qualified method name (e.g., `ai.models.generateContent`, `openai.chat.completions.create`, `openai.images.generate`). Match against `sdk-capability-map.json` entries using exact, case-sensitive equality. + +2. **Detect structured output** — for methods in `structured_output_trio` (`chat.completions.create`, `messages.create`, `generateContent`), check if the call passes any of: `response_format`, `responseSchema`, or `responseMimeType: 'application/json'`. If yes → `structured_output: true`, `capability: "structured_output"`, `confidence: "high"`. If no → `structured_output: false`, `capability: "text_generation"`, `confidence: "medium"`. + +3. **Assign capability from map** — for methods NOT in the structured-output trio, use the map value directly with `confidence: "high"`. For methods not in the map at all → `capability: "unknown"`, `confidence: "low"`. + +4. **Collapse by workload tuple** — group call sites sharing the same `(model_id, sdk_method, structured_output)` into a single workload entry. Record all `call_sites` (file + line). + +5. **Generate workload_id** — deterministic hash: `"wl_" + sha256(model_id + "|" + sdk_method + "|" + ("structured" if structured_output else "plain"))[:6]` + +**Output:** Add a `workloads` array to the `ai-workload-profile.json` output alongside the existing `models[]` field. + +```json +"workloads": [ + { + "workload_id": "wl_3a1f2c", + "model_id": "gemini-2.5-flash", + "sdk_method": "ai.models.generateContent", + "capability": "text_generation", + "capability_confidence": "medium", + "structured_output": false, + "call_sites": [ + { "file": "lib/gemini.ts", "line": 11 } + ] + }, + { + "workload_id": "wl_8e22b4", + "model_id": "gemini-2.5-flash", + "sdk_method": "ai.models.generateContent", + "capability": "structured_output", + "capability_confidence": "high", + "structured_output": true, + "call_sites": [ + { "file": "lib/gemini.ts", "line": 27 } + ] + }, + { + "workload_id": "wl_c97d10", + "model_id": "imagen-3.0-generate-001", + "sdk_method": "ai.models.generateImages", + "capability": "image_generation", + "capability_confidence": "high", + "structured_output": false, + "call_sites": [ + { "file": "lib/imagen.ts", "line": 60 } + ] + } +] +``` + +**Rules:** + +- `workloads[]` is an empty array when no AI call sites are detected +- `call_sites[].file` uses repo-relative POSIX paths +- Every model_id in `workloads[]` MUST also appear in `models[]` (backward compatibility) +- If a method is in the structured-output trio AND the model is multimodal (Gemini, GPT-4o, Claude) AND structured output is NOT detected → set `capability_confidence: "medium"` (ambiguous — Clarify will confirm) + +--- + +**Skip this step entirely if `is_agentic: false` from Step 3.5.** + +For each detected agent in the codebase, extract: + +1. **Agent identifier** (`agent_id`) — Class name, variable name, or function name that defines the agent. Use snake_case normalized form (e.g., `ResearchAgent` → `"research-agent"`, `research_agent` → `"research-agent"`). + +2. **File and line** — Source file and line number where the agent is defined. + +3. **Model ID** — The LLM model this agent uses. Look for: + - Constructor parameter: `Agent(model="gpt-4o")`, `Agent(llm=ChatOpenAI(model="gpt-4o"))` + - Config reference: model string in config file referenced by agent + - Set to `"unknown"` if model is not extractable from code + +4. **Tools list** — Tool names attached to this agent. Look for: + - `tools=[web_search, calculator]` → `["web_search", "calculator"]` + - `@tool` decorated functions passed to agent + - Tool schema objects in `function_declarations` or `tools` parameter + - Set to `[]` if no tools detected + +5. **Memory type** — Classify per-agent memory: + - `"conversation_buffer"` — `ConversationBufferMemory`, `ChatMessageHistory`, message list accumulation + - `"rag"` — Vector store retrieval feeding agent context, `RetrievalQA`, knowledge base lookup + - `"none"` — No memory pattern detected for this agent + - `"unknown"` — Memory detected but type indeterminate + +6. **Role** — Human-readable description of what this agent does. Infer from: + - System prompt or `role=` parameter + - Class/function docstring + - Variable name and surrounding context + - Keep to one sentence + +**Classify orchestration pattern** from the overall agent architecture: + +| Pattern | Evidence | +| -------------- | ------------------------------------------------------------------------------------------------------------------------- | +| `single` | One agent definition, no delegation or coordination | +| `hierarchical` | One orchestrator agent delegates to specialist agents (agents-as-tools, manager/worker) | +| `swarm` | Multiple agents share memory/state, no fixed hierarchy (CrewAI `process=Process.hierarchical` is hierarchical, not swarm) | +| `graph` | Explicit node/edge definitions, conditional routing between agents (`StateGraph`, `GraphBuilder`, `add_edge`) | +| `sequential` | Agents execute in fixed order, output of one feeds next (`process=Process.sequential` in CrewAI, pipeline patterns) | +| `unknown` | Agentic signals detected but orchestration pattern unclear from code | + +**Determine system-level memory:** + +- `has_memory`: `true` if ANY agent has memory OR if shared memory store detected +- `memory_backend`: Classify from imports/config: + - `"redis"` — Redis client imports, `RedisMemory`, Redis URL in config + - `"postgres"` — PostgreSQL connection for memory/history storage + - `"in_memory"` — In-process memory only (default `ConversationBufferMemory`, Python dicts) + - `"vector_store"` — Pinecone, Weaviate, ChromaDB, pgvector for memory retrieval + - `"unknown"` — Memory detected but backend indeterminate + +**Determine human-in-the-loop:** + +- `has_human_in_loop`: `true` if any of: + - `HumanInputRun`, `human_input` tool, `input()` calls in agent loop + - Approval/confirmation patterns before tool execution + - `handoff_to_user` tool or similar delegation to human + +--- + ## Step 6: Map Integration Patterns Determine how the application integrates with AI services: @@ -218,19 +451,31 @@ Determine how the application integrates with AI services: Scan for these patterns and classify: - | Pattern | Gateway Type | Evidence | - | ------------------------------------------------------------ | ---------------- | ---------------------------------- | - | `from litellm import completion` / `litellm` in dependencies | `llm_router` | LiteLLM — multi-provider router | - | `base_url` containing `openrouter.ai` | `llm_router` | OpenRouter — multi-provider router | - | `portkey` imports or `x-portkey-` headers | `llm_router` | Portkey — AI gateway | - | `helicone` imports or `x-helicone-` headers | `llm_router` | Helicone — AI gateway | - | Kong, Apigee, or custom API gateway routing to AI endpoints | `api_gateway` | API gateway proxying AI calls | - | `from vapi_python import Vapi` / Vapi SDK | `voice_platform` | Vapi — voice AI platform | - | `bland` SDK or Bland.ai API calls | `voice_platform` | Bland.ai — voice AI platform | - | `retell` SDK or Retell API calls | `voice_platform` | Retell — voice AI platform | - | `from langchain` with provider imports | `framework` | LangChain orchestration framework | - | `from llama_index` with provider imports | `framework` | LlamaIndex orchestration framework | - | Direct SDK calls only (no router/gateway/framework) | `direct` | Direct API integration | + | Pattern | Gateway Type | Evidence | + | ----------------------------------------------------------------------------------------------------------------- | ---------------- | ----------------------------------------- | + | `from litellm import completion` / `litellm` in dependencies | `llm_router` | LiteLLM — multi-provider router | + | `base_url` containing `openrouter.ai` in source code | `llm_router` | OpenRouter — multi-provider router (code) | + | Env var `OPENAI_BASE_URL` containing `openrouter.ai` in `.env`, `docker-compose.yml`, CI config, or shell scripts | `llm_router` | OpenRouter — env-configured router | + | Env var `OPENROUTER_API_KEY` or `OR_API_KEY` present in `.env`, `docker-compose.yml`, CI config, or shell scripts | `llm_router` | OpenRouter — API key detected | + | Env var `LITELLM_PROXY_BASE_URL` or `LITELLM_API_KEY` present in any config file | `llm_router` | LiteLLM proxy — env-configured | + | `portkey` imports or `x-portkey-` headers | `llm_router` | Portkey — AI gateway | + | `helicone` imports or `x-helicone-` headers | `llm_router` | Helicone — AI gateway | + | Kong, Apigee, or custom API gateway routing to AI endpoints | `api_gateway` | API gateway proxying AI calls | + | `from vapi_python import Vapi` / Vapi SDK | `voice_platform` | Vapi — voice AI platform | + | `bland` SDK or Bland.ai API calls | `voice_platform` | Bland.ai — voice AI platform | + | `retell` SDK or Retell API calls | `voice_platform` | Retell — voice AI platform | + | `from langchain` with provider imports | `framework` | LangChain orchestration framework | + | `from llama_index` with provider imports | `framework` | LlamaIndex orchestration framework | + | Direct SDK calls only (no router/gateway/framework) | `direct` | Direct API integration | + + **Env var scan scope for gateway detection:** Check these files for `OPENAI_BASE_URL`, `OPENROUTER_API_KEY`, `OR_API_KEY`, `LITELLM_PROXY_BASE_URL`, `LITELLM_API_KEY`: + - `.env`, `.env.local`, `.env.production`, `.env.staging`, `.env.*` (any environment file) + - `docker-compose.yml`, `docker-compose.*.yml` + - `.github/workflows/*.yml`, `.gitlab-ci.yml`, `cloudbuild.yaml`, `Jenkinsfile` + - Shell scripts: `*.sh`, `Makefile` + - `fly.toml`, `railway.toml`, `render.yaml`, `vercel.json`, `netlify.toml` + + **Do NOT read secret values** — only check for the presence of the key name. Log: "Detected [KEY_NAME] in [file] — classified as llm_router (OpenRouter/LiteLLM)." Set `gateway_type` to `null` if no AI signals were detected or detection is ambiguous. @@ -251,6 +496,37 @@ A capability is `true` only if there is evidence from code analysis that it is a --- +## Step 6.5: Extract Tool Manifest (Only if `is_agentic: true`) + +**Skip this step entirely if `is_agentic: false` from Step 3.5.** + +Catalog all tool definitions found in agent code. This is inventory only — do NOT include migration recommendations, AWS equivalents, or effort estimates. + +For each tool detected in Step 5.5 (from agent `tools` lists) or Step 3B.7 (tool definitions): + +1. **Name** — Tool name as defined in code (function name, `name` field in schema, decorator argument) + +2. **File and line** — Source file and line number where the tool is defined or imported + +3. **Transport classification** — How the tool communicates: + - `"function"` — Local function with `@tool` decorator, no network calls in body, pure computation or file I/O + - `"api"` — HTTP client calls in tool body (`requests.get`, `httpx.post`, `fetch`, external URL references) + - `"mcp"` — MCP client imports, MCP server URL in tool config, tool served via MCP protocol + - `"unknown"` — Tool schema defined but implementation not inspectable (e.g., imported from external package) + +4. **Auth hint** — Detected authentication pattern in tool implementation: + - `"none"` — No credentials, tokens, or auth headers visible + - `"api_key"` — API key env var (`os.environ["API_KEY"]`), `Authorization: Bearer` header, `x-api-key` header + - `"oauth"` — OAuth flow, token refresh logic, `client_id`/`client_secret` patterns + - `"iam"` — AWS SigV4, boto3 client usage, IAM role assumption, GCP service account credentials + - `"unknown"` — Auth pattern not determinable from code inspection + +5. **Used by agents** — Array of `agent_id` values (from Step 5.5) that reference this tool. A tool may be used by multiple agents. + +**Deduplication:** If the same tool is used by multiple agents, create one `tool_manifest` entry with all agent IDs in `used_by_agents`. Do not duplicate tool entries. + +--- + ## Step 7: Capture Supporting Infrastructure **Only if Terraform files were found (IaC discovery also ran)**, extract AI-related infrastructure resources: @@ -272,6 +548,20 @@ If no Terraform files were provided, set `infrastructure: []`. Load `steering/schema-discover-ai.md` and generate output following the `ai-workload-profile.json` schema. +### Pre-existing IaC profile (`profile_source: "iac_vertex"`) + +If `$MIGRATION_DIR/ai-workload-profile.json` **already exists** with `metadata.profile_source` = `"iac_vertex"`: + +1. Execute Steps 5–7 to build the **code-derived** profile content as usual. +2. **Merge** into the final file written to `$MIGRATION_DIR/ai-workload-profile.json`: + - Set `metadata.profile_source` to **`"merged"`**. + - Set `metadata.sources_analyzed.terraform` to **`true`** when Terraform was present in the project (IaC discovery ran). + - Set `metadata.sources_analyzed.application_code` to **`true`**. + - **Code wins on conflict** for `models[]`, `integration`, `summary.ai_source`, `summary.overall_confidence`, `summary.confidence_level`, and `detection_signals` (code-derived signals take precedence; you may append non-duplicate Terraform `detection_signals` entries). + - **`infrastructure[]`:** Union by resource `address`. Include all Vertex-related entries from the IaC profile plus any from Step 7; where the same `address` appears, prefer the entry with richer `config` (typically Step 7 after code). + - Set `summary.inferred_from_iac` to **`false`** when Step 5–6 populated models or integration from code; if `models[]` is still empty after code analysis, keep `inferred_from_iac` consistent with whether Clarify still needs to disambiguate (default **`false`** once code path ran at ≥70% confidence). +3. If no pre-existing `iac_vertex` file, generate a fresh profile with `metadata.profile_source` = **`"application_code"`**. + **CRITICAL field names** — use EXACTLY these keys: - `model_id` (not model_name, name) @@ -288,13 +578,18 @@ Load `steering/schema-discover-ai.md` and generate output following the `ai-work - `"gemini"` — Only Gemini/Vertex AI generative models detected (patterns 2.3) - `"openai"` — Only OpenAI SDK/models detected (patterns 2.4) -- `"both"` — Both Gemini and OpenAI detected in the same codebase -- `"other"` — Other LLM providers (Anthropic, Cohere, etc.) or traditional ML only (no LLM) +- `"anthropic"` — Only Anthropic SDK detected (pattern 2.5, `anthropic` package, `claude-*` model strings) with no Gemini or OpenAI signals +- `"both"` — Both Gemini and OpenAI detected in the same codebase; or Anthropic + any other provider +- `"other"` — Traditional ML only (custom models, Vision API, Speech API, Document AI) with no LLM SDK detected + +**Note:** Anthropic SDK users are migrating to Bedrock-hosted Claude models, not to SageMaker. Setting `ai_source: "anthropic"` ensures Design routes to the correct Bedrock migration path rather than the traditional ML rubric. **Conditional sections:** - `current_costs` — Include ONLY if billing data was provided (billing discovery ran). Omit entirely if no billing data. - `infrastructure` — Set to `[]` if no Terraform files were provided (IaC discovery did not run). +- `agentic_profile` — Include ONLY if `is_agentic: true` from Step 3.5. Omit entirely if not agentic. +- `tool_manifest` — Include ONLY if `agentic_profile` exists. Set to `[]` if agentic but no tools detected in Step 6.5. After generating the output file, the parent `discover.md` handles the phase status update — do not update `.phase-status.json` here. @@ -302,10 +597,11 @@ After generating the output file, the parent `discover.md` handles the phase sta ## Output Validation Checklist — ai-workload-profile.json +- `metadata.profile_source` is one of: `"application_code"`, `"iac_vertex"`, `"merged"` - `metadata.sources_analyzed` reflects which data sources were actually provided - `summary.overall_confidence` matches the detection confidence from Step 4 - `summary.total_models_detected` matches the length of `models` array -- `summary.ai_source` is set correctly: `"gemini"`, `"openai"`, `"both"`, or `"other"` based on detected LLM SDKs +- `summary.ai_source` is set correctly: `"gemini"`, `"openai"`, `"anthropic"`, `"both"`, or `"other"` based on detected LLM SDKs - Every entry in `models` has `model_id`, `service`, `detected_via`, `evidence`, `capabilities_used`, and `usage_context` - `models[].detected_via` only contains sources that were actually analyzed (`"code"`, `"terraform"`, `"billing"`) - `models[].evidence` array has at least one entry per source listed in `detected_via` @@ -316,6 +612,19 @@ After generating the output file, the parent `discover.md` handles the phase sta - `current_costs` section is present ONLY if billing data was provided; omitted entirely otherwise - `detection_signals` matches the signals found in Step 3 - All field names use EXACT required keys +- `workloads[]` is present (empty array if no AI call sites; one entry per distinct `(model_id, sdk_method, structured_output)` tuple) +- Every `workloads[].model_id` also appears in `models[].model_id` +- `workloads[].capability_confidence` is one of: `"high"`, `"medium"`, `"low"` +- `workloads[].workload_id` matches pattern `wl_[0-9a-f]{6}` +- If `is_agentic: true`: `agentic_profile` section exists with all required fields +- If `is_agentic: true`: `agentic_profile.agent_count` equals length of `agentic_profile.agents[]` +- If `is_agentic: true`: `agentic_profile.tool_count` equals length of `tool_manifest[]` (deduplicated) +- If `is_agentic: true`: `agentic_profile.framework` is one of: `"langgraph"`, `"crewai"`, `"autogen"`, `"openai_agents"`, `"strands"`, `"pydantic_ai"`, `"agno"`, `"custom"`, `"none"` +- If `is_agentic: true`: `agentic_profile.orchestration_pattern` is one of: `"single"`, `"hierarchical"`, `"swarm"`, `"graph"`, `"sequential"`, `"unknown"` +- If `is_agentic: true`: every `agent_id` in `tool_manifest[].used_by_agents` exists in `agentic_profile.agents[].agent_id` +- If `is_agentic: true`: `tool_manifest[].transport` is one of: `"function"`, `"api"`, `"mcp"`, `"unknown"` +- If `is_agentic: true`: `tool_manifest[].auth_hint` is one of: `"none"`, `"api_key"`, `"oauth"`, `"iam"`, `"unknown"` +- If `is_agentic: false`: `agentic_profile` and `tool_manifest` are ABSENT from output (not present with null/false values) --- @@ -323,7 +632,7 @@ After generating the output file, the parent `discover.md` handles the phase sta The Design phase (`steering/design.md`) uses `ai-workload-profile.json`: -1. **`summary.ai_source`** — Routes to the correct design reference: `"gemini"` → `ai-gemini-to-bedrock.md`, `"openai"` → `ai-openai-to-bedrock.md`, `"both"` → load both, `"other"` → `ai.md` (traditional ML) +1. **`summary.ai_source`** — Routes to the correct design reference: `"gemini"` → `design-ref-ai-gemini-to-bedrock.md`, `"openai"` → `design-ref-ai-openai-to-bedrock.md`, `"anthropic"` → `design-ref-ai-anthropic-to-bedrock.md` (Anthropic SDK → Bedrock Converse API client swap), `"both"` → load both Gemini and OpenAI refs, `"other"` → `design-ref-ai.md` (traditional ML / Vision API / Speech API only) 2. **`models`** — Determines which Bedrock models to recommend via the model selection decision tree 3. **`integration.capabilities_summary`** — Validates Bedrock feature parity (e.g., if `function_calling` is `true`, selected Bedrock model must support tool use) 4. **`integration.pattern`** and **`integration.primary_sdk`** — Determines code migration guidance (direct SDK swap vs framework provider swap vs REST endpoint change) @@ -331,6 +640,8 @@ The Design phase (`steering/design.md`) uses `ai-workload-profile.json`: 6. **`integration.frameworks`** — If LangChain is used, migration may be simpler (swap provider, keep chains) 7. **`infrastructure`** — Identifies supporting AWS resources needed (IAM roles, VPC config) 8. **`current_costs.monthly_ai_spend`** — Baseline for cost comparison in estimate phase +9. **`agentic_profile`** (if present) — Routes to agentic design path: determines migration approach (retarget / Harness / Strands), maps tools to AgentCore Gateway targets, configures AgentCore Memory +10. **`tool_manifest`** (if present) — Inventories tools for AgentCore Gateway or Harness tool configuration --- diff --git a/migration-to-aws/steering/discover-billing.md b/migration-to-aws/steering/discover-billing.md index bce83704..47c45d08 100644 --- a/migration-to-aws/steering/discover-billing.md +++ b/migration-to-aws/steering/discover-billing.md @@ -39,12 +39,60 @@ Group by service and calculate monthly totals. --- +## Step 1.5: Identify Commitments and Discounts + +Scan billing line items for GCP Committed Use Discount (CUD) artifacts and other billing-level discounts. These are **not workload costs** — they are financial instruments that must be separated from actual resource usage. + +### Detection patterns + +| Pattern | How to identify | Type | +| ------------------------ | ----------------------------------------------------------------------------------------------- | ---------------- | +| Resource-based CUD fee | SKU contains "Commitment v1:" (e.g., "Commitment v1: E2 Cpu in Americas for 1 Year") | `resource_based` | +| Dollar-based CUD fee | SKU contains "Commitment - dollar based" (e.g., "Commitment - dollar based v1: GCE for 1 year") | `dollar_based` | +| CUD credit (offset) | Column `committedUsageDiscount` or `committedUsageDiscountDollarBase` has non-zero value | credit | +| Sustained usage discount | Column `sustainedUsageDiscount` has non-zero value | credit | +| Subscription benefit | Column `subscriptionBenefit` has non-zero value | credit | +| Free tier credit | Column `freeTier` has non-zero value | credit | + +Additional signals: + +- `resourceGlobalName` contains `project_commitments` → commitment fee row +- `resourceName` starts with `commitment-` → commitment fee row + +### Extraction + +For each detected commitment fee row, extract: + +- **Term**: Parse from SKU (e.g., "1 Year" or "3 Year") +- **Covered service**: Parse from SKU (e.g., "E2 Cpu", "E2 Ram", "GCE", "Cloud SQL") +- **Region**: From the `region` field +- **Monthly fee**: Sum `costAtListUSD` for all rows in that commitment + +For discount credits, sum per type across all services: + +- Total `committedUsageDiscount` (negative values = credits applied) +- Total `sustainedUsageDiscount` +- Total `freeTier` + +### Cost basis determination + +If any CUD-related columns or rows are detected: + +- Use `costAtListUSD` (list price) as the baseline for all service costs in Step 2 +- Record `total_at_list` (sum of all `costAtListUSD`) +- Record `total_net_of_discounts` (list price + all discount credits) +- Calculate `effective_discount_percent`: `(total_at_list - total_net_of_discounts) / total_at_list × 100` + +If no CUD columns are present in the export (older export format), use the available cost column as-is and set `cost_basis.uses_list_price` to `false`. + +--- + ## Step 2: Build Service Usage Profile From the parsed billing data: 1. List all GCP services with non-zero spend -2. Calculate monthly cost per service +2. Calculate monthly cost per service **using list price (`costAtListUSD`) when available** — exclude commitment fee rows from service totals 3. Identify top services by spend (sorted descending) 4. Note usage patterns (consistent vs bursty spend) @@ -70,43 +118,56 @@ Write `$MIGRATION_DIR/billing-profile.json` with the following structure: ```json { "metadata": { - "report_date": "[ISO 8601 date]", - "billing_files_analyzed": ["path/to/billing.csv"], - "currency": "USD" + "report_date": "2026-02-24", + "project_directory": "/path/to/project", + "billing_source": "gcp-billing-export.csv", + "billing_period": "2026-01" }, "summary": { - "total_monthly_spend": 0.00, - "service_count": 0, - "top_services_by_spend": [] + "total_monthly_spend": 2450.00, + "service_count": 8, + "currency": "USD" }, "services": [ { - "service": "Cloud Run", + "gcp_service": "Cloud Run", "gcp_service_type": "google_cloud_run_service", "monthly_cost": 450.00, + "percentage_of_total": 0.18, "top_skus": [ { - "sku": "Cloud Run - CPU Allocation Time", - "monthly_cost": 320.00, - "usage_amount": 1500, - "usage_unit": "vCPU-seconds" + "sku_description": "Cloud Run - CPU Allocation Time", + "monthly_cost": 300.00 + }, + { + "sku_description": "Cloud Run - Memory Allocation Time", + "monthly_cost": 150.00 } ], - "usage_pattern": "consistent" + "ai_signals": [] } ], - "ai_signals": [ - { - "pattern": "3.1", - "service_description": "Vertex AI", - "monthly_cost": 200.00, - "confidence": 0.98 + "commitments": { + "has_active_cuds": false, + "total_monthly_commitment_fees": 0.00, + "total_monthly_cud_credits": 0.00, + "effective_discount_percent": 0.0, + "details": [] + }, + "cost_basis": { + "uses_list_price": true, + "total_at_list": 2450.00, + "total_net_of_discounts": 2450.00, + "discount_breakdown": { + "committed_usage_discount": 0.00, + "sustained_usage_discount": 0.00, + "free_tier": 0.00 } - ], - "ai_detection": { - "has_ai_workload": false, + }, + "ai_signals": { + "detected": false, "confidence": 0, - "ai_monthly_spend": 0.00 + "services": [] } } ``` diff --git a/migration-to-aws/steering/discover-iac.md b/migration-to-aws/steering/discover-iac.md index 666f821e..ade14e97 100644 --- a/migration-to-aws/steering/discover-iac.md +++ b/migration-to-aws/steering/discover-iac.md @@ -27,6 +27,16 @@ Record file paths and types for all files found. **Exit gate:** If NO Terraform files (`.tf`, `.tfvars`, `.tfstate`, `.terraform.lock.hcl`) are found, **exit cleanly**. Return no output artifacts. Other sub-discovery files may still produce artifacts. +**Secret hygiene (HARD — no exceptions):** `.tfstate` and `.tfvars` files may contain database passwords, API keys, TLS private keys, and certificate material in plaintext. + +When `.tfstate` or `.tfvars` files are found: + +1. **Warn the user immediately:** "Found [N] Terraform state/variable file(s). These may contain secrets. They will be read for resource discovery only — raw values will NOT be copied into any migration artifact." +2. **Redact sensitive attributes** before writing to `gcp-resource-inventory.json`. For any `gcp_config` field whose key matches a sensitive pattern (password, secret, key, token, credential, private_key, client_secret, access_key, api_key), replace the value with `"[REDACTED]"`. +3. **Never write raw secret values** into `gcp-resource-inventory.json`, `gcp-resource-clusters.json`, or any other output artifact. + +Sensitive key patterns to redact (case-insensitive): `password`, `passwd`, `secret`, `api_key`, `apikey`, `access_key`, `private_key`, `client_secret`, `token`, `credential`, `auth`. + ## Step 1: Extract Resources from Terraform 1. Read all `.tf`, `.tfvars`, and `.tfstate` files in working directory (recursively) @@ -79,6 +89,13 @@ Count resources matching these types. This is the **primary resource count**. For small projects, skip the full clustering pipeline. Instead: +0. **Exclude Priority 0 resources** before classification. Remove any resources matching the + Excluded Resources list in `clustering-classification-rules.md` (Priority 0). These include: + - `google_identity_platform_*` — Auth provider (keep existing, do not migrate) + - `google_firebase_auth_*` — Auth provider (keep existing, do not migrate) + Log each excluded resource: "Auth provider detected — excluded from migration scope. Keep your existing auth solution." + Do NOT include excluded resources in `gcp-resource-inventory.json` or any cluster. + 1. **Classify resources** using only Priority 1 hardcoded rules from the PRIMARY types list above. - Resources matching the list → PRIMARY - All other resources → SECONDARY with role inferred from type: @@ -120,6 +137,7 @@ phases (clarify, design, estimate, generate) work identically regardless of clus 1. Read `steering/clustering-classification-rules.md` completely 2. For EACH resource from Step 1, apply classification rules in priority order: + - **Priority 0**: Check if in Excluded Resources list → **remove from resource list entirely**. Do not classify, cluster, or include in output. Log: "Auth provider detected — excluded from migration scope." - **Priority 1**: Check if in PRIMARY list → mark `classification: "PRIMARY"`, assign `tier`, continue - **Priority 2**: Check if type matches SECONDARY patterns → mark `classification: "SECONDARY"` with `secondary_role` (one of: `identity`, `access_control`, `network_path`, `configuration`, `encryption`, `orchestration`) - **Priority 3**: Apply fallback heuristics first, then LLM inference → mark as SECONDARY with `secondary_role` and `confidence` field (0.5-0.75) @@ -236,7 +254,48 @@ Include top-level `creation_order` array: 4. Verify all cluster IDs match resource cluster_id assignments 5. Report to user: "Wrote gcp-resource-inventory.json (X resources) and gcp-resource-clusters.json (Y clusters)" -After generating output files, the parent `discover.md` handles the phase status update — do not update `.phase-status.json` here. +### 7d: Optional — Write `ai-workload-profile.json` (Vertex-strong Terraform only) + +Run **only** when all of the following are true: + +1. `gcp-resource-inventory.json` → `ai_detection.has_ai_workload` is `true` +2. **Vertex-strong:** at least one of: + - `ai_detection.ai_services` includes `vertex_ai`, **or** + - Any entry in `ai_detection.signals_found` references a Terraform resource type matching `google_vertex_ai_*` (see Step 2 pattern table) + +Do **not** run this step for AI signals that are **only** BigQuery ML, Document AI, Vision, etc., with **no** Vertex AI service or `google_vertex_ai_*` signal — Category F is scoped to strong Vertex evidence here. + +**If Vertex-strong:** Load `steering/schema-discover-ai.md` and write `$MIGRATION_DIR/ai-workload-profile.json` with a **minimal IaC-inferred** profile: + +| Field | Value | +| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `metadata.profile_source` | `"iac_vertex"` | +| `metadata.sources_analyzed.terraform` | `true` | +| `metadata.sources_analyzed.application_code` | `false` | +| `metadata.sources_analyzed.billing_data` | `false` (billing runs in the parent orchestrator after IaC; app-code or a later merge may set this) | +| `summary.overall_confidence` | Copy from `ai_detection.confidence` | +| `summary.confidence_level` | Copy from `ai_detection.confidence_level` | +| `summary.total_models_detected` | `0` if `models[]` is empty | +| `summary.languages_found` | `[]` | +| `summary.inferred_from_iac` | `true` | +| `summary.ai_source` | `"gemini"` if any Vertex resource type suggests generative/RAG endpoints (e.g. `google_vertex_ai_endpoint`, `google_vertex_ai_index`, `google_vertex_ai_index_endpoint`, metadata stores commonly used with generative search). Use `"other"` if **only** traditional ML resources (e.g. `google_vertex_ai_training_pipeline`, `google_vertex_ai_custom_job`, `google_vertex_ai_batch_prediction_job`) with no generative-type resources. If mixed, prefer `"gemini"` when any generative-type resource exists. | +| `models` | `[]` unless a model ID is explicitly present in Terraform config without guessing | +| `integration.primary_sdk` | `null` | +| `integration.sdk_version` | omit or `null` | +| `integration.frameworks` | `[]` | +| `integration.languages` | `[]` | +| `integration.pattern` | `"unknown"` | +| `integration.gateway_type` | `null` | +| `integration.capabilities_summary` | All keys `false` unless a capability is clearly implied by resource kinds (default: all `false`) | +| `infrastructure` | All `google_vertex_ai_*` resources from the inventory, with `address`, `type`, file path, and `config` as in schema | +| `current_costs` | Omit unless billing data was merged into this run (same rule as app-code schema) | +| `detection_signals` | Mirror `ai_detection.signals_found` into the `detection_signals[]` shape (method `terraform`, confidence, evidence strings) | + +**If `ai-workload-profile.json` already exists** in `$MIGRATION_DIR` with `metadata.profile_source` of `"application_code"` or `"merged"`, **skip Step 7d** (do not overwrite). Otherwise write or replace when Vertex-strong (including replacing a prior `"iac_vertex"` file). + +Report to user when written: "Wrote ai-workload-profile.json (IaC-inferred Vertex AI)." + +After generating output files (including optional Step 7d), the parent `discover.md` handles the phase status update — do not update `.phase-status.json` here. ## Output Validation Checklist @@ -273,11 +332,19 @@ After generating output files, the parent `discover.md` handles the phase status - No cycles in dependency graph - Output is valid JSON +### ai-workload-profile.json (only if Step 7d executed) + +- `metadata.profile_source` is `"iac_vertex"` +- `summary.inferred_from_iac` is `true` +- `integration.pattern` is `"unknown"` unless evidence supports another value +- `models` is `[]` unless Terraform explicitly exposes model IDs +- Valid JSON and matches `steering/schema-discover-ai.md` + --- ## Design Phase Integration -The Design phase (`steering/design.md`) uses both outputs: +The Design phase (`steering/design.md`) uses these outputs: 1. **From gcp-resource-clusters.json:** - `creation_order` — evaluates clusters depth-first (foundational first) @@ -292,8 +359,10 @@ The Design phase (`steering/design.md`) uses both outputs: - `classification` / `secondary_role` — handles primary/secondary differently - `serves` — determines if secondary's primary is mapped - `depth` — validates clustering logic - - `tier` — routes to correct design-ref file (compute.md, database.md, etc.) - - `ai_detection` — determines if AI design phase runs + - `tier` — routes to correct design-ref file (design-ref-compute.md, design-ref-database.md, etc.) + - `ai_detection` — signals for inventory; when Step 7d ran, **`ai-workload-profile.json`** is the driver for AI Clarify/Design + +3. **From `ai-workload-profile.json` (when Step 7d wrote it):** consumed in Phase 2+ per `schema-discover-ai.md` (`profile_source: "iac_vertex"`). --- diff --git a/migration-to-aws/steering/discover-preview.md b/migration-to-aws/steering/discover-preview.md new file mode 100644 index 00000000..ed07a9a4 --- /dev/null +++ b/migration-to-aws/steering/discover-preview.md @@ -0,0 +1,427 @@ +# Migration Preview Heuristic + +> Loaded by `discover.md` Step 3 to compute a lightweight preview signal and rough cost +> estimate from discovery artifacts alone — before Clarify, Design, or Estimate run. +> This is NOT the full complexity tier (that lives in `migration-complexity.md` and requires +> preferences + billing). This is a fast, honest "at a glance" for the user. + +--- + +## Route Detection + +Before executing Steps 1–6, determine which route applies: + +``` +IF gcp-resource-inventory.json does NOT exist + AND ai-workload-profile.json exists +THEN route = "ai_only" + +ELSE + route = "infra" // covers infra-only, hybrid infra+AI, and billing-only +END +``` + +**AI-only route** executes Steps 1A–6A below. +**Infra route** executes Steps 1–6 below (original behavior, unchanged). + +--- + +## AI-Only Route (Steps 1A–6A) + +> Used when only `ai-workload-profile.json` exists — no Terraform, no billing data. +> Infrastructure stays on GCP; only AI/LLM calls move to AWS Bedrock. + +### Step 1A: Compute AI complexity_signal + +Read from `ai-workload-profile.json`: + +| Input | Source | Key | +| ------------------------- | -------------------------- | ---------------------------------------------------------------------------------- | +| `model_count` | `ai-workload-profile.json` | Count of distinct entries in `models[]` | +| `is_agentic` | `ai-workload-profile.json` | `agentic_profile.is_agentic == true` | +| `has_multi_model_routing` | `ai-workload-profile.json` | `integration.gateway_type` is `"openrouter"`, `"litellm"`, `"kong"`, or `"apigee"` | +| `has_multiple_providers` | `ai-workload-profile.json` | `summary.ai_source == "both"` or distinct provider values across `models[]` > 1 | +| `capability_count` | `ai-workload-profile.json` | Count of `true` values in `integration.capabilities_summary` | + +**Classify (first match wins, top to bottom):** + +``` +IF is_agentic == true + OR has_multi_model_routing == true + OR model_count > 3 + OR has_multiple_providers == true +THEN ai_complexity_signal = "complex" + +ELSE IF model_count == 1 + AND is_agentic != true + AND has_multi_model_routing != true + AND capability_count <= 2 +THEN ai_complexity_signal = "likely_simple" + +ELSE + ai_complexity_signal = "standard" +END +``` + +**Fast-path eligibility:** Always `false` for AI-only route — AI profiles always route to full Clarify. + +``` +eligible_for_clarify_fast_path = false +``` + +--- + +### Step 2A: Build per-token price comparison + +**Purpose:** Show the user what their models map to on Bedrock and whether the per-token +price is higher, lower, or roughly equivalent. Do NOT compute a monthly dollar total — +usage volume is unknown at Discover time and will be collected in Clarify (Q3, Q7). + +For each model in `models[]` of `ai-workload-profile.json`, map to the closest Bedrock +equivalent using the table below, then look up both source and Bedrock per-token prices +from `steering/cached-prices.md` (Source Provider Pricing + Bedrock Models sections). + +**Source model → Bedrock equivalent mapping:** + +| Source model pattern | Bedrock equivalent | Bedrock model ID | +| ----------------------------------------------- | -------------------------- | ------------------------------------------ | +| `gpt-4o`, `gpt-4.1`, `gpt-5.*` flagship | Claude Sonnet 4.6 | `anthropic.claude-sonnet-4-6` | +| `gpt-4o-mini`, `gpt-4.1-mini`, `gpt-5.*-mini` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | +| `gpt-3.5-turbo`, `gpt-4.1-nano`, `gpt-5.*-nano` | Amazon Nova Micro | `amazon.nova-micro-v1:0` | +| `o3`, `o4-mini`, reasoning models | Claude Sonnet 4.6 | `anthropic.claude-sonnet-4-6` | +| `gemini-2.5-pro`, `gemini-3.*-pro` | Claude Sonnet 4.6 | `anthropic.claude-sonnet-4-6` | +| `gemini-2.5-flash`, `gemini-2.0-flash` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | +| `gemini-2.0-flash-lite` | Amazon Nova Lite | `amazon.nova-lite-v1:0` | +| `claude-3-5-sonnet`, `claude-sonnet-*` | Claude Sonnet 4.6 | `anthropic.claude-sonnet-4-6` | +| `claude-3-5-haiku`, `claude-haiku-*` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | +| `claude-3-opus`, `claude-opus-*` | Claude Opus 4.6 | `anthropic.claude-opus-4-6-v1` | +| `text-embedding-*`, `*-embedding-*` | Amazon Titan Embeddings v2 | `amazon.titan-embed-text-v2:0` | +| `dall-e-*`, `imagen-*`, image generation | Amazon Nova Canvas | `amazon.nova-canvas-v1:0` | +| `whisper-*`, speech-to-text | Amazon Transcribe | (non-token service — note separately) | +| `tts-*`, text-to-speech | Amazon Polly | (non-token service — note separately) | +| Unknown / other | Amazon Nova Pro | `amazon.nova-pro-v1:0` | + +For each mapped model pair, record `source_model` and `bedrock_equivalent` (model name only). +Do NOT compute or display per-token pricing comparisons at this stage — cost analysis +belongs in the Estimate phase where full usage volume context is available. + +--- + +### Step 3A: Build key_decisions_ahead + +Generate 2-4 bullets based on what was detected in `ai-workload-profile.json`: + +| Signal | Decision bullet | +| ------------------------------------ | ------------------------------------------------------------------------------- | +| Always | "Bedrock model selection for [list detected model IDs, max 3, then '+ N more']" | +| `is_agentic == true` | "Agentic migration path (retarget / AgentCore Harness / Strands)" | +| `has_multi_model_routing == true` | "Multi-model routing strategy on Bedrock (LiteLLM adapter vs native routing)" | +| `has_multiple_providers == true` | "Re-embedding requirements and cascade pair testing across providers" | +| `integration.pattern == "streaming"` | "Streaming transport layer (Bedrock streaming vs current SDK)" | + +Cap at 4 bullets. + +--- + +### Step 4A: Build timeline_hint string + +| ai_complexity_signal | timeline_hint | +| -------------------- | -------------------------------------------------------------------- | +| `likely_simple` | "1-3 weeks (single model swap; confirm after Clarify)" | +| `standard` | "2-6 weeks (multi-model migration; confirm after Clarify)" | +| `complex` | "4-8 weeks (agentic or multi-provider stack; confirm after Clarify)" | + +Always append "(confirm after Clarify)" — full classification requires preferences. + +--- + +### Step 5A: Write migration-preview.json (AI-only) + +Write `$MIGRATION_DIR/migration-preview.json`: + +```json +{ + "preview_version": 1, + "computed_at": "", + "route": "ai_only", + "primary_resource_count": 0, + "complexity_signal": "standard", + "ai_complexity_signal": "standard", + "eligible_for_clarify_fast_path": false, + "services_summary": [], + "ai_summary": { + "model_count": 2, + "model_ids": ["gpt-4o", "text-embedding-3-small"], + "bedrock_targets": [ + { + "source_model": "gpt-4o", + "source_input_per_1m": 2.50, + "source_output_per_1m": 10.00, + "bedrock_equivalent": "Claude Sonnet 4.6", + "bedrock_model_id": "anthropic.claude-sonnet-4-6", + "bedrock_input_per_1m": 3.00, + "bedrock_output_per_1m": 15.00, + "cost_direction": "higher" + }, + { + "source_model": "text-embedding-3-small", + "source_input_per_1m": 0.02, + "source_output_per_1m": null, + "bedrock_equivalent": "Amazon Titan Embeddings v2", + "bedrock_model_id": "amazon.titan-embed-text-v2:0", + "bedrock_input_per_1m": 0.02, + "bedrock_output_per_1m": null, + "cost_direction": "lower" + } + ], + "is_agentic": false, + "has_multi_model_routing": false, + "gateway_type": "direct" + }, + "cost_preview": { + "monthly_estimate": null, + "monthly_estimate_note": "Monthly estimate available after Clarify (usage volume collected in Q3, Q7)", + "disclaimer": "Per-token prices from cached-prices.md; full cost analysis in Estimate phase" + }, + "timeline_hint": "2-6 weeks (multi-model migration; confirm after Clarify)", + "ai_detected": true, + "key_decisions_ahead": [ + "Bedrock model selection for gpt-4o, text-embedding-3-small", + "Streaming transport layer (Bedrock streaming vs current SDK)" + ] +} +``` + +**Field rules:** + +- `route` is `"ai_only"` for this path +- `primary_resource_count` is `0` for AI-only runs (no IaC) +- `complexity_signal` mirrors `ai_complexity_signal` for downstream consumers +- `services_summary` is `[]` for AI-only runs +- `ai_summary.bedrock_targets` lists one entry per distinct source model with actual per-token prices +- `cost_preview.monthly_estimate` is always `null` at Discover time — no invented token volumes +- `eligible_for_clarify_fast_path` is always `false` for AI-only route + +--- + +### Step 6A: Build preview chat message (AI-only) + +Output this block as part of `discover.md` Step 3's user message (chat only — not a file): + +``` +### Your AI migration at a glance *(preview — not final)* + +| | | +|---|---| +| **Models detected** | [model_ids joined by ", "] | +| **Bedrock targets** | [for each bedrock_target: "source_model → bedrock_equivalent"] | +| **Routing** | [if has_multi_model_routing: gateway_type + " (multi-model routing)" else "Direct SDK"] | +| **Monthly estimate** | Available after Estimate phase | +| **Timeline (rough)** | [timeline_hint] | +| **Decisions ahead** | [key_decisions_ahead joined by "; "] | + +*Full cost breakdown in Estimate; runnable adapter code in Generate.* +AI workload detected — full Clarify recommended for best results. +``` + +Do NOT write this to a file. Chat output only. + +--- + +## Infra Route (Steps 1–6) + +> Used when `gcp-resource-inventory.json` exists (infra-only, hybrid infra+AI, or billing-only). +> Original behavior — unchanged. + +## Step 1: Compute complexity_signal + +Read from available discovery artifacts: + +| Input | Source | Key | +| ------------------------ | ------------------------------------------------------- | ------------------------------------------------------------------------------- | +| `primary_resource_count` | `gcp-resource-inventory.json` | Count resources where `classification: "PRIMARY"` | +| `has_database` | `gcp-resource-inventory.json` | Any resource type matching `google_sql_*`, `google_spanner_*`, `google_redis_*` | +| `has_bigquery` | `gcp-resource-inventory.json` or `billing-profile.json` | Any `google_bigquery_*` resource or BigQuery billing SKU | +| `has_ai_profile` | File presence | `ai-workload-profile.json` exists | +| `is_agentic` | `ai-workload-profile.json` | `agentic_profile.is_agentic == true` (if file exists) | +| `billing_monthly_usd` | `billing-profile.json` | `summary.total_monthly_spend` (null if absent) | + +**Classify (first match wins, top to bottom):** + +``` +IF has_bigquery + OR is_agentic == true + OR primary_resource_count > 8 + OR (billing_monthly_usd != null AND billing_monthly_usd > 10000) +THEN complexity_signal = "complex" +ELSE IF primary_resource_count <= 3 + AND has_database == false + AND has_bigquery == false + AND is_agentic != true + AND (billing_monthly_usd == null OR billing_monthly_usd < 1000) +THEN complexity_signal = "likely_simple" +ELSE + complexity_signal = "standard" +END +``` + +**Fast-path eligibility:** + +``` +eligible_for_clarify_fast_path = + complexity_signal == "likely_simple" + AND has_ai_profile == false + +eligible_for_clarify_simple_path = + complexity_signal == "likely_simple" + AND has_ai_profile == true + AND is_agentic != true + AND ai_complexity_signal == "likely_simple" +``` + +**`ai_complexity_signal`** (compute when `ai-workload-profile.json` exists): + +``` +IF agentic_profile.is_agentic == true + OR integration.frameworks is non-empty (LangChain, CrewAI, etc.) + OR models.length > 3 +THEN ai_complexity_signal = "standard" + +ELSE IF integration.pattern in ("direct_sdk", "direct") + AND models.length <= 2 + AND agentic_profile is absent +THEN ai_complexity_signal = "likely_simple" + +ELSE + ai_complexity_signal = "standard" +END +``` + +--- + +## Step 2: Compute rough AWS cost range + +**Purpose:** Give the user a ballpark before Estimate runs. Always label as rough. Never invent GCP spend if billing data is absent. + +### Service type -> dev-tier AWS line item mapping + +For each PRIMARY resource in `gcp-resource-inventory.json`, map to a dev-tier AWS equivalent and look up its monthly cost from `steering/cached-prices.md`: + +| GCP Primary Type | Typical AWS Target | Dev-tier sizing for preview | +| -------------------------------------------------------------------- | --------------------------- | -------------------------------- | +| `google_cloud_run_v2_service` / `google_cloud_run_service` | Fargate | 0.5 vCPU, 1GB RAM, 730 hrs/mo | +| `google_cloudfunctions_function` / `google_cloudfunctions2_function` | Lambda | 1M requests, 128MB, 200ms avg | +| `google_compute_instance` | EC2 t4g.small | On-demand, us-east-1 | +| `google_container_cluster` | EKS (2x t4g.small nodes) | On-demand, us-east-1 | +| `google_sql_database_instance` | RDS db.t4g.micro | Single-AZ, gp3 20GB | +| `google_redis_instance` | ElastiCache cache.t4g.micro | Single-AZ | +| `google_storage_bucket` | S3 | 50GB standard + 10K GET + 1K PUT | +| `google_pubsub_topic` | SQS | 1M requests/mo | +| `google_filestore_instance` | EFS | 10GB standard | +| `google_spanner_instance` | Aurora Serverless v2 | 0.5-1 ACU | +| `google_bigquery_dataset` | Deferred -- specialist | $0 (not estimated) | + +Sum the dev-tier line items to get `aws_monthly_range_usd.low`. Multiply by 1.5 for `high` (accounts for NAT gateway, data transfer, CloudWatch, and sizing variance). + +**If `billing-profile.json` exists:** Set `gcp_monthly_usd` from `summary.total_monthly_spend`. Show both GCP actual and AWS range. + +**If only IaC:** Set `gcp_monthly_usd: null`. Show AWS range only -- never invent GCP spend. + +**If neither IaC nor billing:** Omit cost preview entirely (`cost_preview: null`). + +--- + +## Step 3: Build key_decisions_ahead + +Generate 2-4 bullets based on what was detected. Use only signals present in discovery artifacts: + +| Signal | Decision bullet | +| ------------------------ | ------------------------------------------------------------ | +| Any compute resource | "Target region and deployment model (Fargate vs EKS)" | +| `has_database == true` | "Database migration tooling and cutover window" | +| `has_ai_profile == true` | "Bedrock model selection for [detected model IDs]" | +| `is_agentic == true` | "Agentic migration path (retarget / Harness / Strands)" | +| `has_bigquery == true` | "BigQuery analytics target (specialist engagement required)" | + +Always include "Target region" if any compute is present. Cap at 4 bullets. + +--- + +## Step 4: Build timeline_hint string + +| complexity_signal | timeline_hint | +| ----------------- | -------------------------------------------------------- | +| `likely_simple` | "3-6 weeks (likely simple infra; confirm after Clarify)" | +| `standard` | "8-12 weeks (standard migration; confirm after Clarify)" | +| `complex` | "12-16+ weeks (complex stack; confirm after Clarify)" | + +Always append "(confirm after Clarify)" -- full tier classification requires preferences. + +--- + +## Step 5: Write migration-preview.json + +Write `$MIGRATION_DIR/migration-preview.json`: + +```json +{ + "preview_version": 1, + "computed_at": "", + "primary_resource_count": 3, + "complexity_signal": "likely_simple", + "eligible_for_clarify_fast_path": true, + "eligible_for_clarify_simple_path": false, + "ai_complexity_signal": null, + "services_summary": [ + { "gcp_type": "google_cloud_run_v2_service", "typical_aws_target": "Fargate" }, + { "gcp_type": "google_storage_bucket", "typical_aws_target": "S3" } + ], + "cost_preview": { + "gcp_monthly_usd": 240.00, + "aws_monthly_range_usd": { "low": 120, "high": 180 }, + "disclaimer": "Dev-tier rough estimate (+-30%); full analysis in Estimate phase" + }, + "timeline_hint": "3-6 weeks (likely simple infra; confirm after Clarify)", + "ai_detected": false, + "key_decisions_ahead": [ + "Target region and deployment model (Fargate vs EKS)", + "Cutover window" + ] +} +``` + +**Field rules:** + +- `cost_preview` is `null` if neither IaC nor billing data was available +- `cost_preview.gcp_monthly_usd` is `null` if no billing data (IaC-only run) +- `ai_detected` is `true` if `ai-workload-profile.json` exists +- `services_summary` lists only PRIMARY resources, deduplicated by `gcp_type` +- `eligible_for_clarify_fast_path` is `false` whenever `ai_detected == true`, regardless of infra complexity +- `eligible_for_clarify_simple_path` is `true` only when `ai_detected == true`, `complexity_signal == "likely_simple"`, and `ai_complexity_signal == "likely_simple"` (non-agentic direct SDK, ≤2 models) +- `ai_complexity_signal` is `null` when no AI profile exists; otherwise `"likely_simple"` or `"standard"` + +--- + +## Step 6: Build preview chat message + +Output this block as part of `discover.md` Step 3's user message (chat only -- not a file): + +``` +### Your migration at a glance *(preview -- not final)* + +| | | +|---|---| +| **Services** | [primary_resource_count] resources -> [services_summary as "Fargate, S3"] *(standard pairings)* | +| **AWS cost (rough)** | ~$[low]-$[high]/mo [vs GCP ~$[gcp]/mo if billing present] *(dev-tier estimate, +-30%)* | +| **Timeline (rough)** | [timeline_hint] | +| **AI** | [if ai_detected: "[model IDs] detected -- AI migration path will run" else "None detected"] | +| **Decisions ahead** | [key_decisions_ahead joined by "; "] | + +*Full cost breakdown in Estimate; runnable Terraform in Generate.* + +[if eligible_for_clarify_fast_path: "Your stack looks straightforward -- next step is 3 quick questions."] +[if eligible_for_clarify_simple_path: "Simple stack with lightweight AI detected -- next step is a short question set (~6 questions)."] +[if ai_detected and not eligible_for_clarify_simple_path and not eligible_for_clarify_fast_path: "AI workload detected -- full Clarify recommended for best results."] +``` + +Do NOT write this to a file. Chat output only. diff --git a/migration-to-aws/steering/discover.md b/migration-to-aws/steering/discover.md index 63aa347c..77769f11 100644 --- a/migration-to-aws/steering/discover.md +++ b/migration-to-aws/steering/discover.md @@ -5,8 +5,8 @@ Lightweight orchestrator that delegates to domain-specific discoverers. Each sub ## Sub-Discovery Files -- **discover-iac.md** → `gcp-resource-inventory.json` + `gcp-resource-clusters.json` (if Terraform found) -- **discover-app-code.md** → `ai-workload-profile.json` (if source code with AI signals found) +- **discover-iac.md** → `gcp-resource-inventory.json` + `gcp-resource-clusters.json` (if Terraform found); may also write `ai-workload-profile.json` when **Vertex-strong** (see `discover-iac.md` Step 7d) +- **discover-app-code.md** → `ai-workload-profile.json` when AI confidence ≥ 70% (may **merge** with an existing `iac_vertex` profile) - **discover-billing.md** → `billing-profile.json` (if billing data found) Multiple artifacts can be produced in a single run — they are not mutually exclusive. @@ -20,11 +20,11 @@ Multiple artifacts can be produced in a single run — they are not mutually exc - `[C] Cancel` - **If resuming:** Set `$MIGRATION_DIR` to the selected run's directory. Read its `.phase-status.json` and skip to the appropriate phase per the State Machine in POWER.md. - **If fresh or no existing runs:** Continue to step 2. -2. **Resolve the current local timestamp FIRST** by running the shell command `date "+%m%d-%H%M"` in the user's project directory to get the actual `MMDD-HHMM` value in the user's local timezone. **Do NOT hardcode, guess, or infer the timestamp — always execute the command and use its output.** Then create `.migration/[MMDD-HHMM]/` directory using the resolved value (e.g., if the command returns `0226-1430`, create `.migration/0226-1430/`). Set `$MIGRATION_DIR` to this new directory. +2. Create `.migration/[MMDD-HHMM]/` directory (e.g., `.migration/0226-1430/`) using current timestamp (MMDD = month/day, HHMM = hour/minute). Set `$MIGRATION_DIR` to this new directory. 3. Create `.migration/.gitignore` file (if not already present) with exact content: ``` - # Auto-generated migration state (temporary, should not be committed) + # Auto-generated migration state (temporary, do not commit) * !.gitignore ``` @@ -36,7 +36,8 @@ Multiple artifacts can be produced in a single run — they are not mutually exc ```json { "migration_id": "[MMDD-HHMM]", - "last_updated": "[ISO 8601 timestamp with local timezone offset]", + "last_updated": "[ISO 8601 timestamp]", + "current_phase": "discover", "phases": { "discover": "in_progress", "clarify": "pending", @@ -50,32 +51,28 @@ Multiple artifacts can be produced in a single run — they are not mutually exc 5. Confirm both `.migration/.gitignore` and `.phase-status.json` exist before proceeding to Step 1. -## Step 1: Load Phase Status Schema - -Load `steering/schema-phase-status.md` now. Each sub-discovery file loads its own output schema — do not load them here. - -## Step 2: Scan for Input Sources and Run Sub-Discoveries +## Step 1: Scan for Input Sources and Run Sub-Discoveries Scan the project directory for each input type. Only load sub-discovery files when their input files are present. -**2a. Check for Terraform files:** +**1a. Check for Terraform files:** Glob for: `**/*.tf`, `**/*.tfvars`, `**/*.tfstate`, `**/.terraform.lock.hcl` -- If found → Load `discover-iac.md` +- If found → Load `steering/discover-iac.md` - If not found → Skip. Log: "No Terraform files found — skipping IaC discovery." -**2b. Check for source code / dependency manifests:** +**1b. Check for source code / dependency manifests:** Glob for: `**/*.py`, `**/*.js`, `**/*.ts`, `**/*.jsx`, `**/*.tsx`, `**/*.go`, `**/*.java`, `**/*.scala`, `**/*.kt`, `**/*.rs`, `**/requirements.txt`, `**/setup.py`, `**/pyproject.toml`, `**/Pipfile`, `**/package.json`, `**/go.mod`, `**/pom.xml`, `**/build.gradle` -- If found → Load `discover-app-code.md` +- If found → Load `steering/discover-app-code.md` - If not found → Skip. Log: "No source code found — skipping app code discovery." -**2c. Check for billing data:** +**1c. Check for billing data:** Glob for: `**/*billing*.csv`, `**/*billing*.json`, `**/*cost*.csv`, `**/*cost*.json`, `**/*usage*.csv`, `**/*usage*.json` - If not found → Skip. Log: "No billing files found — skipping billing discovery." -- If found AND **no** Terraform files from 2a → Load `discover-billing.md` (billing is the primary source — needs full processing for the billing-only design path). -- If found AND Terraform files **were** found in 2a → Use lightweight extraction below. Do **not** load `discover-billing.md`. +- If found AND **no** Terraform files from 1a → Load `steering/discover-billing.md` (billing is the primary source — needs full processing for the billing-only design path). +- If found AND Terraform files **were** found in 1a → Use lightweight extraction below. Do **not** load `discover-billing.md`. **Lightweight billing extraction (when IaC is the primary source):** @@ -88,7 +85,7 @@ When Terraform is present, billing data is supplementary — only service-level - Extracts top 3 SKU descriptions per service by cost - Scans service and SKU descriptions (case-insensitive) for AI keywords: `vertex ai`, `ai platform`, `bigquery ml`, `generative ai`, `gemini`, `document ai`, `vision ai`, `speech-to-text`, `natural language`, `dialogflow`, `translation` - Outputs JSON to stdout matching the schema in step 4 -3. Run the script: try `python3 _extract_billing.py` first. If `python3` is not found, try `python _extract_billing.py`. If neither is available, delete the script and fall back to loading `discover-billing.md`. +3. Run the script: try `python3 _extract_billing.py` first. If `python3` is not found, try `python _extract_billing.py`. If neither is available, delete the script and fall back to loading `steering/discover-billing.md`. 4. Write the script's JSON output to `$MIGRATION_DIR/billing-profile.json` with this exact schema: ```json @@ -115,20 +112,56 @@ When Terraform is present, billing data is supplementary — only service-level **If NONE of the three checks found files**: STOP and output: "No GCP sources detected. Provide at least one source type (Terraform files, application code, or billing exports) and try again." -## Step 3: Check Outputs +## Step 2: Check Outputs After all loaded sub-discoveries complete, check what artifacts were produced in `$MIGRATION_DIR/`: 1. Check for output files: - `gcp-resource-inventory.json` — IaC discovery succeeded - `gcp-resource-clusters.json` — IaC discovery produced clusters - - `ai-workload-profile.json` — App code discovery detected AI workloads + - `ai-workload-profile.json` — App code discovery (confidence ≥ 70%) and/or IaC Vertex-strong inference (`discover-iac.md` Step 7d) - `billing-profile.json` — Billing data parsed 2. **If NO artifacts were produced** (sub-discoveries ran but produced no output): STOP and output: "Discovery ran but produced no artifacts. Check that your input files contain valid GCP resources and try again." +3. **Route output gate (fail closed):** For each triggered sub-discovery route, require the expected artifact(s) before completion: + - If `discover-iac.md` ran -> require `gcp-resource-inventory.json` and `gcp-resource-clusters.json` + - If `discover-app-code.md` ran: + - If its Step 4 exit gate applied (overall AI confidence **below** 70%) **and** no `ai-workload-profile.json` exists -> **allow completion** (app-code route may produce no AI profile). + - If Step 4 exit applied with confidence below 70% **but** `ai-workload-profile.json` exists with `metadata.profile_source` = `"iac_vertex"` -> **allow completion** (IaC-inferred profile retained). + - If execution continued to Steps 5–8 (confidence **≥** 70%) -> **require** `ai-workload-profile.json`. + - If full `discover-billing.md` ran OR lightweight billing extraction ran -> require `billing-profile.json` + - If any triggered route is missing its required artifact(s): STOP and output: "Discover route [name] did not produce required artifacts. Resolve the sub-discovery failure before completing Phase 1." + +## Step 3: Migration Preview + +Load and execute `steering/discover-preview.md` to compute the migration preview. This produces `migration-preview.json` and the preview chat block. Skip only if Step 2 found no artifacts (already STOPped). + +## Completion Handoff Gate (Fail Closed) + +Load `steering/handoff-gates.md`. **Re-read from disk** every artifact below before checking. + +**Re-entry guard:** If `preferences.json` exists and `phases.clarify` is `"completed"`: STOP unless the user explicitly confirms re-running Discover. Emit: + +``` +GATE_FAIL | phase=discover | field=preferences.json | reason=stale_downstream +``` + +**Checks (all must PASS):** + +1. At least one discovery artifact exists (`gcp-resource-inventory.json`, `ai-workload-profile.json`, or `billing-profile.json`). +2. Route output gates from Step 2 all pass. +3. If any discovery artifact exists → `migration-preview.json` exists with `complexity_signal` set. + +**On any FAIL:** Emit `GATE_FAIL | phase=discover | field= | reason=`. **Do NOT modify artifacts to pass the gate.** **Do NOT update `.phase-status.json`.** Tell the user which sub-discovery to re-run. + +**On PASS:** Emit `HANDOFF_OK | phase=discover | artifacts=`. ## Step 4: Update Phase Status -In the **same turn** as the output message below, use the Phase Status Update Protocol (Write tool) to write `.phase-status.json` with `phases.discover` set to `"completed"` and all other phases unchanged from their initial values. +Only after `HANDOFF_OK`. In the **same turn** as the output message below, use the Phase Status Update Protocol (read-merge-write) to update `.phase-status.json`: + +- Set `phases.discover` to `"completed"` +- Set `current_phase` to `"clarify"` +- Keep all other phase values unchanged unless already advanced by a resumed run Output to user — build message from whichever artifacts exist: @@ -136,7 +169,9 @@ Output to user — build message from whichever artifacts exist: - If `ai-workload-profile.json` exists: "Detected AI workloads (source: [ai_source])." - If `billing-profile.json` exists: "Parsed billing data ($Z/month across N services)." -Format: "Discover phase complete. [artifact summaries joined by space] Proceeding to Phase 2: Clarify." +Append the preview block from Step 3 to the output message below. + +Format: "Discover phase complete. [artifact summaries joined by space] [preview block from discover-preview.md Step 6] Next required step: Phase 2 — Clarify. Load `steering/clarify.md` now. Do not load Design, Estimate, or Generate until Clarify completes and `.phase-status.json` marks `phases.clarify` as `completed`." ## Output Files @@ -144,8 +179,9 @@ Format: "Discover phase complete. [artifact summaries joined by space] Proceedin 1. `gcp-resource-inventory.json` — from discover-iac.md 2. `gcp-resource-clusters.json` — from discover-iac.md -3. `ai-workload-profile.json` — from discover-app-code.md +3. `ai-workload-profile.json` — from discover-app-code.md (confidence ≥ 70%, optionally merged) and/or discover-iac.md Step 7d (Vertex-strong IaC only) 4. `billing-profile.json` — from discover-billing.md +5. `migration-preview.json` — from discover-preview.md (always written when any artifact exists) **No other files must be created:** diff --git a/migration-to-aws/steering/estimate-ai.md b/migration-to-aws/steering/estimate-ai.md index e53230af..0dc815b0 100644 --- a/migration-to-aws/steering/estimate-ai.md +++ b/migration-to-aws/steering/estimate-ai.md @@ -11,15 +11,23 @@ The parent `estimate.md` selects the pricing mode before loading this file. **Price lookup order:** 1. **`steering/cached-prices.md` (primary)** — Look up Bedrock model pricing and source provider pricing by table. Set `pricing_source: "cached"`. -2. **MCP (secondary)** — If a model is NOT in cached-prices.md and MCP is available, query `get_pricing("AmazonBedrock", "us-east-1")` with model filter. Set `pricing_source: "live"`. +2. **MCP (secondary)** — If a model is NOT in cached-prices.md and MCP is available, query `get_pricing("AmazonBedrock", ...)` with model filter and the user's target region. Set `pricing_source: "live"`. +3. **Cache after MCP failure** — If MCP was attempted but failed, and the model IS in the cache, use the cached price. Set `pricing_source: "cached_fallback"`. +4. **Unavailable** — If a model is NOT in the cache AND MCP failed, set `pricing_source: "unavailable"` and warn the user. For typical migrations (Claude, Llama, Nova, Mistral, DeepSeek, Gemma, OpenAI gpt-oss, Gemini source pricing), ALL prices are in `cached-prices.md`. Zero MCP calls needed. +**Model lifecycle:** When building the model comparison table, check `steering/ai-model-lifecycle.md` and apply the 90-day exclusion rule: + +- **Excluded** (≤90 days to EOL): omit entirely from `model_comparison`, `recommended_model`, and `backup_model`. +- **Legacy** (>90 days to EOL): include in `model_comparison` with `(Legacy — EOL YYYY-MM-DD)` annotation. Do not select as `recommended_model` unless no Active alternative exists. +- **Active**: no restrictions. + ## Prerequisites Read from `$MIGRATION_DIR/`: -- **`ai-workload-profile.json`** — `current_costs.monthly_ai_spend`, `current_costs.services_detected`, `models[]` +- **`ai-workload-profile.json`** — `current_costs.monthly_ai_spend`, `current_costs.services_detected`, `models[]`, `metadata.profile_source`, `summary.inferred_from_iac` - **`preferences.json`** — `ai_constraints.ai_token_volume.value`, `ai_constraints.ai_capabilities_required.value` - **`aws-design-ai.json`** — `metadata.ai_source`, `ai_architecture.honest_assessment`, `ai_architecture.tiered_strategy`, `ai_architecture.bedrock_models[]` (with `source_provider_price`, `bedrock_price`, `honest_assessment`), `ai_architecture.capability_mapping` @@ -33,6 +41,8 @@ Determine current Vertex AI spending from the best available source: 2. **Estimated from token volume** — Use `ai_constraints.ai_token_volume.value` from `preferences.json` with Gemini pricing from `cached-prices.md` (under "Source Provider Pricing"). Apply 60/40 input/output ratio if actual ratio unknown. 3. **Neither available** — Note in output and present model comparison at multiple volume tiers so user can find their range. +**IaC-only profile:** If `metadata.profile_source` is `iac_vertex` or `summary.inferred_from_iac` is true and billing/token data is missing, state explicitly that **current GCP AI spend is unverified** and widen uncertainty bands (use the same multi-tier comparison approach as in case 3). + --- ## Part 2: Build Model Comparison Table @@ -75,7 +85,27 @@ Using the model selected in the design phase, show: --- -## Part 4: ROI Analysis +## Part 4: Human One-Time Migration Costs (Out of Scope) + +**Do not** present human labor, contractors, professional services, or engineering effort as one-time migration **costs** or budget line items (no dollar figures, no "budget for people work" lists, no "one-time migration cost" categories for implementation). + +Populate `migration_cost_considerations.categories` as an **empty array** `[]`. Use `migration_cost_considerations.note` to state that human and professional-services one-time migration costs are intentionally excluded from this advisor. + +**Technical integration complexity** (for internal JSON and risk context only — not framed as money): + +From `ai-workload-profile.json`, record non-monetary factors in `migration_cost_considerations.complexity_factors[]` as short strings, for example: + +- `integration.pattern = "framework"` → lower integration touch surface +- `integration.pattern = "direct_sdk"` → moderate SDK and API pattern changes +- `integration.pattern = "rest_api"` → higher endpoint, auth, and parsing changes +- `summary.total_models_detected` > 3 → multi-model coordination +- `quota_risk = "high"` (from `aws-design-ai.json`) → Bedrock quota increase required before migration; allow 1–5 business days (see `steering/bedrock-quotas.md`) + +Do **not** repeat these as "costs" in the user-facing summary. + +--- + +## Part 5: ROI Analysis Present the monthly and annual cost difference between current GCP AI spend and projected Bedrock cost: @@ -86,9 +116,11 @@ Reference `aws-design-ai.json` → `honest_assessment`. If `"recommend_stay"`, p **Non-cost benefits to present:** model flexibility (30+ models), prompt caching (Claude, 90% savings), AWS ecosystem (Guardrails, Knowledge Bases, Agents), vendor diversification, multi-model strategy. +**Note:** Human/professional-services one-time migration costs are intentionally out of scope for this advisor and excluded from ROI calculations. + --- -## Part 5: Cost Optimization Opportunities +## Part 6: Cost Optimization Opportunities Present applicable optimizations with estimated savings: @@ -103,6 +135,43 @@ Present applicable optimizations with estimated savings: For each applicable optimization, calculate before/after monthly cost and show an `optimized_projection` (best-case monthly with all optimizations). +**Post-migration optimization (do not surface during migration):** Model distillation — training a smaller, faster student model from a larger teacher model — can reduce inference costs up to ~75% for high-volume, stable workloads. Requires production traffic, labeled examples, and a teacher/student eval loop. Mention in the estimate summary as: "Once you have 2–4 weeks of Bedrock production traffic, consider model distillation to further reduce costs. See docs.aws.amazon.com/bedrock/latest/userguide/model-distillation.html." Do not recommend distillation before the startup has migrated and validated their workload. + +--- + +## Part 7: Migration Recommendation (REQUIRED) + +Produce a clear migrate/stay/optimize verdict for the AI workload migration. This is the AI-only equivalent of `estimate-infra.md` Part 7. + +**Decision logic:** + +| Condition | Verdict | `recommendation.path` | +| --------------------------------------------------------------------------------------------------------------------------------- | ------------------- | --------------------- | +| Bedrock cheaper AND capabilities match | Migrate | `migrate_optimized` | +| Bedrock more expensive BUT non-cost benefits justify (vendor diversification, Guardrails, multi-model) AND user priority ≠ `cost` | Migrate with caveat | `migrate_optimized` | +| Bedrock more expensive AND user priority = `cost` AND no compelling non-cost reason | Stay | `stay` | +| Design `honest_assessment` = `recommend_stay` | Stay | `stay` | +| Mixed (some workloads cheaper, some not) | Migrate selectively | `migrate_phased` | + +**Output fields** (add to `estimation-ai.json` top-level): + +```json +"recommendation": { + "path": "migrate_optimized | migrate_phased | stay", + "path_label": "Migrate to Bedrock | Migrate selectively | Stay on current provider", + "migrate_if": "Brief condition under which migration makes sense (1 sentence)", + "stay_if": "Brief condition under which staying makes sense (1 sentence)", + "confidence": "high | medium | low", + "rationale": "2-3 sentence justification citing cost delta and non-cost factors" +} +``` + +**Rules:** + +- MUST emit `recommendation` — never omit. If data is insufficient, set `confidence: "low"` and state why in `rationale`. +- If `honest_assessment` from `aws-design-ai.json` says `recommend_stay`, `recommendation.path` MUST be `stay` regardless of cost numbers. +- For multi-workload runs: if some workloads favor migration and others don't, use `migrate_phased` and list which workloads to migrate vs. keep in `rationale`. + --- ## Output @@ -111,51 +180,69 @@ Write `estimation-ai.json` to `$MIGRATION_DIR/`. **Schema — top-level fields:** -| Field | Type | Description | -| ---------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------- | -| `phase` | string | `"estimate"` | -| `timestamp` | string | ISO 8601 | -| `pricing_source` | string | `"cached"` or `"live"` | -| `accuracy_confidence` | string | `"±5-10%"` or `"±15-25%"` | -| `current_costs` | object | `source`, `gcp_monthly_ai_spend`, `services[]` | -| `token_volume` | object | `source`, `monthly_input_tokens`, `monthly_output_tokens`, ratio | -| `model_comparison` | array | All viable models: `model`, `monthly_cost`, `vs_current`, `quality`, `capabilities_match`, `missing_capabilities[]` | -| `recommended_model` | object | `model`, `monthly_cost`, `breakdown` (input/output/embeddings), `rationale` | -| `backup_model` | object | `model`, `monthly_cost`, `rationale` | -| `embeddings` | object | `model`, `monthly_cost`, `monthly_tokens`, `note` (if applicable) | -| `cost_comparison` | object | `current_gcp_monthly`, `projected_bedrock_monthly`, `monthly_difference`, `annual_difference`, `percent_change` | -| `roi_analysis` | object | `monthly_cost_delta`, `annual_cost_delta`, `justification`, `non_cost_benefits[]` | -| `optimization_opportunities` | array | `opportunity`, `potential_savings_monthly`, `implementation_effort`, `description` | -| `optimized_projection` | object | `monthly_with_optimizations`, `vs_current`, `note` | +| Field | Type | Description | +| ------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------- | +| `phase` | string | `"estimate"` | +| `timestamp` | string | ISO 8601 | +| `pricing_source` | string | `"cached"` or `"live"` | +| `accuracy_confidence` | string | `"±5-10%"` or `"±15-25%"` | +| `current_costs` | object | `source`, `gcp_monthly_ai_spend`, `services[]` | +| `token_volume` | object | `source`, `monthly_input_tokens`, `monthly_output_tokens`, ratio | +| `model_comparison` | array | All viable models: `model`, `monthly_cost`, `vs_current`, `quality`, `capabilities_match`, `missing_capabilities[]` | +| `recommended_model` | object | `model`, `monthly_cost`, `breakdown` (input/output/embeddings), `rationale` | +| `backup_model` | object | `model`, `monthly_cost`, `rationale` | +| `embeddings` | object | `model`, `monthly_cost`, `monthly_tokens`, `note` (if applicable) | +| `cost_comparison` | object | `current_gcp_monthly`, `projected_bedrock_monthly`, `monthly_difference`, `annual_difference`, `percent_change` | +| `migration_cost_considerations` | object | `categories[]` (always `[]`), `complexity_factors[]` (technical integration only), `note` (must state human/pro costs excluded) | +| `roi_analysis` | object | `monthly_cost_delta`, `annual_cost_delta`, `justification`, `non_cost_benefits[]` | +| `optimization_opportunities` | array | `opportunity`, `potential_savings_monthly`, `implementation_effort`, `description` | +| `optimized_projection` | object | `monthly_with_optimizations`, `vs_current`, `note` | +| `recommendation` | object | `path`, `path_label`, `migrate_if`, `stay_if`, `confidence`, `rationale` (see Part 7) | All cost values are numbers, not strings. Output must be valid JSON. ## Validation Checklist +- [ ] `recommendation` field is present with non-empty `path`, `path_label`, `migrate_if`, `stay_if`, and `rationale` +- [ ] `recommendation.path` is one of: `migrate_optimized`, `migrate_phased`, `stay` +- [ ] If Design `honest_assessment` = `recommend_stay`, then `recommendation.path` = `stay` - [ ] `model_comparison` includes ALL viable Bedrock models, not just recommended +- [ ] Legacy models in `model_comparison` are annotated with EOL dates (per `steering/ai-model-lifecycle.md`) +- [ ] `recommended_model` is an Active model (not Legacy) unless no Active alternative exists - [ ] Every model has `capabilities_match` checked against `ai_capabilities_required` - [ ] `recommended_model.rationale` references user's priority, preference, and volume - [ ] `roi_analysis` is honest — if migration increases cost, says so - [ ] `optimization_opportunities` only includes strategies relevant to user's workload - [ ] No compute, database, storage, or networking costs (those belong in `estimate-infra.md`) +- [ ] `migration_cost_considerations.categories` is `[]` — no human one-time migration costs presented + +## Completion Handoff Gate (Fail Closed) + +Before returning control to `estimate.md`, require: + +- `estimation-ai.json` exists and passes the Validation Checklist above. + +If this gate fails: STOP and output: "estimate-ai did not produce a valid `estimation-ai.json`; do not complete Phase 4." ## Present Summary After writing `estimation-ai.json`, present under 25 lines: -1. Current GCP AI spend vs projected Bedrock cost (recommended model) -2. Model comparison table: model name, monthly cost, vs source provider %, capabilities match -3. Recommended model with cost breakdown -4. If migration increases cost: flag honestly with non-cost justification -5. Top 2-3 optimization opportunities with potential savings -6. Optimized projection +1. **Pricing source and accuracy**: State whether prices came from cache or live API, and the accuracy range (±15-25% for AI models from cache, ±5-10% from live API). Example: "AI model estimates based on cached pricing (2026-03-07), accuracy ±15-25%." +2. Current GCP AI spend vs projected Bedrock cost (recommended model) +3. Model comparison table: model name, monthly cost, vs source provider %, capabilities match +4. Recommended model with cost breakdown +5. If migration increases cost: flag honestly with non-cost justification +6. Top 2-3 optimization opportunities with potential savings +7. Optimized projection ## Generate Phase Integration The Generate phase uses `estimation-ai.json`: 1. **`recommended_model`** — Which Bedrock model to provision and test -2. **`optimization_opportunities`** — Which optimizations to implement and when +2. **`migration_cost_considerations`** — `complexity_factors[]` only for integration risk context; **never** present human one-time migration **costs** to the user (`categories` stays `[]`) +3. **`optimization_opportunities`** — Which optimizations to implement and when 4. **`cost_comparison`** — Cost monitoring targets and alerts in production 5. **`model_comparison`** — Fallback options if recommended model doesn't meet quality bar diff --git a/migration-to-aws/steering/estimate-billing.md b/migration-to-aws/steering/estimate-billing.md index 1e38aa28..678ecdd1 100644 --- a/migration-to-aws/steering/estimate-billing.md +++ b/migration-to-aws/steering/estimate-billing.md @@ -52,6 +52,16 @@ Per-service breakdown: This is actual spend data — higher confidence than inferred costs. +### CUD-Aware Baseline Adjustment + +If `billing-profile.json` contains `commitments.has_active_cuds == true`: + +1. **Use list price as baseline**: The `services[].monthly_cost` values already reflect list price (commitment fee rows are excluded). Use these directly — do not subtract CUD credits from the baseline. +2. **Exclude commitment fees from workload costs**: Commitment fee rows (e.g., "Commitment v1: E2 Cpu") are billing artifacts, not workload costs. They are already excluded from `services[]` by the discover phase. +3. **Note the customer's effective discount**: Record `commitments.effective_discount_percent` for the comparison narrative. The customer currently pays this percentage below list price on GCP. + +If `commitments.has_active_cuds == false` or the `commitments` section is absent, proceed with `total_monthly_spend` as-is. + ## Step 2: Generate AWS Cost Ranges For each service in `aws-design-billing.json`, produce low/mid/high estimates: @@ -106,7 +116,39 @@ Difference: Worst case: [high vs GCP] (potential increase of $X/month) ``` -## Step 4: Document Unknowns +### Commitment-to-Commitment Comparison (if CUDs detected) + +If `billing-profile.json` has `commitments.has_active_cuds == true`, add a commitment context note: + +``` +Commitment Context: + GCP current effective rate: [effective_discount_percent]% below list price via CUDs + GCP list price baseline: $[cost_basis.total_at_list]/month + GCP net-of-discounts: $[cost_basis.total_net_of_discounts]/month + + AWS equivalent commitment options: + Compute Savings Plans (Fargate/Lambda/EC2): up to 66% max; typical 20-40% (1yr no-upfront) + Database Savings Plans (Aurora/RDS/etc.): up to 35% (serverless) / up to 20% (provisioned); 1yr no-upfront + RDS Reserved Instances: up to 69% (3yr All Upfront); mutually exclusive with Database SP on same workload + 1-year Compute Savings Plan: typically 20-40% below on-demand + 3-year Compute Savings Plan: typically 40-66% below on-demand + + Fair comparison: GCP list price vs AWS on-demand (both uncommitted) + Committed comparison: GCP net-of-CUD vs AWS with applicable 1yr commitments + Cloud Run / GKE → Fargate note: Do not size Compute Savings Plans from GCP billing — establish 30-90 day AWS baseline post-migration. GKE node pools may inform a rough floor estimate only. +``` + +This ensures the customer understands that their current GCP discount has an AWS equivalent, and the comparison is apples-to-apples. + +## Step 4: Human One-Time Migration Costs (Out of Scope) + +**Do not** present human labor, professional services, engineering, training, discovery/design effort, or similar people-time work as one-time migration **costs** or budget categories. + +Populate `migration_cost_considerations.categories` as an **empty array** `[]`. Use `migration_cost_considerations.note` to state that human and professional-services one-time migration costs are intentionally excluded. You may still recommend IaC discovery in `recommendation.next_steps` or `unknowns` as a **precision** improvement — without framing it as a cost line item. + +**Vendor fees:** If you discuss GCP egress in narrative, describe it only as **vendor/network charges** when grounded in `billing-profile.json` (do not invent dollar amounts). Do not mix human effort into “one-time cost” lists. + +## Step 5: Document Unknowns List what would narrow the cost ranges: @@ -123,7 +165,7 @@ Recommendation: This would narrow total estimate range from ±30-40% to ±10-15%. ``` -## Step 5: Generate Output +## Step 6: Generate Output Write `estimation-billing.json`. @@ -135,7 +177,7 @@ Write `estimation-billing.json`. "timestamp": "[ISO 8601]", "metadata": { "estimate_source": "billing_only", - "pricing_source": "cached|live|fallback", + "pricing_source": "cached|live|cached_fallback|unavailable", "confidence_note": "Estimates have wider ranges due to billing-only source" }, "accuracy_confidence": "±30-40%", @@ -179,6 +221,12 @@ Write `estimation-billing.json`. "worst_case_increase": 0.00 }, + "migration_cost_considerations": { + "categories": [], + "note": "Human and professional-services one-time migration costs are not presented by this advisor. Billing-only source increases estimate variance; IaC discovery narrows recurring cost ranges.", + "complexity_factors": ["billing_only_source", "unknown_infrastructure_config"] + }, + "unknowns": [ { "category": "compute_sizing", @@ -226,17 +274,27 @@ Write `estimation-billing.json`. - No reference to Terraform-based configurations - All unknowns documented with impact and resolution - All cost values are numbers, not strings +- `migration_cost_considerations.categories` is `[]` — no human one-time migration costs presented - Output is valid JSON +## Completion Handoff Gate (Fail Closed) + +Before returning control to `estimate.md`, require: + +- `estimation-billing.json` exists and passes the Output Validation Checklist above. + +If this gate fails: STOP and output: "estimate-billing did not produce a valid `estimation-billing.json`; do not complete Phase 4." + ## Present Summary After writing `estimation-billing.json`, present a concise summary to the user: -1. GCP baseline from billing data (total monthly spend) -2. AWS projected cost ranges: low / mid / high per service -3. Total projection: best case / expected / worst case vs GCP -4. Key unknowns that would narrow the estimates -6. Recommendation: run IaC discovery for tighter estimates (±10-15% vs ±30-40%) +1. **Pricing source and accuracy**: State that estimates are billing-only projections with ±30-40% accuracy due to lack of infrastructure configuration. Example: "Billing-only estimates, accuracy ±30-40%. Provide Terraform files to narrow to ±10-15%." +2. GCP baseline from billing data (total monthly spend) +3. AWS projected cost ranges: low / mid / high per service +4. Total projection: best case / expected / worst case vs GCP +5. Key unknowns that would narrow the estimates +6. Recommendation: run IaC discovery for tighter estimates (±10-15% vs ±30-40%) — as a precision step, not as a human cost estimate Keep it under 20 lines. The user can ask for details or re-read `estimation-billing.json` at any time. @@ -247,3 +305,4 @@ The Generate phase uses `estimation-billing.json`: - Uses wide cost ranges for conservative timeline planning - Recommends IaC discovery as a prerequisite step - Documents unknowns as prerequisites per generation step +- **Do not** surface human one-time migration **costs** from this artifact — `migration_cost_considerations.categories` remains empty in user-facing docs diff --git a/migration-to-aws/steering/estimate-infra.md b/migration-to-aws/steering/estimate-infra.md index e8132312..1d718227 100644 --- a/migration-to-aws/steering/estimate-infra.md +++ b/migration-to-aws/steering/estimate-infra.md @@ -10,8 +10,10 @@ The parent `estimate.md` determines pricing source before loading this file. **Price lookup order for each AWS service in `aws-design.json`:** -1. **`steering/cached-prices.md` (primary)** — Read once. Look up each service by table. If found, use the price directly. No MCP call needed. Set `pricing_source: "cached"`. +1. **`steering/cached-prices.md` (primary)** — Read once. **Before using, check staleness:** compute `days_since_cache = today − cache "Last updated" date`. If `days_since_cache > 30`, set `pricing_source: "cached_stale"` for all AI model prices and prepend a warning to the estimate output: "Pricing cache is more than 30 days old — AI model prices may have changed. Verify via the AWS Pricing MCP server or aws.amazon.com/bedrock/pricing." Infrastructure prices (Fargate, RDS, S3, etc.) remain reliable; only AI model prices need the stale flag. If `days_since_cache ≤ 30`, use the price directly and set `pricing_source: "cached"`. 2. **MCP with recipes (secondary)** — If a service is NOT in cached-prices.md and MCP is available, use the Pricing Recipes table below. Set `pricing_source: "live"`. +3. **Cache after MCP failure** — If MCP was attempted but failed, and the service IS in the cache, use the cached price. Set `pricing_source: "cached_fallback"`. +4. **Unavailable** — If a service is NOT in the cache AND MCP failed, set `pricing_source: "unavailable"`. Add to `services_with_missing_fallback` and warn the user. For typical migrations (Fargate, Aurora/RDS, Aurora Serverless v2, S3, ALB, NAT Gateway, Lambda, Secrets Manager, CloudWatch, ElastiCache, DynamoDB), ALL prices are in `cached-prices.md`. Zero MCP calls needed. @@ -68,16 +70,36 @@ Determine the current GCP monthly infrastructure costs. Use the best available s 1. **`billing-profile.json` (preferred)** — Use actual billing data as the GCP baseline. Highest confidence (±5%). 2. **`gcp-resource-inventory.json` (fallback)** — Estimate costs from discovered resource configurations. Wider range (±20-30%). 3. **`preferences.json` → `gcp_monthly_spend`** — User-provided monthly spend from clarification. -4. **Conservative default** — If none of the above: use `AWS monthly balanced × 1.25`. +4. **Ask the user** — If none of the above are available, ask: "I need your current GCP monthly spend to produce a meaningful cost comparison. What is your approximate GCP monthly infrastructure cost?" Use the user's answer. If the user declines or is unsure, present AWS costs without a GCP comparison and note: "GCP baseline unavailable — AWS costs shown without comparison." Present the GCP baseline as a total and per-service breakdown, noting which source was used. +### CUD-Aware Baseline (when billing data available) + +If `billing-profile.json` contains `commitments.has_active_cuds == true`: + +1. **Use list price as the GCP baseline**: The `services[].monthly_cost` values already use list price (commitment fee rows excluded by discover phase). Use `cost_basis.total_at_list` as the total GCP baseline for comparison. +2. **Do not include commitment fees in workload costs**: Commitment fee rows are billing artifacts with no AWS equivalent — they are already excluded from `services[]`. +3. **Record the effective discount for Part 3**: Note `commitments.effective_discount_percent` — this is the customer's current GCP discount rate that will be compared against AWS Savings Plan rates. + +This ensures the comparison is GCP list price vs. AWS on-demand (both uncommitted baselines), with commitment options presented separately as optimization opportunities. + --- ## Part 2: Calculate Projected AWS Costs +**Security baseline coverage (always required):** Add a `security_baseline` entry to `projected_costs.breakdown` with `service: "AWS Security Baseline (Tier 1)"`, low/mid/high estimates of $3/$15/$30 per month, `accuracy: "±25%"`, and a `components` sub-object breaking down CloudTrail S3 storage (~$1.50/mo mid), GuardDuty (~~$13/mo mid after free trial), AWS Budgets ($0), and the free controls. If `preferences.json.compliance` contains any of `soc2`, `pci`, `hipaa`, `fedramp`, also add a sibling `security_baseline_compliance` entry with low/mid/high estimates of $3/$14/$25 per month, `accuracy: "±25%"`, `emission_reason` field citing the declared compliance values, and a `components` sub-object breaking down AWS Config (~$6/mo mid continuous), Config S3 storage (~~ $0.50/mo mid), Security Hub + FSBP (~$7/mo mid after free trial), and extra standards (free). Per-unit rates are grounded in the AWS Pricing API for us-east-1 as of 2026-05-04 (Config pricing effective 2025-09-01, Security Hub effective 2026-03-01). Cite source as `steering/cached-prices.md § Security Baseline` or live `get_pricing` calls for `AmazonGuardDuty`, `AWSConfig`, and `AWSSecurityHub` service codes. Both line items are added as flat additives to each tier total (Premium/Balanced/Optimized) rather than being tier-dependent. + For each service in `aws-design.json`, calculate monthly cost using rates from `cached-prices.md`. Track `pricing_source` per service. +**Secret Manager coverage (mandatory):** If any mapped resource has `gcp_type` of `google_secret_manager_secret` or `google_secret_manager_secret_version`, ensure an `aws_service` entry for **Secrets Manager** is present in the estimate breakdown. Do not collapse this into a generic "supporting" line item. + +**BigQuery / deferred analytics (mandatory):** For any resource where `aws_service` is exactly **`Deferred — specialist engagement`** OR `gcp_type` starts with `google_bigquery_`: + +- **Do not** apply Athena, Redshift, Glue, or EMR rates as the plugin’s “projected” analytics stack. +- **Exclude** these resources from Premium / Balanced / Optimized **numeric totals** (or list them under a `deferred_services[]` / `excluded_from_totals` section in `estimation-infra.json` with reason: _pending specialist engagement_). +- In the user-facing summary, state that **AWS analytics costs are unknown** until the **AWS account team** and/or **data analytics migration partner** defines the target architecture. + Calculate 3 cost tiers to show the optimization range: | Tier | Description | Examples | @@ -88,33 +110,243 @@ Calculate 3 cost tiers to show the optimization range: **Per-service calculation approach:** -| Domain | Formula | Key inputs from aws-design.json | -| ----------------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------- | -| Compute (Fargate) | (vCPU × vCPU rate + memory GB × memory rate) × 730 hours × instance count | `aws_config.cpu`, `aws_config.memory` | -| Compute (Lambda) | requests × request rate + (requests × duration × memory GB) × GB-second rate | Estimated from usage patterns | -| Database (Aurora) | instance rate × 730 hours × instance count + storage GB × storage rate + I/O estimate | `aws_config.instance_class`, `aws_config.allocated_storage` | -| Database (RDS) | instance rate × 730 hours × instance count + storage GB × storage rate | `aws_config.instance_class`, `aws_config.allocated_storage` | -| Storage (S3) | GB × per-GB rate + request estimates | `aws_config.storage_gb` or source `gcp_config` | -| Networking (ALB) | fixed monthly + LCU estimate | From compute service count | -| Networking (NAT) | fixed monthly × count + GB processed × data rate | From VPC design | -| Supporting | Per-unit rates × quantities (secrets, log GB, metrics) | Inferred from service count | +| Domain | Formula | Key inputs from aws-design.json | +| -------------------------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | +| Compute (Fargate) | (vCPU × vCPU rate + memory GB × memory rate) × 730 hours × instance count | `aws_config.cpu`, `aws_config.memory` | +| Compute (Lambda) | requests × request rate + (requests × duration × memory GB) × GB-second rate | Estimated from usage patterns | +| Database (Aurora) | instance rate × 730 hours × instance count + storage GB × storage rate + I/O estimate | `aws_config.instance_class`, `aws_config.allocated_storage` | +| Database (RDS) | instance rate × 730 hours × instance count + storage GB × storage rate | `aws_config.instance_class`, `aws_config.allocated_storage` | +| Storage (S3) | GB × per-GB rate + request estimates | `aws_config.storage_gb` or source `gcp_config` | +| Networking (ALB) | fixed monthly + LCU estimate | From compute service count | +| Networking (NAT) | fixed monthly × count + GB processed × data rate | From VPC design | +| Security (Secrets Manager) | secrets_count × per-secret monthly rate + api_calls_10k × per-10K API rate | `aws_config.secrets_count`, `aws_config.api_calls_10k` (or inferred defaults) | +| Supporting | Per-unit rates × quantities (secrets, log GB, metrics) | Inferred from service count | Show calculation breakdown per service: rate × quantity = cost. Present all 3 tiers side-by-side. --- +## Part 2B: Observability Cost Estimation (CloudWatch) + +GCP Cloud Operations Suite has generous free tiers (50 GB/month logging free, 150M metric samples free, alerting free, profiling free). AWS CloudWatch charges from the first GB and first custom metric. This section ensures observability costs are not a surprise post-migration. + +**Relationship to Part 2 "Supporting" line item:** The observability entry produced by this section REPLACES any CloudWatch/log/metric portion that would otherwise appear in the "Supporting" row of Part 2. Do NOT include CloudWatch log ingestion, metrics, or alarms in the Supporting line item — they are fully covered here. Supporting retains only Secrets Manager and any non-observability per-unit charges. + +### Pricing source + +All rates from `cached-prices.md § CloudWatch` and `§ X-Ray`. No MCP calls needed. + +### Step 1: Determine log volume + +**IF billing data IS available** (`billing-profile.json` exists): + +Check for Cloud Logging line items: + +- Look for `services[].sku` containing `Log Volume`, `Logging`, or `Cloud Logging` +- Extract volume in GiB from the SKU quantity/units field directly (preferred) +- OR derive from GCP cost: `gcp_logging_cost ÷ $0.50/GiB` (GCP's own list rate above free tier) = volume above free tier, then add 50 GiB for the free portion +- **Important:** Use GCP's own rate ($0.50/GiB) to reverse-engineer volume from GCP billing — not AWS rates. Apply AWS rates ($0.50/GB Standard) to the derived volume for the projected cost. +- Set `observability.log_volume_source: "billing"` + +**IF billing data is NOT available:** + +Use the per-service heuristic table to estimate monthly log volume: + +| AWS Service (from aws-design.json) | Estimated log volume/month | Basis | +| ---------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Fargate task | 3 GB per task | Container stdout/stderr, request logs | +| Lambda function | 0.2 GB per function | Invocation logs at INFO level | +| RDS/Aurora instance | 1 GB per instance | Error log + slow query (audit log off) | +| RDS/Aurora instance (audit on) | 8 GB per instance | If source has `pgaudit.log`, `log_statement = 'all'`, `cloudsql.log_min_duration_statement`, or explicit audit/general log database flags in Terraform | +| ALB | 2 GB per load balancer | Access logs (if enabled) | +| NAT Gateway | 1 GB per gateway | Flow logs (if enabled) | +| ElastiCache | 0.5 GB per node | Slow log only | + +**RDS audit detection:** Check for database flags in the source `google_sql_database_instance` Terraform that indicate audit logging: `pgaudit.log`, `log_statement = 'all'`, `cloudsql.log_min_duration_statement`, or general/audit log flags for MySQL. Do NOT use `cloudsql.iam_authentication` as an audit proxy — IAM auth is access control, not audit logging. + +Sum all applicable services. Set `observability.log_volume_source: "heuristic"`. + +### Step 2: Determine custom metrics count + +**IF billing data IS available:** + +GCP Cloud Monitoring uses sample-based pricing ($0.10/1000 samples above 150M free) which does not map 1:1 to CloudWatch's per-metric-per-month pricing ($0.30/metric/month). Even with billing data, treat custom metrics as heuristic: + +- Count `google_monitoring_alert_policy` resources in `gcp-resource-inventory.json` as a proxy for custom metric complexity +- Multiply alert policy count × 3 (typical metrics per alert: threshold metric + 2 related dimensions) +- Set `observability.metrics_source: "heuristic"` (even when billing data exists — GCP metric pricing is not cleanly convertible to CloudWatch's model) + +**IF billing data is NOT available:** + +Use the heuristic: `custom_metrics_count = (number of services in aws-design.json × 5) + (number of alert policies from source × 2)` + +Default floor: 10 custom metrics. +Default ceiling for small startups: 50 custom metrics. + +Set `observability.metrics_source: "heuristic"`. + +### Step 2b: Determine alarm count + +Count `google_monitoring_alert_policy` resources in `gcp-resource-inventory.json`. + +- If count > 0: `alarm_count = count` +- If count is 0 or no alert policies found: `alarm_count = 5` (baseline — most teams set up basic health/error/latency alarms post-migration) + +Set `observability.alarm_count_source: "inventory"` or `"default"`. + +### Step 3: Determine tracing usage + +**IF billing data IS available:** + +Check for Cloud Trace line items: + +- Extract span volume from SKU quantity/units field directly (preferred) +- OR derive from GCP cost: `gcp_trace_cost ÷ $0.20/million` (GCP's own list rate above 2.5M free tier) = millions of spans above free tier, then add 2.5M +- **Important:** Use GCP's rate ($0.20/M spans) to reverse-engineer volume from GCP billing. Apply AWS X-Ray rate ($5.00/M traces) to that volume for the projected cost. +- Set `observability.tracing_source: "billing"` + +**IF billing data is NOT available:** + +Check `gcp-resource-inventory.json` or application code for tracing libraries: + +- If OpenTelemetry, `@google-cloud/trace-agent`, or `google.cloud.trace` imports detected: assume 1M spans/month (small app baseline) +- If no tracing signals: set `monthly_spans: 0` (tracing not in use; do not add X-Ray costs) + +Set `observability.tracing_source: "heuristic"`. + +### Step 4: Calculate CloudWatch costs + +Default estimate uses Standard log class ($0.50/GB) for all logs. Infrequent Access ($0.25/GB) is an optimization opportunity surfaced in Step 6, not the baseline assumption. + +``` +log_ingestion_cost = monthly_log_gb × $0.50 +log_storage_cost = monthly_log_gb × $0.03 × retention_months (default: 1) +custom_metrics_cost = custom_metrics_count × $0.30 (flat rate; valid for ≤10K metrics at startup scale) +alarms_cost = alarm_count × $0.10 +tracing_cost = max(0, monthly_spans - 100_000) / 1_000_000 × $5.00 (honors X-Ray 100K/month free tier) +dashboard_cost = max(0, dashboards - 3) × $3.00 (default: 0 — assume ≤3) + +total_observability = log_ingestion_cost + log_storage_cost + custom_metrics_cost + alarms_cost + tracing_cost + dashboard_cost +``` + +### Step 5: Add to projected costs + +Add an `observability` entry to `projected_costs.breakdown`. This entry REPLACES any CloudWatch/log/metric portion in the "Supporting" row — do not double-count. + +```json +{ + "service": "CloudWatch + X-Ray (Observability)", + "low": , + "mid": , + "high": , + "accuracy": "±30%", + "pricing_source": "cached", + "components": { + "log_ingestion": , + "log_storage": , + "custom_metrics": , + "alarms": , + "tracing": + }, + "volume_source": " (reflects log volume source — the largest cost component; metrics are always heuristic regardless of this field)", + "note": "GCP Cloud Operations includes 50 GB/month free logging, free alerting, and free profiling. CloudWatch charges from the first GB. Tracing is significantly more expensive on X-Ray ($5/M) vs Cloud Trace ($0.20/M). Actual costs depend on log verbosity and retention policy." +} +``` + +### Step 6: Surface GCP free tier delta in cost comparison + +In Part 3 (Cost Comparison), if observability costs exceed $20/month, add a callout: + +> **Observability cost note:** Your GCP Cloud Operations costs may appear low or zero due to generous free tiers (50 GB/month logging, 150M metric samples, free alerting). The CloudWatch estimate of $X/month reflects the same workload without those free tiers. Consider: +> +> - Reducing log verbosity (WARN-only for production services) to lower ingestion costs +> - Using CloudWatch Logs Infrequent Access class for non-critical logs ($0.25/GB — 50% cheaper than Standard) +> - Evaluating sampling rate for X-Ray traces (10% sampling = 90% cost reduction) +> - CloudWatch Logs Insights for ad-hoc queries instead of always-on metric filters + +### Estimation rules + +- Do NOT emit observability costs as $0 — even minimal apps produce logs on AWS +- Floor: $5/month (absolute minimum for any running Fargate + RDS workload) +- If tracing is not detected in source, do NOT add X-Ray costs (don't upsell) +- Container Insights is NOT included by default — add only if source uses Cloud Monitoring with per-container metrics or if production-tier observability is required +- The observability line item REPLACES CloudWatch entries in the "Supporting" row — never double-count + +--- + ## Part 3: Cost Comparison Present a side-by-side comparison: -- GCP current monthly total +- GCP current monthly total (at list price) - AWS Premium / Balanced / Optimized monthly totals - Difference (savings or increase) per tier vs GCP - Per-service breakdown for the Balanced tier +### Commitment Context (if CUDs detected) + +If `billing-profile.json` has `commitments.has_active_cuds == true`, add a `commitment_context` section to the comparison: + +- **GCP effective committed rate**: The customer currently pays `effective_discount_percent`% below list via CUDs +- **AWS equivalent**: + - **Compute Savings Plans** (Fargate, Lambda, EC2): up to 66% vs On-Demand at maximum term/discount; typical 1-year no-upfront **20–40%** + - **Database Savings Plans** (Aurora, RDS, DynamoDB, ElastiCache, etc.): up to **35%** (serverless) / up to **~20%** (provisioned instances), 1-year no-upfront + - **RDS Reserved Instances** (alternative to Database Savings Plans on the same workload): up to **69%** (3-year All Upfront); locked to instance family/region — mutually exclusive with Database Savings Plans per workload +- **Fair comparison framing**: Present both an uncommitted comparison (GCP list vs. AWS on-demand) and a committed comparison (GCP net-of-CUD vs. AWS with 1yr commitments where applicable) +- **Migration timing note**: If the customer has active CUDs, note that CUD fees continue regardless of usage — migrating mid-commitment means paying both GCP CUD fees and AWS costs until the CUD term expires. For **Cloud Run → Fargate** or **GKE → Fargate** (re-platform) workloads, recommend establishing a 30–90 day AWS compute baseline before purchasing Compute Savings Plans (see Part 6). + +Include in `estimation-infra.json` under `cost_comparison`: + +```json +"commitment_context": { + "gcp_has_active_cuds": true, + "gcp_effective_discount_percent": 8.2, + "gcp_monthly_at_list": 2450.00, + "gcp_monthly_net_of_discounts": 2280.00, + "aws_compute_savings_plan_discount": "up to 66% (Fargate/Lambda/EC2; max term); typical 20-40% (1yr no-upfront)", + "aws_database_savings_plan_discount": "up to 35% (serverless) / up to 20% (provisioned RDS/Aurora)", + "aws_rds_reserved_instance_discount": "up to 69% (specific instance family, 3yr All Upfront)", + "aws_1yr_savings_plan_typical_discount": "20-40%", + "aws_3yr_savings_plan_typical_discount": "40-66%", + "note": "GCP baseline uses list price for apples-to-apples comparison. Customer currently saves 8.2% via CUDs. AWS Savings Plans (compute + database) offer comparable or deeper discounts post-migration. Database Savings Plans and RDS Reserved Instances are mutually exclusive on the same workload. For Cloud Run → Fargate or GKE → Fargate (re-platform), commit Compute Savings Plans after establishing a 30-90 day AWS usage baseline." +} +``` + +If `commitments.has_active_cuds == false` or the section is absent, omit `commitment_context` from the output. + --- -## Part 4: ROI Analysis +## Part 4: GCP Data Transfer Egress (Vendor Fees Only) + +This section covers **GCP vendor/network charges** for outbound data during migration — not human labor or professional-services costs (those are never presented as dollar estimates by this advisor). + +**Billing data check:** Before generating this section, check if `$MIGRATION_DIR/billing-profile.json` exists. + +### IF billing data IS available (`billing-profile.json` exists): + +**Data transfer** — egress fees from GCP during migration. GCP charges for outbound data transfer; volume depends on database sizes and storage to migrate. Use the billing data to estimate the volume of data that needs to move. + +Set `billing_data_available: true` in the output `migration_cost_considerations` object. + +### IF billing data is NOT available (`billing-profile.json` does not exist): + +**Omit GCP data transfer fee estimates.** Without billing data, there is no grounding for egress projections. Instead, include only this note in the output: + +Set `migration_cost_considerations` to: + +```json +{ + "categories": [], + "billing_data_available": false, + "note": "Data transfer cost estimates require GCP billing data. Re-run discovery with a GCP billing export to see GCP egress fee projections." +} +``` + +In the user-facing summary, when billing data is missing, state: "GCP data transfer egress estimates require billing data. Provide a billing export and re-run discovery to see vendor egress projections." + +--- + +## Part 5: ROI Analysis Present the monthly and annual cost difference between GCP baseline and each AWS tier (Premium, Balanced, Optimized). This is the recurring savings (or increase) the customer can expect. @@ -129,24 +361,138 @@ Present the monthly and annual cost difference between GCP baseline and each AWS **Non-cost benefits to present:** operational efficiency, global reach, service breadth, enterprise integration, vendor diversification, scaling flexibility (auto-scaling, spot instances, savings plans). +**Note:** GCP data transfer egress fees (if estimated in Part 4) are **vendor** one-time charges excluded from recurring ROI calculations — not human migration costs. + --- -## Part 5: Cost Optimization Opportunities +## Part 6: Cost Optimization Opportunities -Present applicable optimizations with estimated savings: +**Relationship to cost tiers:** Premium / Balanced / Optimized totals in Part 2 are **pricing scenarios** for the same design. The **Optimized** tier already assumes illustrative trade-offs (e.g. reserved DB pricing, Fargate Spot). Entries in `optimization_opportunities` are **incremental post-migration actions** beyond the Balanced on-demand baseline — do **not** add their savings on top of Optimized tier totals (which already embed assumptions). -| Optimization | Savings Range | Applies To | When | -| ---------------------------------- | ------------- | -------------------------------- | --------------------------------------- | -| Reserved Instances / Savings Plans | 40-60% | RDS, Aurora | Post-migration (after validating usage) | -| Compute Savings Plans | 20-50% | Fargate, Lambda | Post-migration | -| S3 Intelligent-Tiering / S3-IA | 38-50% | S3 storage | During migration | -| Spot Instances | 60-90% | Batch/non-critical EC2 workloads | If batch jobs exist | +Present applicable optimizations with estimated savings: -For each applicable optimization, calculate the before and after monthly cost. +| Optimization | Savings Range | Applies To | When | +| ------------------------------ | ------------------------------------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------- | +| Compute Savings Plans | 20–66% | Fargate, Lambda, EC2 | Post-migration (after 30–90 day usage baseline) | +| Database Savings Plans | Up to 35% (serverless) / ~20% (provisioned) | Aurora, RDS, DynamoDB, ElastiCache, DocumentDB, Neptune, Keyspaces, Timestream, DMS | Post-migration or after instance right-sizing | +| RDS Reserved Instances | Up to 69% | RDS, Aurora (provisioned) | Post-migration (after architecture stabilizes) | +| S3 Intelligent-Tiering / S3-IA | 38–50% | S3 storage | During migration | +| Spot Instances | 60–90% | Batch/non-critical EC2 workloads | If batch jobs exist | + +For each applicable optimization: + +- **Compute Savings Plans:** emit percent range and post-migration timing; **omit `savings_monthly`** unless 30+ days of AWS usage data exist — do not size commitments from GCP Cloud Run or GKE billing alone (see below). +- **Database Savings Plans / RDS RIs:** may include preliminary `savings_monthly` when the Design phase mapped a target instance class **and** projected monthly DB on-demand cost exceeds **$50/month**; otherwise percent-only guidance. +- **S3 / Spot:** calculate before/after monthly cost when quantities are known from design. + +Cross-reference Clarify when available: `cloud_run_traffic_pattern = business-hours` strengthens post-migration baseline guidance for compute; `constant-24-7` allows slightly more confidence in floor sizing — still post-migration for Compute Savings Plans. + +### Compute Savings Plans (Fargate / Lambda) + +**Relevance to GCP migrations:** + +| Source | Fargate commitment sizing from GCP data? | Notes | +| -------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Cloud Run → Fargate** | **Unreliable** | Variable-priced, scales to zero, bills per-request. GCP billing does not map 1:1 to Fargate vCPU-hours. | +| **GKE → Fargate** (re-platform to ECS) | **Unreliable for commitment; useful for sanity check** | Node pools are often 24/7 with known machine types — you can rough-estimate vCPU-hours from node count × machine type. But re-platforming changes task sizing, autoscaling, and bin-packing; do not purchase a Savings Plan from GCP data alone. | +| **GKE → EKS** (keep Kubernetes) | **Partial** | Compute Savings Plans apply to **EC2 worker nodes** or **EKS Fargate profiles**, not the EKS control plane fee (~$0.10/hr per cluster). Post-migration baseline still recommended after right-sizing. | + +**What Compute Savings Plans offer:** + +- Up to **66%** savings on Fargate, Lambda, and EC2 vs On-Demand at maximum term/discount ([source](https://aws.amazon.com/savingsplans/compute-pricing/)) +- Commitment measured in **$/hour** — not tied to specific instance types, regions, or services +- Applies automatically across EC2, Fargate, and Lambda regardless of instance family, size, AZ, region, OS, or tenancy ([source](https://aws.amazon.com/savingsplans/faqs/)) +- 1-year or 3-year terms; deeper discounts for longer terms and higher upfront payment + +**Guidance for workloads migrating to Fargate (Cloud Run or GKE re-platform):** + +The plugin SHALL present Compute Savings Plans as a **post-migration optimization** and SHALL NOT size a commitment from GCP billing data alone: + +1. **Run on On-Demand for 30–90 days post-migration** to establish an AWS usage baseline +2. **Use AWS Cost Explorer Savings Plans Recommendations** — after 30+ days, Cost Explorer generates personalized recommendations based on actual hourly usage floor +3. **Commit to the usage floor, not the average** — cover minimum sustained usage; burst above the commitment bills at On-Demand with no penalty +4. **Consider rolling Savings Plans** — purchase smaller plans staggered quarterly to reduce risk if usage patterns change ([source](https://aws.amazon.com/blogs/aws-cloud-financial-management/how-can-i-use-rolling-savings-plans-to-reduce-commitment-risk/)) + +**Emit in `optimization_opportunities` when Fargate or Lambda is in `aws-design.json`:** + +```json +{ + "opportunity": "Compute Savings Plans", + "type": "compute_savings_plan", + "target_services": ["Fargate", "Lambda"], + "savings_percent": "20-66%", + "savings_monthly": null, + "commitment": "1-year or 3-year", + "timing": "post-migration (after 30-90 days of usage data)", + "implementation_effort": "low", + "prerequisite": "Establish AWS compute usage baseline before committing", + "description": "GCP compute billing (Cloud Run variable pricing or GKE re-platform to Fargate) makes pre-migration commitment sizing unreliable. GKE node pool sizing may sanity-check the floor but do not commit from GCP data alone. Use AWS Cost Explorer recommendations after migration to size the commitment to your actual usage floor.", + "references": [ + "https://aws.amazon.com/savingsplans/compute-pricing/", + "https://aws.amazon.com/savingsplans/faqs/" + ] +} +``` + +### Database Savings Plans and Reserved Instances (Aurora / RDS) + +**Relevance to GCP migrations:** Cloud SQL instances typically run 24/7 with a known instance size. Unlike Cloud Run, Cloud SQL usage translates well to AWS RDS/Aurora steady-state workloads — commitment sizing is more predictable from source configuration. + +**What Database Savings Plans offer:** + +- Up to **35%** on serverless deployments; up to **~20%** on provisioned RDS/Aurora instances ([source](https://aws.amazon.com/blogs/aws/introducing-database-savings-plans-for-aws-databases/)) +- **1-year term only**, no upfront payment required ([source](https://aws.amazon.com/about-aws/whats-new/2025/12/database-savings-plans-savings/)) +- Applies across eligible usage regardless of engine, instance family, size, deployment option, or Region +- **Mutually exclusive with RDS Reserved Instances on the same workload** — choose one discount model per database workload; you may use RIs on one workload and Database Savings Plans on another + +**What RDS Reserved Instances offer (alternative):** + +- Up to **69%** savings for 1-year or 3-year terms ([source](https://aws.amazon.com/rds/reserved-instances/)) +- Payment options: No Upfront (~30%), Partial Upfront (~36% 1yr), All Upfront (~42% 1yr); 3-year terms up to 69% +- Locked to specific instance family, size, and region — less flexible but deeper discounts than Database Savings Plans + +**Guidance for Cloud SQL → Aurora/RDS migrations:** + +When Cloud SQL maps to a prod-tier instance with **> $50/month** projected on-demand DB cost: + +1. Map source instance to AWS instance class (from Design phase) +2. Present Database Savings Plan benefit — 1-year, up to 35% (serverless) or ~20% (provisioned), flexible engine/instance changes +3. Present RI alternative for deeper savings when instance family is stable for 1+ years +4. **Recommend Database Savings Plans over RIs for migrations** — post-migration right-sizing flexibility outweighs RI depth for most customers + +For dev-tier databases (< $50/month on-demand), emit percent-only guidance without dollar `savings_monthly`. + +**Emit in `optimization_opportunities` when RDS or Aurora is in `aws-design.json`:** + +```json +{ + "opportunity": "Database Savings Plans", + "type": "database_savings_plan", + "target_services": ["Aurora", "RDS"], + "savings_percent": "up to 35% (serverless) / up to 20% (provisioned)", + "savings_monthly": 4.50, + "commitment": "1-year no-upfront", + "timing": "immediately post-migration or after instance right-sizing", + "implementation_effort": "low", + "prerequisite": "Confirm target instance class and expected steady-state usage; omit savings_monthly when DB on-demand < $50/month", + "description": "Cloud SQL 24/7 usage is predictable. Database Savings Plans offer flexibility to change engines/instances post-migration. Mutually exclusive with RDS RIs on the same workload.", + "alternative": { + "opportunity": "RDS Reserved Instances", + "type": "rds_reserved_instances", + "savings_percent": "up to 69%", + "trade_off": "Locked to specific instance family and region — less flexibility during post-migration optimization" + }, + "references": [ + "https://aws.amazon.com/savingsplans/database-pricing/", + "https://aws.amazon.com/rds/reserved-instances/", + "https://aws.amazon.com/about-aws/whats-new/2025/12/database-savings-plans-savings/" + ] +} +``` --- -## Part 6: Recommendation +## Part 7: Recommendation Present 3 paths: @@ -159,31 +505,87 @@ Include migrate/stay decision factors: - **Migrate if:** operational efficiency matters, AWS-specific services needed, batch workloads (Spot savings), long-term AWS strategy, growing infrastructure - **Stay if:** cost is the only metric and AWS is more expensive, team deeply experienced with GCP, no need for AWS-specific services +### Persist recommendation to estimation-infra.json + +Part 7 MUST write the following `recommendation` block to `estimation-infra.json`. This is the single source of truth consumed by the HTML migration report (Section 0) and the Estimate chat summary. Do NOT duplicate this logic elsewhere. + +```json +"recommendation": { + "path": "migrate_optimized|migrate_phased|stay", + "path_label": "Migrate with Optimizations|Phased Migration|Stay on GCP", + "roi_justification": "string — one-sentence ROI case from Part 5", + "confidence": "high|medium|low", + "migrate_if": [ + "string — each factor that favors migration for THIS stack" + ], + "stay_if": [ + "string — each factor that favors staying for THIS stack" + ], + "next_steps": [ + "string — actionable items from Part 7" + ] +} +``` + +**Enum normalization for `path`:** + +| Scenario | `path` value | `path_label` (display) | +| -------------------------------------------- | --------------------- | ------------------------------ | +| AWS cheaper or operational benefits justify | `"migrate_optimized"` | `"Migrate with Optimizations"` | +| Complex stack, phase-by-phase safer | `"migrate_phased"` | `"Phased Migration"` | +| AWS more expensive AND costs are sole metric | `"stay"` | `"Stay on GCP"` | + +Use `path` for machine consumption; `path_label` for display in report and chat. + +**Required fields:** `path`, `path_label`, `confidence`, `migrate_if` (non-empty array), `stay_if` (non-empty array), `next_steps` (non-empty array). `roi_justification` is optional (omit when `path` is `"stay"`). + +Tailor `migrate_if` and `stay_if` to THIS stack (deferred services, AI cost delta, CUD lock-in, team GCP depth, etc.) — do not copy the generic Part 7 bullets verbatim unless they apply. + --- ## Output Read `steering/schema-estimate-infra.md` for the `estimation-infra.json` schema and validation checklist, then write `estimation-infra.json` to `$MIGRATION_DIR/`. +## Completion Handoff Gate (Fail Closed) + +Load `steering/handoff-gates.md`. **Re-read from disk** before checking. + +Before returning control to `estimate.md`, require: + +- `estimation-infra.json` exists and passes `steering/schema-estimate-infra.md` validation. +- `recommendation.path` is one of `migrate_optimized`, `migrate_phased`, or `stay` +- `recommendation.path_label` is non-empty +- `recommendation.migrate_if` and `recommendation.stay_if` are non-empty arrays (Part 7 MUST persist `recommendation`) + +**On FAIL:** Emit `GATE_FAIL | phase=estimate | field= | reason=missing`. **Do NOT patch `estimation-infra.json` to pass the gate.** STOP — do not return control to `estimate.md` for phase completion. + +**On PASS:** Emit `HANDOFF_OK | phase=estimate | artifacts=estimation-infra.json` (parent `estimate.md` emits the combined handoff after all routes pass). + ## Present Summary After writing `estimation-infra.json`, present a concise summary to the user: -1. GCP baseline vs AWS projected (balanced tier) — one-line comparison -2. Three-tier table: Premium / Balanced / Optimized with monthly totals -3. Per-service cost breakdown (balanced tier, 1 line per service) -4. Monthly and annual savings (or increase) vs GCP per tier -5. Top 2-3 optimization opportunities with savings amounts +1. **Pricing source and accuracy**: State whether prices came from cache or live API, and the accuracy range (±5-10% for infrastructure from cache/live, ±15-25% if cache is stale). Example: "Estimates based on cached AWS pricing (2026-03-07), accuracy ±5-10%." +2. GCP baseline vs AWS projected (balanced tier) — one-line comparison +3. Three-tier table: **Premium**, **Balanced**, **Optimized** with monthly totals. Under or beside each label, use the **short subtitles**: Premium — _Highest resilience / highest monthly estimate in this model_; Balanced — _Default scenario; compare GCP to this first_; Optimized — _Lower monthly estimate; reservations / Spot / storage trade-offs assumed_. Add a one-line **How to read**: three figures are **pricing scenarios** for the same architecture (high → mid → low); **not** three Terraform stacks. When Terraform is generated later, it aligns with **Balanced**. +4. Per-service cost breakdown (balanced tier, 1 line per service) +5. **If billing data available**: Estimated GCP data transfer egress fees. **If billing data NOT available**: "Data transfer cost estimates require GCP billing data." +6. Monthly and annual savings (or increase) vs GCP per tier +7. Top 2-3 optimization opportunities with savings amounts +8. **Recommendation:** `recommendation.path_label` with one-line ROI justification when present Keep it under 25 lines. The user can ask for details or re-read `estimation-infra.json` at any time. ## Generate Phase Integration -The Generate phase (`steering/generate.md`) uses `estimation-infra.json` as follows: +The Generate phase (`generate.md`) uses `estimation-infra.json` as follows: 1. **`projected_costs.breakdown`** — Budget allocation per cluster migration phase -2. **`optimization_opportunities`** — Which optimizations to implement and when (some during initial migration, some post-migration) +2. **`migration_cost_considerations`** — Data transfer egress cost estimates (if billing data available) +3. **`optimization_opportunities`** — Which optimizations to implement and when (some during initial migration, some post-migration) 4. **`cost_comparison`** — Set cost monitoring targets and alerts for each migrated cluster -5. **`recommendation.next_steps`** — Prerequisites for starting generation +5. **`recommendation`** — Migrate/stay guidance (`path`, `path_label`, `migrate_if`, `stay_if`, `next_steps`); consumed by HTML report Section 0 +6. **Cost tier vs Terraform** — Generated **`terraform/`** implements **one** baseline aligned with the **Balanced** scenario; **Premium** and **Optimized** are **estimate-only** bands unless the user changes IaC. See `generate-artifacts-infra.md` (`terraform/README.md`, `migration_summary` output). The generated artifacts reference the cost estimates to set per-cluster cost monitoring thresholds and validate that actual AWS spend aligns with projections after each cluster migration. diff --git a/migration-to-aws/steering/estimate.md b/migration-to-aws/steering/estimate.md index 48753bc6..958b1f1f 100644 --- a/migration-to-aws/steering/estimate.md +++ b/migration-to-aws/steering/estimate.md @@ -22,12 +22,34 @@ Attempt to reach awspricing with **up to 2 retries** (3 total attempts): 3. **If still fails**: Wait 2 seconds, retry (Attempt 3) 4. **If all 3 attempts fail**: Use cached prices with staleness warning +### Step 0c: MCP Preflight — Surface Status to User (ALWAYS run) + +**Before any sub-estimate file runs**, display the pricing mode to the user so they know what to expect: + +- **If cache ≤ 90 days and MCP not needed**: "Pricing source: cached (updated [date], ±5-25% accuracy). Live pricing API not required." +- **If cache > 90 days and MCP available**: "Pricing source: live API (awspricing MCP). Cache is stale ([date]) — using real-time pricing." +- **If cache > 90 days and MCP unavailable**: "⚠️ Pricing source: stale cache only (updated [date]). The awspricing MCP server is unreachable — ensure `uvx` is installed (`pip install uv` or `brew install uv`) and AWS credentials are configured. Proceeding with cached pricing; accuracy may be ±15-25% for AI models." +- **If cache ≤ 90 days but a required service is NOT in cache and MCP unavailable**: "⚠️ Some services not in pricing cache and MCP unreachable. Those services will show `pricing_source: unavailable` in the estimate." + +This prevents silent failures — the user sees the pricing constraint upfront, not after 5 minutes of estimation work. + ### Pricing Hierarchy Each sub-estimate file uses this lookup order per service: 1. **`steering/cached-prices.md`** (primary) — Cached prices (±5-25% accuracy). Set `pricing_source: "cached"`. Used first because it requires zero API calls and covers most common services. -2. **MCP API** (fallback) — Real-time pricing for services NOT in cached-prices.md (±5-10% accuracy, more precise). Set `pricing_source: "live"`. Only called when the cache lacks the needed service or model. +2. **MCP API** (secondary) — Real-time pricing for services NOT in cached-prices.md (±5-10% accuracy, more precise). Set `pricing_source: "live"`. Only called when the cache lacks the needed service or model. **Region note:** The `.mcp.json` sets `AWS_REGION=us-east-1` as the MCP server default, but each `get_pricing()` call accepts a `region` parameter that overrides it. Always pass the user's target region (from `preferences.json`) in MCP queries. +3. **Cache after MCP failure** — If MCP was attempted but failed (timeout, error), and the service IS in the cache, use the cached price. Set `pricing_source: "cached_fallback"`. This distinguishes intentional cache use from MCP failure recovery. +4. **Unavailable** — If a service is NOT in the cache AND MCP is unavailable, set `pricing_source: "unavailable"` for that service. Add the service to `services_with_missing_fallback` and display a warning to the user: "Pricing unavailable for [service] — not in cache and MCP unreachable. Exclude from totals or provide a manual estimate." + +**`pricing_source` values summary:** + +| Value | Meaning | +| ------------------- | --------------------------------------------------------- | +| `"cached"` | Found in cached-prices.md (normal path) | +| `"live"` | Retrieved from MCP API in real-time | +| `"cached_fallback"` | MCP was attempted but failed; fell back to cache | +| `"unavailable"` | Not in cache AND MCP failed; service excluded from totals | If cache is > 90 days old and MCP is unavailable: @@ -36,7 +58,8 @@ If cache is > 90 days old and MCP is unavailable: ## Step 1: Prerequisites -Read `$MIGRATION_DIR/preferences.json`. If missing: **STOP**. Output: "Phase 2 (Clarify) not completed. Run Phase 2 first." +1. Read `$MIGRATION_DIR/.phase-status.json`. If missing, invalid, or `phases.clarify` is not exactly `"completed"`: **STOP**. Output: "Phase 2 (Clarify) not completed or phase state is missing/invalid. Complete Clarify before Estimate." +2. Read `$MIGRATION_DIR/preferences.json`. If missing: **STOP**. Output: "Phase 2 (Clarify) not completed. Run Phase 2 first." Check which design artifacts exist in `$MIGRATION_DIR/`: @@ -79,7 +102,39 @@ Produces: `estimation-ai.json` ## Phase Completion -After all applicable sub-estimates finish, use the Phase Status Update Protocol (Write tool) to write `.phase-status.json` with `phases.estimate` set to `"completed"` — **in the same turn** as the output message below. +Before marking Estimate complete, enforce route output gates (fail closed): + +1. Determine which estimate routes ran: + - Infra route: `aws-design.json` exists + - Billing-only route: `aws-design-billing.json` exists AND `aws-design.json` does NOT exist + - AI route: `aws-design-ai.json` exists +2. Require at least one route to be active. If none active: STOP. +3. For each active route, require its expected artifact: + - Infra route -> `estimation-infra.json` + - Billing-only route -> `estimation-billing.json` + - AI route -> `estimation-ai.json` +4. If any active route is missing its expected output: STOP and output: "Estimate route [name] did not produce required artifact(s). Re-run the failed sub-estimate before completing Phase 4." + +## Completion Handoff Gate (Fail Closed) + +Load `steering/handoff-gates.md`. **Re-read from disk** each active estimate artifact before checking. + +**Re-entry guard:** If `generation-infra.json` (or sibling generation artifacts) exists and `phases.generate` is not `"pending"`: STOP unless the user explicitly confirms re-running Estimate. Emit `GATE_FAIL | phase=estimate | field=generation-infra.json | reason=stale_downstream`. + +**Infra route additional checks** (when `estimation-infra.json` exists): + +- `recommendation.path` ∈ `{migrate_optimized, migrate_phased, stay}` +- `recommendation.path_label` is non-empty +- `recommendation.migrate_if` and `recommendation.stay_if` are non-empty arrays + +**On any FAIL:** Emit `GATE_FAIL | phase=estimate | field= | reason=missing`. **Do NOT modify artifacts to pass the gate.** **Do NOT update `.phase-status.json`.** Tell the user to re-run `estimate-infra.md` Part 7 (recommendation block). + +**On PASS:** Emit `HANDOFF_OK | phase=estimate | artifacts=`. + +After `HANDOFF_OK`, use the Phase Status Update Protocol (read-merge-write) to update `.phase-status.json` — **in the same turn** as the output message below: + +- Set `phases.estimate` to `"completed"` +- Set `current_phase` to `"generate"` Output to user: "Cost estimation complete. Proceeding to Phase 5: Generate Migration Artifacts." diff --git a/migration-to-aws/steering/feedback.md b/migration-to-aws/steering/feedback.md index cb5f31b7..8e20d537 100644 --- a/migration-to-aws/steering/feedback.md +++ b/migration-to-aws/steering/feedback.md @@ -8,6 +8,32 @@ Builds an anonymized usage trace and directs the user to the Pulse survey form. Read `$MIGRATION_DIR/.phase-status.json`. Verify `phases.discover == "completed"`. If not: **STOP**. Output: "Feedback requires at least the Discover phase to be completed." +## Step 0: Detect IDE Type and Plugin Version + +Detect the IDE type and plugin version for the survey URL. These are passed as hidden fields — the user never sees or enters them. + +### IDE Detection + +Determine which IDE is running: + +- **Claude Code**: Check if the environment indicates Claude Code (e.g., the `CLAUDE_CODE` environment variable is set, or the skill was invoked via `/skill` or Claude Code CLI). Set `ide` to `claude-code`. +- **Cursor**: Check if the environment indicates Cursor (e.g., the `CURSOR_TRACE_ID` environment variable is set, or the editor context is Cursor). Set `ide` to `cursor`. +- **Fallback**: If detection fails, set `ide` to `unknown`. + +### Plugin Version Detection + +Read the plugin version from the plugin manifest: + +- **Claude Code**: Read `.claude-plugin/plugin.json` → `version` field (relative to the plugin install root). +- **Cursor**: Read `.cursor-plugin/plugin.json` → `version` field (relative to the plugin install root). +- **Fallback**: If the manifest cannot be read, set `version` to `0.0.0`. + +### Sanitization + +Values must use only Pulse-safe characters: letters, numbers, dots (`.`), tildes (`~`), hyphens (`-`), and underscores (`_`). Strip or replace any other characters. + +Store the detected values as `$IDE_TYPE` and `$PLUGIN_VERSION` for use in Step 2. + ## Step 1: Build Trace Load `steering/feedback-trace.md` and execute it. This produces `$MIGRATION_DIR/trace.json`. @@ -39,16 +65,18 @@ Then output the single-line minified version for copy-paste: --- End --- ``` -Then provide the survey link: +Then provide the survey link with IDE and version as hidden field query parameters: ``` Open the feedback form in your browser: -https://pulse.amazon/survey/JWX45QZH +https://pulse.amazon/survey/MY0ZY7UA?ide=$IDE_TYPE&version=$PLUGIN_VERSION Answer the 5 quick questions in the form, then paste the trace line above into the "Migration trace (optional)" field and submit. ``` +Replace `$IDE_TYPE` and `$PLUGIN_VERSION` with the actual values detected in Step 0. Example: `https://pulse.amazon/survey/MY0ZY7UA?ide=claude-code&version=1.0.0` + ## Step 3: Write feedback.json Write `$MIGRATION_DIR/feedback.json`: @@ -56,7 +84,7 @@ Write `$MIGRATION_DIR/feedback.json`: ```json { "timestamp": "", - "survey_url": "https://pulse.amazon/survey/JWX45QZH", + "survey_url": "https://pulse.amazon/survey/MY0ZY7UA?ide=$IDE_TYPE&version=$PLUGIN_VERSION", "phases_completed_at_feedback": [""], "trace_included": true } @@ -66,7 +94,14 @@ If trace building failed: set `"trace_included": false`. ## Step 4: Update Phase Status -Use the Phase Status Update Protocol (Write tool) to write `.phase-status.json` with `phases.feedback` set to `"completed"` — **in the same turn** as the output message below. +Before status update, enforce output gate: + +- `feedback.json` must exist. +- If `trace_included` is true, `trace.json` must exist. + +If output gate fails: STOP and output: "Feedback outputs are incomplete. Fix feedback artifacts before completion." + +Use the Phase Status Update Protocol (read-merge-write) to update `.phase-status.json` with `phases.feedback` set to `"completed"` — **in the same turn** as the output message below. Output to user: "Thank you for helping improve this tool." diff --git a/migration-to-aws/steering/generate-ai.md b/migration-to-aws/steering/generate-ai.md index c2777c36..13d17f46 100644 --- a/migration-to-aws/steering/generate-ai.md +++ b/migration-to-aws/steering/generate-ai.md @@ -28,7 +28,14 @@ Check `preferences.json` → `ai_constraints.ai_framework` to determine timeline | Voice Platform (Vapi, Bland.ai) | Check native Bedrock support, update dashboard | Dashboard config | | Framework (LangChain, LlamaIndex) | Swap provider import (e.g., `ChatBedrock` for `ChatVertexAI`) | 1-5 lines of code | -**Direct SDK users (1-3 weeks)** — `ai_framework` = `direct`: +**OpenAI SDK users via Mantle (hours to 1 day)** — `ai_framework` = `direct` AND `ai_source` = `openai` AND `migration_path` = `mantle`: + +- Set `OPENAI_BASE_URL` and `OPENAI_API_KEY` environment variables +- Update model string to Bedrock model ID +- Test in staging, validate responses +- No SDK changes, no new dependencies, no provider adapter needed + +**Direct SDK users (1-3 weeks)** — `ai_framework` = `direct` AND (`ai_source` != `openai` OR `migration_path` = `converse`): - **Week 1:** Enable Bedrock access, create IAM role, develop provider adapter with feature flag, unit test - **Week 2:** Deploy to staging, run A/B comparison, measure latency/quality/cost, tune prompts @@ -44,16 +51,17 @@ Based on `ai-workload-profile.json` → `integration.pattern` and `integration.l **Migration patterns to include (matched to detected language and source):** -| Source SDK | Target | Key Change | -| ------------------ | --------------------------------- | ---------------------------------------------------------------- | -| Vertex AI (Python) | boto3 Bedrock Converse API | `GenerativeModel.generate_content()` → `bedrock.converse()` | -| Vertex AI (JS) | @aws-sdk/client-bedrock-runtime | `model.generateContent()` → `client.send(new ConverseCommand())` | -| Vertex AI (Go) | aws-sdk-go-v2 bedrockruntime | `aiplatform` → `bedrockruntime.Converse()` | -| Vertex AI (Java) | AWS SDK BedrockRuntimeClient | `GenerativeModel` → `BedrockRuntimeClient.converse()` | -| OpenAI SDK | boto3 Bedrock Converse API | `client.chat.completions.create()` → `bedrock.converse()` | -| LiteLLM | LiteLLM config change | `model="gpt-4o"` → `model="bedrock/anthropic.claude-sonnet-4-6"` | -| LangChain | langchain_aws | `ChatOpenAI`/`ChatVertexAI` → `ChatBedrock` | -| LlamaIndex | llama_index.llms.bedrock_converse | `Vertex` → `BedrockConverse` | +| Source SDK | Target | Key Change | +| ------------------ | --------------------------------- | --------------------------------------------------------------------------------- | +| OpenAI SDK | Mantle OpenAI-compat | Set `OPENAI_BASE_URL` + `OPENAI_API_KEY` + model string (zero code changes) | +| Vertex AI (Python) | boto3 Bedrock Converse API | `GenerativeModel.generate_content()` → `bedrock.converse()` | +| Vertex AI (JS) | @aws-sdk/client-bedrock-runtime | `model.generateContent()` → `client.send(new ConverseCommand())` | +| Vertex AI (Go) | aws-sdk-go-v2 bedrockruntime | `aiplatform` → `bedrockruntime.Converse()` | +| Vertex AI (Java) | AWS SDK BedrockRuntimeClient | `GenerativeModel` → `BedrockRuntimeClient.converse()` | +| OpenAI SDK | boto3 Bedrock Converse API | `client.chat.completions.create()` → `bedrock.converse()` (if Mantle unavailable) | +| LiteLLM | LiteLLM config change | `model="gpt-4o"` → `model="bedrock/anthropic.claude-sonnet-4-6"` | +| LangChain | langchain_aws | `ChatOpenAI`/`ChatVertexAI` → `ChatBedrock` | +| LlamaIndex | llama_index.llms.bedrock_converse | `Vertex` → `BedrockConverse` | For each detected language and pattern, generate before/after code examples using actual model IDs from `aws-design-ai.json`. @@ -155,3 +163,68 @@ The parent orchestrator (`generate.md`) uses `generation-ai.json` to: 1. Gate Stage 2 artifact generation — `generate-artifacts-ai.md` requires this file 2. Provide AI migration context to `generate-artifacts-docs.md` for MIGRATION_GUIDE.md 3. Set phase completion status in `.phase-status.json` + +## Part 7: Generate STARTUP_PROGRAMS.md + +Always generate `$MIGRATION_DIR/STARTUP_PROGRAMS.md` when `preferences.json` contains `ai_monthly_spend` or `startup_program_status`. This artifact summarizes applicable AWS startup programs based on the user's detected spend and workload type. + +**Content rules (all amounts from AWS official sources only):** + +```markdown +# AWS Startup Programs for Your Migration + +Based on your migration profile, here are the AWS programs most relevant to you. +Credits apply to Bedrock usage (Claude, Llama, Nova, and other third-party models). + +## AWS Activate Credits + +AWS Activate provides promotional credits to offset AWS costs including Amazon Bedrock. +Apply at: https://aws.amazon.com/startups/credits/ + +### Which tier applies to you + +| Your situation | Package | Credits | How to apply | +| --------------------------------------- | --------------------------- | -------------- | ---------------------------------------------------------------------- | +| Self-funded, no VC/accelerator | Activate Founders | Up to $5,000 | Apply directly at aws.amazon.com/startups/credits — no Org ID needed | +| VC or accelerator-backed (pre-Series B) | Activate Portfolio | Up to $200,000 | Get your Activate Provider Org ID from your VC/accelerator, then apply | +| Ready to scale post-Activate-Portfolio | AWS Credits for AI Startups | $200,000+ | Invite-only — talk with your AWS Account Manager | + +**Activate eligibility (Founders & Portfolio):** Pre-Series B, founded in the last 10 years, AWS Account on Paid Tier Plan, and either new to Activate Credits or requesting more credits than previously received. +Credits expire within 1–2 years. Apply when you're ready to ramp up AWS usage. + +[Conditional — only if ai_monthly_spend is "$2K-$10K" or ">$10K" AND agentic_profile.is_agentic == true:] + +## AWS Generative AI Accelerator + +An adjacent cohort program for agentic AI startups, distinct from the AWS Activate credits-hub funnel above. Highly selective. The 2025 global cohort selected 40 startups; consult the program page for the next cohort cycle. + +- **Credits:** Up to $1,000,000 in AWS credits +- **Program:** 8-week cohort with mentorship, technical support, and go-to-market resources +- **Best for:** Startups with production agentic workloads and significant AI spend +- **Apply:** https://aws.amazon.com/startups/generative-ai/accelerator/ + +[End conditional] + +## How to use credits during this migration + +1. Apply for AWS Activate **before** running `terraform apply` — credits apply automatically to new charges +2. Credits cover eligible AWS services including Fargate, Aurora, S3, CloudWatch, and Bedrock models (both Amazon first-party and third-party foundation models) +3. **Credits do NOT cover upfront Savings Plans or Reserved Instance fees.** If the Estimate phase recommended Savings Plans for cost optimization, those commitments must be paid separately — credits apply to on-demand usage only. +4. Monitor your balance: AWS Console → Billing → Credits +5. Credits do not apply retroactively — apply before incurring costs + +## Next steps + +- [ ] Apply for AWS Activate at https://aws.amazon.com/startups/credits/ +- [ ] If VC/accelerator-backed: get your Activate Provider Org ID from your investor +- [ ] Apply credits before running terraform apply + [- [ ] Apply for AWS Generative AI Accelerator: https://aws.amazon.com/startups/generative-ai/accelerator/ (if agentic and high spend)] +``` + +**Generation rules:** + +- Always include the Activate section +- Only include the Generative AI Accelerator section when `ai_monthly_spend` is `"$2K-$10K"` or `">$10K"` AND `agentic_profile.is_agentic == true` +- If `startup_program_status == "has_credits"`: replace the "Apply for AWS Activate" steps with "You already have AWS Activate credits — ensure they are applied to your account before running terraform apply" +- Do NOT include any credit amounts not sourced from official AWS pages (aws.amazon.com) +- Do NOT reference MAP, IW Migrate, or ISV Workload Migration Program — these are enterprise/partner programs, not startup self-service paths diff --git a/migration-to-aws/steering/generate-artifacts-ai.md b/migration-to-aws/steering/generate-artifacts-ai.md index 3a5b56cf..84ab5d6b 100644 --- a/migration-to-aws/steering/generate-artifacts-ai.md +++ b/migration-to-aws/steering/generate-artifacts-ai.md @@ -12,6 +12,7 @@ Generate migration artifacts from the AI migration plan and design. Artifacts va - `ai-migration/setup_bedrock.sh` — Bedrock model access and IAM setup - `ai-migration/test_comparison.py` — A/B test harness (always Python) +- `ai-migration/bedrock_monitoring.tf` — Bedrock cost budget, anomaly detection, inference profiles (always) **Outputs (direct SDK users — `ai_framework` = `"direct"`):** @@ -36,19 +37,57 @@ Read from `$MIGRATION_DIR/`: If any required file is missing: **STOP**. Output: "Missing required artifact: [filename]. Complete the prior phase that produces it." +**Sparse / IaC-only profile:** If `ai-workload-profile.json` has empty `models[]` or `metadata.profile_source` is `iac_vertex`, use `aws-design-ai.json` for Bedrock targets and example prompts; do **not** fail solely because discovery did not list model IDs. + --- ## Step 0: Determine Artifact Path -Check `preferences.json` → `ai_constraints.ai_framework.value`: +Check `preferences.json` → `ai_constraints.ai_framework.value` and `aws-design-ai.json` → `ai_architecture.code_migration.migration_path`: +- `migration_path` = `"mantle"` → Generate Mantle setup (Step 1M) + setup (Step 3) + test harness (Step 2). Skip provider adapter (Step 1). +- `migration_path` = `"gpt-oss"` → Generate provider adapter (Step 1) targeting the gpt-oss model on Bedrock via the Converse API + setup (Step 3) + test harness (Step 2). Use the Bedrock model ID from `aws-design-ai.json` → `ai_architecture.bedrock_models[].aws_model_id` (the gpt-oss model ID). Do NOT generate a Mantle script — gpt-oss uses the Converse API directly. - `"direct"` or absent → Generate provider adapter (Step 1) + setup (Step 3) + test harness (Step 2) - `"llm_router"`, `"api_gateway"`, `"voice_platform"`, or `"framework"` → Skip Step 1, generate gateway config (Step 3B) instead +**Step 3F runs for ALL paths** (direct, mantle, gpt-oss, gateway, harness, strands) — generate `bedrock_monitoring.tf` regardless of migration path. + **Determine language** (direct SDK users only): Read `ai-workload-profile.json` → `integration.languages` array. Use the first entry: `"python"` → `.py`, `"javascript"`/`"typescript"` → `.js`, `"go"` → `.go`, other/unknown → `.py`. --- +## Step 1M: Generate Mantle Migration Script (OpenAI SDK via Mantle) + +Generate `ai-migration/migrate_to_mantle.sh` — a shell script that configures the OpenAI SDK to use Bedrock's Mantle endpoints. + +**Requirements:** + +- Dry-run by default (`--execute` flag to apply) +- Print the environment variables to set: `OPENAI_BASE_URL=https://bedrock-mantle.{region}.api.aws/v1`, `OPENAI_API_KEY=` +- Print the model string change: current model ID → Bedrock model ID from `aws-design-ai.json` +- Include a quick verification call using the OpenAI SDK against the Mantle endpoint +- **Add a warning comment** about `max_tokens`: OpenAI SDK users migrating via Mantle carry over their existing `max_tokens` value unchanged. If the existing value is 4096 (OpenAI default), this reduces Bedrock concurrency by 5–8x on TPM quota. Instruct users to audit `max_tokens` before production and include the workload-type lookup table (see Step 1M max_tokens guidance above). +- Note: "No code changes required. Your existing OpenAI SDK calls work unchanged." +- Reference [Mantle documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) for API key generation +- **Set `max_tokens` default to 1024** in the generated migration script with an explanatory comment. Do NOT inherit the caller's existing `max_tokens` value (often 4096 from OpenAI defaults) — this reduces concurrency by 5–8x on Bedrock's TPM quota. Include a workload-type lookup table as a comment so users tune before production: + + ```bash + # max_tokens guidance — tune before production: + # | Workload type | Suggested max_tokens | + # |--------------------------------------------|----------------------| + # | Classification / extraction / routing | 256–512 | + # | Chat / Q&A / summarization | 512–1024 (default) | + # | Long-form generation / reports | 2048–4096 | + # | Tool outputs / multi-step reasoning | 2048–8192 | + # | Code generation | 2048–4096 | + # TODO: Review and adjust max_tokens for your workload before deploying to production. + MAX_TOKENS=1024 # Starting point — see table above + ``` + +Skip Step 1 (provider adapter) when this step runs — the OpenAI SDK is the adapter. + +--- + ## Step 1: Generate Provider Adapter (Direct SDK Only) Generate `ai-migration/provider_adapter.{py,js,go}` — an abstraction layer that lets the user switch between the source AI provider and Bedrock via an environment variable. @@ -61,7 +100,17 @@ Generate `ai-migration/provider_adapter.{py,js,go}` — an abstraction layer tha - `streaming: true` → `generate_stream(prompt) → Iterator[str]` - `embeddings: true` → `embed(text) → list[float]` - **Source provider class**: Use SDK imports from `ai-workload-profile.json` → `integration.sdk_imports`. Use model IDs from `ai-workload-profile.json` → `models[].model_id`. -- **Bedrock provider class**: Use `boto3` Converse API (`converse` for generate, `converse_stream` for streaming, `invoke_model` for embeddings with Titan). Use model IDs from `aws-design-ai.json` → `ai_architecture.bedrock_models[].aws_model_id`. Use region from `preferences.json` → `design_constraints.target_region`. +- **Bedrock provider class**: Use `boto3` Converse API (`converse` for generate, `converse_stream` for streaming, `invoke_model` for embeddings with Titan). Use inference profile ARNs from `bedrock_monitoring.tf` output `bedrock_inference_profile_arns` as `modelId` — this enables cost attribution in Cost Explorer. Fall back to raw model IDs from `aws-design-ai.json` → `ai_architecture.bedrock_models[].aws_model_id` if profiles are not yet deployed. Use region from `preferences.json` → `design_constraints.target_region`. + + ```python + # Use inference profile ARN for cost attribution (preferred) + # Set BEDROCK_INFERENCE_PROFILE_ARN from terraform output bedrock_inference_profile_arns + MODEL_ID = os.environ.get( + "BEDROCK_INFERENCE_PROFILE_ARN", + "{aws_model_id}" # fallback — replace with actual ARN from aws-design-ai.json + ) + ``` + - **Shadow mode**: Send requests to both providers, return source response, log Bedrock response for comparison. - Include error handling and logging for API calls. @@ -96,7 +145,8 @@ Generate `ai-migration/setup_bedrock.sh`. - Step 1 — Request model access: List each model from `aws-design-ai.json` → `bedrock_models[].aws_model_id` and the embedding model - Step 2 — Create IAM role: Trust policy for the compute platform (Lambda, ECS, or EC2 based on `aws-design.json` if present). Bedrock policy: `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` scoped to `arn:aws:bedrock:*::foundation-model/*` - Step 3 — Print required environment variables: `AWS_REGION`, `AI_PROVIDER=bedrock`, model IDs -- Step 4 — Verification: Test Bedrock access with a simple `converse` call using the primary model +- Step 4 — Check quota: Query current TPM quota for the primary model via `aws service-quotas get-service-quota`. If `aws-design-ai.json` → `ai_architecture.quota_risk` is `"high"` or `"medium"`, print warning: "⚠️ Your token volume may exceed default Bedrock quotas. Request a quota increase via Service Quotas console (allow 1–5 business days)." Include the `aws service-quotas request-service-quota-increase` command template. +- Step 5 — Verification: Test Bedrock access with a simple `converse` call using the primary model - If `$MIGRATION_DIR/terraform/` exists, print coordination note: "Ensure the IAM role is referenced in compute.tf task definitions" - Use region from `preferences.json` → `design_constraints.target_region` @@ -112,6 +162,26 @@ Skip if `ai_framework` = `"direct"` or absent. Read `preferences.json` → `ai_c - Include embedding model entry if embeddings are used - Note required env vars: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION` +**`"llm_router"` + OpenRouter detected** (evidence: `base_url` containing `openrouter.ai` in `ai-workload-profile.json` → `detection_signals`): + +Check `aws-design-ai.json` → `code_migration.openrouter_path` (set in Design Part 5): + +- `"direct"` → Skip gateway config. Generate provider adapter instead (same as Step 1 for direct SDK users). Note: "Migrating from OpenRouter to direct Bedrock API removes the routing middleman and OpenRouter's margin." +- `"litellm"` → Generate `gateway_config.yaml` in LiteLLM format (standard `llm_router` logic above) + add header comment: "# Migration from OpenRouter to self-hosted LiteLLM with Bedrock backend\n# Install: pip install litellm\n# Run: litellm --config gateway_config.yaml" +- `"keep_openrouter"` → Generate `gateway_config.json` with OpenRouter model ID mappings: + + ```json + { + "models": { + "original_model_id": "amazon/{bedrock_model_id}", + "original_embedding_id": "amazon/{titan_embedding_model_id}" + }, + "notes": "OpenRouter adds margin on top of provider pricing. Compare OpenRouter pricing vs direct Bedrock pricing in your cost estimate." + } + ``` + +- If `openrouter_path` is absent: default to `"litellm"` (standard LiteLLM config generation) + **`"framework"`** → Generate `gateway_config.py`: - Show before/after import swap for the detected framework (`ai-workload-profile.json` → `integration.sdk_imports`) @@ -133,7 +203,167 @@ Skip if `ai_framework` = `"direct"` or absent. Read `preferences.json` → `ai_c --- -## Step 3C: Generate Evaluation Artifacts (If User Opted In) +## Step 3C: Generate AgentCore Harness Artifacts (Harness Users Only) + +**Skip if** `aws-design-ai.json` → `agentic_design` is absent OR `agentic_design.migration_approach != "harness"`. + +Read `aws-design-ai.json` → `agentic_design.harness_config` for all configuration values. + +**Artifact 1: `ai-migration/harness.json`** + +Generate the Harness configuration file: + +```json +{ + "name": "{harness_config.name}", + "model": "{harness_config.model_id}", + "systemPrompt": "{harness_config.system_prompt}", + "tools": [ + // Each tool from harness_config.tools, mapped to Harness tool format: + // remote_mcp: {"type": "remote_mcp", "name": "...", "config": {"remoteMcp": {"url": "..."}}} + // agentcore_browser: {"type": "agentcore_browser", "name": "browser"} + // agentcore_code_interpreter: {"type": "agentcore_code_interpreter", "name": "code_interpreter"} + // agentcore_gateway: {"type": "agentcore_gateway", "name": "...", "config": {"agentCoreGateway": {"gatewayArn": "[TODO: Gateway ARN]"}}} + // inline_function: {"type": "inline_function", "name": "...", "config": {"inlineFunction": {"description": "...", "inputSchema": {...}}}} + ] +} +``` + +Use actual values from `harness_config` — no placeholders except where noted with `[TODO: ...]`. Include a comment header explaining the file's purpose. + +**Artifact 2: `ai-migration/deploy_harness.sh`** + +Generate deployment script. Dry-run by default (`--execute` flag to run for real). + +```bash +#!/bin/bash +# AgentCore Harness Deployment Script +# Generated by GCP-to-AWS migration plugin +# Run with --execute to perform actual deployment (default: dry-run) +set -euo pipefail + +EXECUTE=${1:-""} +HARNESS_NAME="{harness_config.name}" +MODEL_ID="{harness_config.model_id}" +REGION="{target_region from preferences.json}" + +run_cmd() { + if [ "$EXECUTE" = "--execute" ]; then + echo ">>> $*" + eval "$@" + else + echo "[DRY-RUN] $*" + fi +} + +echo "=== AgentCore Harness Deployment: $HARNESS_NAME ===" +echo "Model: $MODEL_ID" +echo "Region: $REGION" +echo "" + +# Step 1: Verify AgentCore CLI is installed +if ! command -v agentcore &> /dev/null; then + echo "AgentCore CLI not found. Install with:" + echo " pip install bedrock-agentcore-cli" + if [ "$EXECUTE" != "--execute" ]; then + echo "[DRY-RUN] Continuing with remaining steps..." + else + exit 1 + fi +fi + +# Step 2: Create project +run_cmd "agentcore create --name $HARNESS_NAME" + +# Step 3: Add harness with model and tools +run_cmd "agentcore add harness --name $HARNESS_NAME \\ + --model-id $MODEL_ID \\ + --system-prompt '{harness_config.system_prompt}' \\ + --tools {comma-separated tool types from harness_config.tools}" + +# Step 4: Deploy +run_cmd "agentcore deploy" + +# Step 5: Test invocation +echo "" +echo "=== Test Invocation ===" +run_cmd "agentcore invoke --harness $HARNESS_NAME \\ + 'Hello, this is a test invocation to verify deployment.'" + +echo "" +echo "=== Deployment Complete ===" +echo "Next steps:" +echo " 1. Test with representative prompts: agentcore invoke --harness $HARNESS_NAME 'your prompt'" +echo " 2. Override model for A/B testing: agentcore invoke --harness $HARNESS_NAME --model-id 'prompt'" +echo " 3. View logs: agentcore logs --harness $HARNESS_NAME" +``` + +**Artifact 3: `ai-migration/incremental_migration.sh`** (only if `harness_config.incremental_migration == true`) + +Generate incremental migration script showing multi-model switching: + +```bash +#!/bin/bash +# Incremental Migration: Source Provider → Bedrock via AgentCore Harness +# This script demonstrates the multi-model switching capability. +# Run each phase manually and validate before proceeding to the next. +set -euo pipefail + +HARNESS_NAME="{harness_config.name}" +SOURCE_PROVIDER="{harness_config.source_model_provider}" +SOURCE_MODEL="{harness_config.source_model_id}" +BEDROCK_MODEL="{harness_config.model_id}" +SESSION_ID=$(uuidgen) + +echo "=== Incremental Migration Plan ===" +echo "Source: $SOURCE_PROVIDER / $SOURCE_MODEL" +echo "Target: Bedrock / $BEDROCK_MODEL" +echo "Session: $SESSION_ID" +echo "" + +# Phase 0: Store source provider API key in AgentCore Identity +echo "--- Phase 0: Configure source provider credentials ---" +echo "Run once:" +echo " agentcore add credential --type api-key --name source-provider-key --api-key \$SOURCE_API_KEY" +echo " agentcore deploy" +echo "" + +# Phase 1: Invoke with source provider model on AgentCore infrastructure +echo "--- Phase 1: Source provider on AgentCore ---" +echo "agentcore invoke --harness $HARNESS_NAME \\" +echo " --model-provider $SOURCE_PROVIDER \\" +echo " --model-id $SOURCE_MODEL \\" +echo " --api-key-arn arn:aws:bedrock-agentcore:{region}:{account}:token-vault/default/apikeycredentialprovider/source-provider-key \\" +echo " --session-id $SESSION_ID \\" +echo " 'Your test prompt here'" +echo "" + +# Phase 2: Same session, switch to Bedrock model +echo "--- Phase 2: Bedrock model (same session, context preserved) ---" +echo "agentcore invoke --harness $HARNESS_NAME \\" +echo " --model-id $BEDROCK_MODEL \\" +echo " --session-id $SESSION_ID \\" +echo " 'Same test prompt for comparison'" +echo "" + +# Phase 3: Update default model +echo "--- Phase 3: Switch default to Bedrock ---" +echo "Edit app/$HARNESS_NAME/harness.json: set \"model\" to \"$BEDROCK_MODEL\"" +echo "agentcore deploy" +echo "" + +# Phase 4: Remove source provider +echo "--- Phase 4: Clean up source provider credentials ---" +echo "After 48h stable on Bedrock:" +echo " Remove API key from AgentCore Identity" +echo " agentcore deploy" +echo "" +echo "=== Migration Complete ===" +``` + +--- + +## Step 3D: Generate Evaluation Artifacts (If User Opted In) Skip if the user did not opt into model evaluation in `generate-ai.md` Part 0. @@ -143,6 +373,366 @@ Skip if the user did not opt into model evaluation in `generate-ai.md` Part 0. --- +## Step 3E: Generate Strands Migration Artifacts (Strands Users Only) + +**Skip if** `aws-design-ai.json` → `agentic_design` is absent OR `agentic_design.migration_approach != "strands"`. + +Read `aws-design-ai.json` → `agentic_design.strands_config` for all configuration values. + +**Artifact 1: `ai-migration/strands_agents.py`** + +Generate Strands agent definitions from the design config. For each agent in `strands_config.agents`: + +```python +""" +Strands Agents Migration — Generated by GCP-to-AWS migration plugin +Source framework: {strands_config.source_framework} +Orchestration: {strands_config.orchestration_primitive} + +Install: pip install strands-agents strands-agents-tools boto3 +Docs: https://strandsagents.com +""" +from strands import Agent, tool +from strands.models import BedrockModel + +# Model configuration +model = BedrockModel( + model_id="{agents[0].model_id}", + region_name="{target_region}" +) + +# --- Tool definitions --- +# Migrate your existing tool functions by adding the @tool decorator. +# Your tool logic stays the same; Strands handles invocation and result parsing. + +@tool +def {tool_name}({params}) -> str: + """[TODO: Your existing tool description]""" + # [TODO: Paste your existing tool implementation here] + pass + +# --- Agent definitions --- +{agent_name} = Agent( + name="{agent_id}", + model=model, + system_prompt="""{system_prompt}""", + tools=[{tool_list}] +) +``` + +**For multi-agent systems**, generate the orchestration based on `orchestration_primitive`: + +- **`agents_as_tools`** (hierarchical): + +```python +# Specialist agents wrapped as tools for the orchestrator +@tool +def {specialist_agent_id}(query: str) -> str: + """[role description]""" + response = {specialist_var}(query) + return str(response) + +# Orchestrator delegates to specialists +orchestrator = Agent( + name="orchestrator", + model=model, + tools=[{specialist_tool_1}, {specialist_tool_2}, ...] +) + +result = orchestrator("Your task here") +``` + +- **`swarm`** (collaborative): + +```python +from strands.multiagent import Swarm + +team = Swarm([{agent_1}, {agent_2}, {agent_3}]) +result = team("Your task here") +``` + +- **`graph`** (deterministic workflow): + +```python +from strands.multiagent import GraphBuilder + +builder = GraphBuilder() +builder.add_node({agent_1}, "{node_1_name}") +builder.add_node({agent_2}, "{node_2_name}") +builder.add_edge("{node_1_name}", "{node_2_name}") # Add conditions if detected +builder.set_entry_point("{entry_node}") +workflow = builder.build() + +result = workflow("Your task here") +``` + +- **`single_agent`**: Just the single Agent definition (no multi-agent orchestration). + +Use actual agent IDs, model IDs, tool names, and system prompts from the design config. Mark TODOs only where implementation details can't be extracted from discovery. + +**Artifact 2: `ai-migration/deploy_strands.sh`** + +Generate AgentCore Runtime deployment script. Dry-run by default. + +```bash +#!/bin/bash +# Strands Agents — AgentCore Runtime Deployment +# Generated by GCP-to-AWS migration plugin +# Run with --execute to perform actual deployment (default: dry-run) +set -euo pipefail + +EXECUTE=${1:-""} +PROJECT_NAME="{agent_id}-strands" +REGION="{target_region}" + +run_cmd() { + if [ "$EXECUTE" = "--execute" ]; then + echo ">>> $*" + eval "$@" + else + echo "[DRY-RUN] $*" + fi +} + +echo "=== Strands Agents — AgentCore Runtime Deployment ===" +echo "Project: $PROJECT_NAME" +echo "Region: $REGION" +echo "Agents: {agent_count}" +echo "" + +# Step 1: Verify dependencies +echo "--- Prerequisites ---" +run_cmd "pip install strands-agents strands-agents-tools boto3 bedrock-agentcore-cli" + +# Step 2: Create AgentCore project +run_cmd "agentcore create --name $PROJECT_NAME" + +# Step 3: Deploy to AgentCore Runtime +echo "" +echo "--- Deploying to AgentCore Runtime ---" +run_cmd "agentcore deploy" + +# Step 4: Test invocation +echo "" +echo "--- Test Invocation ---" +run_cmd "agentcore invoke --name $PROJECT_NAME 'Hello, test invocation'" + +echo "" +echo "=== Deployment Complete ===" +echo "Next steps:" +echo " 1. Run locally first: python strands_agents.py" +echo " 2. Test with representative prompts" +echo " 3. Deploy: ./deploy_strands.sh --execute" +echo " 4. Monitor: agentcore logs --name $PROJECT_NAME" +echo "" +echo "Strands docs: https://strandsagents.com" +echo "AgentCore docs: https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/" +``` + +**Artifact 3: `ai-migration/bridge_retarget.py`** (only if `strands_config.bridge_phase == true`) + +Generate a partial retarget bridge for OpenAI Agents SDK users — routes model calls through Bedrock while keeping existing orchestration temporarily: + +```python +""" +Bridge: OpenAI Agents SDK → Bedrock (via OpenAI-compatible endpoint) +This is a temporary bridge. Use while migrating orchestration to Strands. + +This works because Bedrock's Converse API can be accessed through an +OpenAI-compatible proxy, allowing your existing OpenAI Agents SDK code +to call Bedrock models without code changes. + +After validating Bedrock model quality, migrate to Strands for full +AWS-native agent capabilities. +""" +import os + +# Option A: Use LiteLLM as OpenAI-compatible proxy to Bedrock +# pip install litellm +# Set: OPENAI_API_BASE=http://localhost:4000 (LiteLLM proxy) +# LiteLLM config: model_list: [{model_name: "gpt-4o", litellm_params: {model: "bedrock/{bedrock_model_id}"}}] + +# Option B: Use Bedrock's OpenAI-compatible endpoint (if available) +# Set: OPENAI_API_BASE=https://bedrock-runtime.{region}.amazonaws.com/openai/v1 +# Set: OPENAI_API_KEY= + +# Your existing OpenAI Agents SDK code continues to work unchanged. +# The model calls route through Bedrock instead of OpenAI. + +BRIDGE_CONFIG = { + "source_model": "{source_model_id}", + "target_model": "bedrock/{bedrock_model_id}", + "proxy": "litellm", # or "bedrock_openai_compat" + "region": "{target_region}", + "note": "Temporary bridge. Migrate to Strands (strands_agents.py) when ready." +} + +print(f"Bridge configured: {BRIDGE_CONFIG['source_model']} → {BRIDGE_CONFIG['target_model']}") +print(f"Next step: Validate model quality, then migrate to Strands.") +``` + +--- + +--- + +## Step 3F: Generate Bedrock Cost Monitoring Terraform + +**Runs for ALL migration paths** (direct, mantle, gpt-oss, gateway, harness, strands). +Creates `ai-migration/bedrock_monitoring.tf` — standalone Terraform for AI-only migrations +that have no `baseline.tf` from the infra track. + +> **These are detective controls, not spend caps.** Bedrock has no native spend limit. +> Anomaly Detection has ~24h data lag — you'll know within a day, not at month-end invoice. +> AI-only migrations (infra stays on GCP) have zero cost guardrails without this file. + +**Seed budget threshold:** Read `estimation-ai.json` → `cost_comparison.projected_bedrock_monthly`. +Round up to nearest integer. If absent, use `50` as the default. + +Emit the final computed integer directly in `limit_amount` — do not emit an expression. + +```hcl +# ============================================================ +# Bedrock Cost Monitoring — Generated by GCP to AWS Migration Advisor +# +# DETECTIVE CONTROLS, NOT SPEND CAPS. Bedrock has no native +# spend limit. Anomaly Detection has ~24h data lag. +# ============================================================ + +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 5.66" # aws_bedrock_inference_profile requires >= 5.66 + } + } +} + +provider "aws" { + region = "{target_region}" # from preferences.json design_constraints.target_region +} + +data "aws_caller_identity" "current" {} +data "aws_region" "current" {} + +variable "project_name" { + description = "Project name for resource naming and tagging" + type = string +} + +variable "alert_email" { + description = "Email address for Bedrock cost alert notifications" + type = string +} + +variable "environment" { + description = "Deployment environment (dev, staging, prod)" + type = string + default = "prod" +} + +variable "app_name" { + description = "Application name for cost attribution tags" + type = string + default = "migration" +} + +# Bedrock-scoped monthly budget +# Uses 1.5x multiplier (vs 1.2x in baseline.tf account-wide budget) because +# Bedrock token costs are more volatile than infrastructure. +# For hybrid runs: coexists with baseline.tf account-wide budget — different scopes, no conflict. +resource "aws_budgets_budget" "bedrock_spend" { + name = "${var.project_name}-bedrock-monthly" + budget_type = "COST" + limit_amount = "{projected_bedrock_monthly_times_1.5_as_integer}" # AGENT: replace with ceil(projected * 1.5), min 10 + limit_unit = "USD" + time_unit = "MONTHLY" + + cost_filter { + name = "Service" + values = ["Amazon Bedrock"] + } + + notification { + comparison_operator = "GREATER_THAN" + threshold = 80 + threshold_type = "PERCENTAGE" + notification_type = "ACTUAL" + subscriber_email_addresses = [var.alert_email] + } + + notification { + comparison_operator = "GREATER_THAN" + threshold = 100 + threshold_type = "PERCENTAGE" + notification_type = "FORECASTED" + subscriber_email_addresses = [var.alert_email] + } +} + +# Bedrock anomaly detection — DIMENSIONAL/SERVICE monitors all services independently. +# A Bedrock spend spike will surface as an anomaly for the SERVICE=AmazonBedrock dimension. +# Note: DIMENSIONAL monitors all services; the budget above is Bedrock-scoped via cost_filter. +resource "aws_ce_anomaly_monitor" "bedrock" { + name = "${var.project_name}-bedrock-anomaly" + monitor_type = "DIMENSIONAL" + monitor_dimension = "SERVICE" +} + +resource "aws_ce_anomaly_subscription" "bedrock_alert" { + name = "${var.project_name}-bedrock-anomaly-alert" + frequency = "DAILY" + + monitor_arn_list = [aws_ce_anomaly_monitor.bedrock.arn] + + subscriber { + address = var.alert_email + type = "EMAIL" + } + + threshold_expression { + dimension { + key = "ANOMALY_TOTAL_IMPACT_PERCENTAGE" + values = ["50"] + match_options = ["GREATER_THAN_OR_EQUAL"] + } + } +} + +# Application inference profiles — cost attribution by environment/app +# Tags on these profiles appear in Cost Explorer; tags on API calls do not. +# copy_from must be a full foundation-model ARN or cross-region inference profile ARN. +# For Claude models, use the cross-region inference profile ARN format: +# arn:aws:bedrock:{region}:{account_id}:inference-profile/us.anthropic.claude-sonnet-4-6 +# For Amazon Nova models, use: +# arn:aws:bedrock:{region}:{account_id}:inference-profile/us.amazon.nova-pro-v1:0 +# [AGENT: generate one block per model in aws-design-ai.json → ai_architecture.bedrock_models[]] +resource "aws_bedrock_inference_profile" "primary" { + name = "${var.project_name}-${var.environment}-primary" + + model_source { + # Use cross-region inference profile ARN, not bare model ID + copy_from = "arn:aws:bedrock:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:inference-profile/{cross_region_inference_profile_id}" + # AGENT: replace {cross_region_inference_profile_id} with the correct ID from + # aws-design-ai.json bedrock_models[].aws_model_id, prefixed with "us." for US regions + # e.g., us.anthropic.claude-sonnet-4-6 or us.amazon.nova-pro-v1:0 + } + + tags = { + Project = var.project_name + Environment = var.environment + Application = var.app_name + ManagedBy = "terraform" + } +} + +output "bedrock_inference_profile_arns" { + value = { + primary = aws_bedrock_inference_profile.primary.arn + } + description = "Use these ARNs as BEDROCK_INFERENCE_PROFILE_ARN env var in your adapter for cost attribution" +} +``` + ## Step 4: Self-Check Verify all generated artifacts: @@ -157,6 +747,21 @@ Verify all generated artifacts: - [ ] All scripts default to dry-run mode - [ ] Evaluation artifacts (if generated) have correct model IDs and region - [ ] No hardcoded credentials in any file +- [ ] bedrock_monitoring.tf generated with budget limit_amount as computed integer (not expression) +- [ ] Budget limit_amount = ceil(projected_bedrock_monthly * 1.5), minimum 10 +- [ ] Anomaly monitor uses monitor_type = "DIMENSIONAL" with monitor_dimension = "SERVICE" (no monitor_specification) +- [ ] Inference profile copy_from uses full ARN format, not bare model ID +- [ ] Provider adapter references BEDROCK_INFERENCE_PROFILE_ARN env var with model ID fallback +- [ ] alert_email variable declared in bedrock_monitoring.tf +- [ ] If Harness artifacts generated: `harness.json` uses actual model ID from `agentic_design.harness_config.model_id` +- [ ] If Harness artifacts generated: `harness.json` tool types match `tool_manifest[].transport` mapping +- [ ] If Harness artifacts generated: `deploy_harness.sh` defaults to dry-run +- [ ] If Harness artifacts generated: `incremental_migration.sh` only present when `incremental_migration == true` +- [ ] If OpenRouter path: artifact type matches `code_migration.openrouter_path` value +- [ ] If Strands artifacts generated: `strands_agents.py` uses actual model IDs, agent IDs, and tool names from design config +- [ ] If Strands artifacts generated: orchestration primitive matches `strands_config.orchestration_primitive` +- [ ] If Strands artifacts generated: `bridge_retarget.py` only present when `strands_config.bridge_phase == true` +- [ ] If Strands artifacts generated: `deploy_strands.sh` defaults to dry-run ## Phase Completion @@ -170,6 +775,13 @@ Generated AI migration artifacts: - ai-migration/test_comparison.py - ai-migration/provider_adapter.{py|js|go} # Direct SDK users only - ai-migration/gateway_config.{yaml|py|json} # Gateway users only +- ai-migration/harness.json # Harness users only +- ai-migration/deploy_harness.sh # Harness users only +- ai-migration/incremental_migration.sh # Harness + incremental only +- ai-migration/strands_agents.py # Strands users only +- ai-migration/deploy_strands.sh # Strands users only +- ai-migration/bridge_retarget.py # Strands + OpenAI Agents SDK bridge only +- ai-migration/bedrock_monitoring.tf # Always — Bedrock budget, anomaly detection, inference profiles - ai-migration/eval-prompts.jsonl # If evaluation opted in - ai-migration/run-evaluation.sh # If evaluation opted in diff --git a/migration-to-aws/steering/generate-artifacts-docs.md b/migration-to-aws/steering/generate-artifacts-docs.md index 226a6188..6bff35fd 100644 --- a/migration-to-aws/steering/generate-artifacts-docs.md +++ b/migration-to-aws/steering/generate-artifacts-docs.md @@ -12,6 +12,7 @@ Produce comprehensive migration documentation from all generated artifacts. This - `MIGRATION_GUIDE.md` — Step-by-step migration guide organized by phase - `README.md` — Quick start, artifact catalog, and architecture overview +- `STARTUP_PROGRAMS.md` — AWS startup program eligibility and credit guidance (AI track only) ## Prerequisites @@ -52,6 +53,13 @@ The MIGRATION_GUIDE.md follows this structure: #### Prerequisites Section Content +#### Pre-migration: root user security + +- **Enable root MFA (ACCT.05).** Operators must complete root MFA setup before first console sign-in. The plugin does not manage MFA devices. +- **Restrict use of the root user (ACCT.02).** Store root credentials in a password manager; use only for account recovery or the small handful of tasks that require root. Do not use for daily work. +- **Remove any root access keys.** Root access keys should not exist; if any are present, delete them before any migration work. +- **Plan for day-to-day access via IAM Identity Center (ACCT.03, ACCT.04, ACCT.13).** Creating IAM Identity Center users, group-based permission sets, and short-lived credentials is out of scope for this plan and is covered by the landing-zone spec. + Include these checklists: - AWS Account Setup: account created, IAM user, AWS CLI, Terraform >= 1.5.0 @@ -92,6 +100,10 @@ Secrets Migration (only if `scripts/04-migrate-secrets.sh` exists): `./scripts/0 **Section 4: Service Migration** — Per-cluster migration steps from generation-infra.json, organized by creation_order depth. +**Human Expertise Advisory (BigQuery / deferred analytics)** — If any service has `human_expertise_required: true` for BigQuery or `aws_service` is **`Deferred — specialist engagement`**, include a prominent callout in Section 4 next to that service: + +> **Specialist engagement required (BigQuery):** This plugin **does not** choose an AWS analytics or warehouse target (no Athena/Redshift/Glue recommendation). Engage your **AWS account team** and/or a **data analytics migration partner** before data warehouse, lake, or SQL analytics design. BigQuery work involves query patterns, data movement, ETL/ELT, and BI integration that must be assessed by specialists. + #### IF AI track ran (generation-ai.json exists) Generate the following section: @@ -111,6 +123,10 @@ Generate the following section: ### Common Sections (always included) +**AWS Credits callout (only when generation-ai.json exists):** + +> 💡 **Before incurring costs:** Check [STARTUP_PROGRAMS.md](./STARTUP_PROGRAMS.md) for AWS Activate credits that apply to Bedrock and infrastructure costs. Credits do not apply retroactively — apply before running `terraform apply`. + **Cutover section** with subsections: - Pre-Cutover Checklist (from generation plan) @@ -119,18 +135,36 @@ Generate the following section: **Validation and Cleanup section** with subsections: +#### Post-migration: remove unused default VPC (ACCT.09) + +Confirm first that no resources depend on the default VPC: + +```bash +aws ec2 describe-instances --filters "Name=vpc-id,Values=" \ + --region +``` + +Delete the default VPC (this also deletes default subnets, route tables, and internet gateway associations): + +```bash +aws ec2 delete-vpc --vpc-id --region +``` + +If the command fails with dependency errors, investigate the listed resources before retrying. Do not force-delete. + - Validation Steps checklist: services responding, performance thresholds, data integrity, cost tracking - GCP Teardown checklist (after stability period): archive data, delete resources, disable billing **Troubleshooting section** with a Common Issues table: -| Issue | Cause | Resolution | -| ---------------------------- | -------------------------- | ------------------------------------------ | -| Terraform apply fails | Missing permissions | Check IAM role has required policies | -| Database connection refused | Security group rules | Verify inbound rules allow app subnet CIDR | -| Container image pull fails | ECR authentication | Run `aws ecr get-login-password` | -| Bedrock InvokeModel fails | Model access not enabled | Enable in AWS Console | -| High latency after migration | Suboptimal instance sizing | Review CloudWatch metrics and right-size | +| Issue | Cause | Resolution | +| -------------------------------------------------------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| Terraform apply fails | Missing permissions | Check IAM role has required policies | +| Database connection refused | Security group rules | Verify inbound rules allow app subnet CIDR | +| Container image pull fails | ECR authentication | Run `aws ecr get-login-password` | +| Bedrock InvokeModel fails | Model access not enabled | Enable in AWS Console | +| High latency after migration | Suboptimal instance sizing | Review CloudWatch metrics and right-size | +| `validation-report.json` shows `passed_degraded_offline` | Provider registry was unreachable when Generate ran | From a network-connected shell, run `cd terraform/ && terraform init && terraform validate` to complete the skipped checks | Rollback Procedure subsection (from generation plan — rollback triggers, steps, and RTO). @@ -154,14 +188,33 @@ The README.md follows this structure: - Title: `# GCP to AWS Migration Artifacts` - Subtitle: `> Generated by GCP to AWS Migration Advisor` -#### Quick Start +Immediately after the subtitle, before any other section, write a conditional **Start here** callout block. Include only the lines that apply to what was actually generated: + +```markdown +> **Start here** +> +> - **AI migration:** `cd ai-migration && ./setup_bedrock.sh --dry-run`, then re-run with `--execute` when ready +> - **Infrastructure:** `cd terraform && terraform init && terraform plan -out migration.tfplan`, then `terraform apply migration.tfplan` +> - **Both tracks:** complete AI setup first, then run Terraform +``` + +**Conditional rules:** + +- Omit the AI line entirely if `ai-migration/` was not generated +- Omit the Infra line entirely if `terraform/` was not generated +- Omit the "Both" line if only one track ran +- If neither directory exists (billing-only run), omit the Start here block entirely + +#### Next steps Numbered steps: -1. Review the Migration Guide (link to MIGRATION_GUIDE.md) -1. Deploy infrastructure: `cd terraform/ && terraform init && terraform plan` +1. If AI track ran: `cd ai-migration && ./setup_bedrock.sh` (dry run), then `--execute` when ready +1. If infra track ran: `cd terraform && terraform init && terraform plan`, then `terraform apply migration.tfplan` 1. Run migration scripts: `./scripts/01-validate-prerequisites.sh` 1. If AI track ran: Set up AI: `cd ai-migration/ && ./setup_bedrock.sh` +1. If AI track ran: Review AWS credits: [STARTUP_PROGRAMS.md](./STARTUP_PROGRAMS.md) — apply before incurring costs +1. Review the full Migration Guide: [MIGRATION_GUIDE.md](./MIGRATION_GUIDE.md) #### Artifact Catalog @@ -170,7 +223,7 @@ Table with columns: Artifact, Description, Status. List all generated files/dire Subsections: - Migration Plans (Stage 1): list generation-*.json files -- Infrastructure (Stage 2): list .tf files and migration scripts if they exist +- Infrastructure (Stage 2): list .tf files, **`terraform/README.md`** (when infra Terraform was generated), and migration scripts if they exist - AI Migration (Stage 2): list adapter, test harness, setup script if they exist - Documentation: MIGRATION_GUIDE.md and README.md @@ -182,11 +235,66 @@ Subsections: #### Cost Summary -Table from estimation artifacts with: Current GCP Monthly, Projected AWS Monthly, Annual Savings (or Increase), Timeline. +#### Security baseline costs + +**Scenario A — Tier 1 alone** (no compliance declared in Q2): + +Per-unit pricing verified against the AWS Pricing API for us-east-1 on 2026-05-04; accuracy ±25%. + +| Resource | Monthly cost | Notes | +| ----------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------ | +| Account alternate contacts | $0 | Free (ACCT.01) | +| IAM password policy | $0 | Free (ACCT.06) | +| S3 account-level PAB | $0 | Free (ACCT.08) | +| EBS default encryption | $0 | Free (defense-in-depth; KMS encrypt/decrypt is negligible for EBS) | +| IAM Access Analyzer | $0 | Free for external-access analyzers on your own account (ACCT.11) | +| IMDSv2 account default | $0 | Free (defense-in-depth) | +| CloudTrail trail | $0 for events (first trail per region free); S3 storage ~$0.50–3/mo | Management events only (ACCT.07) | +| S3 bucket for CloudTrail logs | ~$0.50–3/mo | Storage + PUT requests | +| AWS Budgets (1 budget) | $0 | First 2 budgets per account are free (ACCT.10) | +| GuardDuty | $0 for 30 days, then ~$2–25/mo | Scales with VPC traffic and API calls (defense-in-depth) | +| **Total estimate** | **$3–30/mo after trial** | GuardDuty is the dominant line item | + +To skip the baseline, delete `terraform/baseline.tf` before running `terraform apply`. + +**Scenario B — Tier 1 + compliance-conditional** (SOC 2 / PCI / HIPAA / FedRAMP declared in Q2): + +Per-unit pricing verified against the AWS Pricing API for us-east-1 on 2026-05-04; accuracy ±25%. + +Tier 1 subtotal (as in Scenario A above): **$3–30/mo after trial**. + +Additional compliance-conditional resources: + +| Resource | Monthly cost | Notes | +| --------------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------- | +| AWS Config (account-level recorder) | ~$2–10/mo | $0.003 per CI continuous mode; daily mode cheaper, lower signal | +| S3 bucket for Config delivery | ~$0.20–1/mo | Storage + PUT requests; same retention as CloudTrail | +| AWS Security Hub (FSBP + any extra standards) | $0 for 30 days, then ~$1–15/mo | Tiered per-check + per-resource (Fargate-only startups pay nothing for EC2 dimension) | +| **Compliance-conditional subtotal** | **~$3–25/mo after Security Hub trial** | | +| **Grand total** | **~$6–55/mo after both free trials** | | + +**Compliance notes**: + +- Security Hub does NOT provide a HIPAA-specific standard. FSBP covers many overlapping controls; for full HIPAA attestation, engage a qualified HIPAA auditor and review the [AWS HIPAA Eligible Services Reference](https://aws.amazon.com/compliance/hipaa-eligible-services-reference/). +- FedRAMP compliance uses NIST 800-53 at the agency level, but this mapping is not directly subscribable in Security Hub — engage your AWS account team for FedRAMP attestation. + +To skip the compliance-conditional section only, delete the block between `########## Compliance-Conditional ##########` and `########## End Compliance-Conditional ##########` in `terraform/baseline.tf` before `terraform apply`. To skip the baseline entirely, delete `terraform/baseline.tf`. + +Table from estimation artifacts with: Current GCP Monthly, Projected AWS Monthly (use **Balanced** tier for the primary AWS column when `estimation-infra.json` exists), Timeline. **Only include a "GCP data transfer egress (est.)" column when `estimation-infra.json` exists and `migration_cost_considerations.billing_data_available` is `true`.** Do **not** add columns or rows for human labor, professional services, or other people-time migration costs. If billing data is unavailable, add a note below the table: "GCP data transfer egress estimates require billing data. Provide a billing export and re-run discovery to see vendor egress projections." + +**How to read cost tiers** (required when infra estimates include Premium / Balanced / Optimized): + +- The three AWS monthly totals are **scenarios** for the **same** architecture, ordered **high → mid → low** estimate. +- **Premium** — _Highest resilience / highest monthly estimate in this model_ +- **Balanced** — _Default scenario; compare GCP to this first_ +- **Optimized** — _Lower monthly estimate; reservations / Spot / storage trade-offs assumed_ +- **Terraform:** The `terraform/` directory (when present) implements **one** stack, aligned with the **Balanced** scenario. **Premium** and **Optimized** are not separate generated folders — see `terraform/README.md` and the `migration_summary` output in `outputs.tf`. + +Include a compact three-tier row or table if the executive report does, matching figures from `estimation-infra.json`. #### Key Decisions -Bullet list from design and generation artifacts: Compute, Database, Storage, and AI/ML (if applicable) with GCP service, AWS service, and rationale. +Bullet list from design and generation artifacts: Compute, Database, Storage, and AI/ML (if applicable) with GCP service, AWS service, and rationale. For each GCP→AWS mapping, add how it was chosen using `steering/design-ref-fast-path.md` → **User-facing vocabulary**: **Standard pairing**, **Tailored to your setup**, or **Estimated from billing only** (from the design artifact’s `confidence` field). For any service with `human_expertise_required: true`, append: "(Specialist guidance recommended — contact your AWS account team)". #### TODO Items @@ -204,8 +312,8 @@ End with: `Generated by GCP to AWS Migration Advisor` - **Artifact catalog**: List all files actually generated (check for directory/file existence) - **Architecture overview**: Extract from `aws-design.json`, `aws-design-ai.json`, or `aws-design-billing.json` -- **Cost summary**: Extract from `estimation-infra.json`, `estimation-ai.json`, or `estimation-billing.json` -- **Key decisions**: Extract from design artifact `rationale` fields +- **Cost summary**: Extract from `estimation-infra.json`, `estimation-ai.json`, or `estimation-billing.json`; include **How to read cost tiers** when three infra tiers exist; state **Balanced** as primary vs GCP and Terraform alignment per `terraform/README.md` when present +- **Key decisions**: Extract from design artifact `rationale` fields and map `confidence` to user-facing labels per `steering/design-ref-fast-path.md` → **User-facing vocabulary** - **Timeline**: Extract from `generation-*.json` `migration_plan.total_weeks` ## Step 3: Self-Check @@ -225,6 +333,15 @@ After generating documentation, verify: Report the list of generated files to the parent orchestrator. **Do NOT update `.phase-status.json`** — the parent `generate.md` handles phase completion. +Before reporting completion, enforce artifact output gate: + +- `MIGRATION_GUIDE.md` exists. +- `README.md` exists. +- Conditional sections in docs match tracks that actually ran (infra/ai/billing). +- Referenced artifact paths in docs exist. + +If this gate fails: STOP and output: "generate-artifacts-docs did not produce complete documentation artifacts." + Output: ``` diff --git a/migration-to-aws/steering/generate-artifacts-infra.md b/migration-to-aws/steering/generate-artifacts-infra.md index 2885776c..77066237 100644 --- a/migration-to-aws/steering/generate-artifacts-infra.md +++ b/migration-to-aws/steering/generate-artifacts-infra.md @@ -17,7 +17,7 @@ Read from `$MIGRATION_DIR/`: - `preferences.json` (REQUIRED) — User preferences including target region, sizing, compliance - `gcp-resource-clusters.json` (REQUIRED) — Cluster dependency graph for ordering -Reference files (read as needed): `steering/design-ref-index.md` and domain-specific files (`steering/design-ref-compute.md`, `steering/design-ref-database.md`, `steering/design-ref-storage.md`, `steering/design-ref-networking.md`, `steering/design-ref-messaging.md`, `steering/design-ref-ai.md`). +Reference files (read as needed): `steering/design-ref-index.md` and domain-specific files (design-ref-compute.md, design-ref-database.md, design-ref-storage.md, design-ref-networking.md, design-ref-messaging.md, design-ref-security.md, design-ref-ai.md). If any REQUIRED file is missing: **STOP**. Output: "Missing required artifact: [filename]. Complete the prior phase that produces it." @@ -25,45 +25,243 @@ If any REQUIRED file is missing: **STOP**. Output: "Missing required artifact: [ Generate `$MIGRATION_DIR/terraform/` with only the files needed for domains that have resources in `aws-design.json`: -| File | Domain | Contains | -| --------------- | ---------- | ------------------------------------------------ | -| `main.tf` | core | Provider config, backend, data sources | -| `variables.tf` | core | All input variables with types and defaults | -| `outputs.tf` | core | Resource outputs and migration summary | -| `vpc.tf` | networking | VPC, subnets, NAT, security groups, route tables | -| `security.tf` | security | IAM roles, policies, KMS keys | -| `storage.tf` | storage | S3 buckets, EFS, backup vaults | -| `database.tf` | database | RDS/Aurora instances, parameter groups | -| `compute.tf` | compute | Fargate/ECS, Lambda, EC2 | -| `monitoring.tf` | monitoring | CloudWatch dashboards, alarms, log groups | +| File | Domain | Contains | +| --------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `main.tf` | core | Provider config, backend, data sources | +| `variables.tf` | core | All input variables with types and defaults | +| `outputs.tf` | core | Resource outputs and migration summary | +| `baseline.tf` | security baseline | Account-wide security baseline: alternate contacts, password policy, S3 PAB, EBS encryption, Access Analyzer, IMDSv2 default, CloudTrail + S3 log bucket, AWS Budget, GuardDuty. Plus a compliance-conditional section (Config + Security Hub + standards) when `preferences.json.compliance` contains soc2/pci/hipaa/fedramp. Always emitted; users who want to skip it can delete this file before `terraform apply`. | +| `vpc.tf` | networking | VPC, subnets, NAT, security groups, route tables | +| `security.tf` | security | IAM roles, policies, KMS keys, Secrets Manager | +| `storage.tf` | storage | S3 buckets, EFS, backup vaults | +| `database.tf` | database | RDS/Aurora instances, parameter groups | +| `compute.tf` | compute | Fargate/ECS, Lambda, EC2 | +| `monitoring.tf` | monitoring | CloudWatch dashboards, alarms, log groups | +| `README.md` | core | Cost tiers vs this Terraform (one stack; Balanced-aligned) | ## Step 0: Plan Generation Scope Build a generation manifest: read all resources from `aws-design.json` clusters, assign each to its target .tf file by `aws_service`: -| AWS Service | Target File | -| ----------------------------------------------------- | --------------- | -| VPC, Subnet, NAT Gateway, Security Group, Route Table | `vpc.tf` | -| IAM Role, IAM Policy, KMS Key | `security.tf` | -| S3, EFS, Backup Vault | `storage.tf` | -| RDS, Aurora, DynamoDB, ElastiCache | `database.tf` | -| Fargate, ECS, Lambda, EC2 | `compute.tf` | -| CloudWatch, SNS (for alarms) | `monitoring.tf` | +| AWS Service | Target File | +| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | +| Account Alternate Contacts, IAM Account Password Policy, S3 Account Public Access Block, EBS Default Encryption, IAM Access Analyzer (ACCOUNT), EC2 Instance Metadata Defaults, CloudTrail, AWS Budgets, GuardDuty Detector, S3 buckets for CloudTrail and Config logs, AWS Config recorder/delivery/role, AWS Security Hub + standards | `baseline.tf` | +| VPC, Subnet, NAT Gateway, Security Group, Route Table | `vpc.tf` | +| IAM Role, IAM Policy, KMS Key, Secrets Manager | `security.tf` | +| S3, EFS, Backup Vault | `storage.tf` | +| RDS, Aurora, DynamoDB, ElastiCache | `database.tf` | +| Fargate, ECS, Lambda, EC2 | `compute.tf` | +| CloudWatch, SNS (for alarms) | `monitoring.tf` | + +> `baseline.tf` is always emitted. It is NOT driven by `aws-design.json` clusters — the resources are workload-independent account controls. The compliance-conditional subset (Config + Security Hub) is emitted within the same file when `preferences.json.compliance` contains soc2/pci/hipaa/fedramp. The `aws_budgets_budget` resource reads `estimation-infra.json` to set its `limit_amount`. See Step 1.5 below. Users who want to skip the baseline can delete `terraform/baseline.tf` before `terraform apply`. + +**BigQuery / specialist-deferred:** If `aws_service` is **`Deferred — specialist engagement`**, **do not** generate Terraform for that resource (no Glue, Athena, Redshift, or EMR modules from the plugin). Optionally add **`terraform/README-BIGQUERY-DEFERRED.md`** with a short checklist: engage **AWS account team** and/or **data analytics migration partner** before implementing analytics infrastructure. ## Step 1: Generate main.tf **Requirements:** -- `terraform` block: `required_version >= 1.5.0`, `hashicorp/aws ~> 5.0`, commented S3 backend +- **File header comment block (first lines in `main.tf`, before `terraform {`):** Explain that (1) this directory implements the **single** architecture in `aws-design.json`; (2) the migration report’s **Premium / Balanced / Optimized** figures are **three pricing scenarios** from `estimation-infra.json` for that same map — **not** three separate generated stacks; (3) **this Terraform is aligned with the Balanced cost scenario** (default sizing/HA posture used for the middle estimate); (4) **Premium** = higher HA / higher $ model; **Optimized** = cost-optimization assumptions — users must **edit IaC or add modules** to realize those postures. Point readers to `terraform/README.md` and the `migration_summary` output. +- `terraform` block: `required_version >= 1.5.0`, `hashicorp/aws ~> 5.80`, active S3 backend (see Step 1a below — do NOT comment it out) - `provider "aws"` block: `region = var.aws_region`, `default_tags` with Project, Environment, ManagedBy, MigrationId - Data sources: `aws_caller_identity`, `aws_region`, `aws_availability_zones` +## Step 1a: Remote state backend + +Always emit an **active** (not commented-out) S3 backend block in `main.tf`. Local state is not safe for production — `terraform.tfstate` stores resource metadata and sensitive values in plaintext on the local filesystem. + +Emit the following backend block inside the `terraform {}` block in `main.tf`: + +```hcl +backend "s3" { + # Bootstrap: these resources are created by baseline.tf. + # First run: terraform init -backend=false && terraform apply \ + # -target=aws_s3_bucket.tfstate \ + # -target=aws_s3_bucket_versioning.tfstate \ + # -target=aws_s3_bucket_server_side_encryption_configuration.tfstate \ + # -target=aws_s3_bucket_public_access_block.tfstate \ + # -target=aws_dynamodb_table.tfstate_lock + # Then re-run: terraform init (migrates local state to S3) + bucket = "--tfstate-" # TODO: substitute values + key = "migration/terraform.tfstate" + region = "" # TODO: substitute target region + dynamodb_table = "--tfstate-lock" # TODO: substitute values + encrypt = true +} +``` + +Also emit the following resources in `baseline.tf` (append after the always-on resources): + +```hcl +# Remote state backend infrastructure +resource "aws_s3_bucket" "tfstate" { + bucket = "${var.project_name}-${var.environment}-tfstate-${data.aws_caller_identity.current.account_id}" + tags = merge(local.baseline_tags, { Component = "terraform-state" }) +} + +resource "aws_s3_bucket_versioning" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + versioning_configuration { status = "Enabled" } +} + +resource "aws_s3_bucket_server_side_encryption_configuration" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + rule { + apply_server_side_encryption_by_default { + sse_algorithm = "aws:kms" + } + } +} + +resource "aws_s3_bucket_public_access_block" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + block_public_acls = true + block_public_policy = true + ignore_public_acls = true + restrict_public_buckets = true +} + +resource "aws_dynamodb_table" "tfstate_lock" { + name = "${var.project_name}-${var.environment}-tfstate-lock" + billing_mode = "PAY_PER_REQUEST" + hash_key = "LockID" + attribute { + name = "LockID" + type = "S" + } + tags = merge(local.baseline_tags, { Component = "terraform-state" }) +} +``` + +Add a **Bootstrap** section to `terraform/README.md` explaining the two-step init process. + +## Step 1b: Generate terraform/README.md + +**Always create** `$MIGRATION_DIR/terraform/README.md` when generating Terraform (same pass as Step 1). + +**Required sections:** + +1. **What this directory is** — Implements one deployable baseline from `aws-design.json` (and `generation-infra.json` / `preferences.json` as applicable). +2. **Cost tiers in the migration report** — Premium, Balanced, and Optimized are **monthly cost scenarios** in `estimation-infra.json` for the **same** service mapping; order is high → mid → low estimate. +3. **Which scenario this Terraform matches** — **Balanced** (primary comparison to GCP; default migration posture in the advisor model). Premium and Optimized are **not** auto-generated as alternate roots. +4. **If you need Premium or Optimized in production** — Manually adjust instance classes, Multi-AZ, Spot mix, Reserved Instances / Savings Plans, storage classes, etc., then re-estimate. +5. **Artifacts** — Reference `estimation-infra.json`, `migration-report.html` / `MIGRATION_GUIDE.md` for full tier tables. + +Keep it under one screen of text. + +## Step 1.5: Generate baseline.tf + +Always emitted. The baseline applies account-wide security controls that should be in place on any new AWS account. Users who do not want the baseline can delete `terraform/baseline.tf` before `terraform apply`. + +1. **Compute retention.** Read `preferences.json.compliance` (array of strings; may be absent or empty). Compute `cloudtrail_retention_days` using this mapping, taking `max()` across all declared values (use 90 if the array is empty or absent): + - absent / `[]` → 90 + - `soc2` → 365 + - `pci` → 365 + - `hipaa` → 2190 + - `fedramp` → 1095 + - `gdpr` → 365 + +2. **Compute budget limit.** Read `estimation-infra.json.projected_costs.breakdown.total.mid` (or the canonical equivalent). Compute `budget_limit = max(50, ceil(total_mid * 1.2))`. If `estimation-infra.json` is missing or unreadable, use `50` and emit an inline comment noting that the projection was unavailable. + +3. **Choose file-header variant.** If `compliance` contains any of `soc2`, `pci`, `hipaa`, `fedramp`, emit the compliance-expansion header. Otherwise emit the base header. Both variants include a two-sentence provenance note stating per-unit rates were verified against the AWS Pricing API for us-east-1 on 2026-05-04. Substitute the resolved `cloudtrail_retention_days` value into the header. + +4. **Emit `baseline.tf`** starting with the file-header comment block and a `locals` block containing the resolved `cloudtrail_retention_days` integer: + + ```hcl + locals { + cloudtrail_retention_days = + } + ``` + +5. **Append the always-on resources**, in this order. Each resource carries the plugin's standard four default tags plus `Component = "security-baseline"`: + - `aws_account_alternate_contact.operations` (ACCT.01, TODO-email placeholder) + - `aws_account_alternate_contact.billing` (ACCT.01, TODO-email placeholder) + - `aws_account_alternate_contact.security` (ACCT.01, TODO-email placeholder) + - `aws_iam_account_password_policy.baseline` (ACCT.06; `minimum_password_length = 14`, `password_reuse_prevention = 24`, `max_password_age = 90`, all four character-class requirements `true`, `hard_expiry = false`) + - `aws_s3_account_public_access_block.baseline` (ACCT.08; all four flags `true`) + - `aws_ebs_encryption_by_default.baseline` (defense-in-depth; `enabled = true`) + - `aws_accessanalyzer_analyzer.baseline` (ACCT.11; `type = "ACCOUNT"`) + - `aws_ec2_instance_metadata_defaults.baseline` (defense-in-depth; `http_tokens = "required"`, `http_put_response_hop_limit = 2`) + - `aws_cloudtrail.baseline` (ACCT.07; multi-region, management events only, `enable_log_file_validation = true`) + - `aws_s3_bucket.cloudtrail_logs` plus `aws_s3_bucket_public_access_block`, `aws_s3_bucket_server_side_encryption_configuration`, `aws_s3_bucket_versioning`, `aws_s3_bucket_lifecycle_configuration` (transitions driven by `local.cloudtrail_retention_days` per item 7), and `aws_s3_bucket_policy` restricting the CloudTrail service principal by `aws:SourceArn` + - `aws_budgets_budget.monthly_spend` (ACCT.10; `limit_amount = ""` from item 2; three `notification` blocks at 50/80/100% `ACTUAL`; TODO-email placeholders) + - `aws_guardduty_detector.baseline` (defense-in-depth; `enable = true`, `finding_publishing_frequency = "FIFTEEN_MINUTES"`) + +6. **If `compliance` contains any of `soc2`, `pci`, `hipaa`, `fedramp`, append the compliance-conditional section**, wrapped in `########## Compliance-Conditional ##########` / `########## End Compliance-Conditional ##########` dividers: + - `aws_iam_role.config` + `aws_iam_role_policy_attachment` for the managed policy `AWSConfigRole` + - `aws_config_configuration_recorder.baseline` with `recording_group { all_supported = true, include_global_resource_types = true }` + - `aws_config_delivery_channel.baseline` pointing at the Config S3 bucket + - `aws_config_configuration_recorder_status.baseline` with `is_enabled = true` + - `aws_s3_bucket.config_logs` plus PAB, SSE, versioning, lifecycle (same `local.cloudtrail_retention_days`), and a bucket policy allowing the `config.amazonaws.com` service principal + - `aws_securityhub_account.baseline` + - `aws_securityhub_standards_subscription.fsbp` (always emitted in this section) + - `aws_securityhub_standards_subscription.pci_dss` (only if `compliance` contains `pci`) + + Do NOT emit an NIST 800-53 standards subscription, even if `compliance` contains `hipaa` or `fedramp`. Security Hub does not provide a HIPAA-specific standard; FedRAMP attestation is out-of-band. + +7. **Lifecycle rule adjustment.** Omit the `STANDARD_IA` transition block when the resolved retention is less than 90 days. Omit the `GLACIER` transition block when retention is less than 365 days. Both rules apply to both the CloudTrail log bucket and (when emitted) the Config log bucket. + +8. **Attach inline HCL comments**: + - On each `aws_account_alternate_contact.*`: a TODO-email warning. + - On `aws_cloudtrail.baseline`: a collision warning for users who already have a trail in the region. + - On `aws_budgets_budget.monthly_spend`: a TODO-email warning plus the limit-rationale comment (`max(50, ceil(total_mid * 1.2))`; $50 floor prevents alert noise; users may edit `limit_amount` directly post-apply). + - On `aws_guardduty_detector.baseline`: a cost disclosure noting the 30-day free trial and ~$2–25/mo post-trial. + - On `aws_config_configuration_recorder.baseline`: a cost disclosure ($0.003/CI continuous; $0.012/daily-CI as an opt-in for cost-sensitive users). + - On `aws_securityhub_account.baseline`: a cost disclosure noting the 30-day free trial and ~$1–15/mo post-trial. + - On every defense-in-depth resource (EBS encryption, IMDSv2 account default, GuardDuty, Config, Security Hub): the literal token `defense-in-depth` in the inline comment. + +9. **compute.tf modification (runs during Step 3 compute domain, not here):** every `aws_launch_template` emitted for ECS-EC2, EKS node groups, or bare EC2 receives IMDSv2 enforcement unconditionally — the security baseline is always applied: + + ```hcl + metadata_options { + http_tokens = "required" + http_put_response_hop_limit = 1 + http_endpoint = "enabled" + instance_metadata_tags = "enabled" + } + ``` + + Fargate, Lambda, and App Runner do not emit launch templates and are unaffected (no synthetic launch template is created). Hop limit `1` here is intentionally different from the account-level default `2` in `aws_ec2_instance_metadata_defaults.baseline` — strict on templates the plugin owns, permissive at the account default. + +**Emission conditions**: + +- Emit `baseline.tf` even when `aws-design.json` contains only AI or billing-only resources (no infrastructure clusters). The baseline is workload-independent. +- Do NOT probe for existing account resources (CloudTrail trails, Config recorders, Security Hub enrollment). Collision risk is surfaced by the inline comments listed in item 8. + ## Step 2: Generate variables.tf **Global variables (always include):** `aws_region` (from `preferences.json` target_region), `project_name`, `environment` (from `preferences.json`), `migration_id`. **Per-cluster variables:** Extract configurable values from `aws_config` in `aws-design.json`. Infer types (`string`, `number`, `bool`, `list(string)`, `map(string)`). Use `aws_config` values as defaults. Deduplicate shared variables. Add GCP source as comment (e.g., `# GCP source: db-custom-2-7680`). +## Step 2b: Generate terraform.tfvars.example and .gitignore + +Always emit `$MIGRATION_DIR/terraform/terraform.tfvars.example` alongside `variables.tf`. Populate it with actual values from `aws-design.json`, `preferences.json`, and `estimation-infra.json` where available. Use descriptive placeholder strings (not empty values) for anything that cannot be inferred. Format: + +```hcl +# Copy this file to terraform.tfvars and fill in the values before running terraform plan. +# Do NOT commit terraform.tfvars to source control — it may contain sensitive values. + +aws_region = "" # from preferences.json target_region +project_name = "" # TODO: set your project name +environment = "production" # TODO: dev | staging | production +migration_id = "" # from migration run ID + +# One entry per variable in variables.tf, with source annotation as comment +``` + +Also emit `$MIGRATION_DIR/terraform/.gitignore` with: + +``` +# Never commit actual variable values — may contain sensitive data +terraform.tfvars +*.tfvars +!terraform.tfvars.example +.terraform/ +*.tfstate +*.tfstate.backup +``` + ## Step 3: Generate Per-Domain .tf Files For each domain with resources in the generation manifest: @@ -73,24 +271,52 @@ For each domain with resources in the generation manifest: - Consult `steering/design-ref-*.md` for AWS configuration best practices - A single GCP resource may map to multiple AWS resources (1:Many expansion) - Use `gcp_config` values from `aws-design.json` to populate resource attributes -- For `confidence: "inferred"` resources, add comment: `# Inferred mapping — verify configuration` +- For `confidence: "inferred"` resources, add comment: `# Tailored to your setup — verify configuration (JSON confidence: inferred)` +- For `confidence: "deterministic"` resources, optional comment: `# Standard pairing (fixed mapping list)` - Include `secondary_resources` from the cluster (IAM roles, security groups) - Tag every resource: Project, Environment, ManagedBy, MigrationId **Domain-specific rules:** -| Domain | Key Rules | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------- | -| Networking | At least 2 AZs; public + private subnets; NAT gateway for private subnet internet | -| Security | Least-privilege IAM (specific ARNs, never wildcards); per-service roles for Fargate/Lambda | -| Storage | Versioning enabled; SSE-S3 or SSE-KMS encryption; block public access; lifecycle policies | -| Database | Private subnets; subnet group + parameter group + security group; backups; encryption | -| Compute | Fargate in private subnets; task definitions from `aws_config` CPU/memory; auto-scaling | -| Monitoring | Log groups per service; dashboard with key metrics; alarms from `generation-infra.json` success_metrics; 30-day log retention | +| Domain | Key Rules | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Networking | At least 2 AZs; public + private subnets; NAT gateway for private subnet internet; internet-facing ALB must terminate TLS on 443 and HTTP 80 must redirect to HTTPS; when `preferences.json.compliance` contains `pci`, `hipaa`, or `fedramp`, emit `aws_flow_log` for the VPC targeting a CloudWatch log group — add inline cost disclosure comment: `# VPC Flow Logs: ~$0.50/GB ingested. Enabled for compliance. Disable if cost is a concern and compliance posture allows.` — omit for non-compliance stacks | +| Security | Least-privilege IAM (specific ARNs, never wildcards); per-service roles for Fargate/Lambda; Secrets Manager resources with no plaintext defaults; when `preferences.json.compliance` contains `soc2`, `pci`, `hipaa`, or `fedramp`, emit a companion `aws_secretsmanager_secret_rotation` block for every `aws_secretsmanager_secret` resource with `automatically_after_days = 30` and a TODO comment for the rotation Lambda ARN — omit the rotation block for non-compliance stacks to keep the generated Terraform immediately applyable; when `preferences.json.compliance` contains `pci`, `hipaa`, or `fedramp`, generate a customer-managed KMS key (`aws_kms_key`) and reference it via `kms_key_id` on every `aws_secretsmanager_secret` — omit for non-compliance stacks (AWS-managed key is sufficient) | +| Storage | Versioning enabled; SSE-S3 or SSE-KMS encryption; block public access by default; lifecycle policies; if public content is required use CloudFront/OAC instead of public bucket policy; when `preferences.json.compliance` contains `pci`, `hipaa`, or `fedramp`, emit `aws_s3_bucket_logging` for every application S3 bucket (not the CloudTrail/Config log buckets themselves) targeting a dedicated access-log bucket — add inline cost disclosure comment: `# S3 access logging: ~$0.023/GB stored. Enabled for compliance. Disable if cost is a concern and compliance posture allows.` — omit for non-compliance stacks | +| Database | Private subnets; subnet group + parameter group + security group; backups; encryption at rest (`storage_encrypted = true`); `deletion_protection = true` by default; **`publicly_accessible = false` always** — never emit `publicly_accessible = true` unless the user has explicitly requested it via a compliance exception in `preferences.json`; **never emit a security group rule allowing ingress on port 5432 or 3306 from `0.0.0.0/0`** — database ports must only allow ingress from the application security group CIDR or security group ID; if GCP Cloud SQL `authorized_networks` contains `0.0.0.0/0`, emit a `warnings[]` entry in `aws-design.json`: "Cloud SQL authorized_networks includes 0.0.0.0/0 — mapped to private RDS with no public access" (add inline comment: `# Set to false only when intentionally destroying this cluster`); **never use `master_password = var.database_master_password`** — instead generate the master password into Secrets Manager and reference it via data source: emit `aws_secretsmanager_secret` + `aws_secretsmanager_secret_version` (with `secret_string = jsonencode({password = random_password.db_master.result})`) and `resource "random_password" "db_master"`, then set `master_password = jsondecode(data.aws_secretsmanager_secret_version.db_master.secret_string)["password"]` on the cluster — this keeps the password out of `terraform.tfvars` and Terraform state plaintext | +| Compute | Fargate in private subnets; task definitions from `aws_config` CPU/memory; auto-scaling; for EKS clusters set `endpoint_private_access = true` and `endpoint_public_access = false` by default — add inline comment: `# Public endpoint disabled. To enable kubectl access from outside the VPC set endpoint_public_access = true and restrict public_access_cidrs to known CIDRs.`; every `aws_ecr_repository` resource must include `image_scanning_configuration { scan_on_push = true }` — ECR basic scanning is free and catches known CVEs before images reach production | +| Monitoring | Log groups per service; dashboard with key metrics; alarms from `generation-infra.json` success_metrics; 30-day log retention | ## Step 4: Generate outputs.tf -Output identifiers for key resources (VPC ID, database endpoint, ECS cluster name, etc.) plus a `migration_summary` output with region, VPC ID, environment, service count, and migration ID. +Output identifiers for key resources (VPC ID, database endpoint, ECS cluster name, etc.) plus a **`migration_summary` output** (object) including at minimum: + +| Key | Type / example | Purpose | +| ------------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `aws_region` | string | From `var.aws_region` | +| `environment` | string | From `var.environment` | +| `migration_id` | string | From `var.migration_id` | +| `service_count` | number | Count of primary logical services / resources represented | +| `aligned_with_estimate_tier` | string | Always **`"balanced"`** for this advisor — generated IaC matches the **Balanced** scenario in `estimation-infra.json` | +| `cost_scenarios_modeled_in_terraform` | string | e.g. **`"design_baseline_only"`** — only one stack generated; Premium/Optimized exist as **pricing** scenarios in estimates, not as additional Terraform trees | + +Add VPC ID or other IDs when known from resources. Descriptions on every output. + +**Example shape:** + +```hcl +output "migration_summary" { + description = "Migration run metadata and cost-tier alignment (Balanced baseline)" + value = { + aws_region = var.aws_region + environment = var.environment + migration_id = var.migration_id + service_count = + aligned_with_estimate_tier = "balanced" + cost_scenarios_modeled_in_terraform = "design_baseline_only" + } +} +``` ## Step 5: Self-Check @@ -102,22 +328,55 @@ Verify these quality rules before reporting completion: - [ ] Tags on every resource (Project, Environment, ManagedBy, MigrationId) - [ ] Encryption at rest on all storage (S3, EBS, RDS) - [ ] Databases and internal services use private subnets +- [ ] All RDS/Aurora resources have `publicly_accessible = false` +- [ ] No security group rule allows ingress on port 5432 or 3306 from `0.0.0.0/0` +- [ ] ALB listeners enforce HTTPS (443) and HTTP (80) only redirects to HTTPS +- [ ] No S3 bucket policy with `Principal = "*"` unless explicitly approved by user requirements - [ ] No `0.0.0.0/0` ingress except ALB port 443 - [ ] Every variable has `type` and `description` - [ ] Every output has `description` - [ ] Region from `var.aws_region`, never hardcoded +- [ ] `terraform/README.md` exists with cost-tier vs Terraform explanation +- [ ] `main.tf` begins with the required cost-tier / Balanced alignment comment block +- [ ] `migration_summary` output includes `aligned_with_estimate_tier` and `cost_scenarios_modeled_in_terraform` + +- [ ] `baseline.tf` exists. +- [ ] `baseline.tf` contains `aws_account_alternate_contact` for each of OPERATIONS, BILLING, SECURITY, plus `aws_iam_account_password_policy`, `aws_s3_account_public_access_block`, `aws_ebs_encryption_by_default`, `aws_cloudtrail`, `aws_guardduty_detector`, `aws_accessanalyzer_analyzer`, `aws_ec2_instance_metadata_defaults`, and `aws_budgets_budget`. +- [ ] `baseline.tf` contains a `locals` block with `cloudtrail_retention_days` set to a positive integer, and the lifecycle `expiration.days` on `aws_s3_bucket_lifecycle_configuration.cloudtrail_logs` equals `local.cloudtrail_retention_days`. +- [ ] `aws_budgets_budget.monthly_spend.limit_amount` equals `max(50, ceil(estimation-infra.json.projected_costs.breakdown.total.mid * 1.2))` as a string. +- [ ] If `compute.tf` contains any `aws_launch_template`, every such launch template has `metadata_options { http_tokens = "required", http_put_response_hop_limit = 1 }` — this applies unconditionally (the security baseline is always emitted). +- [ ] Every `aws_ecr_repository` in `compute.tf` has `image_scanning_configuration { scan_on_push = true }`. +- [ ] If `compliance` contains soc2/pci/hipaa/fedramp, `baseline.tf` contains `aws_config_configuration_recorder`, `aws_config_delivery_channel`, `aws_config_configuration_recorder_status`, `aws_securityhub_account`, `aws_securityhub_standards_subscription` for FSBP. +- [ ] If `compliance` contains pci, an additional `aws_securityhub_standards_subscription` for PCI DSS exists. +- [ ] `baseline.tf` does NOT contain any `aws_securityhub_standards_subscription` whose `standards_arn` references `nist-800-53`, regardless of compliance values. +- [ ] If `compliance` is empty, absent, or contains only gdpr, `baseline.tf` does NOT contain any `aws_config_*` or `aws_securityhub_*` resources. +- [ ] `baseline.tf` does NOT contain any invented SSB control IDs. Search for `ACCT.IAM`, `ACCT.S3`, `ACCT.EBS`, `ACCT.CT`, `ACCT.GD`, `ACCT.CFG`, `ACCT.SH`, `WKLD.EC2.01` — all MUST have zero matches. Only bare `ACCT.01` through `ACCT.13` identifiers are permitted. +- [ ] `baseline.tf` does NOT mention "Trusted Advisor" anywhere (Trusted Advisor is docs-action only and out of scope). +- [ ] Security Hub subscribes to FSBP (always when the compliance-conditional section is emitted) and PCI DSS (only when `compliance` contains `pci`). No other standards subscriptions. ## Phase Completion Report generated files to the parent orchestrator. **Do NOT update `.phase-status.json`** — the parent `generate.md` handles phase completion. +Before reporting completion, enforce artifact output gate: + +- `terraform/` directory exists. +- At minimum: `terraform/main.tf`, `terraform/variables.tf`, and `terraform/outputs.tf` exist. +- At least one domain file exists among: `vpc.tf`, `security.tf`, `storage.tf`, `database.tf`, `compute.tf`, `monitoring.tf`. +- `terraform/baseline.tf` MUST exist (baseline is always emitted). + +If this gate fails: STOP and output: "generate-artifacts-infra did not produce required Terraform artifacts; do not complete Generate Stage 2." + ``` Generated terraform artifacts: +- terraform/README.md - terraform/main.tf - terraform/variables.tf - terraform/outputs.tf - terraform/[domain].tf (for each domain with resources) +- validation-report.json (status: ) Total: [N] Terraform files +Validation: (attempts=, errors_fixed=) TODO markers: [N] items requiring manual configuration ``` diff --git a/migration-to-aws/steering/generate-artifacts-report.md b/migration-to-aws/steering/generate-artifacts-report.md new file mode 100644 index 00000000..28a40291 --- /dev/null +++ b/migration-to-aws/steering/generate-artifacts-report.md @@ -0,0 +1,572 @@ +# Generate Phase: HTML Migration Report + +> Loaded by generate.md AFTER generate-artifacts-docs.md completes. + +**Execute ALL steps in order. Do not skip or optimize.** + +## Overview + +Generate a single self-contained HTML report (`migration-report.html`) combining an executive summary with detailed appendix. The HTML file uses inline CSS — no external dependencies required. Users can open it in any browser and use "Print to PDF" if a PDF is needed. + +**Output:** + +- `migration-report.html` — Self-contained HTML report with executive summary and detailed appendix + +**Non-blocking:** If report generation fails after `VALIDATE_OK` (HTML build error only), log a warning and continue. Validation `GATE_FAIL` is **not** a silent skip — always surface to the user. Do NOT fail the Generate phase for report issues. + +## Step 0: Validate Artifacts (Read Only) + +Load and execute `steering/validate-artifacts.md` **before** building report content. + +- Run all **required** checks (field presence only — do not rewrite artifact prose). +- On any `GATE_FAIL`: output failure lines to the user, **do NOT write** `migration-report.html`, **do NOT patch artifacts**, return to parent `generate.md`. +- On `VALIDATE_OK`: proceed to Step 1. + +## Prerequisites + +At least one of these must exist in `$MIGRATION_DIR/`: + +- Design artifact: `aws-design.json`, `aws-design-ai.json`, or `aws-design-billing.json` +- Estimation artifact: `estimation-infra.json`, `estimation-ai.json`, or `estimation-billing.json` +- Generation plan: `generation-infra.json`, `generation-ai.json`, or `generation-billing.json` + +If **none** exist: skip report generation. Output: "Skipping HTML report — no migration artifacts found." + +## Data Sources + +Gather data from all available artifacts. Each section below notes which artifact provides the data. + +| Data Point | Primary Source | Fallback | +| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | +| GCP services detected | `aws-design.json` clusters[].resources[] | `aws-design-billing.json` services[] | +| AWS service mappings | `aws-design.json` resources[].aws_service | `aws-design-billing.json` services[].aws_service | +| Rationale per service | `aws-design.json` resources[].rationale | `aws-design-billing.json` services[].rationale | +| Current GCP monthly cost | `estimation-infra.json` current_costs.gcp_monthly | `estimation-billing.json` | +| Projected AWS monthly cost | `estimation-infra.json` projected_costs.aws_monthly_balanced | `estimation-billing.json` | +| Cost breakdown per service | `estimation-infra.json` projected_costs.breakdown | `estimation-billing.json` | +| Cost tiers (premium/balanced/optimized) | `estimation-infra.json` cost_comparison | — | +| Optimization opportunities | `estimation-infra.json` optimization_opportunities | — | +| Migration timeline | `generation-infra.json` migration_plan.total_weeks | `generation-billing.json` | +| Top risks | `generation-infra.json` risk_assessment | `generation-billing.json` | +| Human expertise flags | Design artifact resources[].human_expertise_required | — | +| AI model mappings | `aws-design-ai.json` | — | +| AI cost estimates | `estimation-ai.json` | — | +| Migration decision / recommendation | `estimation-infra.json` → `recommendation` | `financial_summary.recommendation` (string fallback) | +| Complexity and timeline hint | `migration-preview.json` → `complexity_signal`, `timeline_hint` | — | +| Key decisions ahead | `migration-preview.json` → `key_decisions_ahead` | — | +| User configuration choices | `preferences.json` (read `.value` from wrapped fields) | — | +| AI capabilities and integration | `ai-workload-profile.json` → `models[]`, `integration`, `agentic_profile` | — | +| Deferred services | Design artifact `resources[].aws_service == "Deferred — specialist engagement"` | — | +| Observability cost callout | `estimation-infra.json` → `projected_costs.breakdown` (array: `service` contains "Observability"; object: key contains `observability` or `cloudwatch`) | — | + +## Step 1: Build Executive Summary Section + +The executive summary is the first thing visible when opening the report. Design it to fit approximately 1–2 printed pages. + +### Executive Summary Content + +**Header:** "GCP to AWS Migration Assessment" with subtitle "Executive Summary" and generation date. + +**Target length:** approximately 1–2 printed pages. If content exceeds 2 pages, move the Security Capabilities table to Appendix G and keep only the teaser in the exec summary. + +**Section 0 — Migration Decision Summary (REQUIRED):** + +Pull from `estimation-infra.json` → `recommendation` block. Fallback chain if `recommendation` is absent: + +1. `estimation-infra.json` → `financial_summary.recommendation` (string) — use as `path_label`; synthesize `migrate_if` / `stay_if` from Part 7 prose defaults in `estimate-infra.md` +2. `migration-preview.json` — show complexity + timeline only; label: "Full recommendation requires Estimate phase — run Phase 4" + +Content when `recommendation` block exists: + +1. **Verdict badge:** `recommendation.path_label` — render as colored badge (green for `migrate_optimized`, blue for `migrate_phased`, amber for `stay`) +2. **Complexity:** from `migration-preview.json` → `complexity_signal` ("Simple", "Moderate", "Complex") — colored badge +3. **Cost headline:** from `estimation-infra.json` → `cost_comparison.option_b_balanced` vs GCP baseline, OR legacy `comparison.aws_balanced_monthly_usd` vs `comparison.gcp_monthly_usd`. Do NOT use `migration-preview.json` → `cost_preview` when estimation artifact exists (preview is superseded). If only preview exists: show labeled "Early estimate (±30%) — full analysis not yet run." +4. **Timeline:** from `generation-infra.json` → `migration_plan.total_weeks` (preferred), OR `migration-preview.json` → `timeline_hint`. Do NOT use `recommendation.next_steps` as timeline — those are action items, not duration. +5. **Migrate if / Stay if:** from `recommendation.migrate_if` and `recommendation.stay_if`. Render as two compact lists. +6. **Key decisions ahead:** from `migration-preview.json` → `key_decisions_ahead` — bullet list +7. **Next steps (optional):** from `recommendation.next_steps` — compact bullet list separate from timeline + +**Deferred services flag:** If ANY resource in the design artifact has `aws_service == "Deferred — specialist engagement"`, add a prominent callout: + +> ⚠️ **Specialist engagement required:** [service name] does not have an automated AWS mapping. Engage your AWS account team before including this in cost projections or migration timelines. + +**Startup credits callout:** If `STARTUP_PROGRAMS.md` exists in `$MIGRATION_DIR/` OR `$MIGRATION_DIR/ai-migration/STARTUP_PROGRAMS.md` OR `preferences.json` contains `startup_program_status.value` other than `"unknown"`: + +> 💡 **AWS Activate credits:** You may be eligible for $1K–$100K in AWS credits that apply to Bedrock, Fargate, RDS, and other services used in this migration. See `STARTUP_PROGRAMS.md` for program tiers and application links. + +Do not show this callout if none of the conditions are met. + +Source: estimation artifact `recommendation`, `migration-preview.json`, design artifact + +**Section 1 — Current Stack Overview:** + +- Count of PRIMARY GCP services detected (from design artifact — filter to primary classification only; exclude secondary/supporting resources like default VPC firewalls) +- List each PRIMARY GCP service with its type (e.g., "Cloud Run (compute)", "Cloud SQL (database)") +- Source: design artifact (filtered) + +**Section 2 — Recommended AWS Architecture:** + +- Table with columns: GCP Service, AWS Service, **How we chose this** +- **How we chose this** values: use `steering/design-ref-fast-path.md` → **User-facing vocabulary** — **Standard pairing** (`deterministic`), **Tailored to your setup** (`inferred`), **Estimated from billing only** (`billing_inferred`). Show the **bold phrase** in the table; JSON value optional in a tooltip or footnote for technical readers only. +- One row per mapped service +- If any service has `human_expertise_required: true`, mark it with a warning indicator and footnote: "Specialist guidance recommended — contact your AWS account team" +- Source: design artifact + +**Section 3 — Cost Comparison:** + +- Side-by-side display: Current GCP Monthly vs Projected AWS Monthly (**Balanced** tier — the default scenario for comparing to GCP) +- Percent change (savings or increase) +- **How to read cost tiers (callout box — required when infra estimation with three tiers exists):** The three AWS monthly figures are **pricing scenarios** for the **same** mapped architecture (same services in `aws-design.json`), not three different generated Terraform stacks. **Order = highest → middle → lowest** monthly estimate in this model. Use **Balanced** as the **primary** row vs GCP; **Premium** and **Optimized** are **bounds** (higher HA / newer skew vs cost-optimization skew). When `terraform/` is present, it implements **one** infrastructure baseline aligned with the **Balanced** cost scenario (see `terraform/README.md` and `migration_summary` output). +- If 3 tiers available: show **Premium**, **Balanced**, and **Optimized** with **short subtitles** (second line or subtext under each label): + - **Premium** — _Highest resilience / highest monthly estimate in this model_ + - **Balanced** — _Default scenario; compare GCP to this row first_ + - **Optimized** — _Lower monthly estimate; reservations, Spot, or storage trade-offs assumed_ +- **Footnote (required):** _Only one Terraform configuration is generated (Balanced-aligned baseline). Premium and Optimized are what-if cost models in `estimation-infra.json` — adjust IaC yourself if you want those postures in production._ +- **Only include "GCP data transfer egress (est.)" when the infra estimation artifact has `migration_cost_considerations.billing_data_available === true`.** Never present human one-time migration costs. If `false` or only non-infra estimates exist, footnote: "GCP data transfer egress estimates require billing data and the infra estimate path." +- If observability entry exists in `projected_costs.breakdown` (tolerant lookup: array where `service` contains "Observability" OR object where key contains `observability` or `cloudwatch`) AND the entry's `note` field mentions GCP free tiers: + +> **Observability cost note:** [Pull the `note` field verbatim] + +- Source: estimation artifact + +**Section 4 — Security & Cost Guardrails (teaser — full table in Appendix G):** + +Show top controls as a compact teaser: + +| Control | What it does for you | Monthly cost | +| ------------------------------ | ------------------------------------------------------------------ | ------------------------------------ | +| GuardDuty | Detects compromised credentials and crypto mining within minutes | ~$2–25/mo | +| CloudTrail | Immutable audit log of every API call — required for SOC 2 | ~$0.50–3/mo | +| Budget alerts | Email when spend exceeds threshold — catches runaway resources | $0 | +| Bedrock cost anomaly detection | Alerts within ~24h if AI spend spikes unexpectedly (AI track only) | $0 (Cost Explorer anomaly detection) | + +The fourth row is **conditional** — only render when `$MIGRATION_DIR/ai-migration/bedrock_monitoring.tf` exists on disk. Do NOT render based on `generation-ai.json` alone. + +> See Appendix G for full security and cost guardrails table with GCP equivalents. + +If `preferences.json` contains compliance values (`soc2`, `pci`, `hipaa`, `fedramp`): + +> **Compliance note:** Your declared compliance requirement ([standard]) triggers additional controls (AWS Config + Security Hub) at ~$3–25/mo. See Appendix G. + +**Do NOT include step-by-step enablement** — that belongs in `terraform/README.md` and `MIGRATION_GUIDE.md`. + +Source: static content + `preferences.json` compliance values + +**Section 5 — Operational Changes (conditional rows only):** + +Only render rows for service types PRESENT in the design artifact. Do not show rows for services not in the migration. + +| GCP Service | AWS Service | What stays the same | What's new | +| ------------------ | ----------- | --------------------------------------------------- | ------------------------------------------------------------------------------------ | +| Cloud Run | Fargate | Fully managed containers, auto-scaling, pay-per-use | Task definitions replace service.yaml; ALB for HTTP routing; ECR replaces GCR | +| Cloud SQL | RDS/Aurora | Managed DB, automated backups, PITR | Parameter groups replace database flags; Security Groups replace authorized networks | +| Cloud Storage | S3 | Object storage, lifecycle policies, versioning | Bucket policies replace IAM conditions; CloudFront needed for public CDN | +| Vertex AI / Gemini | Bedrock | Managed API, pay-per-token | IAM-based access (no API keys); SDKs differ | + +**Conditional rendering:** Check design artifact for each `gcp_type` / `aws_service` pair. Only include rows where the GCP source type exists in the design. If a service category has no match, skip that row entirely. + +Source: static template filtered by design artifact service types + +**Section 6 — Timeline:** + +- Total migration weeks +- Migration approach (phased/fast-track/conservative) +- Source: generation plan + +**Section 7 — Top Risks:** + +- Up to 3 highest-severity risks +- Each with: risk name, severity, one-line mitigation +- Source: generation plan risk_assessment + +## Step 2: Build Detailed Appendix + +The appendix follows the executive summary, clearly separated with an "Appendix: Detailed Migration Analysis" header. + +### Appendix Section A — Service Recommendations + +For each mapped service, include: + +- GCP service name and type +- AWS service recommendation +- **How the mapping was chosen** — use **Standard pairing**, **Tailored to your setup**, or **Estimated from billing only** (`steering/design-ref-fast-path.md` → User-facing vocabulary); JSON `confidence` may appear in parentheses for support +- Full rationale text from design artifact +- If the mapping was **Tailored to your setup** (`inferred`) and `rubric_applied` is present: list the 6 criteria evaluations (appendix detail — optional in executive summary) +- If `human_expertise_required: true`: include the specialist guidance callout + +Source: design artifact (aws-design.json or aws-design-billing.json) + +### Appendix Section B — Cost Estimates + +**Per-service cost breakdown table** with columns: Service Category, AWS Service, Monthly Cost (Balanced), Alternative, Alternative Cost, Potential Savings. + +Source: estimation artifact projected_costs.breakdown + +**Three-tier comparison table** with columns: **Tier** (name + subtitle as in Section 3), Monthly Cost, vs GCP Monthly, Annual Difference. + +Repeat the **How to read cost tiers** callout from Section 3 here or include a one-line pointer: _See executive summary — three tiers are scenario $ only; generated Terraform matches **Balanced** baseline._ + +Source: estimation artifact cost_comparison + +**Optimization opportunities table** with columns: Optimization, Target Services, Monthly Savings, Commitment, Effort. + +Source: estimation artifact optimization_opportunities + +> **Security baseline costs** are included as a line item in the breakdown above. For what each control does and GCP equivalents, see Section 4 (exec summary teaser) or Appendix G (full capabilities table). + +### Appendix Section C — Migration Steps + +Numbered migration phases from the generation plan, each with: + +- Phase name and description +- Services included +- Estimated duration +- Dependencies and prerequisites + +Source: generation plan + +**Rollback procedure** — triggers, steps, and RTO from generation plan. + +### Appendix Section D — AI Migration (conditional) + +**Only include if `aws-design-ai.json` or `estimation-ai.json` exists.** + +**D.1 — AI Stack Detected:** + +Pull from `ai-workload-profile.json` when present: + +| Aspect | Detected | +| ------------------- | ---------------------------------------------------------------------------------------------------------- | +| AI source | `summary.ai_source` | +| Models | `models[].model_id` — comma-separated list | +| Capabilities in use | `integration.capabilities_summary` — keys where value is `true` | +| Integration pattern | `integration.pattern` | +| Gateway/router | `integration.gateway_type` or "None (direct SDK)" | +| Frameworks | `integration.frameworks` or "None" | +| Agentic | If `agentic_profile` exists: "Yes — [framework], [orchestration_pattern], [agent_count] agents"; else "No" | + +**D.2 — Why Bedrock (conditional):** + +Show this section when `aws-design-ai.json` → `ai_architecture.honest_assessment` contains ANY of: `"recommend_stay"`, `"weak_migrate"`, or `"moderate_migrate"` where any model's Bedrock price exceeds the source provider price. + +> **Why migrate to Bedrock when [source] may be cheaper per token?** +> +> - **Single-vendor billing:** One AWS bill instead of separate provider invoices +> - **VPC-private inference:** Model calls stay in your VPC — no data over public internet +> - **IAM access control:** No API keys to rotate; permissions follow your AWS IAM model +> - **Model evaluation:** A/B test models with Bedrock's built-in evaluation framework +> - **Guardrails:** Content filtering and PII detection at the platform level +> - **Commitment pricing:** Provisioned Throughput for predictable costs at scale +> +> These benefits matter most when: you handle sensitive data, need AI call audit trails, or want to consolidate vendors. + +**D.3 — Model Mappings:** + +- Model mappings (GCP model to AWS Bedrock model) +- AI cost estimates + +**D.4 — Migration approach:** + +- Migration approach (adapter pattern, A/B testing) + +**D.5 — Post-migration AI cost optimization (conditional):** + +**Only include if `generation-ai.json` exists in `$MIGRATION_DIR/` (AI track ran).** + +After migration is validated and stable, three optimization levers are available (typical ranges from plugin guidance — validate against your traffic): + +| Optimization | Estimated savings | When to apply | Prerequisite | +| -------------------------- | ------------------------ | ------------------------------------------------ | ------------------------------------------------------------------------ | +| Intelligent Prompt Routing | 10–30% | After 2+ weeks of production traffic | Same model family in multiple tiers (e.g., Claude Sonnet + Haiku) | +| Prompt caching | 20–50% on eligible calls | When prompts have long repeated context | Minimum ~1K–4K tokens cacheable prefix; per-model minimums and TTL apply | +| Model distillation | Up to ~75% | After 30+ days of stable, high-volume production | Stable prompts, evaluation dataset, sufficient call volume | + +These are not migration steps — they are post-migration optimizations. Do not block migration on these. Surface as a "Month 2–3" roadmap item. + +**Prompt caching caveat:** Caching only helps for long, repeated context windows. Evaluate actual prompt patterns before assuming savings. + +**Full detail:** Open `ai-workload-profile.json` in this directory. + +Source: `ai-workload-profile.json`, `aws-design-ai.json`, `estimation-ai.json` + +### Appendix Section E — Generated Artifacts Catalog + +List all files and directories generated during the Generate phase: + +- `terraform/` — list .tf files and **`README.md`** +- `scripts/` — list migration scripts +- `ai-migration/` — list adapter files (if applicable) +- `MIGRATION_GUIDE.md`, `README.md` + +Check for actual file/directory existence before listing. + +**Data artifacts (for detailed review):** + +| Artifact | Contents | +| ----------------------------- | ---------------------------------------------------------- | +| `preferences.json` | All migration configuration choices and their sources | +| `ai-workload-profile.json` | Full AI model inventory, capabilities, evidence | +| `gcp-resource-inventory.json` | Complete GCP resource inventory with classifications | +| `estimation-infra.json` | Detailed cost model, recommendation, per-service breakdown | +| `aws-design.json` | Full architecture design with rationale per service | + +Open any JSON file with a text editor or `cat | python3 -m json.tool` for formatted output. + +**AI migration artifacts (conditional — only list if they exist on disk):** + +| Artifact | Description | +| ----------------------------------------------------------- | --------------------------------------------------------------------------------- | +| `ai-migration/bedrock_monitoring.tf` | Bedrock budget alerts, anomaly detection, inference profiles for cost attribution | +| `ai-migration/STARTUP_PROGRAMS.md` or `STARTUP_PROGRAMS.md` | AWS Activate credit tiers, application URLs, eligibility guidance | +| `ai-migration/setup_bedrock.sh` | Bedrock model access setup script | +| `ai-migration/test_comparison.py` | A/B comparison harness for source vs Bedrock quality | + +Do not list files that were not generated. + +### Appendix Section F — Your Configuration (conditional) + +**Only include if `preferences.json` exists in `$MIGRATION_DIR/`.** + +Key decisions that shaped this migration plan. Each value is read from `preferences.json` using the `.value` field of wrapped preference objects (e.g., `availability.value`, not `availability` directly). + +| Decision | Your choice | Impact on plan | +| ------------------------ | ------------------------------------------------------ | --------------------------------------------------------------- | +| Target AWS region | `design_constraints.target_region.value` or equivalent | All resources deployed here; Bedrock model availability checked | +| Availability requirement | `availability.value` | Drives RDS single-AZ vs Multi-AZ vs Aurora selection | +| Monthly GCP spend | From estimation source or `gcp_monthly_spend.value` | Cost comparison baseline | +| Framework | `ai_framework.value` (if AI track ran) | Determines migration effort for AI workloads | +| AI priority | `ai_priority.value` (if present) | Drives Bedrock model selection | +| Compliance | `compliance.value` (if present) | Triggers Config + Security Hub in baseline.tf | + +**Source indicators:** Each preference shows `chosen_by`: `"user"` (explicitly answered), `"extracted"` (inferred from IaC/code), or `"default"` (plugin default applied). + +**Full detail:** Open `preferences.json` in this directory. + +Source: `preferences.json` + +### Appendix Section G — Security Capabilities + +Full security baseline capabilities. Executive summary shows a teaser; this appendix provides the complete picture. + +| Control | What it does for you | Threat mitigated | GCP equivalent | Monthly cost | +| ---------------------- | ----------------------------------------------------------------------------------- | ------------------------------------- | ------------------------------------------ | ------------ | +| GuardDuty | Detects compromised credentials, crypto mining, unusual API patterns within minutes | Credential theft, resource hijacking | Security Command Center (paid Premium) | ~$2–25/mo | +| CloudTrail | Immutable audit log of every API call — who did what, when | Unauthorized changes, compliance gaps | Cloud Audit Logs (free for admin activity) | ~$0.50–3/mo | +| IMDSv2 enforcement | Blocks SSRF attacks from stealing instance/container credentials | Server-side request forgery | N/A (GCP uses different metadata model) | $0 | +| Access Analyzer | Alerts when IAM policies or S3 buckets become publicly accessible | Accidental public exposure | IAM Recommender (partial) | $0 | +| EBS default encryption | All storage volumes encrypted at rest by default | Data exposure from stolen disks | Default encryption (GCP default) | $0 | +| Budget alerts | Email when spend exceeds threshold | Bill shock from runaway resources | GCP Budgets (free) | $0 | +| S3 Block Public Access | Account-wide prevention of any bucket being made public | Accidental data leaks | Uniform bucket-level access (opt-in) | $0 | +| IAM password policy | 14-char min, rotation, complexity for console users | Weak password compromise | Cloud Identity policies | $0 | + +**Compliance-conditional (only when SOC 2/PCI/HIPAA/FedRAMP declared in preferences):** + +| Control | What it adds | Compliance gap covered | Monthly cost | +| ------------------- | ------------------------------------------------------ | ---------------------------------- | ------------ | +| AWS Config | Continuous recording of resource configuration changes | Change audit trail for SOC 2 / PCI | ~$2–10/mo | +| Security Hub + FSBP | Automated security checks against AWS best practices | Baseline posture scoring | ~$1–15/mo | + +**Cost guardrails (when `$MIGRATION_DIR/ai-migration/bedrock_monitoring.tf` exists):** + +| Control | What it does for you | Threat mitigated | GCP equivalent | Monthly cost | +| ------------------------------- | ----------------------------------------------------------------------- | -------------------------------------------------- | ---------------------------------------------- | ------------ | +| Bedrock budget (1.5× projected) | Hard spend alert at 150% of estimated AI costs — fires before month-end | Runaway token spend from buggy retry loops | GCP Budgets (free, but no per-service scoping) | $0 | +| Cost anomaly detection | Daily digest when AI spend deviates from baseline (~24h data lag) | Gradual cost creep, unexpected model-price changes | None (no GCP per-service anomaly equivalent) | $0 | +| Inference profiles (tagged) | Per-model cost attribution in Cost Explorer | Invisible cost distribution across models | None | $0 | + +These are detective controls, not spend caps. You will know within ~24 hours if something goes wrong — not at invoice time. + +**What the baseline does NOT cover (you still need):** + +- SOC 2: Qualified auditor, formal policies, employee training, vendor management +- HIPAA: BAA with AWS, qualified HIPAA auditor, data handling policies +- PCI: QSA assessment, network segmentation validation, pen testing +- FedRAMP: Agency-level NIST 800-53 controls, ATO process + +**For detailed cost breakdown:** See Appendix B (cost estimates include security baseline as a line item). + +**To enable:** Run `terraform apply` on `baseline.tf`. To skip: delete `baseline.tf` before applying. See `terraform/README.md` for details. + +Source: static content + `preferences.json` compliance values + +## Step 3: Generate HTML + +### Pre-Write Sanity Check (mandatory) + +Immediately before writing the file, **re-read from disk**: + +1. `estimation-infra.json` → `recommendation.path_label` present OR Step 1 fallback documented. +2. `migration-preview.json` → `complexity_signal` present (if file exists). +3. Assembled HTML string contains `
`. + +If any check fails: emit `GATE_FAIL | phase=generate | field= | reason=missing`, do **not** write the file, return to parent. + +Write the complete HTML to `$MIGRATION_DIR/migration-report.html`. + +### HTML Structure (required section IDs) + +The output MUST include these `id` attributes (content from Steps 1–2; gates check **presence only**): + +| Section ID | Content | +| -------------------- | -------------------------------------- | +| `decision-summary` | Section 0 — Migration Decision Summary | +| `exec-services` | Primary services summary | +| `exec-costs` | Cost comparison headline | +| `exec-timeline` | Timeline | +| `exec-risks` | Top risks | +| `appendix-services` | Appendix A | +| `appendix-costs` | Appendix B | +| `appendix-steps` | Appendix C | +| `appendix-artifacts` | Appendix E | + +Optional IDs (include when data exists): `appendix-ai`, `appendix-config`, `appendix-security`. + +```html + + + + + + GCP to AWS Migration Assessment + + + +
+
+
+
+
+
+
+
+
+
+
+
+ +
+ + +
+
+ Generated by GCP to AWS Migration Advisor — draft for review; verify figures against source JSON artifacts before executive sign-off. +
+
+ + +``` + +### CSS Specification + +The inline CSS must include: + +**Layout:** + +- `body`: font-family system-ui, -apple-system, sans-serif; max-width 900px; margin 0 auto; padding 40px 20px; color #1a1a2e; background #ffffff; line-height 1.6 +- `.report`: single container + +**Typography:** + +- `h1`: font-size 1.8rem; color #1a1a2e; border-bottom 3px solid #ff9900; padding-bottom 8px +- `h2`: font-size 1.4rem; color #232f3e; margin-top 2rem +- `h3`: font-size 1.1rem; color #545b64 + +**Tables:** + +- `table`: width 100%; border-collapse collapse; margin 1rem 0 +- `th`: background #232f3e; color white; padding 10px 12px; text-align left; font-size 0.85rem +- `td`: padding 8px 12px; border-bottom 1px solid #e8e8e8; font-size 0.85rem +- `tr:hover`: background #f5f5f5 + +**Cards (for executive summary metrics):** + +- `.metric-card`: display inline-block; background #f8f9fa; border 1px solid #e8e8e8; border-radius 8px; padding 16px 24px; margin 8px; text-align center; min-width 160px +- `.metric-value`: font-size 1.6rem; font-weight bold; color #232f3e +- `.metric-label`: font-size 0.8rem; color #687078; text-transform uppercase + +**Cost comparison highlight:** + +- `.cost-savings`: color #067d68 (green for savings) +- `.cost-increase`: color #d13212 (red for increase) + +**Warning callout (for human_expertise_required):** + +- `.callout-warning`: background #fff8e1; border-left 4px solid #ff9900; padding 12px 16px; margin 1rem 0; border-radius 0 4px 4px 0 + +**Confidence badges (visible text = user-facing vocabulary, not JSON):** + +- `.badge`: display inline-block; padding 2px 8px; border-radius 12px; font-size 0.75rem; font-weight 600 +- `.badge-deterministic`: background #e6f4ea; color #137333 — label **Standard pairing** +- `.badge-inferred`: background #fef7e0; color #b05a00 — label **Tailored to your setup** +- `.badge-billing`: background #fce8e6; color #c5221f — label **Estimated from billing only** + +**Verdict badges (Section 0):** + +- `.badge-verdict-migrate`: background #e6f4ea; color #137333 — `migrate_optimized` +- `.badge-verdict-phased`: background #e8f0fe; color #1a73e8 — `migrate_phased` +- `.badge-verdict-stay`: background #fef7e0; color #b05a00 — `stay` +- `.badge-complexity`: background #f1f3f4; color #545b64 — complexity signal + +**Print styles:** + +- `@media print`: hide nothing, adjust margins, ensure page breaks before `.appendix` + +**Footer:** + +- `footer`: margin-top 3rem; padding-top 1rem; border-top 1px solid #e8e8e8; text-align center; color #687078; font-size 0.8rem + +### Content Rules + +1. **All data must come from artifacts** — do not invent numbers or services. If an artifact field is missing, omit that section. +2. **Currency formatting**: All cost values displayed as `$X,XXX.XX` with dollar sign and commas. +3. **Percentage formatting**: Include `+` or `-` prefix. Use green styling for savings, red for increases. +4. **No external resources**: No CDN links, no external fonts, no images. Everything inline. +5. **Valid HTML5**: Output must be valid, well-formed HTML5. + +## Step 4: Self-Check + +After generating the HTML file, verify: + +1. **Required section IDs**: `decision-summary`, `exec-services`, `exec-costs`, `exec-timeline`, `exec-risks`, `appendix-services`, `appendix-costs`, `appendix-steps`, `appendix-artifacts` each appear exactly once as `
`. If any missing: treat as build failure (warn user; do not fail Generate phase). +2. **Data accuracy**: Cost figures in HTML match the estimation artifact values exactly +3. **Conditional sections**: AI appendix only present if AI artifacts exist; billing caveats shown when billing_data_available is false; Bedrock monitoring row only when `bedrock_monitoring.tf` exists; startup credits callout only when `STARTUP_PROGRAMS.md` or preference indicates eligibility +4. **Section 0**: Migration Decision Summary present when estimation or preview artifacts exist; uses `recommendation.path_label` when block present +5. **Human expertise flags**: Warning callouts appear for all services with `human_expertise_required: true` +6. **Valid HTML**: Opening and closing tags match, no broken table structures +7. **No placeholders**: No `[placeholder]` or `TODO` text in the report output +8. **Footer disclaimer**: Footer contains "draft for review" + +## Step 5: Open Report in Browser + +After writing the HTML file, open it in the user's default browser so they can view it immediately. + +Run: `open "$MIGRATION_DIR/migration-report.html"` (macOS) or `xdg-open "$MIGRATION_DIR/migration-report.html"` (Linux). + +If the open command fails, fall back to presenting the full file path to the user: + +``` +Migration report ready — open in your browser: +file://$MIGRATION_DIR/migration-report.html +``` + +## Completion + +Report to the parent orchestrator. **Do NOT update `.phase-status.json`** — the parent `generate.md` handles phase completion. + +Output: + +``` +Migration report saved to $MIGRATION_DIR/migration-report.html + +Report sections: +- Executive Summary: Section 0 Migration Decision Summary, [services count] primary services, [cost comparison], [timeline] +- Appendix A: Service Recommendations +- Appendix B: Cost Estimates +- Appendix C: Migration Steps +- [Appendix D: AI Migration — if applicable] +- Appendix E: Artifacts Catalog +- [Appendix F: Your Configuration — if preferences.json exists] +- Appendix G: Security Capabilities +``` diff --git a/migration-to-aws/steering/generate-artifacts-scripts.md b/migration-to-aws/steering/generate-artifacts-scripts.md index 501a6f00..6447920c 100644 --- a/migration-to-aws/steering/generate-artifacts-scripts.md +++ b/migration-to-aws/steering/generate-artifacts-scripts.md @@ -81,12 +81,27 @@ Based on database and storage resources in `aws-design.json`: **Cloud SQL to RDS/Aurora** — include only if `has_databases`: +Read `preferences.json` → `design_constraints.db_size.value` to select the migration tool: + +- `"<10GB"` → use **pg_dump/pg_restore** +- `"10-100GB"` or `"100-500GB"` → use **pgcopydb** (parallel copy; requires `wal_level=logical` on Cloud SQL) +- `">500GB"` → use **AWS DMS** (continuous replication; generate DMS task config instead of a shell export script) +- `"unknown"` → default to **pgcopydb** with a TODO comment to verify size before running + +Generate the script with conditional branches based on `db_size`: + ```bash #!/usr/bin/env bash set -euo pipefail - # Cloud SQL → RDS data migration # Usage: ./02-migrate-data.sh [--execute] +# +# Tool selection based on database size (preferences.json design_constraints.db_size.value): +# <10GB: pg_dump/pg_restore +# 10-500GB: pgcopydb (parallel copy, 3-5x faster than pg_dump) +# >500GB: AWS DMS recommended — see README-DMS.md if generated +# unknown: pgcopydb (safer default at unknown scale) +# TODO: Verify database size before running — wrong tool choice can exceed your maintenance window. DRY_RUN=true [[ "${1:-}" == "--execute" ]] && DRY_RUN=false @@ -94,31 +109,96 @@ DRY_RUN=true echo "=== Database Migration: Cloud SQL → RDS ===" echo "Mode: $([ "$DRY_RUN" = true ] && echo 'DRY RUN' || echo 'EXECUTE')" -# TODO: Configure source and target connection details -SOURCE_HOST="" # TODO: Set Cloud SQL IP -TARGET_HOST="" # From terraform output database_endpoint -DATABASE_NAME="" # TODO: Set database name +SOURCE_HOST="" # TODO: Set Cloud SQL IP +TARGET_HOST="" # From terraform output database_endpoint +DATABASE_NAME="" # TODO: Set database name +``` + +Then emit ONE of the following tool-specific blocks based on `db_size`: + +**If `db_size` is `"<10GB"`** — emit pg_dump block: + +```bash +# Tool: pg_dump/pg_restore (database < 10GB) +if [ "$DRY_RUN" = true ]; then + echo "[DRY RUN] Would export from Cloud SQL using pg_dump: $DATABASE_NAME" + echo "[DRY RUN] Would import to RDS using pg_restore: $TARGET_HOST" +else + echo "Exporting from Cloud SQL with pg_dump..." + PGPASSWORD="$SOURCE_DB_PASSWORD" pg_dump \ + -h "$SOURCE_HOST" -U postgres -d "$DATABASE_NAME" \ + -Fc -f /tmp/migration_dump.pgc + echo "Importing to RDS with pg_restore..." + PGPASSWORD="$TARGET_DB_PASSWORD" pg_restore \ + -h "$TARGET_HOST" -U postgres -d "$DATABASE_NAME" \ + -Fc /tmp/migration_dump.pgc + rm -f /tmp/migration_dump.pgc +fi +``` + +**If `db_size` is `"10-100GB"` or `"100-500GB"`** — emit pgcopydb block: + +```bash +# Tool: pgcopydb (database 10GB–500GB — parallel copy, 3-5x faster than pg_dump) +# Requires: pgcopydb installed, wal_level=logical on Cloud SQL source +# See: https://pgcopydb.readthedocs.io/ +if [ "$DRY_RUN" = true ]; then + echo "[DRY RUN] Would copy database using pgcopydb: $DATABASE_NAME" + echo "[DRY RUN] Source: $SOURCE_HOST → Target: $TARGET_HOST" +else + echo "Copying database with pgcopydb (parallel)..." + pgcopydb copy db \ + --source "postgresql://postgres:${SOURCE_DB_PASSWORD}@${SOURCE_HOST}/${DATABASE_NAME}" \ + --target "postgresql://postgres:${TARGET_DB_PASSWORD}@${TARGET_HOST}/${DATABASE_NAME}" \ + --table-jobs 4 \ + --index-jobs 4 +fi +``` + +**If `db_size` is `">500GB"`** — emit DMS guidance block instead of a shell script: +```bash +# Tool: AWS DMS (database > 500GB — continuous replication recommended) +# Single-pass export/import at this scale is high-risk and likely to exceed any maintenance window. +# AWS DMS provides continuous replication with a minimal cutover window (minutes, not hours). +# +# Steps: +# 1. Create a DMS replication instance in the target VPC +# 2. Create source endpoint (Cloud SQL PostgreSQL) and target endpoint (Aurora PostgreSQL) +# 3. Create a full-load + CDC replication task +# 4. Run full load, then let CDC catch up to < 1 second lag +# 5. Cut over DNS during the minimal lag window +# +# TODO: Configure DMS endpoints and task — see AWS DMS documentation: +# https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html +echo "Database size > 500GB: AWS DMS is strongly recommended." +echo "See README-DMS.md for setup instructions." +echo "This script does not perform the migration directly at this scale." +``` + +**If `db_size` is `"unknown"`** — emit pgcopydb block with prominent TODO: + +```bash +# Tool: pgcopydb (database size unknown — safer default than pg_dump at unknown scale) +# TODO: Verify your database size before running. If > 500GB, use AWS DMS instead. +# Requires: pgcopydb installed, wal_level=logical on Cloud SQL source if [ "$DRY_RUN" = true ]; then - echo "[DRY RUN] Would export from Cloud SQL: $SOURCE_HOST" - echo "[DRY RUN] Would import to RDS: $TARGET_HOST" - echo "[DRY RUN] Database: $DATABASE_NAME" + echo "[DRY RUN] Would copy database using pgcopydb: $DATABASE_NAME" + echo "[DRY RUN] TODO: Verify database size before executing" else - # Export from Cloud SQL - echo "Exporting from Cloud SQL..." - gcloud sql export sql "$SOURCE_HOST" "gs://migration-bucket/export.sql" \ - --database="$DATABASE_NAME" - - # Import to RDS - echo "Importing to RDS..." - # TODO: Use pg_restore, mysql, or appropriate tool for your database engine - # psql -h "$TARGET_HOST" -U admin -d "$DATABASE_NAME" < export.sql + echo "Copying database with pgcopydb..." + pgcopydb copy db \ + --source "postgresql://postgres:${SOURCE_DB_PASSWORD}@${SOURCE_HOST}/${DATABASE_NAME}" \ + --target "postgresql://postgres:${TARGET_DB_PASSWORD}@${TARGET_HOST}/${DATABASE_NAME}" \ + --table-jobs 4 \ + --index-jobs 4 fi +``` +```bash # Verification echo "=== Verification ===" echo "TODO: Compare row counts between source and target" -echo "TODO: Run checksum validation on critical tables" ``` **BigQuery to S3** — include only if `has_databases`: @@ -219,16 +299,25 @@ for SECRET_NAME in "${SECRETS[@]}"; do echo "[DRY RUN] Would migrate secret: $SECRET_NAME" else echo "Reading secret from GCP: $SECRET_NAME" - SECRET_VALUE=$(gcloud secrets versions access latest --secret="$SECRET_NAME") + # Write to a restricted temp file — avoids secret values appearing in shell + # variables, process lists (ps aux), or shell history. Cleaned up on exit. + TMPFILE=$(mktemp) + chmod 600 "$TMPFILE" + trap 'rm -f "$TMPFILE"' EXIT + + gcloud secrets versions access latest --secret="$SECRET_NAME" > "$TMPFILE" - echo "Creating secret in AWS: $SECRET_NAME" + echo "Creating/updating secret in AWS: $SECRET_NAME" aws secretsmanager create-secret \ --name "$SECRET_NAME" \ - --secret-string "$SECRET_VALUE" \ + --secret-string "file://$TMPFILE" \ --tags Key=MigrationSource,Value=gcp-secret-manager 2>/dev/null || \ aws secretsmanager put-secret-value \ --secret-id "$SECRET_NAME" \ - --secret-string "$SECRET_VALUE" + --secret-string "file://$TMPFILE" + + rm -f "$TMPFILE" + trap - EXIT fi done diff --git a/migration-to-aws/steering/generate-billing.md b/migration-to-aws/steering/generate-billing.md index a3b920d9..b20c3ca0 100644 --- a/migration-to-aws/steering/generate-billing.md +++ b/migration-to-aws/steering/generate-billing.md @@ -8,10 +8,10 @@ ## Overview -This file produces a **conservative migration plan** with wider timelines and lower confidence thresholds than the infrastructure path. Billing-only data provides service-level spend but lacks configuration details (instance sizes, replication settings, networking topology). The plan accounts for this uncertainty with: +This file produces a **complexity-scaled migration plan** with wider timelines and lower confidence thresholds than the infrastructure path. Billing-only data provides service-level spend but lacks configuration details (instance sizes, replication settings, networking topology). The plan accounts for this uncertainty with: -- Longer discovery refinement phase upfront -- Wider success criteria thresholds +- Complexity tier classification to right-size the timeline (small: 2-4 weeks, medium: 6-10 weeks, large: 12-18 weeks) +- Wider success criteria thresholds scaled by tier - Explicit recommendation to run IaC discovery before executing the plan ## Prerequisites @@ -50,65 +50,91 @@ If any required file is missing: **STOP**. Output: "Missing required artifact: [ > This billing-only plan is suitable for initial budgeting and stakeholder discussions, > but must be refined with IaC discovery before executing the actual migration. -## Part 2: Conservative Timeline +## Part 2: Complexity-Scaled Timeline -A 15-week timeline with extended discovery and parallel-run phases to account for billing-only uncertainty. +> **Before building the timeline, load `steering/migration-complexity.md` and classify the migration tier using the inputs from `aws-design-billing.json`, `billing-profile.json`, and `preferences.json`.** -### Stage 1: Discovery Refinement (Weeks 1-4) +Use the **Billing-Only Path** stage templates from `migration-complexity.md` for the determined tier. The three tiers produce different timelines: -- **Week 1-2**: Audit current GCP infrastructure manually - - Document instance sizes, database configs, networking topology - - Map dependencies between services - - Identify stateful vs stateless services - - Catalog secrets, environment variables, API keys -- **Week 3-4**: Refine AWS design based on discovered configurations - - Update `aws-design-billing.json` unknowns with actual values - - Re-estimate costs with refined configuration data - - Identify any services that need different AWS targets - - Consider running IaC discovery if Terraform files become available +### Small Tier (2-4 weeks) -### Stage 2: Service Migration (Weeks 5-9) +No parallel-run stage. Discovery and provisioning overlap. Suitable for migrations with <=3 services, <$1K/month, no databases, single-AZ, no compliance. -- **Week 5-6**: Provision AWS infrastructure - - Set up VPC, subnets, security groups (based on discovery findings) - - Provision compute, database, and storage resources - - Configure IAM roles and policies -- **Week 7-8**: Deploy applications - - Migrate application code and configurations - - Set up CI/CD pipelines - - Deploy to AWS staging environment -- **Week 9**: Integration testing - - End-to-end testing on AWS - - Performance baseline measurement - - Data migration dry run +- **Stage 1: Discovery + Provisioning (Week 1)** + - Quick audit of GCP infrastructure (few services, low complexity) + - Provision AWS VPC, compute, and supporting resources + - Configure IAM roles and security groups +- **Stage 2: Deploy + Test (Week 2)** + - Deploy applications to AWS + - Run functional and integration tests + - Validate cost tracking against estimates +- **Stage 3: Cutover + Validation (Weeks 3-4)** + - Execute cutover during maintenance window (DNS switch) + - 24-hour intensive monitoring + - Stabilization and GCP teardown planning -### Stage 3: Parallel Run (Weeks 10-12) +### Medium Tier (6-10 weeks) -- Run both GCP and AWS simultaneously -- Compare performance, reliability, and costs -- Validate data consistency between environments -- Build confidence in AWS deployment -- Monitor for 2+ weeks before cutover decision +Shortened parallel run. Discovery takes 1-2 weeks instead of 4. For migrations with 4-8 services, $1K-$10K/month, or databases present. -### Stage 4: Cutover and Validation (Weeks 13-15) - -- **Week 13**: Execute cutover (DNS switch, traffic migration) -- **Week 14**: Intensive monitoring (48-hour watch period) -- **Week 15**: Stabilization and GCP teardown planning +- **Stage 1: Discovery Refinement (Weeks 1-2)** + - Audit current GCP infrastructure + - Document instance sizes, database configs, networking topology + - Map dependencies between services + - Refine AWS design based on discovered configurations +- **Stage 2: Service Migration (Weeks 3-5)** + - Provision AWS infrastructure (VPC, compute, databases, storage) + - Deploy applications, set up CI/CD pipelines + - Integration testing and data migration dry run +- **Stage 3: Parallel Run (Weeks 6-7)** + - Run both environments simultaneously + - Compare performance, reliability, and costs + - Validate data consistency +- **Stage 4: Cutover and Validation (Weeks 8-10)** + - Execute cutover (DNS switch, traffic migration) + - 48-hour intensive monitoring + - Stabilization and GCP teardown planning + +### Large Tier (12-18 weeks) + +Full conservative plan with extended discovery and parallel-run phases. For migrations with 9+ services, >$10K/month, multi-region, compliance, or AI alongside infrastructure. + +- **Stage 1: Discovery Refinement (Weeks 1-4)** + - Weeks 1-2: Manual infrastructure audit, dependency mapping, configuration documentation + - Weeks 3-4: Refine AWS design, re-estimate costs, identify services needing different AWS targets + - Consider running IaC discovery if Terraform files become available +- **Stage 2: Service Migration (Weeks 5-9)** + - Weeks 5-6: Provision AWS infrastructure (VPC, compute, databases, storage) + - Weeks 7-8: Deploy applications, set up CI/CD, migrate to staging + - Week 9: Integration testing, performance baseline, data migration dry run +- **Stage 3: Parallel Run (Weeks 10-12)** + - Run both GCP and AWS simultaneously + - Compare performance, reliability, and costs + - Validate data consistency between environments + - Monitor for 2+ weeks before cutover decision +- **Stage 4: Cutover and Validation (Weeks 13-15+)** + - Execute cutover (DNS switch, traffic migration) + - 48-hour intensive monitoring + - Stabilization and GCP teardown planning ## Part 3: Risk Assessment -Risks are higher for billing-only migrations due to configuration uncertainty. +Risks are scaled by complexity tier. Use the **Risk Scaling by Tier** table in `steering/migration-complexity.md` to set probability values. The table below shows risk templates — replace the probability column with the tier-appropriate value. + +### Standard Risks + +| Risk | Small Prob. | Medium Prob. | Large Prob. | Impact | Mitigation | +| ------------------------------------------------------ | ----------- | ------------ | ----------- | ------ | ------------------------------------------------------------------------------------------------------------------- | +| Incorrect service sizing | low | medium | high | high | Discovery phase audit; right-size after validation | +| Missing dependencies discovered late | low | medium | high | medium | Manual dependency mapping in discovery; buffer in timeline | +| Data migration complexity underestimated | n/a | medium | high | high | Dry run before cutover; parallel run as safety net (medium/large only) | +| Cost overrun due to unknown configurations | low | medium | high | medium | Set billing alerts at 80% of high estimate; weekly cost reviews | +| Performance regression from incorrect sizing | low | medium | medium | high | Parallel run comparison (medium/large); resize before cutover | +| Longer timeline than planned | low | medium | high | medium | Build buffer into schedule; communicate planned timeline upfront | +| Unmapped services block migration | low | medium | medium | high | Address unknowns in discovery refinement | +| BigQuery migration complexity (if BigQuery in billing) | high | high | high | high | Engage AWS account team for specialist guidance on query patterns, data volumes, ETL pipelines, and BI integrations | -| Risk | Probability | Impact | Mitigation | -| -------------------------------------------- | ----------- | ------ | ------------------------------------------------------------------- | -| Incorrect service sizing | high | high | Extended discovery phase (Weeks 1-4); right-size after parallel run | -| Missing dependencies discovered late | high | medium | Manual dependency mapping in Week 1-2; extra buffer in timeline | -| Data migration complexity underestimated | medium | high | Dry run in Week 9; parallel run (Weeks 10-12) as safety net | -| Cost overrun due to unknown configurations | high | medium | Set billing alerts at 80% of high estimate; weekly cost reviews | -| Performance regression from incorrect sizing | medium | high | Parallel run comparison; resize before cutover | -| Longer timeline than planned | high | medium | Build 3-week buffer into schedule; communicate 15-week plan upfront | -| Unmapped services block migration | medium | high | Address unknowns in discovery refinement (Weeks 1-4) | +For the determined tier, include only risks with probability > "n/a". Use the probability from that tier's column. ## Part 4: Per-Service Migration Steps @@ -119,7 +145,7 @@ For each service in `aws-design-billing.json.services[]`, generate a migration s ``` Service: [gcp_service] → [aws_service] Monthly Cost: $[monthly_cost] (GCP) → $[aws_mid] estimated (AWS) -Confidence: billing_inferred +How chosen: Estimated from billing only (JSON: billing_inferred) — see steering/design-ref-fast-path.md User-facing vocabulary Steps: 1. [ ] Determine actual configuration (instance size, storage, etc.) @@ -140,7 +166,7 @@ Unknowns: ``` Service: Cloud Run → Fargate Monthly Cost: $450.00 (GCP) → $270-$630 estimated (AWS) -Confidence: billing_inferred +How chosen: Estimated from billing only (JSON: billing_inferred) SKU Hints: CPU Allocation Time, Memory Allocation Time Steps: @@ -179,79 +205,82 @@ Action Required: ## Part 5: Success Criteria -Relaxed thresholds reflecting billing-only uncertainty. +Thresholds scaled by complexity tier. Simpler migrations have fewer unknowns and tighter targets. Use the **Success Criteria Scaling by Tier > Billing-Only Path** table in `steering/migration-complexity.md`. -| Criteria | Target | Note | -| --------------------------- | -------------------------- | ------------------------------------------------------------- | -| Performance within baseline | Within 20% of GCP | Wider than infra path (10%) due to sizing uncertainty | -| Monitoring stability | 48-hour watch period | Longer than infra path (24 hours) | -| Post-migration stability | 45-day observation | Longer than infra path (30 days) | -| Cost variance | Within 40% of mid estimate | Wider than infra path (15%) due to billing-only confidence | -| Data integrity | 100% | Same as infra path — no compromise on data | -| Service availability | 99% | Lower than infra path (99.9%) initially, improve after tuning | +| Criteria | Small | Medium | Large | +| --------------------------- | -------------------------- | -------------------------- | -------------------------- | +| Performance within baseline | Within 15% of GCP | Within 20% of GCP | Within 20% of GCP | +| Monitoring stability | 24-hour watch period | 48-hour watch period | 48-hour watch period | +| Post-migration stability | 14-day observation | 30-day observation | 45-day observation | +| Cost variance | Within 25% of mid estimate | Within 30% of mid estimate | Within 40% of mid estimate | +| Data integrity | 100% | 100% | 100% | +| Service availability | 99% | 99% | 99% | + +Apply the column matching the determined complexity tier. ## Part 6: Output Format -Generate `generation-billing.json` in `$MIGRATION_DIR/` with the following schema: +Generate `generation-billing.json` in `$MIGRATION_DIR/` with the following schema. + +The example below shows a **small** tier migration. Adjust `complexity_tier`, `complexity_inputs`, `total_weeks`, `approach`, phases, risk probabilities, success metric thresholds, and `estimated_total_effort_hours` according to the determined tier (see Part 2, Part 3, Part 5, and `steering/migration-complexity.md`). ```json { "phase": "generate", "generation_source": "billing_only", "confidence": "low", + "complexity_tier": "small", + "complexity_inputs": { + "service_count": 2, + "monthly_spend": 75.71, + "has_databases": false, + "has_stateful_storage": false, + "has_ai_workloads": false, + "availability": "single-az", + "compliance": "none", + "multi_region": false + }, "timestamp": "2026-02-26T14:30:00Z", "migration_plan": { - "total_weeks": 15, - "approach": "conservative_with_discovery", + "total_weeks": 4, + "approach": "compressed", "phases": [ { - "name": "Discovery Refinement", - "weeks": "1-4", + "name": "Discovery + Provisioning", + "weeks": "1", "key_activities": [ - "Manual infrastructure audit", - "Dependency mapping", - "Configuration documentation", - "Design refinement" - ], - "note": "Extended discovery to compensate for missing IaC data" - }, - { - "name": "Service Migration", - "weeks": "5-9", - "key_activities": [ - "AWS infrastructure provisioning", - "Application deployment", - "Integration testing", - "Data migration dry run" + "Quick infrastructure audit", + "AWS provisioning", + "IAM and security configuration" ] }, { - "name": "Parallel Run", - "weeks": "10-12", + "name": "Deploy + Test", + "weeks": "2", "key_activities": [ - "Dual environment operation", - "Performance comparison", - "Cost validation", - "Confidence building" + "Application deployment", + "Functional and integration testing", + "Cost validation" ] }, { - "name": "Cutover and Validation", - "weeks": "13-15", + "name": "Cutover + Validation", + "weeks": "3-4", "key_activities": [ - "DNS switch", - "48-hour intensive monitoring", - "Stabilization" + "DNS switch during maintenance window", + "24-hour intensive monitoring", + "Stabilization and GCP teardown planning" ] } ], "services": [ { - "gcp_service": "Cloud Run", - "aws_service": "Fargate", - "monthly_cost_gcp": 450.00, - "estimated_cost_aws_mid": 450.00, + "gcp_service": "Compute Engine", + "aws_service": "EC2", + "monthly_cost_gcp": 75.46, + "estimated_cost_aws_mid": 75.46, "confidence": "billing_inferred", + "human_expertise_required": false, "unknowns": ["instance sizing", "scaling config"] } ] @@ -259,22 +288,22 @@ Generate `generation-billing.json` in `$MIGRATION_DIR/` with the following schem "risks": [ { "category": "incorrect_sizing", - "probability": "high", + "probability": "low", "impact": "high", - "mitigation": "Extended discovery phase; right-size after parallel run", - "phase_affected": "Discovery Refinement" + "mitigation": "Discovery audit; right-size after validation", + "phase_affected": "Discovery + Provisioning" } ], "success_metrics": { - "performance_threshold": "within 20% of GCP baseline", - "monitoring_period_hours": 48, - "stability_period_days": 45, - "cost_variance_threshold": "within 40% of mid estimate", + "performance_threshold": "within 15% of GCP baseline", + "monitoring_period_hours": 24, + "stability_period_days": 14, + "cost_variance_threshold": "within 25% of mid estimate", "data_integrity": "100%", "availability_target": "99%" }, "recommendation": { - "approach": "Conservative migration with extended discovery", + "approach": "Compressed migration", "confidence": "low", "iac_discovery_offered": true, "note": "For tighter estimates and a shorter timeline, provide Terraform files and re-run discovery.", @@ -283,7 +312,7 @@ Generate `generation-billing.json` in `$MIGRATION_DIR/` with the following schem "Missing dependency information", "Cost variance due to unknown sizing" ], - "estimated_total_effort_hours": 720 + "estimated_total_effort_hours": 60 } } ``` @@ -293,16 +322,28 @@ Generate `generation-billing.json` in `$MIGRATION_DIR/` with the following schem - `phase` is `"generate"` - `generation_source` is `"billing_only"` - `confidence` is `"low"` -- `migration_plan.total_weeks` is 12-18 (conservative range) -- `migration_plan.phases` includes Discovery Refinement as first phase +- `complexity_tier` is one of `"small"`, `"medium"`, `"large"` +- `complexity_inputs` object is present with all required fields (service_count, monthly_spend, has_databases, has_stateful_storage, has_ai_workloads, availability, compliance, multi_region) +- `migration_plan.total_weeks` is within the tier's allowed range: small 2-4, medium 6-10, large 12-18 +- `migration_plan.approach` matches tier: small = `"compressed"`, medium = `"standard_with_discovery"`, large = `"conservative_with_discovery"` +- `migration_plan.phases` stage names match the tier template from Part 2 - `migration_plan.services` covers every service from `aws-design-billing.json` -- `risks` array has at least 4 entries (more than infra path, reflecting higher uncertainty) -- Each risk `probability` is appropriately elevated (most are "medium" or "high") -- `success_metrics` has relaxed thresholds compared to infrastructure path +- `risks` array has at least 2 entries (small), 4 entries (medium), or 5 entries (large) +- Each risk `probability` matches the tier column from Part 3 +- `success_metrics` thresholds match the tier column from Part 5 - `recommendation.iac_discovery_offered` is `true` - `recommendation.confidence` is `"low"` +- `recommendation.estimated_total_effort_hours` is within the tier's range from `migration-complexity.md` - Output is valid JSON +## Completion Handoff Gate (Fail Closed) + +Before returning control to `generate.md`, require: + +- `generation-billing.json` exists and passes the Output Validation Checklist above. + +If this gate fails: STOP and output: "generate-billing did not produce a valid `generation-billing.json`; do not continue Generate Stage 2." + ## Generate Phase Integration The parent orchestrator (`generate.md`) uses `generation-billing.json` to: diff --git a/migration-to-aws/steering/generate-infra.md b/migration-to-aws/steering/generate-infra.md index 81a7a232..15ff04da 100644 --- a/migration-to-aws/steering/generate-infra.md +++ b/migration-to-aws/steering/generate-infra.md @@ -17,14 +17,49 @@ If any required file is missing: **STOP**. Output: "Missing required artifact: [ ## Part 1: Migration Timeline -Build an 8-12 week migration timeline based on: +> **Before building the timeline, load `steering/migration-complexity.md` and classify the migration tier using inputs from `aws-design.json`, `gcp-resource-clusters.json`, `estimation-infra.json`, and `preferences.json`.** + +Build a timeline using the **Infrastructure Path** ranges for the determined tier. Also consider: - **Cluster count** from `gcp-resource-clusters.json` — more clusters = longer infrastructure phase - **Dependency depth** from `creation_order` — deeper graphs need more sequential work - **Service complexity** from `aws-design.json` — databases and stateful services take longer - **Cutover strategy** from `preferences.json` — maintenance window vs. blue-green affects timeline -### Stage 1: Setup (Weeks 1-2) +### Small Tier (3-6 weeks) + +Compressed setup. PoC is a 2-day smoke test integrated into deployment. No data migration stage (small tier excludes databases by definition). + +#### Stage 1: Setup (Week 1) + +- Provision VPC, subnets, routing, IAM, monitoring baseline +- Set up CI/CD pipeline for Terraform deployments + +#### Stage 2: Deploy + Smoke Test (Week 2) + +- Deploy all clusters (few services, shallow dependency graph) +- Validate application connectivity and performance +- Run integration tests +- Confirm cost tracking matches `estimation-infra.json` projections +- **Go/No-Go checkpoint**: If smoke test fails acceptance criteria, stop and reassess + +#### Stage 3: Cutover (Weeks 3-4) + +- Execute cutover per `preferences.json` cutover_strategy +- Monitor for 24-48 hours post-cutover +- Keep GCP resources running as hot standby + +#### Stage 4: Validation + Cleanup (Weeks 5-6) + +- Monitor AWS performance for 1 week +- Compare actual costs against projections +- Begin GCP teardown planning (execute teardown after 2-week stability period) + +### Medium Tier (8-12 weeks) + +Standard phased plan. Apply the data-migration skip rule below. + +#### Stage 1: Setup (Weeks 1-2) - Finalize AWS account structure and billing alerts - Provision foundational infrastructure: VPC, subnets, routing, NAT gateways @@ -33,7 +68,7 @@ Build an 8-12 week migration timeline based on: - Set up CI/CD pipeline for Terraform deployments - Configure monitoring baseline (CloudWatch, alarms) -### Stage 2: Proof of Concept (Weeks 3-4) +#### Stage 2: Proof of Concept (Weeks 3-4) - Deploy the **shallowest-depth cluster** (lowest `creation_order_depth`) to AWS - Validate application connectivity and performance @@ -42,7 +77,7 @@ Build an 8-12 week migration timeline based on: - Confirm cost tracking matches `estimation-infra.json` projections - **Go/No-Go checkpoint**: If PoC fails acceptance criteria, stop and reassess -### Stage 3: Infrastructure Deployment (Weeks 5-7) +#### Stage 3: Infrastructure Deployment (Weeks 5-7) - Deploy remaining clusters in `creation_order` sequence (depth-first) - For each cluster: @@ -53,10 +88,10 @@ Build an 8-12 week migration timeline based on: - Implement monitoring and logging per cluster - Establish backup and restore procedures -### Stage 4: Data Migration (Weeks 8-9) +#### Stage 4: Data Migration (Weeks 8-9) **Include this phase ONLY if `aws-design.json` contains database or storage resources -(see resource detection rules in generate-artifacts-scripts.md Step 4).** +(see resource detection rules in generate-artifacts-scripts.md Step 1).** **If no data migration is needed, compress the timeline: move Cutover to Weeks 8-9 and Validation to Week 10. Reduce `total_weeks` accordingly.** @@ -70,7 +105,7 @@ and Validation to Week 10. Reduce `total_weeks` accordingly.** - **Secrets**: Migrate secrets from Secret Manager to AWS Secrets Manager - Establish dual-write pattern for production data -### Stage 5: Cutover (Weeks 10-11, or Weeks 8-9 if Stage 4 skipped) +#### Stage 5: Cutover (Weeks 10-11, or Weeks 8-9 if Stage 4 skipped) - Pre-cutover validation: - All clusters deployed and healthy on AWS @@ -83,7 +118,7 @@ and Validation to Week 10. Reduce `total_weeks` accordingly.** - Monitor for 24-48 hours post-cutover - Keep GCP resources running as hot standby -### Stage 6: Validation and Cleanup (Week 12) +#### Stage 6: Validation and Cleanup (Week 12) - Monitor AWS performance for 1 full week - Compare actual costs against `estimation-infra.json` projections @@ -91,6 +126,63 @@ and Validation to Week 10. Reduce `total_weeks` accordingly.** - Document any drift or issues - Begin GCP teardown planning (execute teardown after 2-week stability period) +### Large Tier (12-16 weeks) + +Extended timeline for complex migrations. Extra time for multi-cluster orchestration, large data volumes, and extended validation. Apply the data-migration skip rule from Stage 4. + +#### Stage 1: Setup (Weeks 1-2) + +Same as Medium. + +#### Stage 2: Proof of Concept (Weeks 3-4) + +Same as Medium. + +#### Stage 3: Infrastructure Deployment (Weeks 5-8) + +Extended by 1 week for additional clusters and complex cross-cluster networking. + +- Deploy remaining clusters in `creation_order` sequence (depth-first) +- For each cluster: + - Deploy infrastructure per `aws-design.json` resource mappings + - Configure cross-cluster networking and security groups + - Validate service health checks + - Run integration tests +- Implement monitoring and logging per cluster +- Establish backup and restore procedures + +#### Stage 4: Data Migration (Weeks 9-11) + +**Include this phase ONLY if `aws-design.json` contains database or storage resources.** +**If no data migration is needed, compress the timeline: move Cutover to Weeks 9-10 +and Validation to Weeks 11-12. Reduce `total_weeks` accordingly.** + +Extended for large data volumes and complex replication topologies. + +- **Databases**: Set up continuous replication (Cloud SQL to RDS/Aurora) + - Initial full snapshot transfer + - Enable ongoing replication (DMS or native replication) + - Validate data integrity (row counts, checksums) +- **Object storage**: Sync GCS buckets to S3 + - Use AWS DataSync or `gsutil`/`aws s3 sync` for bulk transfer + - Enable ongoing sync for new objects +- **Secrets**: Migrate secrets from Secret Manager to AWS Secrets Manager +- Establish dual-write pattern for production data + +#### Stage 5: Cutover (Weeks 12-13) + +Same structure as Medium, adjusted week numbers. + +#### Stage 6: Validation and Cleanup (Weeks 14-16) + +Extended monitoring before GCP teardown. + +- Monitor AWS performance for 2 full weeks +- Compare actual costs against `estimation-infra.json` projections +- Run final data integrity checks +- Document any drift or issues +- Begin GCP teardown planning (execute teardown after 2-week stability period) + ## Part 2: Risk Assessment Build a risk matrix from the discovered infrastructure and migration complexity. @@ -111,7 +203,6 @@ For each risk, assess: | Performance regression on AWS | medium | high | PoC testing (Weeks 3-4); load testing (Week 5); performance baseline comparison | | Extended downtime during cutover | medium | high | Practice cutover in staging; automate DNS switch; rollback procedure on standby | | Cost overrun vs estimates | medium | medium | Set billing alerts at 80% and 100% of projected; weekly cost review | -| Team capacity constraints | medium | medium | Allocate 2 FTE engineers dedicated for 12 weeks; identify backup resources | | Cross-region latency | low | medium | Validate latency in PoC phase; consider same-region deployment for latency-sensitive services | | Terraform state corruption | low | high | Remote state with locking (S3 + DynamoDB); state backups before each apply | @@ -123,6 +214,7 @@ Add additional risks based on discovered infrastructure: - If **stateful services** (Redis, Elasticsearch): Add "State migration data loss" (low/critical) - If **multiple regions** in GCP: Add "Cross-region dependency during migration" (medium/medium) - If **AI workloads** coexist: Add "Model performance drift on Bedrock" (medium/high) +- If **BigQuery** resources exist (`google_bigquery_*` or `aws_service` is **`Deferred — specialist engagement`** in `aws-design.json`): Add "BigQuery migration complexity" (high/high) with mitigation: "**No automated AWS analytics target** — engage **AWS account team** and/or **data analytics migration partner** before architecture or cost commitments; plugin does not prescribe Athena/Redshift/Glue." ## Part 3: Success Metrics @@ -204,12 +296,6 @@ Initiate rollback if ANY of: **R** = Responsible, **A** = Accountable, **C** = Consulted, **I** = Informed -### Staffing Estimate - -- **Minimum**: 2 FTE engineers for 12 weeks -- **Recommended**: 3 FTE (1 infra, 1 database/data, 1 app/QA) for 12 weeks -- **With AI track**: Add 1 FTE ML engineer for weeks 3-8 - ## Part 6: Go/No-Go Framework Each phase gate requires explicit approval before proceeding. @@ -284,6 +370,17 @@ Generate `generation-infra.json` in `$MIGRATION_DIR/` with the following schema: { "phase": "generate", "generation_source": "infrastructure", + "complexity_tier": "medium", + "complexity_inputs": { + "service_count": 5, + "monthly_spend": 3500.00, + "has_databases": true, + "has_stateful_storage": false, + "has_ai_workloads": false, + "availability": "single-az", + "compliance": "none", + "multi_region": false + }, "timestamp": "2026-02-26T14:30:00Z", "migration_plan": { "total_weeks": 12, @@ -304,7 +401,8 @@ Generate `generation-infra.json` in `$MIGRATION_DIR/` with the following schema: "migration_week": 5, "cluster_id": "compute_cloudrun_us-central1_001", "estimated_effort_hours": 40, - "data_migration_required": false // derive from resource detection flags (has_databases, has_storage) + "data_migration_required": false, // derive from resource detection flags (has_databases, has_storage) + "human_expertise_required": false // propagate from aws-design.json; true for BigQuery mappings } ], "critical_path": [ @@ -356,12 +454,6 @@ Generate `generation-infra.json` in `$MIGRATION_DIR/` with the following schema: "post_cutover_rto": "2-4 hours", "rollback_window": "Reversible until 48 hours post-DNS cutover" }, - "team_roles": { - "minimum_fte": 2, - "recommended_fte": 3, - "duration_weeks": 12, - "roles": ["Migration Lead", "Infrastructure Engineer", "Database Engineer"] - }, "go_no_go_criteria": [ { "gate": "G1", @@ -387,20 +479,29 @@ Generate `generation-infra.json` in `$MIGRATION_DIR/` with the following schema: - `phase` is `"generate"` - `generation_source` is `"infrastructure"` -- `migration_plan.total_weeks` is a positive integer (8-16 range) -- `migration_plan.phases` array has at least 4 entries (Setup, PoC, Infrastructure, Cutover, Validation — plus Data Migration if data resources exist) +- `complexity_tier` is one of `"small"`, `"medium"`, `"large"` +- `complexity_inputs` object is present with all required fields (service_count, monthly_spend, has_databases, has_stateful_storage, has_ai_workloads, availability, compliance, multi_region) +- `migration_plan.total_weeks` is within the tier's allowed range: small 3-6, medium 8-12, large 12-16 +- `migration_plan.phases` array has at least 4 entries; stage names match the tier template from Part 1 - `migration_plan.services` covers every service from `aws-design.json` - `migration_plan.critical_path` is non-empty - `migration_plan.dependencies` reflect `gcp-resource-clusters.json` creation_order - `risks` array has at least 3 entries with probability, impact, mitigation - `success_metrics` has both `per_service` and `overall` sections - `rollback_procedures` has trigger conditions and RTO values -- `team_roles` has minimum and recommended FTE counts -- `go_no_go_criteria` has at least 4 gates +- `go_no_go_criteria` has at least 2 gates (small) or 4 gates (medium/large) - `post_migration` specifies monitoring duration and teardown timing - All cluster IDs reference valid clusters from `gcp-resource-clusters.json` - Output is valid JSON +## Completion Handoff Gate (Fail Closed) + +Before returning control to `generate.md`, require: + +- `generation-infra.json` exists and passes the Output Validation Checklist above. + +If this gate fails: STOP and output: "generate-infra did not produce a valid `generation-infra.json`; do not continue Generate Stage 2." + ## Generate Phase Integration The parent orchestrator (`generate.md`) uses `generation-infra.json` to: diff --git a/migration-to-aws/steering/generate.md b/migration-to-aws/steering/generate.md index ccc7d72d..9062b95e 100644 --- a/migration-to-aws/steering/generate.md +++ b/migration-to-aws/steering/generate.md @@ -13,7 +13,8 @@ Both stages must complete for the phase to succeed. ## Prerequisites -Read `$MIGRATION_DIR/preferences.json`. If missing: **STOP**. Output: "Phase 2 (Clarify) not completed. Run Phase 2 first." +1. Read `$MIGRATION_DIR/.phase-status.json`. If missing, invalid, or `phases.clarify` is not exactly `"completed"`: **STOP**. Output: "Phase 2 (Clarify) not completed or phase state is missing/invalid. Complete Clarify before Generate." +2. Read `$MIGRATION_DIR/preferences.json`. If missing: **STOP**. Output: "Phase 2 (Clarify) not completed. Run Phase 2 first." Check which estimation artifacts exist in `$MIGRATION_DIR/`: @@ -25,6 +26,20 @@ If **none** of these estimation artifacts exist: **STOP**. Output: "No estimatio ## Stage 1: Migration Planning +**Dirty-state tracking**: Before producing any Stage 1 outputs, set `dirty_state` in `.phase-status.json`: + +```json +"dirty_state": { + "phase": "generate", + "stage": "stage_1_planning", + "started_at": "", + "partial_outputs": [], + "missing_outputs": ["generation-infra.json", "generation-ai.json", "generation-billing.json"] +} +``` + +Trim `missing_outputs` to only the artifacts expected for the active routes. Update `partial_outputs` and `missing_outputs` after each sub-file completes. + Route based on which estimation artifacts exist. Multiple paths can run independently. ### Infrastructure Migration Plan @@ -55,6 +70,20 @@ Produces: `generation-billing.json` **MUST proceed only after Stage 1 completes.** Route based on generation plans + design artifacts. +**Dirty-state tracking**: Before producing any Stage 2 outputs, update `dirty_state` in `.phase-status.json`: + +```json +"dirty_state": { + "phase": "generate", + "stage": "stage_2_artifacts", + "started_at": "", + "partial_outputs": ["generation-infra.json"], + "missing_outputs": ["terraform/", "scripts/", "MIGRATION_GUIDE.md", "README.md"] +} +``` + +Carry forward `partial_outputs` from Stage 1. Trim `missing_outputs` to only the artifacts expected for the active routes plus mandatory docs. Update after each sub-file completes. + ### Infrastructure Artifacts IF `generation-infra.json` AND `aws-design.json` exist: @@ -84,7 +113,7 @@ IF `generation-billing.json` AND `aws-design-billing.json` exist: Produces: `terraform/skeleton.tf` (with TODO markers) -### Documentation (ALWAYS runs last) +### Documentation (ALWAYS runs after artifact generation) AFTER all above artifact generation sub-files complete: @@ -92,24 +121,75 @@ AFTER all above artifact generation sub-files complete: Produces: `MIGRATION_GUIDE.md`, `README.md` +### HTML Report (ALWAYS runs last, after documentation) + +AFTER generate-artifacts-docs.md completes: + +> Load `generate-artifacts-report.md` + +Produces: `migration-report.html` + +**Validation gate:** Report generation runs `steering/validate-artifacts.md` first. If validation emits `GATE_FAIL`: log the failure to the user, **do not write** `migration-report.html`, and continue to Phase Completion (report is optional output; validation failure is not a silent skip). Do **NOT** patch artifacts to pass validation. + ## Phase Completion +Load `steering/handoff-gates.md`. **Re-read from disk** before checking. + Verify both stages are complete: -1. **Stage 1**: At least one `generation-*.json` file exists -2. **Stage 2**: At least one artifact directory or file was produced, plus documentation +1. **Stage 1 route gates (fail closed)**: + - If `estimation-infra.json` exists -> require `generation-infra.json` + - If `estimation-ai.json` exists -> require `generation-ai.json` + - If `estimation-billing.json` exists -> require `generation-billing.json` +2. **Stage 2 route gates (fail closed)**: + - If infra artifact route is active (`generation-infra.json` AND `aws-design.json`) -> require `terraform/`, `scripts/`, and `validation-report.json` (with `status` in `{passed, passed_degraded_offline, skipped_user_continue}`) + - If AI artifact route is active (`generation-ai.json` AND `aws-design-ai.json`) -> require `ai-migration/` + - If billing artifact route is active (`generation-billing.json` AND `aws-design-billing.json`) -> require `terraform/skeleton.tf` +3. **Documentation gate (always)**: + - Require `MIGRATION_GUIDE.md` and `README.md` +4. If any active route is missing expected outputs: Emit `GATE_FAIL | phase=generate | field= | reason=missing`. **Do NOT modify artifacts.** STOP — do not mark phase complete. + +**On PASS:** Emit `HANDOFF_OK | phase=generate | artifacts=`. -Use the Phase Status Update Protocol (Write tool) to write `.phase-status.json` with `phases.generate` set to `"completed"` — **in the same turn** as the summary below. +After `HANDOFF_OK`, use the Phase Status Update Protocol (read-merge-write) to update `.phase-status.json` — **in the same turn** as the summary below: + +- Set `phases.generate` to `"completed"` +- Set `current_phase` to `"complete"` ## Summary -Present final summary to user: +**Use structured completion reporting** (see `steering/validate-artifacts.md` Section 3). Present final summary to user: + +``` +Phase 5 (Generate) complete. + +✓ Produced: + - generation-infra.json: [X]-week migration plan + - terraform/: [N] files (list key files) + - scripts/: [N] files + - MIGRATION_GUIDE.md: [N] sections + - README.md: artifact catalog + quick start + - migration-report.html: executive summary + - migration-report.pdf: PDF version [or "skipped — no converter available"] + +⊘ Skipped (not applicable): + - [artifact]: [reason] + +⚠ Skipped (non-blocking failure): + - migration-report.html: [failure reason] ← only if report generation failed +``` + +After the structured block, include: 1. **Plans generated** — List all `generation-*.json` files produced -2. **Artifacts generated** — List all directories and files created (terraform/, scripts/, ai-migration/, MIGRATION_GUIDE.md, README.md) -3. **Key timelines** — Highlight migration timeline from the generation plans -4. **Key risks** — Highlight top risks from the generation plans -5. **TODO markers** — Note any TODO markers in generated artifacts that require manual attention -6. **Next steps** — Recommend reviewing generated artifacts, customizing TODO sections, and beginning migration execution +2. **Artifacts generated** — List all directories and files created (terraform/, scripts/, ai-migration/, MIGRATION_GUIDE.md, README.md). Include `migration-report.html` only if it exists. +3. **Validation status** — If `$MIGRATION_DIR/validation-report.json` exists, report its `status` field (`passed`, `passed_degraded_offline`, or `skipped_user_continue`). If `status == "passed_degraded_offline"`, add: "Provider registry was unreachable; `terraform validate` was skipped. Re-run `terraform init && terraform validate` from a network-connected shell to complete validation." +4. **Key timelines** — Highlight migration timeline from the generation plans +5. **Key risks** — Highlight top risks from the generation plans +6. **TODO markers** — Note any TODO markers in generated artifacts that require manual attention +7. **Next steps** — Recommend reviewing generated artifacts, customizing TODO sections, and beginning migration execution + +Output to user: -Output to user: "Migration artifact generation complete. All phases of the GCP-to-AWS migration analysis are complete." +- If `migration-report.html` exists: "Migration artifact generation complete. All phases of the GCP-to-AWS migration analysis are complete. Your migration report is ready at $MIGRATION_DIR/migration-report.html" +- If `migration-report.html` is missing: "Migration artifact generation complete. All phases of the GCP-to-AWS migration analysis are complete. Markdown documentation is available at $MIGRATION_DIR/MIGRATION_GUIDE.md and $MIGRATION_DIR/README.md. (HTML report generation is optional and non-blocking.)" diff --git a/migration-to-aws/steering/handoff-gates.md b/migration-to-aws/steering/handoff-gates.md new file mode 100644 index 00000000..8d55b022 --- /dev/null +++ b/migration-to-aws/steering/handoff-gates.md @@ -0,0 +1,84 @@ +# Handoff Gates (Fail Closed) + +All phases MUST pass a handoff gate before marking `phases.` as `"completed"` in `.phase-status.json`. Load this file when executing any phase completion step. + +--- + +## Gate protocol + +1. **Re-read from disk** — Open each required artifact with the Read tool from `$MIGRATION_DIR/`. Do not rely on chat memory or prior summaries. +2. **Check every item** in the phase-specific checklist (defined in that phase's orchestrator or sub-file). +3. **On failure** — emit exactly one line per failure using the parseable format below. Do **NOT** mark the phase complete. Do **NOT** advance `current_phase`. +4. **On success** — emit one success line, then update `.phase-status.json` in the same turn. + +### Parseable failure format (required) + +``` +GATE_FAIL | phase= | field= | reason= +``` + +Examples: + +``` +GATE_FAIL | phase=estimate | field=recommendation.path | reason=missing +GATE_FAIL | phase=clarify | field=design_constraints.availability.value | reason=missing +GATE_FAIL | phase=discover | field=preferences.json | reason=stale_downstream +``` + +### Success format (required) + +``` +HANDOFF_OK | phase= | artifacts= +``` + +Example: + +``` +HANDOFF_OK | phase=estimate | artifacts=estimation-infra.json +``` + +--- + +## On GATE_FAIL — user action only (CRITICAL) + +When any gate check fails: + +1. Output the `GATE_FAIL` line(s) to the user in plain language (what is missing and which phase to re-run). +2. **Do NOT modify artifacts** to pass the gate (no inventing `recommendation`, no defaulting `availability`, no patching JSON inline). +3. **Do NOT continue** to the next phase. +4. Tell the user: **"Re-run Phase N (phase name) to produce the missing field, then continue."** + +Patching artifacts to satisfy a gate defeats fail-closed validation and produces reports that look complete but are not. + +--- + +## Phase re-entry (idempotent runs) + +| Situation | Rule | +| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Re-run **Discover** after **Clarify** completed | **STOP** unless user explicitly confirms. Emit `GATE_FAIL \| phase=discover \| field=preferences.json \| reason=stale_downstream`. Downstream artifacts may be stale. | +| Re-run **Clarify** after **Design** completed | Same — confirm with user; Design/Estimate may need re-run. | +| Re-run **Estimate** after **Generate** started | Same — confirm with user; report and Terraform may be stale. | +| Re-run a phase **before** downstream phase completed | Allowed. Overwrite that phase's artifacts; downstream phases remain `"pending"` or must be re-run. | + +When user confirms intentional re-run: set downstream phases back to `"pending"` in `.phase-status.json` before proceeding. + +--- + +## Phase-specific checklists (summary) + +Detailed checklists live in each phase file. Minimum gates: + +| Phase | Key checks | +| ------------ | ----------------------------------------------------------------------------------------------------------------- | +| **discover** | At least one discovery artifact; `migration-preview.json` when any artifact exists; route output gates (existing) | +| **clarify** | `preferences.json` valid; Cloud SQL in inventory → `design_constraints.availability.value` set | +| **design** | Active route artifacts present (existing gates) | +| **estimate** | Active route artifacts present; infra route → `recommendation.path` + non-empty `migrate_if` / `stay_if` | +| **generate** | Load `steering/validate-artifacts.md` before report; report pre-write sanity (see `generate-artifacts-report.md`) | + +--- + +## Orchestrator rule (POWER.md) + +The top-level skill MUST NOT load the next phase until the previous phase's output includes `HANDOFF_OK | phase=`. A phase completion message without `HANDOFF_OK` is not valid handoff. diff --git a/migration-to-aws/steering/migration-complexity.md b/migration-to-aws/steering/migration-complexity.md new file mode 100644 index 00000000..01b84809 --- /dev/null +++ b/migration-to-aws/steering/migration-complexity.md @@ -0,0 +1,234 @@ +# Migration Complexity Tiers + +Shared classification loaded by `generate-billing.md` and `generate-infra.md` to right-size migration timelines. The AI path (`generate-ai.md`) self-sizes from workload profiles and does not use this file. + +## Inputs + +Collect these values from prior-phase artifacts before classifying: + +| Input | Source Artifact | Key | +| -------------------- | ------------------------------------------------- | --------------------------------------------------------------------------------------- | +| Service count | `aws-design-billing.json` or `aws-design.json` | `metadata.total_services` | +| Monthly spend | `billing-profile.json` or `estimation-infra.json` | `summary.total_monthly_spend` or `current_costs.gcp_monthly` | +| Has databases | Design artifact `services[]` | `aws_service` in {RDS, Aurora, DynamoDB, ElastiCache, DocumentDB, MemoryDB, OpenSearch} | +| Has stateful storage | Design artifact `services[]` | `aws_service` in {EFS, FSx, S3} with replication or versioning hints in `sku_hints` | +| Has AI workloads | `estimation-ai.json` exists | File presence | +| Availability | `preferences.json` | `design_constraints.availability` | +| Compliance | `preferences.json` | `design_constraints.compliance` | +| Multi-region | Design artifact `services[]` | More than one distinct `aws_config.region` value | + +## Tier Definitions + +Evaluate from **Large down to Small**. The first tier whose condition matches is the result (highest-matching-tier wins). + +### Large + +ANY of the following: + +- Service count >= 9 +- Monthly spend > $10,000 +- Multi-region deployment (services span 2+ AWS regions) +- AI workloads coexist with infrastructure (`estimation-ai.json` exists AND (`estimation-infra.json` OR `estimation-billing.json`) also exists) +- Compliance requirements present (`compliance` is not empty/none) + +### Medium + +NOT Large, and ANY of the following: + +- Service count 4-8 +- Monthly spend $1,000-$10,000 +- Has databases +- Availability is `multi-az` + +### Small + +NOT Large, NOT Medium. Equivalently, ALL of: + +- Service count <= 3 +- Monthly spend < $1,000 +- No databases or stateful storage with replication +- No AI workloads alongside infrastructure +- Availability is `single-az` or unspecified +- No compliance requirements + +## Timeline Ranges + +### Billing-Only Path + +| Tier | Weeks | Effort Hours | Approach | +| ------ | ----- | ------------ | ----------------------------- | +| Small | 2-4 | 40-80 | `compressed` | +| Medium | 6-10 | 160-400 | `standard_with_discovery` | +| Large | 12-18 | 480-720 | `conservative_with_discovery` | + +### Infrastructure Path + +| Tier | Weeks | Effort Hours | Approach | +| ------ | ----- | ------------ | -------------------------- | +| Small | 3-6 | 80-160 | `compressed` | +| Medium | 8-12 | 240-480 | `phased_cluster_migration` | +| Large | 12-16 | 400-640 | `phased_cluster_migration` | + +## Stage Templates + +### Billing-Only Path + +#### Small (2-4 weeks) + +No parallel-run stage. Discovery and provisioning overlap. Cutover uses maintenance window. + +- **Stage 1: Discovery + Provisioning (Week 1)** + - Quick audit of GCP infrastructure (few services, low complexity) + - Provision AWS VPC, compute, and supporting resources + - Configure IAM roles and security groups +- **Stage 2: Deploy + Test (Week 2)** + - Deploy applications to AWS + - Run functional and integration tests + - Validate cost tracking against estimates +- **Stage 3: Cutover + Validation (Weeks 3-4)** + - Execute cutover during maintenance window (DNS switch) + - 24-hour intensive monitoring + - Stabilization and GCP teardown planning + +#### Medium (6-10 weeks) + +Shortened parallel run. Discovery takes 1-2 weeks instead of 4. + +- **Stage 1: Discovery Refinement (Weeks 1-2)** + - Audit current GCP infrastructure + - Document instance sizes, database configs, networking topology + - Map dependencies between services + - Refine AWS design based on discovered configurations +- **Stage 2: Service Migration (Weeks 3-5)** + - Provision AWS infrastructure + - Deploy applications and configure CI/CD + - Integration testing and data migration dry run +- **Stage 3: Parallel Run (Weeks 6-7)** + - Run both environments simultaneously + - Compare performance, reliability, and costs + - Validate data consistency +- **Stage 4: Cutover and Validation (Weeks 8-10)** + - Execute cutover (DNS switch, traffic migration) + - 48-hour intensive monitoring + - Stabilization and GCP teardown planning + +#### Large (12-18 weeks) + +Full conservative plan. Extended discovery, full parallel run. + +- **Stage 1: Discovery Refinement (Weeks 1-4)** + - Weeks 1-2: Manual infrastructure audit, dependency mapping, configuration documentation + - Weeks 3-4: Refine AWS design, re-estimate costs, identify services needing different AWS targets +- **Stage 2: Service Migration (Weeks 5-9)** + - Weeks 5-6: Provision AWS infrastructure (VPC, compute, databases, storage) + - Weeks 7-8: Deploy applications, set up CI/CD, migrate to staging + - Week 9: Integration testing, performance baseline, data migration dry run +- **Stage 3: Parallel Run (Weeks 10-12)** + - Run both GCP and AWS simultaneously + - Compare performance, reliability, and costs + - Validate data consistency between environments + - Monitor for 2+ weeks before cutover decision +- **Stage 4: Cutover and Validation (Weeks 13-15+)** + - Execute cutover (DNS switch, traffic migration) + - 48-hour intensive monitoring + - Stabilization and GCP teardown planning + +### Infrastructure Path + +#### Small (3-6 weeks) + +Compressed setup. PoC is a 2-day smoke test, not a 2-week phase. No data migration stage (small tier excludes databases by definition). + +- **Stage 1: Setup (Week 1)** + - Provision VPC, subnets, IAM, monitoring baseline + - Set up CI/CD pipeline for Terraform +- **Stage 2: Deploy + Smoke Test (Week 2)** + - Deploy all clusters (few services, shallow dependency graph) + - Run integration tests and validate connectivity + - Confirm cost tracking matches estimates + - Go/No-Go checkpoint +- **Stage 3: Cutover (Weeks 3-4)** + - Execute cutover per `preferences.json` strategy + - 24-48 hour monitoring + - Keep GCP as hot standby +- **Stage 4: Validation + Cleanup (Weeks 5-6)** + - Monitor AWS performance for 1 week + - Compare costs to projections + - Begin GCP teardown planning + +#### Medium (8-12 weeks) + +Standard phased plan. Same as the existing `generate-infra.md` default stages. Apply the existing data-migration skip rule: if no databases/storage, compress Cutover to Weeks 8-9 and Validation to Week 10. + +#### Large (12-16 weeks) + +Extended infrastructure deployment. Extra time for complex dependency graphs, multi-cluster orchestration, and extended parallel validation. + +- **Stage 1: Setup (Weeks 1-2)** — same as medium +- **Stage 2: Proof of Concept (Weeks 3-4)** — same as medium +- **Stage 3: Infrastructure Deployment (Weeks 5-8)** — extended by 1 week for additional clusters and cross-cluster networking +- **Stage 4: Data Migration (Weeks 9-11)** — extended for large data volumes and complex replication topologies (skip if no databases/storage) +- **Stage 5: Cutover (Weeks 12-13)** — same structure, adjusted week numbers +- **Stage 6: Validation and Cleanup (Weeks 14-16)** — extended monitoring before GCP teardown + +## Risk Scaling by Tier + +Risk probabilities should be adjusted based on complexity tier: + +| Risk Category | Small | Medium | Large | +| -------------------------------- | ----------- | ------ | ------ | +| Incorrect service sizing | low | medium | high | +| Missing dependencies | low | medium | high | +| Data migration complexity | n/a (no DB) | medium | high | +| Cost overrun | low | medium | high | +| Performance regression | low | medium | medium | +| Timeline overrun | low | medium | high | +| Unmapped services block progress | low | medium | medium | + +## Success Criteria Scaling by Tier + +Tighter thresholds for simpler migrations (fewer unknowns, less variance). + +### Billing-Only Path + +| Criteria | Small | Medium | Large | +| --------------------------- | -------------------------- | -------------------------- | -------------------------- | +| Performance within baseline | Within 15% of GCP | Within 20% of GCP | Within 20% of GCP | +| Monitoring stability | 24-hour watch period | 48-hour watch period | 48-hour watch period | +| Post-migration stability | 14-day observation | 30-day observation | 45-day observation | +| Cost variance | Within 25% of mid estimate | Within 30% of mid estimate | Within 40% of mid estimate | +| Data integrity | 100% | 100% | 100% | +| Service availability | 99% | 99% | 99% | + +### Infrastructure Path + +| Criteria | Small | Medium | Large | +| --------------------------- | -------------------------- | -------------------------- | -------------------------- | +| Performance within baseline | Within 10% of GCP | Within 10% of GCP | Within 10% of GCP | +| Monitoring stability | 24-hour watch period | 24-hour watch period | 48-hour watch period | +| Post-migration stability | 14-day observation | 30-day observation | 30-day observation | +| Cost variance | Within 10% of mid estimate | Within 15% of mid estimate | Within 15% of mid estimate | +| Data integrity | 100% | 100% | 100% | +| Service availability | 99.9% | 99.9% | 99.9% | + +## Output + +After classification, the consuming generate file must include a `complexity_tier` field in its output JSON: + +```json +{ + "complexity_tier": "small", + "complexity_inputs": { + "service_count": 2, + "monthly_spend": 75.71, + "has_databases": false, + "has_stateful_storage": false, + "has_ai_workloads": false, + "availability": "single-az", + "compliance": "none", + "multi_region": false + } +} +``` + +These fields go at the top level of the generation JSON (alongside `phase`, `generation_source`, etc.). diff --git a/migration-to-aws/steering/pricing-fallback.md b/migration-to-aws/steering/pricing-fallback.md new file mode 100644 index 00000000..c62a4721 --- /dev/null +++ b/migration-to-aws/steering/pricing-fallback.md @@ -0,0 +1,37 @@ +# Pricing Fallback + +> Loaded by `ai-migration-guardrails.md` as the tertiary pricing source when both +> `cached-prices.md` (primary) and the `awspricing` MCP server (secondary) are unavailable. + +## When This File Is Used + +Use this fallback **only** when: + +1. `cached-prices.md` is stale (>30 days since `Last updated`) **and** +2. The `awspricing` MCP `get_pricing` call fails or times out + +## Fallback Behavior + +When this fallback is active: + +1. Set `pricing_source: "unavailable"` on all AI model cost estimates in `estimation-ai.json` +2. Surface a visible warning to the user: + + > ⚠️ **Pricing data unavailable.** The pricing cache is stale and the live pricing API could not be reached. AI cost estimates in this report are omitted. Re-run the Estimate phase when connectivity is restored, or check [aws.amazon.com/bedrock/pricing](https://aws.amazon.com/bedrock/pricing) manually. + +3. Do **not** fabricate or guess token prices — emit `null` for all per-token cost fields +4. Infrastructure pricing (Fargate, RDS, S3, etc.) from `cached-prices.md` may still be used — infrastructure prices change rarely and the cache remains reliable beyond 30 days + +## MCP Retry Path + +Before falling back to this file, attempt the MCP call with one retry: + +``` +get_pricing(service="bedrock", model_id="", region="") +``` + +If the retry also fails, proceed with `pricing_source: "unavailable"` as above. + +## Do Not Guess + +Never emit fabricated prices. A missing cost estimate is better than a wrong one. diff --git a/migration-to-aws/steering/retarget-gotchas.md b/migration-to-aws/steering/retarget-gotchas.md new file mode 100644 index 00000000..5da4d40d --- /dev/null +++ b/migration-to-aws/steering/retarget-gotchas.md @@ -0,0 +1,152 @@ +# Retarget Gotchas: Framework-Specific Migration Pitfalls + +> Loaded by `design-ai.md` Step 0.6 when `agentic_profile.is_agentic == true` AND `ai_constraints.agentic.migration_approach == "retarget"`. +> Also useful context for `generate-ai.md` when generating retarget migration plans. + +These are real issues startups hit when keeping their existing framework and swapping the model layer to Bedrock. A simple "swap `ChatOpenAI` for `ChatBedrock`" understates the work. This file documents what actually breaks and how to fix it. + +--- + +## LangChain / LangGraph + +### Tool calling schema differences + +**Problem:** OpenAI and Claude handle tool calls differently. OpenAI returns `tool_calls` as a list with `function.arguments` as a JSON string. Claude via Bedrock returns tool use blocks with `input` as a parsed object. LangChain's `ChatBedrock` abstracts most of this, but edge cases leak through. + +**What breaks:** + +- Custom output parsers that assume OpenAI's `tool_calls` format +- Code that accesses `response.additional_kwargs["tool_calls"]` directly instead of using LangChain's `ToolMessage` abstraction +- Parallel tool calls: OpenAI supports multiple tool calls in one response; Claude does too, but some Bedrock models (Nova, Llama) may not + +**Fix:** Use LangChain's standardized tool calling interface (`bind_tools()`, `ToolMessage`). Don't access raw response format. Test parallel tool calls explicitly with your target Bedrock model. + +### `with_structured_output` behavior + +**Problem:** `ChatOpenAI.with_structured_output(schema)` uses OpenAI's native JSON mode or function calling. `ChatBedrock.with_structured_output(schema)` uses Claude's tool use to enforce structure. The output is the same, but error handling differs — Claude may refuse to fill required fields if the prompt doesn't provide enough context, returning a partial object instead of raising an error. + +**Fix:** Validate structured output after every call. Add explicit instructions in the prompt for each required field. Test with edge-case inputs where the model might not have enough context to fill all fields. + +### Streaming differences + +**Problem:** `ChatBedrock` streaming works but the chunk format differs from `ChatOpenAI`. Code that processes `AIMessageChunk` objects and checks for `tool_call_chunks` may need adjustment. + +**Fix:** Use LangChain's `astream_events` API (v2) which normalizes events across providers. Avoid directly inspecting chunk internals. + +### Async behavior + +**Problem:** `ChatBedrock` wraps synchronous boto3 calls in threads for async. It's not truly async like `ChatOpenAI`'s native async client. Under high concurrency, this can cause thread pool exhaustion. + +**Fix:** For high-concurrency LangGraph deployments, increase the thread pool size or use `ChatBedrockConverse` (the newer Converse API-based client) which has better async support. Monitor thread usage in production. + +### LangGraph checkpointer compatibility + +**Problem:** LangGraph checkpoints serialize the full state including message objects. If you switch models mid-thread (e.g., testing Bedrock on an existing thread), the deserialized messages may have provider-specific metadata that the new model doesn't understand. + +**Fix:** Start fresh threads when switching providers. Don't resume an OpenAI-started thread with a Bedrock model. Use the feature flag approach (separate threads per provider) rather than mid-thread switching. + +--- + +## CrewAI + +### Model configuration format + +**Problem:** CrewAI uses LiteLLM under the hood for model routing. The model string format for Bedrock is `bedrock/model-id` (e.g., `bedrock/us.anthropic.claude-sonnet-4-6-20250514-v1:0`). Getting the format wrong produces cryptic LiteLLM errors. + +**Fix:** Use the exact format: `bedrock/{full_model_id}`. Set `AWS_REGION_NAME` environment variable. Verify with a simple LiteLLM test call before running the full crew. + +### Tool schema formatting + +**Problem:** CrewAI formats tool descriptions into the prompt differently per model. Claude expects specific XML-like formatting for best tool use results. If CrewAI's internal prompt formatting doesn't match what Claude expects, tool selection accuracy drops. + +**Fix:** Update to latest CrewAI version (tool formatting improvements ship frequently). Test tool selection accuracy with your specific tools — if accuracy drops, add more explicit tool descriptions with examples. Consider using `@tool` decorator with detailed `description` parameter. + +### Process type and model capability + +**Problem:** CrewAI's `Process.hierarchical` requires the manager model to reliably delegate tasks. Some Bedrock models (especially smaller ones like Nova Lite, Llama variants) are less reliable at hierarchical delegation than GPT-4o or Claude Sonnet. + +**Fix:** For hierarchical processes, use Claude Sonnet 4.6 as the manager model (best tool-use reliability on Bedrock). Specialist agents can use cheaper models. Don't use Nova Micro or Llama Scout as the manager in hierarchical crews. + +### Memory and context window + +**Problem:** CrewAI accumulates conversation history across tasks. With GPT-4o (128K context), this rarely overflows. If you switch to a model with smaller context (Nova Pro 300K is fine, but Nova Micro 128K or Haiku 200K might be tight for long crews), tasks later in the sequence may lose context. + +**Fix:** Check your crew's typical context accumulation. If tasks are long, use a model with >= 200K context for the crew, or enable CrewAI's memory summarization to keep context compact. + +--- + +## AutoGen + +### Response format assumptions + +**Problem:** AutoGen's `ConversableAgent` parses model responses expecting OpenAI's format. When using Bedrock via LiteLLM, the response translation is usually correct, but edge cases exist — especially around `function_call` (legacy) vs `tool_calls` (current) format. + +**Fix:** Use AutoGen >= 0.4 which has better multi-provider support. Configure the model client explicitly rather than relying on auto-detection. Test the full conversation flow, not just single turns. + +### GroupChat speaker selection + +**Problem:** AutoGen's `GroupChat` uses the model to select the next speaker. This is a meta-task that requires the model to understand the conversation flow and pick the right agent. Claude handles this well; smaller Bedrock models may select speakers inconsistently. + +**Fix:** Use Claude Sonnet 4.6 for the GroupChat manager. If using a cheaper model, consider switching to `round_robin` or custom `speaker_selection_method` instead of model-based selection. + +### Code execution sandbox + +**Problem:** AutoGen's `UserProxyAgent` with `code_execution_config` executes code locally. This is unrelated to the model swap, but startups often discover during migration that their code execution setup is fragile. The model swap can change code generation patterns (Claude generates slightly different Python than GPT-4o), breaking assumptions in the execution environment. + +**Fix:** Test code generation + execution end-to-end. Claude tends to be more verbose in code comments and may use different library idioms. Ensure your execution sandbox has all required packages. + +--- + +## OpenAI SDK (Direct — not Agents SDK) + +### Converse API vs Chat Completions API + +**Problem:** The Bedrock Converse API is similar to OpenAI's Chat Completions but not identical. Key differences: + +- No `n` parameter (multiple completions per request) +- No `logprobs` +- No `response_format: { type: "json_schema", schema: {...} }` (use tool use for structured output instead) +- Different token counting (Bedrock uses model-specific tokenizers) +- No `seed` parameter for reproducibility + +**Fix:** Audit your OpenAI API calls for parameters that don't have Converse API equivalents. Replace `response_format` with tool use for structured output. Remove `n > 1` calls (make multiple sequential calls instead). Accept that exact reproducibility (`seed`) is not available. + +### Function calling format + +**Problem:** OpenAI uses `functions` (legacy) or `tools` with `type: "function"`. Bedrock Converse uses `toolConfig` with `tools` array. The schema format is similar but not identical — Bedrock requires `inputSchema` with a JSON Schema object, while OpenAI uses `parameters`. + +**Fix:** If using the provider adapter pattern (generated by this plugin), the adapter handles format translation. If migrating directly, map: `tools[].function.parameters` → `tools[].toolSpec.inputSchema.json`. + +### Streaming event format + +**Problem:** OpenAI streams `ChatCompletionChunk` objects with `delta` fields. Bedrock Converse streams `contentBlockDelta`, `contentBlockStart`, `contentBlockStop`, `messageStop` events. The event structure is fundamentally different. + +**Fix:** Use the provider adapter's `generate_stream()` method which normalizes both to a simple string iterator. If migrating directly, rewrite your stream processing loop to handle Bedrock's event types. + +--- + +## Common Across All Frameworks + +### Prompt sensitivity + +**Problem:** Prompts optimized for GPT-4o may not perform identically on Claude Sonnet 4.6. Claude tends to be more literal and instruction-following; GPT-4o is more "creative" with ambiguous instructions. System prompts that rely on GPT-4o's implicit behaviors may need adjustment. + +**Fix:** Test your top 20 most-used prompts on the target Bedrock model. Adjust prompts that produce degraded output. Claude generally responds better to explicit, structured instructions with clear formatting requirements. + +### Rate limiting and throttling + +**Problem:** OpenAI rate limits are per-API-key with clear headers. Bedrock throttling is per-account per-model per-region with different error codes (`ThrottlingException`). Retry logic built for OpenAI's `429` responses may not handle Bedrock's throttling correctly. + +**Fix:** Update retry logic to handle `ThrottlingException` and `ServiceUnavailableException`. Use exponential backoff. For high-throughput workloads, request quota increases via AWS Support or use provisioned throughput. + +### Token counting + +**Problem:** OpenAI's `tiktoken` library gives exact token counts for GPT models. Bedrock models use different tokenizers (Claude uses its own, Llama uses SentencePiece, etc.). Code that pre-calculates token budgets using `tiktoken` will be wrong for Bedrock models. + +**Fix:** Remove `tiktoken` dependency for Bedrock models. Use Bedrock's response metadata (`usage.inputTokens`, `usage.outputTokens`) for post-hoc counting. For pre-calculation, use conservative estimates (4 chars per token as rough heuristic) or the model-specific tokenizer if available. + +### Error handling + +**Problem:** OpenAI errors are well-documented HTTP errors (400, 401, 429, 500). Bedrock errors are AWS SDK exceptions (`ValidationException`, `ThrottlingException`, `ModelNotReadyException`, `AccessDeniedException`). Error handling code built for OpenAI's error taxonomy won't catch Bedrock failures. + +**Fix:** Map error handlers: OpenAI `401` → Bedrock `AccessDeniedException`, OpenAI `429` → Bedrock `ThrottlingException`, OpenAI `400` → Bedrock `ValidationException`. Add handlers for Bedrock-specific errors like `ModelNotReadyException` (model access not yet enabled). diff --git a/migration-to-aws/steering/schema-discover-ai.md b/migration-to-aws/steering/schema-discover-ai.md index 65b64c38..8dc65110 100644 --- a/migration-to-aws/steering/schema-discover-ai.md +++ b/migration-to-aws/steering/schema-discover-ai.md @@ -1,6 +1,9 @@ # AI Discovery Schema -Schema for `ai-workload-profile.json`, produced by `discover-app-code.md` when AI confidence >= 70%. +Schema for `ai-workload-profile.json`. Produced by: + +- **`discover-app-code.md`** when application-code AI confidence >= 70%, or +- **`discover-iac.md`** when **Vertex-strong** Terraform evidence is present (see that file’s Step 7d), producing a minimal **IaC-inferred** profile so Clarify Category F can run without app code. **Convention**: Values shown as `X|Y` in examples indicate allowed alternatives — use exactly one value per field, not the literal pipe character. @@ -8,13 +11,14 @@ Schema for `ai-workload-profile.json`, produced by `discover-app-code.md` when A ## ai-workload-profile.json (Phase 1 output) -Focused profile of AI/ML workloads including models, capabilities, integration patterns, and supporting infrastructure. Generated by `discover-app-code.md` when AI confidence >= 70%. +Focused profile of AI/ML workloads including models, capabilities, integration patterns, and supporting infrastructure. ```json { "metadata": { "report_date": "2026-02-26", "project_directory": "/path/to/project", + "profile_source": "application_code|iac_vertex|merged", "sources_analyzed": { "terraform": true, "application_code": true, @@ -27,7 +31,8 @@ Focused profile of AI/ML workloads including models, capabilities, integration p "confidence_level": "very_high", "total_models_detected": 2, "languages_found": ["python"], - "ai_source": "gemini|openai|both|other" + "ai_source": "gemini|openai|anthropic|both|other", + "inferred_from_iac": false }, "models": [ @@ -129,6 +134,11 @@ Focused profile of AI/ML workloads including models, capabilities, integration p } ``` +**Profile source and IaC inference:** + +- `metadata.profile_source` — `"application_code"` (full profile from code path), `"iac_vertex"` (minimal profile from Terraform Vertex-strong discovery only), or `"merged"` (code + pre-existing IaC profile combined in `discover-app-code.md`). +- `summary.inferred_from_iac` — `true` when the profile was produced or substantially completed without application code analysis (IaC-only or unknown integration until Clarify). `false` when code analysis drove model/SDK detection. + **CRITICAL Field Names** (use EXACTLY these keys): - `model_id` — Model identifier string (NOT `model_name`, `name`) @@ -136,17 +146,17 @@ Focused profile of AI/ML workloads including models, capabilities, integration p - `detected_via` — Array of detection sources (NOT `detection_method`, `source`) - `capabilities_used` — Array of capability strings per model (NOT `capabilities`, `features`) - `usage_context` — Human-readable description of what the model does (NOT `description`, `purpose`) -- `pattern` — Integration pattern in `integration` object (NOT `integration_type`, `method`) +- `pattern` — Integration pattern in `integration` object: `direct_sdk`, `framework`, `rest_api`, `mixed`, or `unknown` when not inferable (typical for IaC-only profiles) (NOT `integration_type`, `method`) - `gateway_type` — Gateway/router type in `integration` object: `"llm_router"`, `"api_gateway"`, `"voice_platform"`, `"framework"`, `"direct"`, or `null` - `capabilities_summary` — Boolean map in `integration` object (NOT `capabilities`, `feature_flags`) -- `ai_source` — Source AI provider in `summary` object: `"gemini"`, `"openai"`, `"both"`, or `"other"` +- `ai_source` — Source AI provider in `summary` object: `"gemini"`, `"openai"`, `"anthropic"`, `"both"`, or `"other"` **Key Fields:** - `metadata.sources_analyzed` — Which data sources were provided (affects which sections are populated) - `summary.overall_confidence` — Combined detection confidence from all signals -- `models[]` — Each distinct AI model/service detected, with evidence and capabilities -- `integration.pattern` — How the app connects to AI (`direct_sdk`, `framework`, `rest_api`, `mixed`) +- `models[]` — Each distinct AI model/service detected, with evidence and capabilities. **May be empty** for `profile_source: "iac_vertex"` when no model IDs can be inferred from Terraform alone; Clarify and Design must not assume non-empty. +- `integration.pattern` — How the app connects to AI (`direct_sdk`, `framework`, `rest_api`, `mixed`, or `unknown` for IaC-only) - `integration.capabilities_summary` — Union of all capabilities across all models - `infrastructure[]` — Terraform resources related to AI (empty array if no Terraform provided) - `current_costs` — Present ONLY if billing data was provided; omitted entirely otherwise @@ -156,3 +166,220 @@ Focused profile of AI/ML workloads including models, capabilities, integration p - `current_costs` — Include ONLY if billing data was provided (billing discovery ran). Omit entirely if no billing data. - `infrastructure` — Set to `[]` if no Terraform files were provided (IaC discovery did not run). +- `agentic_profile` — Include ONLY if agentic signals detected (`is_agentic: true`). Omit entirely otherwise. +- `tool_manifest` — Include ONLY if `agentic_profile` exists. Set to `[]` if agentic but no tools detected. +- `workloads` — Always present. Empty array `[]` if no AI call sites detected. + +--- + +## workloads[] (per-workload disambiguation) + +Generated by `discover-app-code.md` Step 5B. Splits the detected AI usage into distinct workloads — one per unique `(model_id, sdk_method, structured_output)` tuple. Enables downstream phases to produce one Bedrock recommendation per workload. + +```json +{ + "workloads": [ + { + "workload_id": "wl_3a1f2c", + "model_id": "gemini-2.5-flash", + "sdk_method": "ai.models.generateContent", + "capability": "text_generation", + "capability_confidence": "medium", + "structured_output": false, + "call_sites": [ + { "file": "lib/gemini.ts", "line": 11 } + ] + }, + { + "workload_id": "wl_8e22b4", + "model_id": "gemini-2.5-flash", + "sdk_method": "ai.models.generateContent", + "capability": "structured_output", + "capability_confidence": "high", + "structured_output": true, + "call_sites": [ + { "file": "lib/gemini.ts", "line": 27 } + ] + } + ] +} +``` + +**CRITICAL Field Names** (use EXACTLY these keys): + +- `workload_id` — Deterministic hash: `"wl_" + sha256(model_id + "|" + sdk_method + "|" + structured_flag)[:6]` (NOT `id`, `name`) +- `model_id` — Same model_id as in `models[]` (NOT `model_name`, `source_model`) +- `sdk_method` — Fully qualified SDK method name (NOT `method`, `api_call`) +- `capability` — Inferred capability: `"text_generation"`, `"structured_output"`, `"image_generation"`, `"embedding"`, `"speech_to_text"`, `"text_to_speech"`, `"unknown"` (NOT `capability_inferred`, `type`) +- `capability_confidence` — `"high"`, `"medium"`, or `"low"` (NOT `confidence`, `confidence_level`) +- `structured_output` — Boolean: true iff structured-output indicators detected in call arguments +- `call_sites` — Array of `{ "file": string, "line": integer }` — repo-relative POSIX paths + +**Capability assignment rules (from `steering/sdk-capability-map.json`):** + +| SDK Method | Default Capability | Confidence | Notes | +| ----------------------------------------------------------------- | ------------------- | ---------- | --------------------------------- | +| Methods in `sdk-capability-map.json` (not structured-output trio) | Map value | `high` | Direct map lookup | +| Structured-output trio + structured indicators present | `structured_output` | `high` | Argument inspection overrides map | +| Structured-output trio + no structured indicators | `text_generation` | `medium` | Ambiguous — Clarify confirms | +| Method not in map | `unknown` | `low` | Clarify fallback | + +**Validation rules:** + +- `workloads[]` is always present (empty array if no AI detected) +- Every `workloads[].model_id` MUST exist in `models[].model_id` +- `workload_id` is unique within the array and deterministic across re-runs +- `call_sites` is non-empty for every workload entry +- `call_sites[].file` uses repo-relative POSIX paths (forward slashes, no absolute paths) + +--- + +## agentic_profile (conditional — present only when agentic signals detected) + +Generated by `discover-app-code.md` Step 3.5 when agentic framework signals are found. Captures agent architecture, orchestration patterns, and memory usage. + +```json +{ + "agentic_profile": { + "is_agentic": true, + "framework": "langgraph|crewai|autogen|openai_agents|strands|custom|none", + "agents": [ + { + "agent_id": "research-agent", + "file": "src/agents/researcher.py", + "line": 45, + "model_id": "gpt-4o", + "tools": ["web_search", "document_reader"], + "memory_type": "conversation_buffer|rag|none|unknown", + "role": "Researches topics and gathers information" + }, + { + "agent_id": "writer-agent", + "file": "src/agents/writer.py", + "line": 22, + "model_id": "gpt-4o-mini", + "tools": ["file_write"], + "memory_type": "none", + "role": "Writes reports from research output" + } + ], + "orchestration_pattern": "hierarchical", + "agent_count": 2, + "tool_count": 3, + "has_human_in_loop": false, + "has_memory": true, + "memory_backend": "in_memory" + } +} +``` + +**CRITICAL Field Names** (use EXACTLY these keys): + +- `agent_id` — Agent identifier (NOT `name`, `agent_name`, `id`) +- `framework` — Agentic framework detected (NOT `agentic_framework`, `agent_framework`, `orchestration_framework`) +- `orchestration_pattern` — How agents coordinate (NOT `pattern`, `orchestration_type`, `coordination`) +- `memory_type` — Per-agent memory type (NOT `memory`, `memory_kind`) +- `memory_backend` — System-level memory storage (NOT `memory_store`, `backend`) +- `has_human_in_loop` — Boolean (NOT `human_in_loop`, `hitl`) + +**Allowed values:** + +- `framework`: `"langgraph"`, `"crewai"`, `"autogen"`, `"openai_agents"`, `"strands"`, `"custom"`, `"none"` +- `orchestration_pattern`: `"single"`, `"hierarchical"`, `"swarm"`, `"graph"`, `"sequential"`, `"unknown"` +- `agents[].memory_type`: `"conversation_buffer"`, `"rag"`, `"none"`, `"unknown"` +- `memory_backend`: `"redis"`, `"postgres"`, `"in_memory"`, `"vector_store"`, `"unknown"` + +**Classification rules for `orchestration_pattern`:** + +| Pattern | Evidence | +| -------------- | ------------------------------------------------------------------------------------------------------------------- | +| `single` | One agent definition, no delegation or coordination | +| `hierarchical` | One orchestrator agent delegates to specialist agents (agents-as-tools, manager/worker) | +| `swarm` | Multiple agents share memory/state, no fixed hierarchy | +| `graph` | Explicit node/edge definitions, conditional routing between agents (LangGraph `StateGraph`, Strands `GraphBuilder`) | +| `sequential` | Agents execute in fixed order (pipeline), output of one feeds next | +| `unknown` | Agentic signals detected but orchestration pattern unclear | + +**Key fields:** + +- `is_agentic` — Always `true` when this section exists (redundant but explicit for downstream gates) +- `framework` — Primary agentic framework; if multiple detected, use the one with the most agent definitions +- `agents[]` — Each distinct agent/role detected; include all that can be identified from code +- `agents[].model_id` — The LLM model this agent uses; `"unknown"` if not extractable from code +- `agents[].tools` — Tool names attached to this agent; empty array `[]` if no tools detected +- `agent_count` — Must equal length of `agents[]` array +- `tool_count` — Total unique tools across all agents (deduplicated) + +--- + +## tool_manifest (conditional — present only when agentic_profile exists) + +Generated by `discover-app-code.md` Step 6.5. Inventories tool definitions found in agent code. This is discovery only — no migration recommendations. + +```json +{ + "tool_manifest": [ + { + "name": "web_search", + "file": "src/tools/search.py", + "line": 10, + "transport": "api", + "auth_hint": "api_key", + "used_by_agents": ["research-agent"] + }, + { + "name": "document_reader", + "file": "src/tools/reader.py", + "line": 5, + "transport": "function", + "auth_hint": "none", + "used_by_agents": ["research-agent"] + }, + { + "name": "file_write", + "file": "src/tools/writer.py", + "line": 8, + "transport": "function", + "auth_hint": "none", + "used_by_agents": ["writer-agent"] + } + ] +} +``` + +**CRITICAL Field Names** (use EXACTLY these keys): + +- `name` — Tool name as defined in code (NOT `tool_name`, `tool_id`, `id`) +- `transport` — How the tool communicates (NOT `transport_type`, `type`, `protocol`) +- `auth_hint` — Detected authentication pattern (NOT `auth`, `auth_type`, `authentication`) +- `used_by_agents` — Array of `agent_id` values that reference this tool (NOT `agents`, `consumers`) + +**Allowed values:** + +- `transport`: `"function"` (local Python/JS/Go function), `"api"` (HTTP/REST call), `"mcp"` (MCP server), `"unknown"` +- `auth_hint`: `"none"`, `"api_key"`, `"oauth"`, `"iam"`, `"unknown"` + +**Classification rules for `transport`:** + +| Transport | Evidence | +| ---------- | ---------------------------------------------------------------------------------------------- | +| `function` | `@tool` decorator on a local function, inline function definition, no network call in body | +| `api` | HTTP client calls (`requests.get`, `httpx`, `fetch`) inside tool body, external URL references | +| `mcp` | MCP client imports, MCP server URL in tool config, `mcp.client` usage | +| `unknown` | Tool schema defined but implementation not inspectable or ambiguous | + +**Classification rules for `auth_hint`:** + +| Auth hint | Evidence | +| --------- | ------------------------------------------------------------------- | +| `none` | No credentials, tokens, or auth headers in tool implementation | +| `api_key` | API key env var, `Authorization: Bearer` header, `x-api-key` header | +| `oauth` | OAuth flow, token refresh, `client_id`/`client_secret` patterns | +| `iam` | AWS SigV4, boto3 client usage, IAM role assumption | +| `unknown` | Auth pattern not determinable from code inspection | + +**Validation rules:** + +- Every `agent_id` in `used_by_agents` MUST exist in `agentic_profile.agents[].agent_id` +- `tool_manifest` length MUST equal `agentic_profile.tool_count` (deduplicated tool count) +- If `agentic_profile` exists but no tools detected: set `tool_manifest: []` and `agentic_profile.tool_count: 0` diff --git a/migration-to-aws/steering/schema-discover-billing.md b/migration-to-aws/steering/schema-discover-billing.md index 1886958b..e8bb6e1b 100644 --- a/migration-to-aws/steering/schema-discover-billing.md +++ b/migration-to-aws/steering/schema-discover-billing.md @@ -76,6 +76,40 @@ Cost breakdown derived from GCP billing export CSV. Provides service-level spend "ai_signals": ["vertex_ai", "generative_ai"] } ], + "commitments": { + "has_active_cuds": true, + "total_monthly_commitment_fees": 150.00, + "total_monthly_cud_credits": -120.00, + "effective_discount_percent": 8.2, + "details": [ + { + "type": "resource_based", + "term": "1_year", + "covered_services": ["Compute Engine"], + "region": "us-central1", + "monthly_fee": 75.00, + "sku_description": "Commitment v1: E2 Cpu in Americas for 1 Year" + }, + { + "type": "resource_based", + "term": "1_year", + "covered_services": ["Compute Engine"], + "region": "us-central1", + "monthly_fee": 75.00, + "sku_description": "Commitment v1: E2 Ram in Americas for 1 Year" + } + ] + }, + "cost_basis": { + "uses_list_price": true, + "total_at_list": 2450.00, + "total_net_of_discounts": 2280.00, + "discount_breakdown": { + "committed_usage_discount": -120.00, + "sustained_usage_discount": -50.00, + "free_tier": 0.00 + } + }, "ai_signals": { "detected": true, "confidence": 0.85, @@ -86,12 +120,23 @@ Cost breakdown derived from GCP billing export CSV. Provides service-level spend **Key Fields:** -- `summary.total_monthly_spend` — Total monthly GCP spend from the billing export +- `summary.total_monthly_spend` — Total monthly GCP spend from the billing export (at list price when available) - `summary.service_count` — Number of distinct GCP services with charges - `services[].gcp_service_type` — Terraform resource type equivalent for the service (used by downstream phases) -- `services[].monthly_cost` — Monthly cost for this service -- `services[].top_skus` — Highest-cost line items within the service +- `services[].monthly_cost` — Monthly cost for this service (at list price; excludes commitment fee rows) +- `services[].top_skus` — Highest-cost line items within the service (excludes commitment fee SKUs) - `services[].ai_signals` — AI-related keywords found in SKU descriptions for this service +- `commitments.has_active_cuds` — Whether any CUD commitment fees or credits were detected +- `commitments.total_monthly_commitment_fees` — Sum of commitment fee line items (positive values) +- `commitments.total_monthly_cud_credits` — Sum of CUD credits applied (negative values) +- `commitments.effective_discount_percent` — Overall discount rate from all commitments +- `commitments.details[]` — Individual commitment contracts with type, term, covered services, and monthly fee +- `commitments.details[].type` — `"resource_based"` (vCPU/RAM commitments) or `"dollar_based"` (spend-based) +- `commitments.details[].term` — `"1_year"` or `"3_year"` +- `cost_basis.uses_list_price` — Whether `costAtListUSD` was available and used as the baseline +- `cost_basis.total_at_list` — Total spend at list price (before discounts) +- `cost_basis.total_net_of_discounts` — Total spend after all discounts applied +- `cost_basis.discount_breakdown` — Per-discount-type credit totals (negative values = savings) - `ai_signals.detected` — Whether any AI/ML services were found in the billing data - `ai_signals.confidence` — Confidence that the project uses AI (derived from billing SKU analysis) - `ai_signals.services` — List of AI-related GCP services found diff --git a/migration-to-aws/steering/schema-estimate-infra.md b/migration-to-aws/steering/schema-estimate-infra.md index b4635aa6..2bd7da71 100644 --- a/migration-to-aws/steering/schema-estimate-infra.md +++ b/migration-to-aws/steering/schema-estimate-infra.md @@ -4,6 +4,22 @@ Schema for `estimation-infra.json`, produced by `estimate-infra.md`. --- +## Cost tiers (`projected_costs` / `cost_comparison`) + +The fields **`aws_monthly_premium`**, **`aws_monthly_balanced`**, **`aws_monthly_optimized`** (under `projected_costs`) and **`option_a_premium`**, **`option_b_balanced`**, **`option_c_optimized`** (under `cost_comparison`) are **three pricing scenarios** for the **same** GCP->AWS mapping in `aws-design.json`. They are **not** three alternative Terraform roots. + +| Tier key | User-facing label | Subtitle (use in reports / MIGRATION_GUIDE) | +| --------------- | ----------------- | -------------------------------------------------------------------------- | +| **`premium`** | Premium | _Highest resilience / highest monthly estimate in this model_ | +| **`balanced`** | Balanced | _Default scenario; compare GCP to this first_ | +| **`optimized`** | Optimized | _Lower monthly estimate; reservations / Spot / storage trade-offs assumed_ | + +**How to read:** Scenario order is **highest -> middle -> lowest** monthly AWS estimate for the modeled architecture. **Balanced** is the **primary** comparison row vs the GCP baseline. **Premium** and **Optimized** are **bounds** (HA vs cost-optimization skew). + +**Terraform:** When the Generate phase produces `terraform/`, it implements **one** infrastructure baseline aligned with the **Balanced** scenario (`aligned_with_estimate_tier` in the `migration_summary` output). **Premium** and **Optimized** remain **estimate-only** unless the customer edits IaC. See `steering/generate-artifacts-infra.md` (`terraform/README.md`, `main.tf` header comment). + +--- + ## estimation-infra.json schema ```json @@ -12,8 +28,8 @@ Schema for `estimation-infra.json`, produced by `estimate-infra.md`. "design_source": "infrastructure", "timestamp": "2026-02-24T14:00:00Z", "pricing_source": { - "status": "cached|live|fallback", - "message": "Using cached prices from 2026-03-04 (±5-10% accuracy)|Using live AWS pricing API|Using cached rates from 2026-02-24 (±15-25% accuracy)", + "status": "cached|live|cached_fallback|unavailable", + "message": "Using cached prices from 2026-03-04 (±5-10% accuracy)|Using live AWS pricing API|MCP unavailable, using cached rates (±5-25% accuracy)|Pricing unavailable for [service]", "fallback_staleness": { "last_updated": "2026-02-24", "days_old": 3, @@ -30,7 +46,7 @@ Schema for `estimation-infra.json`, produced by `estimate-infra.md`. "accuracy_confidence": "±5-10%|±15-25%", "current_costs": { - "source": "billing_data|inventory_estimate|preferences|default", + "source": "billing_data|inventory_estimate|preferences|user_provided|unavailable", "gcp_monthly": 300, "gcp_annual": 3600, "baseline_note": "From billing-profile.json actual spend data", @@ -82,9 +98,29 @@ Schema for `estimation-infra.json`, produced by `estimate-infra.md`. "monthly_difference": -106, "annual_difference": -1272, "percent_change": "-35%" + }, + "commitment_context": { + "gcp_has_active_cuds": true, + "gcp_effective_discount_percent": 8.2, + "gcp_monthly_at_list": 300, + "gcp_monthly_net_of_discounts": 275, + "aws_compute_savings_plan_discount": "up to 66% (Fargate/Lambda/EC2; max term); typical 20-40% (1yr no-upfront)", + "aws_database_savings_plan_discount": "up to 35% (serverless) / up to 20% (provisioned RDS/Aurora)", + "aws_rds_reserved_instance_discount": "up to 69% (specific instance family, 3yr All Upfront)", + "aws_1yr_savings_plan_typical_discount": "20-40%", + "aws_3yr_savings_plan_typical_discount": "40-66%", + "note": "GCP baseline uses list price for apples-to-apples comparison. Customer currently saves 8.2% via CUDs. Database Savings Plans and RDS RIs are mutually exclusive per workload. For Cloud Run → Fargate, establish 30-90 day AWS baseline before Compute Savings Plans." } }, + "migration_cost_considerations": { + "billing_data_available": true, + "categories": [ + "Data transfer (GCP egress fees based on migration volume)" + ], + "note": "GCP charges for outbound data transfer during migration. Volume depends on database sizes and storage to migrate." + }, + "roi_analysis": { "recurring_savings": { "monthly_difference_balanced": -35, @@ -105,18 +141,48 @@ Schema for `estimation-infra.json`, produced by `estimate-infra.md`. "Better enterprise tool integration", "Vendor diversification (reduce single-vendor risk)", "Auto-scaling, spot instances, savings plans flexibility" - ] + ], + "note": "GCP data transfer egress fees (if estimated) are vendor one-time charges excluded from recurring ROI calculations. Human/professional-services migration costs are not modeled here." }, "optimization_opportunities": [ { - "opportunity": "Reserved Instances", + "opportunity": "Database Savings Plans", + "type": "database_savings_plan", "target_services": ["RDS", "Aurora"], - "savings_monthly": 58, - "savings_percent": "40%", - "commitment": "1-year", + "savings_monthly": 15, + "savings_percent": "up to 20% (provisioned)", + "commitment": "1-year no-upfront", + "timing": "post-migration or after instance right-sizing", + "implementation_effort": "low", + "prerequisite": "Confirm target instance class; omit savings_monthly when DB on-demand < $50/month", + "description": "Cloud SQL 24/7 usage is predictable. Database Savings Plans offer flexibility post-migration. Mutually exclusive with RDS RIs on the same workload.", + "alternative": { + "opportunity": "RDS Reserved Instances", + "type": "rds_reserved_instances", + "savings_percent": "up to 69%", + "trade_off": "Locked to specific instance family and region" + }, + "references": [ + "https://aws.amazon.com/savingsplans/database-pricing/", + "https://aws.amazon.com/rds/reserved-instances/" + ] + }, + { + "opportunity": "Compute Savings Plans", + "type": "compute_savings_plan", + "target_services": ["Fargate", "Lambda"], + "savings_monthly": null, + "savings_percent": "20-66%", + "commitment": "1-year or 3-year", + "timing": "post-migration (after 30-90 days of usage data)", "implementation_effort": "low", - "description": "Commit to 1-year reserved capacity for predictable workloads" + "prerequisite": "Establish AWS compute usage baseline before committing", + "description": "GCP compute billing (Cloud Run or GKE re-platform) makes pre-migration commitment sizing unreliable. Use Cost Explorer recommendations after migration.", + "references": [ + "https://aws.amazon.com/savingsplans/compute-pricing/", + "https://aws.amazon.com/savingsplans/faqs/" + ] }, { "opportunity": "S3 Infrequent Access", @@ -135,15 +201,6 @@ Schema for `estimation-infra.json`, produced by `estimate-infra.md`. "commitment": "none", "implementation_effort": "medium", "description": "Use Spot instances for fault-tolerant batch processing jobs" - }, - { - "opportunity": "Compute Savings Plans", - "target_services": ["Fargate", "Lambda"], - "savings_monthly": 20, - "savings_percent": "25%", - "commitment": "1-year", - "implementation_effort": "low", - "description": "AWS Savings Plans covering Fargate and Lambda usage" } ], @@ -158,9 +215,19 @@ Schema for `estimation-infra.json`, produced by `estimate-infra.md`. }, "recommendation": { - "path": "Full Infrastructure with Optimizations", - "roi_justification": "Optimized tier saves $106/month ($1,272/year) vs GCP with operational efficiency benefits", + "path": "migrate_optimized", + "path_label": "Migrate with Optimizations", + "roi_justification": "2.6 month payback with operational efficiency; $475K 5-year savings", "confidence": "high", + "migrate_if": [ + "operational efficiency matters", + "AWS-specific services needed", + "long-term AWS strategy" + ], + "stay_if": [ + "cost is the only metric and AWS is more expensive", + "team deeply experienced with GCP" + ], "next_steps": [ "Review financial case with stakeholders", "Confirm service tier selections (Aurora vs RDS, Fargate vs Lambda)", @@ -171,21 +238,83 @@ Schema for `estimation-infra.json`, produced by `estimate-infra.md`. } ``` +### recommendation block in estimation-infra.json + +The `recommendation` block is the single source of truth for migrate/stay guidance. Consumed by Estimate chat output AND HTML migration report (Section 0). Do not duplicate this logic in the report template. + +| `path` value | `path_label` (display) | +| --------------------- | ------------------------------ | +| `"migrate_optimized"` | `"Migrate with Optimizations"` | +| `"migrate_phased"` | `"Phased Migration"` | +| `"stay"` | `"Stay on GCP"` | + +Validation: + +- `path` is one of: `"migrate_optimized"`, `"migrate_phased"`, `"stay"` +- `path_label` matches the corresponding display string for `path` +- `migrate_if` and `stay_if` are non-empty arrays of strings +- `next_steps` is a non-empty array of strings +- Block is **REQUIRED** in `estimation-infra.json` output (Part 7 must write it) + +## Observability Entry in `projected_costs.breakdown` + +When Part 2B of `estimate-infra.md` produces an observability cost, it is included as an entry in `projected_costs.breakdown[]` with this shape: + +```json +{ + "service": "CloudWatch + X-Ray (Observability)", + "low": 7.00, + "mid": 10.00, + "high": 15.00, + "accuracy": "±30%", + "pricing_source": "cached", + "components": { + "log_ingestion": 5.00, + "log_storage": 0.45, + "custom_metrics": 3.00, + "alarms": 0.50, + "tracing": 0.00 + }, + "volume_source": "heuristic", + "note": "GCP Cloud Operations includes 50 GB/month free logging, free alerting, and free profiling. CloudWatch charges from the first GB." +} +``` + +**Validation for observability entry:** + +- `components` keys are exactly: `log_ingestion`, `log_storage`, `custom_metrics`, `alarms`, `tracing` +- `volume_source` is one of: `"heuristic"`, `"billing"` (reflects log volume source — the largest cost component; metrics are always heuristic regardless of this field) +- `tracing` is 0 when no tracing signals detected in source — do not add X-Ray costs unprompted +- `mid` equals the sum of all `components` values +- This entry REPLACES any CloudWatch/log/metric portion in the "Supporting" row — never both + ## Output Validation Checklist - `design_source` is `"infrastructure"` -- `pricing_source.status` is `"cached"`, `"live"`, or `"fallback"` +- `pricing_source.status` is `"cached"`, `"live"`, `"cached_fallback"`, or `"unavailable"` - `accuracy_confidence` matches the pricing mode (±5-10% for cached/live, ±15-25% for fallback) -- `current_costs.source` is `"billing_data"` if `billing-profile.json` was used, `"inventory_estimate"`, `"preferences"`, or `"default"` otherwise +- `current_costs.source` is `"billing_data"` if `billing-profile.json` was used, `"inventory_estimate"`, `"preferences"`, `"user_provided"` (asked during estimate), or `"unavailable"` (user declined) otherwise - `current_costs.gcp_monthly` matches billing-profile.json total (if used) or is a reasonable estimate - `projected_costs` has all three tiers (premium, balanced, optimized) -- `projected_costs.breakdown` covers compute, database, storage, networking, and supporting services +- **Tier semantics:** Three totals are **scenario $** only (same design); **Balanced** matches generated Terraform baseline — see **Cost tiers** section above; user-facing labels must use the subtitles there (also `estimate-infra.md` Present Summary / `generate-artifacts-report.md`) +- `projected_costs.breakdown` covers compute, database, storage, networking, supporting services, and observability - Every service in `aws-design.json` is represented in the cost breakdown +- `projected_costs.breakdown` observability entry (when present) REPLACES any CloudWatch/log/metric costs in the "Supporting" row — never double-count - `cost_comparison` shows all three options with monthly and annual differences +- `cost_comparison.commitment_context` is present if `billing-profile.json` has `commitments.has_active_cuds == true`; omitted otherwise +- `migration_cost_considerations.billing_data_available` is `true` if `billing-profile.json` exists, `false` otherwise +- If `billing_data_available` is `true`: `migration_cost_considerations.categories` lists **GCP vendor egress / data transfer** only (never human or professional-services costs) +- If `billing_data_available` is `false`: `migration_cost_considerations.categories` is empty; `note` explains that billing data is required for GCP egress fee estimates - `roi_analysis` presents recurring monthly/annual savings (or increase) per tier - `roi_analysis` is honest — if migration increases cost, say so and justify with non-cost benefits - `optimization_opportunities` only includes strategies relevant to the designed architecture +- Each `optimization_opportunities[]` entry includes required fields: `opportunity`, `target_services`, `savings_percent`, `implementation_effort`, `description`. Optional fields: `type`, `savings_monthly` (null when post-migration sizing unavailable), `commitment`, `timing`, `prerequisite`, `references`, `alternative` +- Compute Savings Plans entries for Cloud Run migrations MUST NOT include `savings_monthly` sized from GCP billing — use `savings_monthly: null` and `timing: post-migration` +- Database Savings Plans entries MAY include `savings_monthly` only when projected DB on-demand exceeds $50/month +- `optimization_opportunities` savings are incremental to **Balanced** on-demand totals — not additive on **Optimized** tier (which already embeds reservation/Spot assumptions) - `financial_summary` provides a clear executive-level view +- `recommendation` block exists with `path`, `path_label`, `migrate_if`, `stay_if`, and `next_steps` all populated +- `recommendation.path` is one of: `"migrate_optimized"`, `"migrate_phased"`, `"stay"` - `recommendation.next_steps` includes actionable items - No references to AI-specific costs (those belong in `estimate-ai.md`) - No references to billing-only estimates (those belong in `estimate-billing.md`) diff --git a/migration-to-aws/steering/sdk-capability-map.json b/migration-to-aws/steering/sdk-capability-map.json new file mode 100644 index 00000000..7b64f00a --- /dev/null +++ b/migration-to-aws/steering/sdk-capability-map.json @@ -0,0 +1,29 @@ +{ + "schema_version": 1, + "description": "Deterministic mapping from SDK method to AI capability. Used by Discover to disambiguate workloads by SDK call pattern rather than model ID alone.", + "entries": { + "ai.models.generateContent": "text_generation", + "ai.models.generateImages": "image_generation", + "ai.models.embedContent": "embedding", + "openai.chat.completions.create": "text_generation", + "openai.completions.create": "text_generation", + "openai.embeddings.create": "embedding", + "openai.images.generate": "image_generation", + "openai.audio.speech.create": "text_to_speech", + "openai.audio.transcriptions.create": "speech_to_text", + "anthropic.messages.create": "text_generation", + "bedrock.invoke_model": "unknown", + "cohere.embed": "embedding", + "cohere.generate": "text_generation" + }, + "structured_output_trio": [ + "openai.chat.completions.create", + "anthropic.messages.create", + "ai.models.generateContent" + ], + "structured_output_indicators": [ + "response_format", + "responseSchema", + "responseMimeType" + ] +} diff --git a/migration-to-aws/steering/terraform-validation.md b/migration-to-aws/steering/terraform-validation.md new file mode 100644 index 00000000..47fd3cb6 --- /dev/null +++ b/migration-to-aws/steering/terraform-validation.md @@ -0,0 +1,252 @@ +# Terraform Validation Protocol (Shared Reference) + +> Canonical definition of the `fmt → init → validate → fix-and-retry → offline-fallback` protocol used by any phase that emits Terraform. Referenced by `steering/generate-artifacts-infra.md` Step 6 and reusable by future artifact generators. + +## When to Use + +Any step that writes a `terraform/` directory and wants to block phase completion on HCL-level defects while still degrading gracefully when the provider registry is unreachable. + +## Protocol + +Working directory: the `terraform/` directory under test. All commands run non-interactively (`-input=false -no-color` where supported). + +### Stage A — Format + +1. `terraform fmt -recursive` (auto-apply). +2. `terraform fmt -recursive -check`. If non-zero, enter the Fix-and-Retry loop targeting fmt failures. On success, advance to Stage B. + +### Stage B — Initialize (no backend) + +1. `terraform init -backend=false -input=false -no-color`, capturing stderr. +2. On non-zero exit, run the **Offline Detection** algorithm below on the captured stderr. + - If classified network-unavailable: set `validation_status = "passed_degraded_offline"`, emit warning, SKIP Stage C, proceed to Stage E. Do NOT enter the retry loop. + - Otherwise: enter the Fix-and-Retry loop targeting init failures. +3. On success, advance to Stage C. + +### Stage C — Validate + +1. `terraform validate -json`, capturing stdout. +2. On non-zero exit, parse `.diagnostics[]` and enter the Fix-and-Retry loop targeting validate failures. +3. On success, set `validation_status = "passed"` and advance to Stage E. + +### Stage D — Fix-and-Retry Loop + +Attempt budget: **3 attempts per batch**. Hardcoded. Not configurable via `preferences.json`. + +Per attempt: + +1. Read the failing command's error output: + - fmt: the diff shown by `fmt -recursive -check` (list of files that would change). + - init: the stderr captured from `terraform init`. + - validate: the JSON diagnostics array from `terraform validate -json`. +2. **Group errors by file path.** For each file, open it once, apply all targeted edits for that file, close. Never rewrite a file wholesale; only edit the lines/blocks reported. +3. Re-run only the failing command (fmt -check, init, or validate). +4. If it passes, exit the loop and return to the calling stage's success path. +5. If the same `(file, line, summary)` reappears on consecutive attempts, emit a "same error recurring" signal in the attempt log — this is a warning only; continue to the next attempt. + +On the 3rd consecutive failure in a batch, prompt the user: + +``` +Terraform validation failed after 3 automated fix attempts. +Last error: +[retry] attempt 3 more fixes +[skip] proceed with warning (validation_status = skipped_user_continue) +[abort] stop, do NOT write .phase-status.json +Choose [retry/skip/abort]: +``` + +User choices: + +- **retry** — reset the per-batch attempt counter to 0, grant 3 more attempts, continue. The cumulative `attempts` field in `validation-report.json` is NOT reset (it keeps incrementing). +- **skip** — set `validation_status = "skipped_user_continue"`, emit warning, proceed to Stage E. Phase Completion is allowed. +- **abort** — set `validation_status = "skipped_user_abort"`, write `validation-report.json` with that status, STOP. **Do NOT write to `.phase-status.json`.** The caller (generate.md) relies on seeing no completion signal. + +### Stage E — Emit validation-report.json + +Write `$MIGRATION_DIR/validation-report.json` following the schema in the **Report Schema** section below. + +## Offline Detection + +``` +FUNCTION isNetworkUnavailable(init_stderr) + INPUT: init_stderr (string, captured stderr of `terraform init`) + OUTPUT: boolean + + patterns ← ["lookup", "dial tcp", "connection refused", "timeout", "no such host"] + + IF init_stderr IS NULL OR trim(init_stderr) = "" THEN + RETURN false // empty stderr → treat as non-network failure (let retry loop handle) + END IF + + haystack ← toLowerCase(init_stderr) + + FOR EACH p IN patterns DO // first-match-wins; order does not matter for correctness + IF contains(haystack, toLowerCase(p)) THEN + RETURN true + END IF + END FOR + + RETURN false +END FUNCTION +``` + +**Rules**: + +- **Source stream**: stderr of `terraform init` only. Do not read stdout for classification (terraform writes progress to stdout, errors to stderr). +- **Case sensitivity**: case-insensitive. Lowercase both the haystack and the patterns before comparing. +- **Match semantics**: first-match-wins. Any pattern hit short-circuits to `true`. Full-stderr scan, not per-line. +- **Empty stderr**: treat as non-network failure. The retry loop runs. This prevents silent offline-fallback when terraform fails for an unrelated reason (e.g., a panic) and produces no stderr. + +## Fix-and-Retry Algorithm (pseudocode) + +``` +FUNCTION fixAndRetry(stage, initial_error_output) + INPUT: + stage ∈ {"fmt", "init", "validate"} + initial_error_output — captured output from the failing command + OUTPUT: + terminal_status ∈ {"passed", "skipped_user_continue", "skipped_user_abort"} + cumulative_attempts (int) + + cumulative_attempts ← 0 + last_error_output ← initial_error_output + recurring_errors ← ∅ + + LOOP // outer loop: retry user choice may re-enter + batch_attempt ← 0 + last_batch_errors ← ∅ + + WHILE batch_attempt < 3 DO + batch_attempt ← batch_attempt + 1 + cumulative_attempts ← cumulative_attempts + 1 + + errors ← parseErrors(stage, last_error_output) + errors_by_file ← groupBy(errors, e → e.file) + + FOR EACH (file, file_errors) IN errors_by_file DO + applyTargetedEdits(file, file_errors) // LLM edits only reported sites + END FOR + + (exit_code, new_output) ← run(stage) // rerun only the failing command + + IF exit_code = 0 THEN + // advance the caller to the next stage; this function returns "passed" + // once the caller reaches Stage C success. For fmt/init this means + // success at the current stage; the caller chains into the next. + RETURN ("passed", cumulative_attempts) + END IF + + // Detect recurring errors for logging (warning signal, not a control-flow change) + new_errors_set ← set of (file, line, summary) from parseErrors(stage, new_output) + recurring ← last_batch_errors ∩ new_errors_set + IF recurring ≠ ∅ THEN + emitWarning("same error recurring: " + recurring) + recurring_errors ← recurring_errors ∪ recurring + END IF + last_batch_errors ← new_errors_set + last_error_output ← new_output + END WHILE + + choice ← promptUser("retry/skip/abort") + + IF choice = "retry" THEN + // reset per-batch counter; outer loop continues for 3 more attempts + CONTINUE + ELSE IF choice = "skip" THEN + RETURN ("skipped_user_continue", cumulative_attempts) + ELSE IF choice = "abort" THEN + writeValidationReport(status="skipped_user_abort", attempts=cumulative_attempts, ...) + STOP_WITHOUT_PHASE_STATUS_WRITE() // MUST NOT update .phase-status.json + END IF + END LOOP +END FUNCTION +``` + +**Key contract points**: + +- The cumulative `attempts` counter in `validation-report.json` counts every rerun across all batches, including after a user `retry`. It never resets. +- The per-batch counter (triggering the user prompt) resets to 0 on `retry`. +- On `abort`, the function terminates the whole run without touching `.phase-status.json`. The calling phase (`generate.md`) must see the absence of a completion write and NOT advance the state machine. +- "Progress" vs "same error recurring": progress is when `exit_code == 0` OR when `last_batch_errors \ new_errors_set ≠ ∅` (at least one error disappeared). A fully-overlapping error set across consecutive attempts triggers the warning but does not change control flow. + +## Report Schema + +```json +{ + "$schema": "validation-report/v1", + "status": "passed | passed_degraded_offline | skipped_user_continue | skipped_user_abort", + "attempts": 0, + "errors_found": [ + { + "file": "string (relative to terraform/)", + "line": "integer (1-based, 0 if unknown)", + "severity": "error | warning", + "summary": "string (≤200 chars, first line of diagnostic)" + } + ], + "errors_fixed": [ + { + "file": "string", + "line": "integer", + "severity": "error | warning", + "summary": "string", + "attempt": "integer (1-indexed; which retry attempt repaired it)" + } + ], + "offline_fallback_used": false, + "timestamp": "ISO 8601 UTC (e.g., 2026-02-26T15:35:22Z)", + "terraform_version": "string (output of `terraform version -json | .terraform_version`; empty string if unavailable)" +} +``` + +**Field rules**: + +- `status`: MUST match one of the four enum values. MUST equal the terminal `validation_status`. +- `attempts`: integer ≥ 0. Counts total fix-and-retry attempts across all batches (a `retry` user choice that grants 3 more attempts continues to increment this counter; it does NOT reset). Value 0 means fmt/init/validate all passed on first try. +- `errors_found`: every distinct diagnostic emitted across all attempts, deduplicated by `(file, line, summary)`. +- `errors_fixed`: subset of `errors_found` that did not reappear after the indicated `attempt`. +- `offline_fallback_used`: `true` iff `status == "passed_degraded_offline"`. +- `terraform_version`: populated by parsing `terraform version -json`; if that command fails, fall back to empty string (never crash on this). + +**Example** (passed after 2 retry attempts repaired a missing brace and an unresolved reference): + +```json +{ + "$schema": "validation-report/v1", + "status": "passed", + "attempts": 2, + "errors_found": [ + { + "file": "vpc.tf", + "line": 47, + "severity": "error", + "summary": "Missing close brace on resource \"aws_subnet\" \"private_a\"" + }, + { + "file": "compute.tf", + "line": 12, + "severity": "error", + "summary": "Reference to undeclared resource aws_security_group.app" + } + ], + "errors_fixed": [ + { + "file": "vpc.tf", + "line": 47, + "severity": "error", + "summary": "Missing close brace on resource \"aws_subnet\" \"private_a\"", + "attempt": 1 + }, + { + "file": "compute.tf", + "line": 12, + "severity": "error", + "summary": "Reference to undeclared resource aws_security_group.app", + "attempt": 2 + } + ], + "offline_fallback_used": false, + "timestamp": "2026-02-26T15:37:04Z", + "terraform_version": "1.9.5" +} +``` diff --git a/migration-to-aws/steering/validate-artifacts.md b/migration-to-aws/steering/validate-artifacts.md new file mode 100644 index 00000000..84288d8f --- /dev/null +++ b/migration-to-aws/steering/validate-artifacts.md @@ -0,0 +1,67 @@ +# Validate Artifacts (Pre-Report) + +> **Read-only validation.** Load at the start of `generate-artifacts-report.md` (Step 0) and before writing `migration-report.html`. Do NOT modify artifacts during this step. + +On any failure: emit `GATE_FAIL` per `steering/handoff-gates.md`, skip report generation, tell the user which phase to re-run. **Do NOT patch JSON to pass validation.** + +--- + +## How to run + +1. Read each file below from `$MIGRATION_DIR/` using the Read tool. +2. Mark each check PASS or FAIL based on file contents (not memory). +3. If **any required** check FAILs → stop; do not write `migration-report.html`. +4. If all required checks PASS → proceed to report generation. + +Optional checks: skip section in report if FAIL (do not halt). + +--- + +## Required checks (max 15 — halt report on FAIL) + +| # | Check | PASS when | On FAIL | +| -- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | +| 1 | Estimation exists | At least one of `estimation-infra.json`, `estimation-ai.json`, `estimation-billing.json` exists | `GATE_FAIL \| phase=generate \| field=estimation-* \| reason=missing` — re-run Estimate | +| 2 | Design exists | At least one of `aws-design.json`, `aws-design-ai.json`, `aws-design-billing.json` exists | `GATE_FAIL \| phase=generate \| field=aws-design-* \| reason=missing` — re-run Design | +| 3 | Preferences | `preferences.json` exists and parses as JSON | `GATE_FAIL \| phase=generate \| field=preferences.json \| reason=missing` — re-run Clarify | +| 4 | Recommendation (infra) | If `estimation-infra.json` exists: `recommendation.path` ∈ `{migrate_optimized, migrate_phased, stay}` | `GATE_FAIL \| phase=estimate \| field=recommendation.path \| reason=missing` | +| 5 | Recommendation lists | If `estimation-infra.json` exists: `migrate_if` and `stay_if` are non-empty arrays | `GATE_FAIL \| phase=estimate \| field=recommendation.migrate_if \| reason=missing` | +| 6 | Preview complexity | If `migration-preview.json` exists: `complexity_signal` is present | `GATE_FAIL \| phase=discover \| field=complexity_signal \| reason=missing` | +| 7 | Cloud SQL availability | If `gcp-resource-inventory.json` contains `google_sql_database_instance` AND `estimation-infra.json` exists: `preferences.json` → `design_constraints.availability.value` is set | `GATE_FAIL \| phase=clarify \| field=design_constraints.availability.value \| reason=missing` | +| 8 | AI profile consistency | If `aws-design-ai.json` exists → `ai-workload-profile.json` exists | `GATE_FAIL \| phase=discover \| field=ai-workload-profile.json \| reason=missing` | +| 9 | Generation plan (infra) | If `estimation-infra.json` exists → `generation-infra.json` exists | `GATE_FAIL \| phase=generate \| field=generation-infra.json \| reason=missing` | +| 10 | Docs | `MIGRATION_GUIDE.md` and `README.md` exist | `GATE_FAIL \| phase=generate \| field=MIGRATION_GUIDE.md \| reason=missing` | + +--- + +## Optional checks (do not halt — omit report sections) + +| # | Check | If FAIL | +| -- | ----------------------- | ------------------------------------------------------------------------------------------------- | +| 11 | AI generation plan | If `estimation-ai.json` exists but not `generation-ai.json` → omit Appendix D.5 | +| 12 | Startup credits | If no `STARTUP_PROGRAMS.md` and no `startup_program_status` in preferences → omit credits callout | +| 13 | Bedrock monitoring | If `ai-migration/bedrock_monitoring.tf` missing → omit Bedrock anomaly row | +| 14 | Observability breakdown | If no observability key in `projected_costs.breakdown` → omit observability note | +| 15 | Deferred services | If no `Deferred — specialist engagement` in design → omit deferred callout | + +--- + +## Pre-write sanity (repeat immediately before HTML write) + +After building report content in memory, re-read from disk and confirm these **three** checks (defense in depth — context may drift during long Generate): + +1. `estimation-infra.json` → `recommendation.path_label` OR fallback documented in Step 1 is available for Section 0. +2. `migration-preview.json` → `complexity_signal` present (if file exists). +3. Planned HTML includes `
` — see `generate-artifacts-report.md` HTML skeleton. + +If any pre-write check fails: do not write the file; emit `GATE_FAIL` and stop. + +--- + +## Output when all required checks pass + +``` +VALIDATE_OK | checks=10/10 | ready=migration-report.html +``` + +Then proceed to `generate-artifacts-report.md` Step 1.