Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,11 @@ PIKI 의 상품 추출 서비스. 상품 URL(또는 S3 이미지)을 받아 fetc
./gradlew test # Docker 불필요 (DB 없음)
./gradlew bootRun # 기본 포트 8090
```

## 운영 (prod 박스)

- 박스 자체: `infra/extractor/` (terraform, EC2·SG·IAM).
- 앱 컨테이너 기동: `infra/scripts/run-extractor.sh` — 컨테이너 (재)기동의 SSOT.
- 관측(Alloy) 배선: `infra/scripts/provision-observability.sh` — 공통 Alloy 블록을 이 박스 값으로 호출.
- 둘 다 박스 안에서 SSM(run-command 또는 세션)으로 실행하며, 시크릿은 SSM Parameter Store(`/piki-extractor/*`)에서 읽는다.
- 관측 계약(라벨 opt-in 등)은 TeamPiKi/infra 의 `contracts/observability.md` 가 SSOT.
90 changes: 90 additions & 0 deletions infra/scripts/provision-observability.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
#!/usr/bin/env bash
#
# extractor prod 박스 관측(Alloy) 배선 - 이 스크립트가 값의 SSOT다.
#
# TeamPiKi/infra 의 공통 Alloy 블록(blocks/alloy/config.alloy + provision-alloy.sh)을
# fetch 하고, Grafana Cloud 자격을 SSM Parameter Store 에서 읽어 env 로 얹은 뒤
# extractor 의 값(environment=prod, box=piki-extractor)으로 공통 블록을 호출한다.
#
# 실행 위치: extractor 박스 안 (SSM run-command 또는 세션 접속 후 직접 실행).
# 이 박스에는 aws cli 가 없어 SSM 조회는 dockerized aws cli 로 한다
# (인스턴스 IAM 롤 + IMDSv2 hop limit 2 라 컨테이너 안에서도 자격 획득 가능).
#
# 전제: TeamPiKi/infra 의 공통 Alloy 블록 PR 이 머지돼 있어야 한다. 머지 전에는
# --ref 로 그 PR 의 브랜치명을 지정해 선검증할 수 있다.
#
# 사용 예:
# provision-observability.sh # main(머지된 공통 블록) 사용
# provision-observability.sh --ref feat/alloy-common-block # 머지 전 브랜치로 선검증
#
# 인자:
# --ref (선택) TeamPiKi/infra git ref. 기본 main
#
# 종료 코드: 성공 0, fetch/SSM 조회/공통 블록 실행 실패 1, 인자 오류 2

set -euo pipefail

REF="main"

while [ $# -gt 0 ]; do
case "$1" in
--ref) REF="${2:-}"; shift 2;;
*) echo "unknown arg: $1" >&2; exit 2;;
esac
done
Comment on lines +29 to +34

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Same missing-value edge case as run-extractor.sh's --image parsing.

If --ref is passed with no following value, shift 2 on a single remaining positional param fails under set -e, crashing before the [ -n "$REF" ] check at line 36 can produce the intended "인자 오류" message.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/scripts/provision-observability.sh` around lines 29 - 34, Update the
argument parser’s --ref branch in the while loop to validate that a following
value exists before assigning REF and shifting two arguments. When the value is
missing, exit through the existing argument-error path so the intended “인자 오류”
message is emitted instead of letting shift fail.


[ -n "$REF" ] || { echo "--ref 값이 비었다" >&2; exit 2; }

WORK_DIR="/tmp/piki-obs"
mkdir -p "$WORK_DIR"

INFRA_RAW_BASE="https://raw.githubusercontent.com/TeamPiKi/infra/$REF/blocks/alloy"

# --- 1. 공통 Alloy 블록 fetch (repo public, 인증 불필요) ---
curl -fsSL "$INFRA_RAW_BASE/config.alloy" -o "$WORK_DIR/config.alloy" \
|| { echo "config.alloy fetch 실패 (ref=$REF)" >&2; exit 1; }
curl -fsSL "$INFRA_RAW_BASE/provision-alloy.sh" -o "$WORK_DIR/provision-alloy.sh" \
|| { echo "provision-alloy.sh fetch 실패 (ref=$REF)" >&2; exit 1; }
Comment on lines +44 to +47

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== infra/scripts/provision-observability.sh ==\n'
wc -l infra/scripts/provision-observability.sh
sed -n '1,140p' infra/scripts/provision-observability.sh

printf '\n== Search for REF defaults and integrity checks ==\n'
rg -n --no-heading -S 'REF=|--ref|checksum|sha256|signature|gpg|cosign|main' infra/scripts

Repository: TeamPiKi/extractor

Length of output: 4591


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== infra/scripts/run-extractor.sh ==\n'
wc -l infra/scripts/run-extractor.sh
sed -n '1,220p' infra/scripts/run-extractor.sh

Repository: TeamPiKi/extractor

Length of output: 3331


Pin the fetched observability scripts instead of defaulting to main

  • infra/scripts/provision-observability.sh:27,44-47 fetches config.alloy and provision-alloy.sh from TeamPiKi/infra/main by default, then executes the fetched provision-alloy.sh directly.
  • infra/scripts/run-extractor.sh:84 does the same for blocks/healthcheck.sh.
  • Pin the default ref to a tagged release or commit SHA; keep --ref for intentional overrides.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/scripts/provision-observability.sh` around lines 44 - 47, Update the
default ref configuration used by provision-observability.sh and
run-extractor.sh so fetched observability scripts resolve to a tagged release or
commit SHA instead of main. Preserve the existing --ref override behavior, and
ensure config.alloy, provision-alloy.sh, and blocks/healthcheck.sh continue
using the selected ref.


# --- 2. Grafana Cloud 자격을 SSM Parameter Store 에서 조회 ---
# 박스엔 aws cli 가 없어 dockerized aws cli 로 조회한다.
get_ssm_param() {
local param_name="$1"
docker run --rm --network host \
-e AWS_DEFAULT_REGION=ap-northeast-2 \
amazon/aws-cli ssm get-parameter \
--name "$param_name" \
--with-decryption \
--query 'Parameter.Value' \
--output text
}

# metrics/logs/token 5종은 필수 - 없으면 관측이 성립하지 않으므로 즉시 실패.
GRAFANA_METRICS_URL=$(get_ssm_param /piki-extractor/grafana-metrics-url) \
|| { echo "필수 SSM 파라미터 조회 실패: /piki-extractor/grafana-metrics-url" >&2; exit 1; }
GRAFANA_METRICS_USER=$(get_ssm_param /piki-extractor/grafana-metrics-user) \
|| { echo "필수 SSM 파라미터 조회 실패: /piki-extractor/grafana-metrics-user" >&2; exit 1; }
GRAFANA_LOGS_URL=$(get_ssm_param /piki-extractor/grafana-logs-url) \
|| { echo "필수 SSM 파라미터 조회 실패: /piki-extractor/grafana-logs-url" >&2; exit 1; }
GRAFANA_LOGS_USER=$(get_ssm_param /piki-extractor/grafana-logs-user) \
|| { echo "필수 SSM 파라미터 조회 실패: /piki-extractor/grafana-logs-user" >&2; exit 1; }
GRAFANA_CLOUD_TOKEN=$(get_ssm_param /piki-extractor/grafana-cloud-token) \
|| { echo "필수 SSM 파라미터 조회 실패: /piki-extractor/grafana-cloud-token" >&2; exit 1; }

# traces 2종은 선택 - 없으면 빈 값으로 둔다. 공통 config.alloy 가 더미 endpoint 로
# fallback 해 trace export 만 비활성화되고(metrics/logs 는 정상 동작), 스크립트는 실패하지 않는다.
GRAFANA_TRACES_URL=$(get_ssm_param /piki-extractor/grafana-traces-url) || GRAFANA_TRACES_URL=""
GRAFANA_TRACES_USER=$(get_ssm_param /piki-extractor/grafana-traces-user) || GRAFANA_TRACES_USER=""

export GRAFANA_METRICS_URL GRAFANA_METRICS_USER
export GRAFANA_LOGS_URL GRAFANA_LOGS_USER
export GRAFANA_TRACES_URL GRAFANA_TRACES_USER
export GRAFANA_CLOUD_TOKEN

# --- 3. 공통 블록 호출 - 이 박스는 prod 전용이라 environment=prod 고정 ---
# (dev extractor 는 core dev 박스에 동거하며 그쪽 배선은 core 소관, 여기서 다루지 않는다)
bash "$WORK_DIR/provision-alloy.sh" \
--config "$WORK_DIR/config.alloy" \
--name piki-alloy \
--environment prod \
--box piki-extractor
91 changes: 91 additions & 0 deletions infra/scripts/run-extractor.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
#
# extractor 앱 컨테이너 (재)기동 - 이 스크립트가 컨테이너 기동의 신설 SSOT다.
#
# 지금까지 앱 컨테이너(piki-extractor)는 임시(수동 SSM)로 기동돼 run 파라미터가
# repo 어디에도 없었다. 라벨·env 는 이 스크립트 한 곳에만 박는다 - 이 스크립트
# 밖에서 env 를 더하지 않는다.
#
# 실행 위치: extractor 박스 안 (SSM run-command 또는 세션 접속 후 직접 실행).
# 박스엔 aws cli 가 없어 SSM 조회는 dockerized aws cli 로 한다
# (인스턴스 IAM 롤 + IMDSv2 hop limit 2 라 컨테이너 안에서도 자격 획득 가능).
#
# 사용 예:
# run-extractor.sh --image piki-extractor:latest
#
# 인자:
# --image (필수) 기동할 이미지:태그. default 없음
#
# 종료 코드: 성공(헬스체크 통과) 0, SSM 조회/헬스체크 실패 1, 인자 오류 2

set -euo pipefail

IMAGE=""

while [ $# -gt 0 ]; do
case "$1" in
--image) IMAGE="${2:-}"; shift 2;;
*) echo "unknown arg: $1" >&2; exit 2;;
esac
done
Comment on lines +25 to +30

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

--image with no value crashes ambiguously instead of a clean usage error.

If --image is the last token (no value follows), shift 2 runs with only one positional param left. Under set -euo pipefail, shift's nonzero exit (n > $#) triggers immediate script exit — before reaching the --image is required check at line 32 — with a confusing failure instead of the documented "인자 오류".

🐛 Proposed fix
   case "$1" in
-    --image) IMAGE="${2:-}"; shift 2;;
+    --image)
+      [ $# -ge 2 ] || { echo "--image requires a value" >&2; exit 2; }
+      IMAGE="$2"; shift 2;;
     *) echo "unknown arg: $1" >&2; exit 2;;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
while [ $# -gt 0 ]; do
case "$1" in
--image) IMAGE="${2:-}"; shift 2;;
*) echo "unknown arg: $1" >&2; exit 2;;
esac
done
while [ $# -gt 0 ]; do
case "$1" in
--image)
[ $# -ge 2 ] || { echo "--image requires a value" >&2; exit 2; }
IMAGE="$2"; shift 2;;
*) echo "unknown arg: $1" >&2; exit 2;;
esac
done
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/scripts/run-extractor.sh` around lines 25 - 30, Update the --image
branch in the argument-parsing loop to validate that a following value exists
before assigning IMAGE and shifting two arguments. When --image is the final
token, emit the script’s documented argument-error usage message and exit
cleanly instead of invoking shift 2 with insufficient parameters; preserve
existing handling for valid values and unknown arguments.


[ -n "$IMAGE" ] || { echo "--image is required" >&2; exit 2; }

# --- 1. GEMINI_API_KEY 를 SSM Parameter Store 에서 조회 ---
GEMINI_API_KEY=$(
docker run --rm --network host \
-e AWS_DEFAULT_REGION=ap-northeast-2 \
amazon/aws-cli ssm get-parameter \
--name /piki-extractor/gemini-api-key \
--with-decryption \
--query 'Parameter.Value' \
--output text
) || { echo "필수 SSM 파라미터 조회 실패: /piki-extractor/gemini-api-key" >&2; exit 1; }

# --- 2. 기존 컨테이너 제거 후 재기동 (없으면 무시) ---
docker rm -f piki-extractor >/dev/null 2>&1 || true

docker run -d --name piki-extractor --restart unless-stopped \
-p 8090:8090 \
--add-host=host.docker.internal:host-gateway \
--label piki.observe=true \
--label piki.service=piki-extractor \
--label piki.metrics.port=8090 \
--label piki.metrics.path=/actuator/prometheus \
-e AWS_REGION=ap-northeast-2 \
-e GEMINI_API_KEY="$GEMINI_API_KEY" \
-e TRACING_OTLP_ENABLED=true \
-e OTLP_ENDPOINT=http://host.docker.internal:4318/v1/traces \
-e LOGGING_STRUCTURED_FORMAT_CONSOLE=ecs \
"$IMAGE"
Comment on lines +48 to +60

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Secret leaked via process command line.

GEMINI_API_KEY is passed with -e GEMINI_API_KEY="$GEMINI_API_KEY" on the docker run invocation. This value becomes part of the docker process's argv, which is visible to any local user via ps aux / /proc/<pid>/cmdline (and shell history if run interactively) while the command executes — a classic CWE-214-style secret exposure. docker inspect will still show the resolved env either way, but avoiding argv exposure is a straightforward hardening step.

🛡️ Proposed fix using an env-file with restricted permissions
+ENV_FILE=$(mktemp)
+chmod 600 "$ENV_FILE"
+trap 'rm -f "$ENV_FILE"' EXIT
+cat > "$ENV_FILE" <<EOF
+AWS_REGION=ap-northeast-2
+GEMINI_API_KEY=$GEMINI_API_KEY
+TRACING_OTLP_ENABLED=true
+OTLP_ENDPOINT=http://host.docker.internal:4318/v1/traces
+LOGGING_STRUCTURED_FORMAT_CONSOLE=ecs
+EOF
+
 docker run -d --name piki-extractor --restart unless-stopped \
   -p 8090:8090 \
   --add-host=host.docker.internal:host-gateway \
   --label piki.observe=true \
   --label piki.service=piki-extractor \
   --label piki.metrics.port=8090 \
   --label piki.metrics.path=/actuator/prometheus \
-  -e AWS_REGION=ap-northeast-2 \
-  -e GEMINI_API_KEY="$GEMINI_API_KEY" \
-  -e TRACING_OTLP_ENABLED=true \
-  -e OTLP_ENDPOINT=http://host.docker.internal:4318/v1/traces \
-  -e LOGGING_STRUCTURED_FORMAT_CONSOLE=ecs \
+  --env-file "$ENV_FILE" \
   "$IMAGE"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
docker run -d --name piki-extractor --restart unless-stopped \
-p 8090:8090 \
--add-host=host.docker.internal:host-gateway \
--label piki.observe=true \
--label piki.service=piki-extractor \
--label piki.metrics.port=8090 \
--label piki.metrics.path=/actuator/prometheus \
-e AWS_REGION=ap-northeast-2 \
-e GEMINI_API_KEY="$GEMINI_API_KEY" \
-e TRACING_OTLP_ENABLED=true \
-e OTLP_ENDPOINT=http://host.docker.internal:4318/v1/traces \
-e LOGGING_STRUCTURED_FORMAT_CONSOLE=ecs \
"$IMAGE"
ENV_FILE=$(mktemp)
chmod 600 "$ENV_FILE"
trap 'rm -f "$ENV_FILE"' EXIT
cat > "$ENV_FILE" <<EOF
AWS_REGION=ap-northeast-2
GEMINI_API_KEY=$GEMINI_API_KEY
TRACING_OTLP_ENABLED=true
OTLP_ENDPOINT=http://host.docker.internal:4318/v1/traces
LOGGING_STRUCTURED_FORMAT_CONSOLE=ecs
EOF
docker run -d --name piki-extractor --restart unless-stopped \
-p 8090:8090 \
--add-host=host.docker.internal:host-gateway \
--label piki.observe=true \
--label piki.service=piki-extractor \
--label piki.metrics.port=8090 \
--label piki.metrics.path=/actuator/prometheus \
--env-file "$ENV_FILE" \
"$IMAGE"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/scripts/run-extractor.sh` around lines 48 - 60, Remove the inline
GEMINI_API_KEY assignment from the docker run command in run-extractor.sh.
Create a temporary environment file with restricted permissions, write the key
there, pass it using Docker’s env-file option, and ensure the file is cleaned up
after the container starts while preserving the existing container
configuration.


# 라벨 4종(piki.observe/piki.service/piki.metrics.port/piki.metrics.path)은 공통 Alloy 의
# label opt-in 계약(TeamPiKi/infra contracts/observability.md) - 이 라벨이 있어야 박스
# Alloy 가 이 컨테이너를 스크레이프 대상으로 인식한다. piki.metrics.port 는 컨테이너
# 내부 포트가 아니라 "호스트 127.0.0.1 에서 닿는 포트" 다.
#
# -p 8090:8090 전체 바인드: 이 박스는 SG 로 인바운드가 앱 SG 에서만 허용되니(SG 격리)
# 외부 노출이 없다. core 앱은 이 박스의 private IP 로 이 포트를 호출한다.
#
# --add-host=host.docker.internal:host-gateway + OTLP_ENDPOINT=...host.docker.internal:4318:
# 트레이스는 앱이 박스 로컬 Alloy(4318, OTLP HTTP)로 능동적으로 push 하는 유일한 신호다.
# 이 컨테이너는 bridge 네트워크라 host-gateway 를 거쳐야 호스트의 Alloy 에 닿는다.
#
# LOGGING_STRUCTURED_FORMAT_CONSOLE=ecs: 공통 Loki 파이프라인이 ECS JSON 포맷을 전제로
# level 라벨링·trace 상관을 한다(Spring Boot 구조화 로깅, relaxed binding 으로 이 env 하나면
# 활성화된다). 형식이 다르면 그쪽 파이프라인이 파싱하지 못한다.
#
# 이 스크립트를 박스에 첫 적용할 때는 기존 임시 기동 컨테이너를
# `docker inspect piki-extractor` 로 대조해, 누락된 env 가 있으면 여기(이 스크립트)에
# 흡수시킨다 - 이 스크립트 밖에서 별도로 env 를 더하지 않는다.

# --- 3. 헬스 대기 - 공통 healthcheck 블록을 dogfooding ---
HEALTHCHECK_SH="/tmp/piki-run-extractor-healthcheck.sh"
curl -fsSL https://raw.githubusercontent.com/TeamPiKi/infra/main/blocks/healthcheck.sh -o "$HEALTHCHECK_SH" \
|| { echo "healthcheck.sh fetch 실패" >&2; exit 1; }
Comment on lines +82 to +85

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== run-extractor excerpt ==\n'
sed -n '60,120p' infra/scripts/run-extractor.sh

printf '\n== provision-observability excerpt ==\n'
sed -n '1,220p' infra/scripts/provision-observability.sh

printf '\n== search for healthcheck.sh and ref override ==\n'
rg -n "healthcheck\.sh|--ref|raw\.githubusercontent|bash .*healthcheck|HEALTHCHECK_SH" infra/scripts

Repository: TeamPiKi/extractor

Length of output: 6109


Pin the fetched healthcheck script instead of pulling main. run-extractor.sh downloads blocks/healthcheck.sh from TeamPiKi/infra/main and runs it with bash, so any change on that branch is immediately trusted on the prod extractor box. Add a --ref/commit-SHA pin or checksum/signature check like provision-observability.sh.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/scripts/run-extractor.sh` around lines 82 - 85, Update the healthcheck
download flow in run-extractor.sh around HEALTHCHECK_SH to pin the fetched
healthcheck.sh to a trusted immutable ref or validate it with a
checksum/signature, following the existing approach in
provision-observability.sh; ensure the script is only executed after that pin or
verification succeeds.


bash "$HEALTHCHECK_SH" \
--url http://localhost:8090/actuator/health \
--interval 5 \
--attempts 24 \
--expect-body '"status":"UP"'
Loading