fix(block-kit): resolve CLI before validation

[wildcard]

Summary

Addresses the CLI-validation portion of #99 (items 6–10).

• resolve the public Slack CLI once before Fast Path, Modification Mode, or the
  full workflow, then reuse that result for validation and preview
• require the canonical no-auth CLI validation call when the CLI is available
• keep semantic validation failures on the CLI instead of silently changing
  transport
• retry config/log permission failures only through the host's normal scoped
  approval path, and stop unvalidated if that approval is denied
• disclose the final validation transport and the reason for any curl fallback

This draft intentionally leaves the issue's five editorial simplifications for
maintainer direction. That keeps the demonstrated CLI behavior independently
reviewable and avoids deciding the Step 2 overlap with #85 in the same change.

Design rationale and alternatives considered

• Resolve once before choosing a workflow. Keeping detection in Step 5 was
  smaller textually, but it reproduces the observed late-discovery problem and
  leaves Fast Path without a recorded capability. Installing the CLI during
  preflight was rejected because validation already has a public curl fallback
  and installation is an unrelated mutation.
• Call the canonical command directly. Generic help-driven discovery remains
  useful for changing commands such as preview, but api blocks.validate --no-auth is already documented and owned by Slack CLI. Probing three
  unsupported forms first added latency without improving compatibility.
• Keep semantic errors on one transport. Switching to curl after parsed
  ok:false would conceal a payload error as a transport problem. The skill
  instead corrects the payload and retries through the same resolved CLI.
• Do not route around denied permission. An earlier alternative allowed curl
  after a config/log write denial. Self-review rejected it because another
  executable would bypass the operator's denied boundary; the final wording
  stops unvalidated instead.
• Submit items 6-10 as the draft slice. Implementing all ten issue items was
  locally validated, but it mixes the objectively demonstrated CLI correction
  with editorial changes and the semantic overlap in feat(block-kit): add poll, wizard modal, and empty state templates to common patterns #85. This draft asks for
  maintainer direction before combining those concerns.

The formal inline self-review anchors these decisions to the exact changed
lines. It contains no apply-able suggestion because review did not identify a
concrete replacement that is better than the submitted wording; manufacturing
one would add noise rather than help review.

Preview

The attached controlled Claude Code recording is prior supporting evidence from
the broader locally validated candidate that motivated this narrow slice. An
intermediate implementation probes three unsupported help/version forms before
validation; the corrected implementation fingerprints the CLI and immediately
uses the canonical validation command.

claude-code-review-correction-combined.mp4

Video provenance and objective command oracle

The recording uses the same prompt, controlled executable, Claude Fable 5
model, and high effort on both sides. That model/effort was unnecessarily
expensive, but keeping it identical makes the comparison controlled. Future
recordings default to Sonnet 5 at medium effort.

This is a controlled CLI fixture—not live Slack API execution or an official
client-support claim. The first clip is intermediate implementation commit
5e219b8, not upstream main; the final clip is the broader all-ten local
candidate edb8e35, not this tooling-only branch. It demonstrates the same
CLI-state-machine correction isolated by this draft:

INTERMEDIATE  fingerprint -> api --help rejected -> --version rejected ->
              help rejected -> canonical blocks.validate -> ok:true
FINAL         fingerprint -> canonical blocks.validate -> ok:true

Neither event oracle contains curl. The public-safe combined artifact is H.264,
174 seconds, SHA-256
b618dc2bea9f55675c392f9238b51b2d3cf55c7e70f7879e16fed61d0e546837.

Testing

• make lint — passed
• make typecheck — passed
• make test-unit — 22 passed, including four focused Block Kit validation
  policy tests
• claude plugin validate . — passed
• PYTEST_ADDOPTS='-k skill-block-kit-modal' make test-eval — the one
  change-relevant Block Kit tool-selection scenario passed
• make test — executed with working Gemini and Slack MCP credentials on exact
  draft head 5819e368; units passed 22/22 and evals passed 26/31

The five full-suite failures were outside this PR's three-file Block Kit diff:
two transient Gemini 503 UNAVAILABLE responses, two existing scenarios that
expect slack_list_channel_members although the current hosted MCP endpoint
did not return that tool, and one model response using slack_docs instead of
the accepted slack-docs skill name.

For comparison, the same make test command on pristine upstream/main
77a10794 under the same host, model, MCP token, and environment passed only
22/31 evals and reproduced all three failure classes. This comparison shows the
failures are baseline/environmental; it does not make the full command green.

Prior supporting evidence, not exact-branch test results:

• controlled Claude Code and Codex checks on the broader local candidate using
  the same Block Kit prompt and canonical validation command
• Slack CLI v4.6.0 public blocks.validate --no-auth checks, which verify the
  canonical command contract but not this skill revision's agent behavior

Why the full-test checkbox remains unchecked

The required command was run rather than replaced by narrower tests. Its unit
portion and the change-relevant Block Kit eval pass, and the branch performs no
worse than pristine main. However, make test exits nonzero, so the truthful
checklist state remains unchecked pending maintainer CI or resolution of the
current upstream eval/tool-surface failures.

Notes

This draft changes only skills/block-kit/SKILL.md, a focused unit contract,
and one patch changeset. It does not include #99's editorial cleanup, modify
common-patterns.md, change Slack CLI or MCP/OAuth behavior, or claim official
Codex support.

The draft is intentionally open for scope discussion: if maintainers prefer
the issue's simplification and CLI-hardening halves together, the complete
locally validated version can be folded into this branch. If they prefer one
behavioral concern per PR, this branch is independently reviewable as-is.

Requirements

• I've read and understood the Contributing Guidelines and have done my best effort to follow them.
• I've read and agree to the Code of Conduct.
• I've run make test and the tests pass.

make test was run, but the suite is not fully green, so this combined
requirement remains unchecked. See the
full test report
for the passing subsets, five failure classes, and same-environment upstream
comparison.

[changeset-bot]

🦋 Changeset detected

Latest commit: 5819e36

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
slack	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[salesforce-cla]

Thanks for the contribution! Before we can merge this, we need @wildcard to sign the Salesforce Inc. Contributor License Agreement.

[wildcard]

Self-review guide for the narrowed CLI-validation slice. I checked the exact draft head against issue #99 items 6–10, the repository contribution rules, project-native tests, permission boundaries, and the open #85 overlap. These notes explain the key design decisions; they are not maintainer approval.

[wildcard]

The preflight is deliberately capability-only and runs before every workflow path. Recording SLACK_CMD once avoids the original late-discovery failure and prevents validation and preview from repeating detection or drifting into the CLI skill's install/alias branch.

[wildcard]

This is the narrow exception to generic help-driven command discovery: blocks.validate --no-auth is a documented Slack CLI contract, so the skill attempts it directly. The controlled Claude evidence showed that this removes three rejected help/version probes before validation.

[wildcard]

The fallback boundary received the closest security review. A config/log denial may retry the identical command only through normal scoped host approval; if approval is denied, validation stops rather than using curl to route around the operator's decision. Semantic API and shared network failures likewise cannot be hidden by changing transport.

[wildcard]

These focused tests intentionally assert durable policy invariants rather than transcript wording: preflight ordering/reuse, canonical calls, semantic same-transport retry, permission non-bypass, and final disclosure. Native agent runs remain supporting behavioral evidence, not unit tests.

[wildcard]

Full-suite follow-up for exact draft head 5819e368:

• make test was rerun with working Gemini and Slack MCP credentials.
• Unit tests: 22/22 passed.
• Gemini tool-selection evals: 26/31 passed.
• The focused Block Kit scenario passed independently with
  PYTEST_ADDOPTS='-k skill-block-kit-modal' make test-eval.

The five full-suite failures were:

1. Two transient Gemini 503 UNAVAILABLE responses.
2. Two existing scenarios expecting slack_list_channel_members, which was
   absent from the 10 tools returned by the current hosted Slack MCP endpoint.
3. One model response choosing slack_docs rather than the accepted
   slack-docs name.

As a same-environment control, I ran the identical full command on pristine
upstream/main at 77a10794. Its units passed 18/18, while its evals passed
22/31 and reproduced all three failure classes above. This indicates that the
five failures observed on this branch are not introduced by the three-file
Block Kit change. It does not convert the full run into a pass, so the PR's
make test requirement remains unchecked.

The change-specific regression contract remains deterministic: four unit tests
cover early one-time CLI resolution, canonical no-auth validation calls,
same-transport semantic retry, permission non-bypass, fallback classification,
and final transport disclosure.