Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 31 additions & 59 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,82 +1,54 @@
# lightcone-cli

**lightcone-cli** is Lightcone Research's agentic execution layer for
**ASTRA** (Agentic Schema for Transparent Research Analysis). It ships
the `lc` executable, a small set of Claude Code skills, and the
provenance/integrity machinery that ties an `astra.yaml` spec to a tree
**lightcone-cli** is [Lightcone Research][lr]'s agentic execution layer for
[**ASTRA**][astra] (Agentic Schema for Transparent Research Analysis).
It serves as the machinery that ties an analysis `astra.yaml` specification to a tree
of materialized outputs.

This site has two halves.
## Choose your path to the documentation

## I'm a researcher and I want to use this thing
<div class="grid cards" markdown>

Start at the [User Guide](user/index.md). Friendly, step-by-step, with
worked examples. You will not need to read any Python.
- __I want to try it out__ – :lucide-rocket:

The shortest possible path:
---

=== "uv"
```bash
uv tool install lightcone-cli
lc init my-analysis && cd my-analysis
claude # then, inside Claude Code: /lc-new
```
Installation instructions, step-by-step tutorial, and fast tour of the lightcone framework and its agentic and workflow capabilities.

=== "pip"
```bash
pip install lightcone-cli
lc init my-analysis && cd my-analysis
claude # then, inside Claude Code: /lc-new
```
[User Guide](user/index.md){ .md-button .md-button--primary }

## I work on lightcone-cli
- __I want to contribute__ – :lucide-cog:

Welcome — keep reading. The rest of this page is a fast tour for
contributors and maintainers; deep dives live in the sub-trees of the
nav.
---

---
In depth tour of the software architecture, agentic skills and API docs, as well as contribution instructions, aimed for
contributors and maintainers.

[Developer corner](maintainer.md){ .md-button .md-button--primary }

## Two packages, one toolchain
</div>

---

| Layer | Package | Role |
|-------|---------|------|
| **ASTRA** | `astra-tools` | Pure specification: schema, validation, prior insights & findings, evidence verification helpers, the `astra` CLI. |
| **lightcone-cli** | `lightcone-cli` | Agentic layer: project scaffolding, Snakemake-based execution, Dask cluster management, container builds, Claude Code skills. |
## Two libraries, one toolchain

`lightcone-cli` depends on `astra-tools`. The `astra` CLI handles the
spec itself (validation, paper management, evidence verification); the
`lc` CLI handles execution and the agent surface.
<div class="grid cards" markdown>

## What every materialized output gets
- __lightcone-cli__

A sidecar `.lightcone-manifest.json` next to its data, recording:
The library that ships the `lc` CLI which handle the agent surface (skills, plugins, guardrails) as well as the workflow execution layer. Depends on [**astra-tools**][astra-tools], the SDK for working with ASTRA analysis specifications.

- `code_version` = `sha256(recipe + container_image + decisions)`
- `data_version` = `sha256_dir(output_dir)` excluding the manifest itself
- `input_versions` for each declared input (chained data_version when the
input is another materialized output, `mtime-size` or `sha256` for
external files)
- `container_image`, `recipe`, `decisions`, `git_sha`, `lc_version`,
`host`, `slurm_job_id`, `finished_at`
[:fontawesome-brands-github: Repository][cli]{ .md-button }

`lc verify` recomputes `data_version` and walks the chain. Failures
surface as `tampered_data`, `broken_chain`, or `missing_manifest`. `lc
status` reads only manifests — works offline, no Snakemake or DB needed.
- __astra-tools__

## Development setup
The SDK for working with [**ASTRA**][astra] analysis specifications. This library provides the `astra` CLI which handles the [**ASTRA**][astra] lifecycle and validation process (schema, prior insights & findings, evidence verification helpers).

```bash
just install # uv sync --all-groups
just test # uv run pytest
just lint # ruff + mypy
just docs-serve # live docs preview
```
[:fontawesome-brands-github: Repository][astra-tools]{ .md-button }

## Where to read next
</div>

- [Architecture](architecture.md) — the full execution and integrity story
- [CLI Reference](cli/index.md) — every command currently shipped
- [Python API](api/index.md) — the engine modules
- [Skills](skills/index.md) — what each `/lc-*` skill does (including the `/lc-from-*` family)
- [Contributing](contributing/setup.md) — getting the dev loop running
[lr]: https://lightconeresearch.org/
[astra]: https://astra-spec.org/latest/
[astra-tools]: https://github.com/LightconeResearch/astra-tools
[cli]: https://github.com/LightconeResearch/lightcone-cli
48 changes: 48 additions & 0 deletions docs/maintainer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Developer corner

`lightcone-cli` is a thin shim over Snakemake that owns provenance. This guide
covers everything below the user surface: how the execution and integrity layers
work, what each engine module does, how the Claude Code plugin is structured, and
how to get a working dev loop.

If you're looking for the user-facing docs, the
[user guide](user/index.md) is the other half of this site.

## What this covers

- [Architecture](architecture.md) — the three subsystems (Snakefile generation,
manifest layer, cluster management) and the invariants that hold them together.
- [CLI Reference](cli/index.md) — every `lc` command: flags, options, and the
exact Snakemake invocation each one triggers.
- [Python API](api/index.md) — the `lightcone.engine.*` modules: public
signatures, common entry points, and module responsibilities.
- [Skills](skills/index.md) — what each `/lc-*` Claude Code skill does,
including the full `/lc-from-paper` reproduction bundle.
- [HPC & SLURM](hpc/index.md) — how the Dask cluster manager adapts to local,
SLURM, and external schedulers.
- [Contributing](contributing/setup.md) — clone, install, run the test suite,
lint, and build the docs locally.

## Get started in three commands

!!! tip "Dev loop"

```bash
git clone https://github.com/LightconeResearch/lightcone-cli.git
cd lightcone-cli
just install # uv sync --all-groups
just test # pytest
```

Run `just` with no arguments to see all available recipes.

## What lightcone-cli *owns*

The codebase is intentionally small. Snakemake handles DAG construction,
parallelism, cluster submission, staleness detection, locking, and log capture —
we do not replicate any of that. The parts that are ours:

- **Snakefile generator** — translates `astra.yaml` into `.lightcone/Snakefile`.
- **Manifest layer** — writes and verifies `.lightcone-manifest.json` per output.
- **Cluster manager** — picks local / SLURM / external Dask shape at runtime.
- **Claude Code plugin** — skills, hooks, and agents bundled into the wheel.
27 changes: 15 additions & 12 deletions docs/user/index.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,22 @@
# Welcome
# Welcome to the user guide

`lightcone-cli` is a small toolchain that turns a research question into
a reproducible analysis. You describe what you're trying to learn,
a cli agent helps you turn that into a precise specification, and the
`lc` command line keeps the resulting code, decisions, and outputs in
sync — forever.
sync thanks to the [**ASTRA**][astra] specification.

You don't need to write Python or YAML by hand. The agent handles the
implementation; **you stay in charge of the scientific choices**.
No need to write code by hand, **you stay in charge of the scientific choices**, the agent handles the implementation.

## What this guide covers

- [Install](install.md) — get `lc` and Claude Code running on your
machine.
- [Install](install.md) — get the `lc` command line and Claude Code running on your
machine or on a cluster.
- [Getting Started](getting-started.md) — create your first project,
run it end-to-end, and understand what each piece does.
- [The Agentic Workflow](agent-workflow.md) — `/lc-new`,
`/lc-from-code`, `/lc-from-paper`, and `/lc-feedback` — what each
one does and when to reach for it.
command does and when to reach for it.
- [Running on a Cluster](cluster.md) — taking your analysis to a SLURM
HPC system, including Perlmutter-specific notes.
- [Troubleshooting](troubleshooting.md) — common issues and how to
Expand All @@ -28,22 +27,24 @@ implementation; **you stay in charge of the scientific choices**.
## What you'll do, in three lines

!!! tip "Quick start"

=== "uv"
```bash
uv tool install lightcone-cli
lc init my-analysis && cd my-analysis
claude # then, inside Claude Code: /lc-new
claude
# then, inside Claude Code, run /lc-new
```

=== "pip"
```bash
pip install lightcone-cli
lc init my-analysis && cd my-analysis
claude # then, inside Claude Code: /lc-new
claude
# then, inside Claude Code: /lc-new
```

That's the shortest possible path. The rest of the guide is the
unhurried version.
That's the shortest possible path. The rest of the guide is the unhurried version.

## What lightcone-cli is *not*

Expand All @@ -56,4 +57,6 @@ unhurried version.
inside the agent harness (Claude Code for now).

If you'd rather skim the design and architecture, the
[maintainer docs](../index.md) are the other half of this site.
[maintainer docs](../maintainer.md) are the other half of this site.

[astra]: https://astra-spec.org/latest/
3 changes: 2 additions & 1 deletion zensical.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ site_name = "lightcone-cli"
site_description = "Agentic execution layer for ASTRA research pipelines"
site_author = "Lightcone Research Team"
repo_url = "https://github.com/LightconeResearch/lightcone-cli"
repo_name = "LightconeResearch/lightcone-cli"
copyright = "&copy; 2026 Lightcone Research"
docs_dir = "docs"
extra_css = ["stylesheets/extra.css"]
Expand Down Expand Up @@ -60,6 +61,7 @@ nav = [
{"Authoring Skills" = "skills/authoring.md"},
]},
]},
{"ASTRA docs" = "https://astra-spec.org/latest/"},
]

[project.theme]
Expand All @@ -69,7 +71,6 @@ favicon = "assets/favicon.svg"
features = [
"navigation.tabs",
"navigation.sections",
"navigation.expand",
"navigation.top",
"search.highlight",
"content.code.copy",
Expand Down
Loading