LLM Inference Handbook

This repository contains the source for the LLM Inference Handbook, a practical guide for understanding, optimizing, scaling, and operating LLM inference systems.

The handbook is a standalone Docusaurus site styled with the Modular design language system. Its future canonical source home is modular/llm-inference-handbook.

Local preview

Install dependencies and start the local development server:

pnpm install
pnpm start

The site runs at http://localhost:3000/.

To build or serve the production output locally:

pnpm build
pnpm serve

Markdown linting

Markdown files are linted with rumdl. Prose is wrapped at 80 characters. Run the same checks as CI locally:

pnpm lint:md       # check for issues
pnpm lint:md:fix   # auto-fix

Some line-length violations (long URLs, API references) may need manual reflow or <!-- rumdl-disable MD013 --> blocks.

Editor integration

rumdl supports in-editor linting via:

• VS Code extension: Install from the marketplace, or run pnpm exec rumdl vscode. See VS Code setup below to match CI rules and enable fix-on-save.
• LSP server: Run pnpm exec rumdl server and point your editor's LSP client at it.

VS Code setup

By default the extension uses its own built-in rules, not this repo's config. To match CI, point it at .rumdl.toml and apply rumdl fixes on save.

This repo ships recommended settings in .vscode/recommended_settings.json. Either copy those keys into .vscode/settings.json (workspace-scoped, gitignored) or into your user settings (~/Library/Application Support/Cursor/User/settings.json on macOS):

{
  "rumdl.configPath": ".rumdl.toml",
  "[markdown]": {
    "editor.formatOnSave": false,
    "editor.codeActionsOnSave": { "source.fixAll.rumdl": "explicit" }
  },
  "[mdx]": {
    "editor.formatOnSave": false,
    "editor.codeActionsOnSave": { "source.fixAll.rumdl": "explicit" }
  },
  "editor.rulers": [80]
}

Install the recommended extensions when prompted, or open the Extensions view and run Extensions: Show Recommended Extensions.

Contributing

Contributions are welcome! Feel free to open issues, suggest improvements, or submit pull requests.

Licenses

This handbook follows the upstream dual-license posture:

• All files in the docs/ folder are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) License.
• All other files are licensed under the Apache License 2.0.