diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..45a9e33 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,7 @@ +# SPDX-FileCopyrightText: 2026 [ernolf] Raphael Gradenwitz +# SPDX-License-Identifier: MIT + +# Store and check out every text file with LF, regardless of a contributor's +# core.autocrlf setting, so line endings never leak into a diff. Binary files +# are detected by text=auto and left untouched. +* text=auto eol=lf diff --git a/README.md b/README.md index 1f7e2a7..1b58d99 100644 --- a/README.md +++ b/README.md @@ -8,330 +8,58 @@ The Swiss Army knife for Nextcloud app development: one generic `Makefile` for building, packaging, deploying, versioning and App Store management of a Nextcloud app. -Everything is derived from the app itself, so there is nothing to configure for a standard app: drop it in, run `make`, done. The package managers run in throwaway containers, so the host needs neither PHP nor Node. +Everything is derived from the app itself, so a standard app needs no build config at all โ€” drop it in, run `make`, done. Composer and npm run in throwaway containers, so the host needs neither PHP nor Node. > [!TIP] -> **TL;DR** โ€” Commit the [bootstrap stub](#-installation) as your app's `Makefile`, then `make` gives you build, packaging, test-deploy, release and App Store targets โ€” all in throwaway containers, so no PHP or Node on your host. `make` alone prints a colorized, annotated help; `make help-` explains one target in depth. +> **๐Ÿ“– The full documentation lives in the [ncmake wiki](https://github.com/ernolf/ncmake/wiki).** This page is just the quick start โ€” every guide, reference and background piece is a wiki page, kept current there. -- [Installation](#-installation) -- [What happens when you run make](#-what-happens-when-you-run-make) -- [How ncmake understands your app](#-how-ncmake-understands-your-app) -- [The container runtime](#-the-container-runtime) -- [Building](#-building) -- [Packaging: the shipped file set](#-packaging-the-shipped-file-set) -- [Deploying to a test instance](#-deploying-to-a-test-instance) -- [Releasing](#-releasing) -- [CI workflows](#-ci-workflows) -- [The Installation section for your app's README](#-the-installation-section-for-your-apps-readme) -- [App Store management](#-app-store-management) -- [Per-app tuning](#-per-app-tuning) -- [Variables](#-variables) -- [Target reference](#-target-reference) -- [Requirements](#-requirements) -- [Show that your app uses ncmake](#-show-that-your-app-uses-ncmake) -- [Documentation](#-documentation) +## Quick start -## ๐Ÿงฉ Installation - -### The bootstrap stub (recommended) - -Put the bootstrap stub into the root of your app repository, once: +Commit the bootstrap stub as your app's `Makefile`, once: ```sh curl -fLO https://raw.githubusercontent.com/ernolf/ncmake/main/bootstrap/Makefile git add Makefile ``` -The stub is a dozen lines that never change. It fetches the real Makefile into a per-machine cache and includes it from there. Every developer who clones your app and runs `make` automatically gets the current ncmake, on every machine, for every app, from one shared cache. - -The stub is the only thing you install by hand. Everything else ncmake contributes to your repository โ€” the CI workflows (see [CI workflows](#-ci-workflows)) โ€” is installed and updated through `make` targets once the stub is in place. - -> [!IMPORTANT] -> **Only the stub lands in your repository.** The stub file you committed stays byte-identical forever; the fetched Makefile lives in `~/.cache/ncmake/`, outside of every project. Running `make` creates or modifies nothing in your checkout (apart from the usual build outputs such as `build/`, `js/` and `vendor/`, which belong in your `.gitignore` anyway, as in every Nextcloud app). `git status` stays clean; there is nothing extra to ignore. - -**How the cache stays current.** At most once per day (`NCMAKE_TTL_MIN`, default 1440 minutes) the cached Makefile checks upstream with a conditional GET (ETag): unchanged or offline keeps the cache, a new version replaces it and is used from the next run on. `make self-update` forces a refresh at any time. - -**Pinning a version.** By default the stub follows the `main` branch. To pin your app to a fixed ncmake version, set `NCMAKE_REF` in the stub to a tag: - -```make -NCMAKE_REF ?= v1.0.0 -``` - -### The full copy (self-contained alternative) - -If you prefer a repository without any fetch-at-build-time behavior, commit the full `Makefile` instead: - -```sh -curl -fLo Makefile https://raw.githubusercontent.com/ernolf/ncmake/main/core/Makefile -``` - -A committed copy never modifies itself. `make self-update` downloads the newest version over it; review the diff and commit it like any other change. - -## ๐Ÿ” What happens when you run make - -```mermaid -flowchart TD - A["make <target>"] --> B{Stub or full copy?} - B -- stub --> C{"~/.cache/ncmake/
Makefile present?"} - C -- no --> D[fetch once from GitHub] - C -- yes --> E{"older than
NCMAKE_TTL_MIN?"} - E -- yes --> F["conditional GET (ETag):
new version โ†’ refresh cache
unchanged/offline โ†’ keep cache"] - E -- no --> G - D --> G[include cached Makefile] - F --> G - B -- full copy --> H[use the committed Makefile as is] - G --> I[run the target] - H --> I -``` - -The first `make` after a fresh `git clone` needs network once (to fill the cache); after that everything works offline. - -## ๐Ÿง  How ncmake understands your app - -Nothing is configured twice, everything is read from files your app has anyway: - -| Fact | Source | -|---|---| -| App id | `` in `appinfo/info.xml` | -| Version | `` in `appinfo/info.xml` | -| PHP build needed? | `composer.json` declares runtime requirements (anything besides `php` and `ext-*`) | -| Frontend build needed? | `package.json` has a `build` script | -| Is `js/` (or `vendor/`) a build artifact? | `.gitignore` (evaluated via `git check-ignore`) | -| PHP container images | build image from the min-version in `appinfo/info.xml`; analysis on the current composer image | -| Node container image | `engines.node` in `package.json` | - -The `.gitignore` line deserves a word: when `js/` is gitignored, it is a build output and must exist before packaging (`make dist` refuses otherwise and tells you to run `make build`). When `js/` is committed, as in apps that ship their built frontend in git, a fresh checkout is already complete and packages without building. The same logic applies to `vendor/`. - -The tarball and the deployed directory are always named after the **app id**, regardless of what your checkout directory is called. - -## ๐Ÿณ The container runtime - -`composer` and `npm` never run on your host by default. Each invocation starts a throwaway container (`--rm`), does its work in your bind-mounted checkout and disappears. Your host needs no PHP, no Node, no version juggling, and you can build against exactly the PHP the app declares as its minimum. - -The runtime is auto-detected (podman preferred, then docker) and can be chosen per call, for example `make build RUNTIME=docker`: - -| `RUNTIME=` | What it is | Notes | -|---|---|---| -| `podman` | rootless podman (default when podman exists) | daemonless, no idle cost, files owned by you | -| `docker` | standard rootful docker | ncmake maps your uid/gid into the container, so no root-owned files appear | -| `docker-rootless` | rootless docker | | -| `bare` | no container | composer and npm must be on the PATH | - -Two PHP images are picked per task, both maintained upstream so ncmake ships no image of its own. The **build** runs in the Nextcloud CI image for your declared min-version (`ghcr.io/nextcloud/continuous-integration-php`): fully tooled (composer, git, unzip) and resolving dependencies against the support floor, which is what packaging needs โ€” its frozen patch level is irrelevant for shipping runtime deps. Everything **interactive** (`make composer`, `make psalm`) runs in the official `composer` image (`docker.io/library/composer:2`), which always ships the newest PHP patch alongside composer, git and unzip; psalm needs a current runtime, while the analysed PHP level stays pinned through `psalm.xml`, so the image's fixed PHP version does not affect the result. Node runs in `node:` from your `engines.node`. An app that needs a specific PHP version or an extra extension (`ext-gd`, `ext-intl`, โ€ฆ) points `php_image`/`analysis_image` at another image in `ncmake.mk`. All images can be overridden (see [Variables](#-variables)). - -> [!CAUTION] -> On SELinux hosts (Fedora, RHEL) bind mounts may need a `:z` label. If you hit permission errors there, run with `RUNTIME=bare` or adjust your container policy. - -## ๐Ÿ”จ Building - -```sh -make build -``` - -runs the detected build commands, each in its container: - -- `composer install --no-dev --no-scripts --prefer-dist --no-progress` when `composer.json` declares runtime requirements -- `npm ci && npm run build` when `package.json` has a `build` script - -When a side does not apply, it is skipped with a note. Apps with special build steps override the commands in `ncmake.mk` (see [Per-app tuning](#-per-app-tuning)). - -For everything beyond the release build there are generic pass-through targets running in the same throwaway containers โ€” the host needs no toolchain even for the dev setup: - -```sh -make composer ARGS=install # install dependencies INCLUDING dev tools (vendor-bin etc.) -make composer ARGS="cs:check" # run a composer script -make psalm # run static analysis (psalm) on the current composer image -make npm ARGS=ci # install frontend dependencies -make npm ARGS="run test" # run the frontend tests -``` - -`make psalm` runs `composer psalm` on the current composer image, because psalm needs a newer runtime than the build floor; run `make composer ARGS=install` once beforehand to install the dev tools. Pass extra flags straight to psalm with `make psalm ARGS="..."`; they are forwarded past composer's own options (`composer psalm -- ...`), so e.g. `make psalm ARGS="--show-info=true"` reaches psalm rather than being eaten by composer. `make composer` uses that same current image by default, so dev tools just work; only `make build` drops to the min-version for package-correct resolution. Add `PHP=min` to route any other composer command through the build floor instead. - -`make dist-clean` resets to a pristine checkout first (it removes every git-ignored build output: `vendor/`, `node_modules/`, `js/`, caches), so - -```sh -make dist-clean && make build -``` - -is the reproducible from-scratch build. - -## ๐Ÿ“ฆ Packaging: the shipped file set - -What ends up in a release is defined as an **allowlist** (the keep model), not as an exclude list. Shipped are the standard app paths, each only when it exists: - -```text -appinfo/ lib/ l10n/ templates/ img/ css/ js/ vendor/ LICENSES/ -CHANGELOG.md AUTHORS.md REUSE.toml COPYING COPYING.md LICENSE LICENSE.md -``` - -> [!NOTE] -> A new dev file in your repository can never leak into the tarball, because it is not on the list. A missing runtime directory fails loudly instead of silently shipping a broken app. - -```mermaid -flowchart LR - A[working tree] -- "allowlist
+ .nextcloudignore" --> B["build/stage/<app_id>/"] - B -- "tar (make dist)" --> C["build/artifacts/dist/
<app_id>-<version>.tar.gz"] - B -- "rsync (make rsync)" --> D["<apps-dir>/<app_id>/"] - B -- "docker cp (make cp)" --> E["<container>:<apps-dir>/<app_id>/"] -``` - -`make dist` materializes the file set once into a staging directory and packs it; `make rsync` and `make cp` deploy the very same staging directory. One mechanism, one source of truth: what you deploy for testing is byte-for-byte what a release ships. - -## ๐Ÿš€ Deploying to a test instance - -```sh -make build -make rsync TARGET=/var/www/nextcloud/apps OCC=1 -``` - -`make rsync` deploys the shipped file set straight into an `apps/` directory (local or over SSH); `make cp` does the same into a running container such as Nextcloud All-in-One. `OCC=1` wraps the sync into the full `occ app:disable โ†’ chown โ†’ occ app:enable` refresh cycle, so `info.xml` is re-read and migrations run. `make dist`, `make rsync` and `make cp` all deploy the very same staged file set โ€” one source of truth, byte-for-byte what a release ships. - -The full walkthrough โ€” `TARGET` forms, remote SSH, `ENGINE`, `web_user`, the `-it`/TTY detail โ€” lives in the install guide: **[Method 3: `make rsync`](doc/INSTALL.md#-method-3--deploy-with-make-rsync)** and **[Method 4: `make cp`](doc/INSTALL.md#-method-4--deploy-into-a-running-container-with-make-cp)**. - -> [!TIP] -> That guide is written for the people who *install* your app rather than develop it, so every ncmake app can link its users straight to [doc/INSTALL.md](doc/INSTALL.md) and [keep its own README down to a couple of lines](#-the-installation-section-for-your-apps-readme). - -## ๐Ÿšข Releasing - -The release flow assumes a protected `main` (required checks, no direct pushes), which is good practice anyway: - -```mermaid -flowchart LR - A["make version
(on main)"] -- "branch ncmake/release/X.Y.Z
bump + lockfile sync + commit" --> B["make changelog
review, extend, commit"] - B --> C[push, PR, merge] - C --> D["git pull
make tag"] - D -- "signed tag vX.Y.Z" --> E[GitHub release
+ tarball asset] - E --> F["make publish
(App Store)"] -``` - -**`make version`** (run on `main`) prompts for the new version, validates it against the latest tag (`sort -V`, must be greater), branches off into `ncmake/release/X.Y.Z` (branches created by ncmake always carry the `ncmake/` prefix, so they are immediately distinguishable from hand-made branches) and commits the bump there: `appinfo/info.xml`, plus `composer.json`/`package.json` when present, plus the re-synced lockfiles (synced inside the containers, so the bump commit is complete and CI-clean). - -**`make changelog`** (on the release branch) generates the `## [X.Y.Z]` section for the version in `info.xml` from the conventional commits since the last tag, and inserts it above the previous release, together with its `[X.Y.Z]:` link reference to the GitHub release tag (the repository URL is derived from the origin remote). Only user-visible changes make it in: `feat` becomes *Added*, `fix` becomes *Fixed*, `perf` becomes *Changed*; build, ci, test, chore, docs, refactor, style, merge commits and the daily Transifex bot commits are left out. The rest of the file is never touched, so the generated section can be freely edited and extended before committing, and hand-written history survives. It also prints the exact commit command: while the bump commit from make version is still unpushed, the changelog is folded into it via git commit --amend --no-edit (one commit per release); otherwise it suggests a separate build(release): update changelog for X.Y.Z commit. Rerunning is safe: an existing section is not duplicated, and when nothing user-visible happened since the last tag it says so (add a hand-written section then, for example for translation updates). It runs `git-cliff` via `npx` in the node container; an app-provided `cliff.toml` overrides the built-in configuration. - -**`make tag`** (back on `main`, after the merge) refuses to re-tag, refuses when `CHANGELOG.md` has no `## [X.Y.Z]` section, shows a fat reminder that a tag freezes the current commit, then creates and pushes the **signed** `vX.Y.Z` tag after your confirmation. - -`make dist` builds the tarball to attach to the GitHub release; `make sign` (base64 signature) and `make release` (dist + sign) come from the [appstore module](#-app-store-management). - -## ๐Ÿค– CI workflows - -The workflows of an ncmake app are managed by the **workflow manager**, a developer module: `make dev-init` fetches the ncmake modules once per machine, then +That single dozen-line file is all your repository carries: it fetches the real Makefile into a per-machine cache and includes it from there, so every clone of your app runs the current ncmake from one shared cache โ€” nothing else lands in your checkout. Then: ```sh -make workflows-list -make workflows-install W="release reuse lint-php" -git add .github/workflows/ +make # colorized, annotated help with your app id, version and cert status +make build # composer + npm, each in a throwaway container +make dist # stage the runtime file set and pack the release tarball ``` -lists everything the sources offer โ€” ncmake's own workflow-updater- and release workflow plus the official [nextcloud/.github workflow templates](https://github.com/nextcloud/.github/tree/master/workflow-templates) โ€” installs your pick and records the provenance in a lock file, so `make workflows-update` later distinguishes upstream updates from your local edits. Discovery is live via the GitHub API: new upstream templates appear in the list without any ncmake update. +New here? Start with **[Getting started](https://github.com/ernolf/ncmake/wiki/Getting-started)** and the **[step-by-step walkthrough](https://github.com/ernolf/ncmake/wiki/Step-by-step)**. -The full guide โ€” module setup, sources, status model, the lock file, placeholder handling and the workflow-updater- and release workflows ncmake ships itself โ€” lives in **[doc/WORKFLOWS.md](doc/WORKFLOWS.md)**. +## What ncmake does -ncmake can keep the workflows current for you: the [workflow updater](doc/AUTOUPDATE_WORKFLOW.md) opens a pull request when they drift from upstream. It replaces Dependabot for `.github/workflows/` and authenticates with a [GitHub App](doc/GITHUB_APP.md). +- **No host toolchain** โ€” composer and npm run in throwaway containers on the PHP and Node versions the app declares; you need only podman or docker. +- **Nothing to configure** โ€” app id, version, build steps and the shipped file set are read from `info.xml`, `composer.json`, `package.json` and `.gitignore`. +- **Keep-model packaging** โ€” a release ships an allowlist of runtime paths, so a stray dev file can never leak in, and the same staged set feeds `dist`, `rsync` and `cp`. +- **One shared, self-updating Makefile** โ€” a dozen-line stub per app; update ncmake once and every app on the machine follows. +- **The full release lifecycle** โ€” a validated version bump, a changelog from your conventional commits, a signed tag, and App Store signing and publishing. +- **Managed CI workflows** โ€” installed from upstream templates, tracked against local edits, and kept current by an auto-updater that opens pull requests. -## ๐Ÿ“ The Installation section for your app's README +Wondering how it compares to [krankerl](https://github.com/ChristophWurst/krankerl), or why it is built this way? โ†’ **[Why ncmake](https://github.com/ernolf/ncmake/wiki/Why-ncmake)** -Every ncmake app's install instructions should read the same and stay short: one line for the App Store, one that points at the shared [install guide](doc/INSTALL.md). Do not repeat tarball or `make` steps in the app README โ€” they live in the guide, in one place, so a change is made once. +## Documentation -If the app is in the App Store: +Everything is in the **[wiki](https://github.com/ernolf/ncmake/wiki)**; its sidebar has the full set. Good entry points: -```markdown -## Installation - -The app is published in the [App Store](https://apps.nextcloud.com/apps/). Install it through [Nextcloud's app management UI](https://docs.nextcloud.com/server/latest/admin_manual/apps_management.html#managing-apps) (**Apps** โ†’ search for **** โ†’ Install) or with `occ app:enable `. - -It is built with [ncmake](https://github.com/ernolf/ncmake). To build and install it from source โ€” release tarball, `make rsync` or `make cp` โ€” see the [installation guide](https://github.com/ernolf/ncmake/blob/main/doc/INSTALL.md). -``` - -If it is not (yet) in the App Store, drop the first paragraph: - -```markdown -## Installation +- **New to ncmake** โ†’ [Getting started](https://github.com/ernolf/ncmake/wiki/Getting-started) ยท [Step by step](https://github.com/ernolf/ncmake/wiki/Step-by-step) +- **Building & releasing** โ†’ [How ncmake understands your app](https://github.com/ernolf/ncmake/wiki/How-ncmake-understands-your-app) ยท [Building and packaging](https://github.com/ernolf/ncmake/wiki/Building-and-packaging) ยท [Releasing](https://github.com/ernolf/ncmake/wiki/Releasing) ยท [Per-app tuning](https://github.com/ernolf/ncmake/wiki/Per-app-tuning) ยท [Target reference](https://github.com/ernolf/ncmake/wiki/Target-reference) +- **CI workflows** โ†’ [Workflows](https://github.com/ernolf/ncmake/wiki/Workflows) ยท [Workflow updater](https://github.com/ernolf/ncmake/wiki/Workflow-updater) ยท [GitHub App](https://github.com/ernolf/ncmake/wiki/GitHub-App) ยท [GitHub PAT](https://github.com/ernolf/ncmake/wiki/GitHub-PAT) ยท [Deleting merged branches](https://github.com/ernolf/ncmake/wiki/Deleting-merged-branches) +- **App Store** โ†’ [App Store](https://github.com/ernolf/ncmake/wiki/App-Store) +- **Installing an ncmake app** (for your app's users) โ†’ [Installation](https://github.com/ernolf/ncmake/wiki/Installation) -This app is not yet in the App Store. It is built with [ncmake](https://github.com/ernolf/ncmake). To build and install it from source โ€” release tarball, `make rsync` or `make cp` โ€” see the [installation guide](https://github.com/ernolf/ncmake/blob/main/doc/INSTALL.md). -``` - -Replace `` with the app id and `` with the app's display name (the exact term users search for in the App Store). Anything genuinely app-specific โ€” a migration note, a link to the app's own developer docs โ€” follows as its own subsection. - -## ๐Ÿช App Store management - -Everything that talks to the [App Store](https://apps.nextcloud.com) or needs the signing key lives in the **appstore module**, another developer module: `make dev-init` once, then - -```sh -make csr # one-time: key + certificate request -make register # one-time: register app id and certificate -make publish GH=1 # per release: sign and submit the GitHub release asset -``` +## Requirements -plus the read-only queries (`list-releases`, `ratings`, ...), `delete-release` and the signing building blocks `sign` and `release`. `NIGHTLY=1` switches `publish` and `delete-release` to the store's nightly channel. `make help` shows whether token, certificate and key are in place (green check or red cross, with the real filename). +GNU make, git, curl, openssl, rsync, python3; optionally `xmllint` (ncmake falls back to `grep` without it). For containerized builds: podman or docker โ€” otherwise `RUNTIME=bare` with composer and npm on the `PATH`. -The full guide โ€” certificate directory, onboarding walkthrough, the publish flow and why it signs the downloaded bytes โ€” lives in **[doc/APPSTORE.md](doc/APPSTORE.md)**. +## Show that your app uses ncmake -## ๐Ÿ”ง Per-app tuning - -Most apps need none of this. - -**`.nextcloudignore`** removes files from within the shipped set (rsync exclude syntax, one pattern per line), for example test ballast inside shipped vendor packages: - -```text -/vendor/*/*/tests/ -``` - -**`ncmake.mk`** in the app root overrides single variables in plain make syntax. It is included first, so anything set there wins: - -```make -keep_extra = resources # extra runtime paths to ship -php_build_cmd = composer install --no-dev && php bin/generate.php -node_build_cmd = # empty = skip the npm build -php_image = ghcr.io/nextcloud/continuous-integration-php8.2:latest # build image -analysis_image = docker.io/library/composer:2.7 # pin the composer/psalm image -``` - -## ๐Ÿ“‹ Variables - -Set on the command line (`make build RUNTIME=bare`), in the environment, or persistently in `ncmake.mk`. - -| Variable | Default | Purpose | -|---|---|---| -| `RUNTIME` | auto (`podman`, else `docker`) | container runtime: `podman`, `docker`, `docker-rootless`, `bare` | -| `TARGET` | (required by `make rsync`/`make cp`) | apps parent directory: local, `user@host:` (rsync) or `:` (cp) | -| `OCC` | (unset) | `OCC=1` wraps a deploy into occ app:disable โ†’ chown โ†’ occ app:enable | -| `web_user` | `www-data` | web server user of the target instance (file owner, runs occ) | -| `ENGINE` | auto (`docker`, else `podman`) | container CLI for `make cp` (independent of `RUNTIME`) | -| `cert_dir` | `~/.nextcloud/certificates` | location of certificate, key and API token | -| `php_image` | `ghcr.io/nextcloud/continuous-integration-php` | build image (min-version, fully tooled) | -| `analysis_image` | `docker.io/library/composer:2` | image for `composer` and `psalm` (current PHP, fully tooled) | -| `PHP` | `max` | `max` โ†’ `analysis_image`, `min` โ†’ `php_image` (build floor); `make build` forces `min` | -| `node_image` | `node:` | Node container image | -| `keep_extra` | (empty) | additional paths for the shipped file set | -| `php_build_cmd` | auto-detected | PHP-side build command, empty skips | -| `node_build_cmd` | auto-detected | frontend build command, empty skips | -| `NCMAKE` | `$(NCMAKE_DIR)/Makefile-$(NCMAKE_REF)` | path to the core Makefile to include; point it at a local file to bypass the download and run an uncommitted change (development only, see [CONTRIBUTING](CONTRIBUTING.md)) | -| `NCMAKE_REF` | `main` | branch or tag the stub fetches | -| `NCMAKE_DIR` | `$XDG_CACHE_HOME/ncmake`, else `~/.cache/ncmake` | cache location of the shared Makefile | -| `NCMAKE_TTL_MIN` | `1440` | minutes between upstream freshness checks | - -## ๐ŸŽฏ Target reference - -`make` without a target prints the annotated, colorized help, including the detected app, version and certificate status. Colors turn off automatically when stdout is not a terminal and honor `NO_COLOR`. `make help-` (for example `make help-rsync`) prints extended help for a single target โ€” options and worked examples where they help. - -| Area | Targets | -|---|---| -| Release versioning | `version`, `changelog`, `tag` | -| Build | `build`, `dist`, `composer ARGS=...`, `npm ARGS=...`, `reuse` | -| Local deploy | `rsync TARGET=...`, `cp TARGET=...` | -| App Store (module) | `csr`, `register`, `publish`, `sign`, `release`, `list-releases`, `list-releases-full`, `list-for-author`, `delete-release`, `ratings` | -| CI workflows (module) | `workflows-list`, `workflows-install W=...`, `workflows-update` | -| Utility | `clean`, `dist-clean`, `self-update`, `dev-init`, `dev-clean`, `help`, `help-` | - -Targets marked `[m]` in the help are maintainer-only: they need repository write access and/or the App Store signing key. Everything else works for anyone who clones the app. The areas marked *(module)* come from the developer modules (`make dev-init`, see [CI workflows](#-ci-workflows) and [App Store management](#-app-store-management)); without the modules those targets simply do not exist โ€” which keeps the help of a plain checkout down to build, deploy and utility. - -## โœ… Requirements - -GNU make, git, curl, openssl, rsync, python3. Optional but recommended: `xmllint` (libxml2) for reading `info.xml` โ€” ncmake falls back to `grep` when it is missing, so a bare CI runner works too. Optional: podman or docker for containerized builds (strongly recommended; without them use `RUNTIME=bare` and provide composer and npm yourself). - -## ๐Ÿ… Show that your app uses ncmake - -If ncmake is useful to you, add the badge to your app's README: +If ncmake is useful to you, add the badge to your app's README (see [Getting started](https://github.com/ernolf/ncmake/wiki/Getting-started#-show-that-your-app-uses-ncmake)): [![built with ncmake](https://cdn.jsdelivr.net/gh/ernolf/ncmake@main/img/ncmake-badge.svg)](https://github.com/ernolf/ncmake) @@ -339,31 +67,6 @@ If ncmake is useful to you, add the badge to your app's README: [![built with ncmake](https://cdn.jsdelivr.net/gh/ernolf/ncmake@main/img/ncmake-badge.svg)](https://github.com/ernolf/ncmake) ``` -The badge is a small SVG served from this repository through the jsDelivr CDN and links here; it is purely cosmetic and reports nothing back. To actually find the apps that use ncmake, search GitHub's code search for the fetch URL every stub carries โ€” that signal does not depend on the badge: - - - -The same from the command line needs a recent `gh` (2.10 or newer, for the `search` command): - -```sh -gh search code 'raw.githubusercontent.com/ernolf/ncmake' --json repository --jq '.[].repository.nameWithOwner' | sort -u -``` - -> [!NOTE] -> Both queries hit code-search indexes that only contain public repositories GitHub has already picked up, so a freshly created consumer can take weeks to appear. The bootstrap stub is committed regardless, so a consumer becomes findable the moment its repo is indexed. - -## ๐Ÿ“š Documentation - -The guides that live in [`doc/`](doc/), collected in one place: - -- **[INSTALL.md](doc/INSTALL.md)** โ€” building and deploying an app from source; written for the people who install your app. -- **[WORKFLOWS.md](doc/WORKFLOWS.md)** โ€” the CI workflows module: sources, status model, the lock file, installing and updating, the `COMMIT`/`PR` flags. -- **[AUTOUPDATE_WORKFLOW.md](doc/AUTOUPDATE_WORKFLOW.md)** โ€” the workflow updater that opens a pull request when your workflows drift from upstream. -- **[GITHUB_APP.md](doc/GITHUB_APP.md)** โ€” the GitHub App the workflow updater authenticates with. -- **[GITHUB_PAT.md](doc/GITHUB_PAT.md)** โ€” the Personal Access Token alternative to the GitHub App. -- **[APPSTORE.md](doc/APPSTORE.md)** โ€” the App Store module: certificate onboarding, signing and the publish flow. -- **[DELETE_MERGED_BRANCHES.md](doc/DELETE_MERGED_BRANCHES.md)** โ€” the GitHub setting that removes a branch once its pull request is merged. - -## ๐Ÿ“„ License +## License [MIT](LICENSE) diff --git a/doc/APPSTORE.md b/doc/APPSTORE.md deleted file mode 100644 index 94e1137..0000000 --- a/doc/APPSTORE.md +++ /dev/null @@ -1,125 +0,0 @@ - -# ๐Ÿช App Store management for ncmake apps - -This guide covers everything that connects an ncmake app to the [Nextcloud App Store](https://apps.nextcloud.com): the one-time onboarding (key, certificate, registration), publishing releases, and the read-only queries. - -> [!TIP] -> **TL;DR** โ€” `make dev-init` once per machine, `make csr` once per app (submit the printed CSR upstream, save the issued certificate), `make register` once, and from then on every release is a `make publish GH=1` after the GitHub release exists. `make help` always shows whether token, certificate and key are in place. - -- [The developer module](#-the-developer-module) -- [The certificate directory](#-the-certificate-directory) -- [One-time onboarding](#-one-time-onboarding) -- [Publishing a release](#-publishing-a-release) -- [Nightly releases](#-nightly-releases) -- [Queries and housekeeping](#-queries-and-housekeeping) -- [FAQ](#-faq) - -## ๐Ÿงฉ The developer module - -The App Store targets are not part of the core Makefile: they are a **developer module**. Someone who clones your app to build and install it never needs them, so they stay out of their `make help`. As the maintainer you fetch the modules once per machine: - -```sh -make dev-init -``` - -This caches all ncmake modules next to the core Makefile in `~/.cache/ncmake/` and loads them on every subsequent `make` run โ€” in every ncmake app on this machine, kept current by the same TTL/ETag mechanism as the core, pinned by `NCMAKE_REF` like everything else. `make dev-clean` removes them again. - -## ๐Ÿ“ The certificate directory - -All credentials live in one directory, `~/.nextcloud/certificates/` by default (change with `cert_dir=`, e.g. in `ncmake.mk`): - -| File | Purpose | -|---|---| -| `.crt` (or `.cert`, both are accepted) | the app certificate issued via [app-certificate-requests](https://github.com/nextcloud/app-certificate-requests) | -| `.key` | the private key | -| `appstore_api-token` | your API token from the [App Store account page](https://apps.nextcloud.com/account/token) | - -`make help` shows for each of the three whether it was found (green check or red cross, with the real filename), so a missing piece is visible before any target fails. - -> [!IMPORTANT] -> The key signs everything the App Store trusts about your app. Keep the directory out of every repository and back it up privately; a lost key means a new certificate request, a leaked key means strangers can publish releases in your app's name. - -## ๐Ÿ”‘ One-time onboarding - -**1. Key and certificate request:** - -```sh -make csr -``` - -generates `.key` (chmod 600) and prints the certificate signing request. It refuses to overwrite an existing key or CSR. Submit the CSR as `/.csr` in a pull request to [nextcloud/app-certificate-requests](https://github.com/nextcloud/app-certificate-requests); when the PR is merged, save the issued certificate as `.crt` in the certificate directory. - -**2. Registration:** - -```sh -make register -``` - -signs your app id with the key and registers id and certificate on the App Store. Running it again after a certificate change updates the registration. - -## ๐Ÿšข Publishing a release - -The App Store does not host your tarball โ€” it stores a download URL plus your signature over the file behind it. The natural flow with ncmake: finish the release ([make version โ†’ changelog โ†’ tag](../README.md#-releasing)), publish the GitHub release so the [release workflow](WORKFLOWS.md#-the-ncmake-release-workflow) attaches the tarball asset, then: - -```sh -make publish GH=1 -``` - -`GH=1` pre-fills the canonical GitHub asset URL for the current version, so you only confirm with Enter. The target **always downloads the asset from the given URL and signs exactly those bytes** โ€” the signature can never be computed over anything else than what the App Store will fetch. Without `GH=1` it prompts for any URL (your own server works just as well); `URL=...` sets it non-interactively. A GitHub asset is downloaded via `gh` when installed, so private repos work too. - -For a manually assembled release there are the building blocks: - -```sh -make sign # sign the local tarball from make dist, print the base64 signature -make release # dist + sign in one step -``` - -## ๐ŸŒ™ Nightly releases - -The App Store has a dedicated nightly channel, and `publish` targets it with a flag: - -```sh -make publish GH=1 NIGHTLY=1 -``` - -Nightlies follow their own rules, straight from the store's API: - -- The store keeps **exactly one nightly per app** โ€” publishing a new one replaces the previous, no cleanup needed. -- A nightly does **not** need a higher version than the one before; for identical versions the upload time decides. -- Stable releases are completely unaffected: the nightly lives next to them in its own channel. - -To catch mix-ups, `publish` cross-checks GitHub when it can: for a GitHub asset URL (with `gh` installed) it reads the release's **pre-release flag** and asks before publishing when it contradicts `NIGHTLY` โ€” a pre-release without `NIGHTLY=1`, or `NIGHTLY=1` on a regular release. For non-GitHub URLs the flag alone decides. - -> [!TIP] -> Mark your nightly releases as **pre-release** on GitHub. That keeps them off the repository's "Latest" badge and gives `make publish` the signal for the cross-check. - -Removing a nightly from the store works through the same flag: - -```sh -make delete-release NIGHTLY=1 -``` - -## ๐Ÿ” Queries and housekeeping - -| Target | What it does | -|---|---| -| `make list-releases` | your published releases, compact JSON | -| `make list-releases-full` | the full App Store entry | -| `make list-for-author` | all apps of an author (prompts for a name) | -| `make ratings` | ratings and comments for the app | -| `make delete-release` | deletes one release, interactively, with confirmation | - -The read-only targets cache `apps.json` with ETag revalidation under `build/cache/`, so repeated calls are fast and gentle to the API. - -## โ“ FAQ - -**Can I publish without a GitHub release?** Yes โ€” `make publish` accepts any URL that serves the tarball. GitHub is just the convenient default because the ncmake release workflow already builds and attaches the asset there. - -**Why does publish download the tarball instead of signing my local build?** Because the App Store verifies the signature over the bytes it downloads. Signing the served file rules out the entire class of "local build differs from published asset" failures. - -**The App Store rejected the signature (HTTP 400) โ€” what now?** Almost always a mismatch between the URL's content and the signature: the asset was re-uploaded after signing, or the URL redirects somewhere unexpected. Re-run `make publish` so download and signature happen in one go; if it persists, check that the certificate in the store (`make register`) matches your key. - -**Where do the maintainer release targets live?** `version`, `changelog` and `tag` are core targets โ€” they need repository rights, not the App Store key. Everything that touches the store or the signing key is in this module. diff --git a/doc/AUTOUPDATE_WORKFLOW.md b/doc/AUTOUPDATE_WORKFLOW.md deleted file mode 100644 index 6e85219..0000000 --- a/doc/AUTOUPDATE_WORKFLOW.md +++ /dev/null @@ -1,69 +0,0 @@ - -# ๐Ÿ”„ The workflow updater - -The [workflow manager](WORKFLOWS.md) lets you install and update your CI workflows from their upstream templates by hand (`make workflows-update`). The **workflow updater** automates the update side: on a schedule it runs `make workflows-update` for you and opens a pull request whenever a managed workflow has changed upstream. It is what replaces Dependabot for the files under `.github/workflows/`. - -> [!TIP] -> **TL;DR** โ€” Install with `make workflows-install W=workflow-updater`, set up the [GitHub App](GITHUB_APP.md) it authenticates with, and from then on you get a pull request whenever your managed workflows drift from upstream. Locally modified workflows are left untouched. - -- [What it does](#-what-it-does) -- [Installing it](#-installing-it) -- [The GitHub App it needs](#-the-github-app-it-needs) -- [When it runs](#-when-it-runs) -- [What a run does](#-what-a-run-does) -- [The pull request it opens](#-the-pull-request-it-opens) -- [Why not Dependabot](#-why-not-dependabot) - -## ๐Ÿ” What it does - -`workflow-updater.yml` is a workflow ncmake ships itself (source `ncmake`, like `release.yml`). Once installed and scheduled it: - -1. fetches the ncmake modules (`make dev-init`), -2. refreshes the managed workflows from their upstream templates (`make workflows-update`), -3. opens a pull request if anything changed. - -It only ever touches workflows that are **managed** (listed in `.ncmake-workflows.json`) and **not locally modified**. Anything you edited by hand is left alone, exactly as with a manual `make workflows-update`. - -## ๐Ÿงฉ Installing it - -It is a normal ncmake-provided workflow, so it installs through the manager: - -```sh -make workflows-install W=workflow-updater -git add .github/workflows/ -``` - -Commit and merge that like any other workflow adoption. From then on `workflow-updater.yml` lives in `.github/workflows/` and is itself managed, so a later run keeps it up to date too. - -## ๐Ÿ”‘ The GitHub App it needs - -The updater changes files under `.github/workflows/`, which the automatic `GITHUB_TOKEN` may not push, and its commit must be verified when your branch protection requires signed commits. A **GitHub App** covers both: its token pushes the workflow files, and the pull request is committed as the app's bot with a verified signature. It also triggers your CI checks (a real actor, unlike the `GITHUB_TOKEN`), so you see whether a template update breaks anything before you merge. - -> [!IMPORTANT] -> Set the app up once, following **[A GitHub App for the workflow updater](GITHUB_APP.md)**. It needs Contents, Pull requests and Workflows (each Read and write), installed on the repository, with its Client ID and private key stored as the `NCMAKE_UPDATER_CLIENT_ID` and `NCMAKE_UPDATER_PRIVATE_KEY` secrets. Without it the run fails the moment there is a workflow change to push. - -## โฐ When it runs - -- **On a schedule:** daily at 05:30 UTC (the `cron` line in the workflow). GitHub runs scheduled workflows on a best-effort basis and [delays them under load](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#schedule), so in practice the run starts a couple of hours after the cron time, not on the minute. That is expected, not a failure: it only needs to catch an upstream change within a day, so the delay does not matter here. A run with nothing to do is a quiet no-op, and standard runners are free on public repositories, so a daily check costs nothing. -- **On demand:** *Actions* tab โ†’ **ncmake workflow update** โ†’ **Run workflow** (`workflow_dispatch`). - -The manual trigger only appears once the workflow is on your default branch. That is how `workflow_dispatch` works, and it is why installing the updater means merging it to `main` first. It stays idle until the schedule fires or you trigger it. - -## โš™๏ธ What a run does - -The single job mints the app token, checks out the repository, runs `make dev-init` then `make workflows-update`, and hands any resulting changes to the pull-request step. `ubuntu-latest` already carries make, git, curl and python3, so there is nothing to set up. The commit is signed by the app's bot (so it is verified) and carries a matching `Signed-off-by`, so the DCO check passes. - -## ๐Ÿš€ The pull request it opens - -If `make workflows-update` changed anything, the updater opens a pull request titled **`ci: update managed CI workflows from upstream`** on the branch `ncmake/ci/workflow-update`, containing only the refreshed files under `.github/workflows/` (plus the updated lock). Review the diff and merge it like the original adoption. - -If nothing changed upstream, no pull request is opened and the run is quiet. - -Once you merge that pull request, the updater deletes its own `ncmake/ci/workflow-update` branch right away, so it never leaves a stale branch behind between runs. This is the updater tidying up after *itself* โ€” it is a different mechanism from the repository-wide [Automatically delete head branches](DELETE_MERGED_BRANCHES.md) setting, which removes the head branch of *every* merged pull request. Either one keeps this branch from piling up; enabling both is harmless, as the branch is simply already gone by the time the other would act. - -## ๐Ÿค Why not Dependabot - -Dependabot's `github-actions` ecosystem and this updater both change the same files, so running both makes them fight: each edit flags the workflows as locally modified for the other. The upstream templates already keep their action pins current, and `make workflows-update` brings those along, so the updater is the single source of truth. Remove the `github-actions` ecosystem from `dependabot.yml` and let the updater own `.github/workflows/`; keep npm and Composer under Dependabot. diff --git a/doc/DELETE_MERGED_BRANCHES.md b/doc/DELETE_MERGED_BRANCHES.md deleted file mode 100644 index 07ab654..0000000 --- a/doc/DELETE_MERGED_BRANCHES.md +++ /dev/null @@ -1,42 +0,0 @@ - -# ๐Ÿงน Automatically deleting merged branches - -A merged pull request leaves its source branch โ€” the *head branch* โ€” behind. Nothing removes it on its own, so over time a repository collects a long list of stale branches: your own feature branches, and the `ncmake/โ€ฆ` branches ncmake opens for you (`make version` releases on `ncmake/release/x.y.z`, the [COMMIT/PR flags](WORKFLOWS.md#-committing-and-opening-a-pr-commit-and-pr) and the [workflow updater](AUTOUPDATE_WORKFLOW.md) on `ncmake/ci/โ€ฆ`). - -GitHub has a single repository setting that cleans them up: it deletes each head branch the moment its pull request is merged. This is a general GitHub feature, not an ncmake one โ€” it applies to *every* branch merged through a pull request โ€” but ncmake apps benefit from it, which is why it is documented here. - -> [!TIP] -> **TL;DR** โ€” **Settings โ†’ General โ†’ Pull Requests โ†’ tick "Automatically delete head branches".** From then on every branch merged through a pull request is removed automatically. Nothing else is touched, and a deleted branch can be restored from its pull request. - -- [Turning it on](#-turning-it-on) -- [What it deletes, and what it does not](#-what-it-deletes-and-what-it-does-not) -- [Restoring a branch](#-restoring-a-branch) -- [One repository at a time](#-one-repository-at-a-time) - -## โš™๏ธ Turning it on - -1. Open the repository's **Settings** tab. -2. On **General** (the default page), scroll to the **Pull Requests** section. -3. Tick **Automatically delete head branches**. - -That is the whole setup. It takes effect immediately and needs no workflow, token or permission of its own. - -## ๐ŸŽฏ What it deletes, and what it does not - -It deletes **only the head branch of a pull request, and only once that pull request is merged**. Concretely: - -- The base branch of the merge (`main`) is never touched โ€” only the branch that was merged *into* it. -- A branch is left alone as long as it still has **another open pull request**; it is removed only when no open PR points at it any more. -- Only branches **in this repository** are affected. A pull request opened from a fork never has the fork's branch deleted. -- Branches that were **never opened as a pull request** are outside its scope entirely โ€” it is a merge-time cleanup, nothing more. - -## โ™ป๏ธ Restoring a branch - -Deletion is not final. Every merged pull request keeps a **Restore branch** button on its page, so a branch removed by mistake โ€” or one you want back to keep working on โ€” is one click away, with its commits intact. - -## ๐Ÿ” One repository at a time - -The setting lives on the repository, so it is enabled **per repository**; a new ncmake app does not inherit it. Turn it on once in each repo where you want merged branches cleaned up. diff --git a/doc/GITHUB_APP.md b/doc/GITHUB_APP.md deleted file mode 100644 index 6fc515f..0000000 --- a/doc/GITHUB_APP.md +++ /dev/null @@ -1,81 +0,0 @@ - -# ๐Ÿค– A GitHub App for the workflow updater - -The [workflow updater](AUTOUPDATE_WORKFLOW.md) authenticates with a **GitHub App** rather than a token you paste in. This page explains why, and walks through creating the app, installing it and wiring up its two secrets. For the general background on tokens and secrets, see [GitHub tokens and PAT](GITHUB_PAT.md). - -> [!TIP] -> **TL;DR** โ€” Create a GitHub App with **Contents**, **Pull requests** and **Workflows** set to *Read and write*, install it on your ncmake repositories, and store its **Client ID** and **private key** as the secrets **`NCMAKE_UPDATER_CLIENT_ID`** and **`NCMAKE_UPDATER_PRIVATE_KEY`**. The updater mints a fresh one-hour token from them on every run. - -- [Why a GitHub App](#-why-a-github-app) -- [Creating the app](#-creating-the-app) -- [Installing it on your repositories](#-installing-it-on-your-repositories) -- [Storing its credentials as secrets](#-storing-its-credentials-as-secrets) -- [How the workflow uses it](#-how-the-workflow-uses-it) -- [The private key: safety](#-the-private-key-safety) - -## ๐Ÿงญ Why a GitHub App - -The updater has two hard requirements, and a GitHub App is the only clean way to satisfy both at once: - -1. **It changes files under `.github/workflows/`.** GitHub does not let the automatic `GITHUB_TOKEN` push workflow files. A token with the *Workflows* permission is required. -2. **Verified (signed) commits.** If your branch protection requires signed commits, the updater's commit has to be verified. - -A fine-grained PAT can do the first but **not** the second: commit signing does not work with a PAT. A GitHub App does both. Its token can push workflow files, and the pull-request action signs the commit as the app's bot, so it shows up as **Verified**. As a bonus it is more secure than a PAT: the app mints a **fresh, one-hour token per run** instead of a long-lived credential sitting in a secret. This is the same pattern Dependabot and Renovate use. - -## ๐Ÿ› ๏ธ Creating the app - -**Settings** (your account) โ†’ **Developer settings** โ†’ **GitHub Apps** โ†’ **New GitHub App**, and work down the form. Most fields belong to features the updater does not use, so the short version is: set the name, the homepage, the three permissions and the install scope, turn the webhook off, and leave everything else at its default. - -- **GitHub App name:** something globally unique, for example `ncmake updater (yourname)`. -- **Description:** optional, leave it empty. -- **Homepage URL** (required): any valid URL, your repository is fine. -- **Callback URL, Expire user authorization tokens, Request user authorization (OAuth), Enable Device Flow:** leave all at their defaults. These belong to the user-login (OAuth) flow, which the updater does not use. (`Expire user authorization tokens` stays ticked; that is the secure default and has no effect here.) -- **Setup URL, Redirect on update:** leave empty / unticked. -- **Webhook โ†’ Active:** **untick it.** The app receives no events, so there is no webhook URL or secret to set. (Left ticked without a URL, the form rejects the save.) -- **Permissions โ†’ Repository permissions**, each set to **Read and write**: - - | Permission | Why it is needed | - |---|---| - | **Contents** | push the update branch | - | **Pull requests** | open the pull request | - | **Workflows** | required to change files under `.github/workflows/` | - - *Metadata: Read-only* is selected automatically and is mandatory. Leave every other repository permission at **No access**, and the Organization and Account permissions at their defaults. -- **Where can this GitHub App be installed:** **Only on this account**. - -Then **Create GitHub App**. On the app's page afterwards: - -1. Under **About**, note the **Client ID** (a 20-character alphanumeric string). It goes into the `NCMAKE_UPDATER_CLIENT_ID` secret. GitHub also shows a shorter, purely numeric **App ID** and still accepts it, but it is being phased out: GitHub itself points you to the Client ID, and `actions/create-github-app-token` marks its `app-id` input deprecated. Both are public identifiers, not secrets (only the private key authenticates), so this is about which one stays supported, not about safety. Use the Client ID. -2. Scroll to **Private keys** โ†’ **Generate a private key**. A `.pem` file downloads. That file is the app's credential and goes into the `NCMAKE_UPDATER_PRIVATE_KEY` secret. - -## ๐Ÿ“ฆ Installing it on your repositories - -On the app's page โ†’ **Install App** โ†’ install on your account โ†’ **Only select repositories** โ†’ pick your ncmake apps โ†’ **Install**. - -## ๐Ÿ”‘ Storing its credentials as secrets - -In **each** repository where the updater runs (**Settings โ†’ Secrets and variables โ†’ Actions โ†’ New repository secret**): - -| Secret name | Value | -|---|---| -| `NCMAKE_UPDATER_CLIENT_ID` | the **Client ID** (the alphanumeric string from the app's About page) | -| `NCMAKE_UPDATER_PRIVATE_KEY` | the **full contents** of the downloaded `.pem`, including the `-----BEGIN...` and `-----END...` lines | - -Personal repositories cannot share an organization secret, so add both to every repo. (Move to an organization later and org-level secrets cover all of them at once.) - -## โš™๏ธ How the workflow uses it - -`workflow-updater.yml` mints a token from the two secrets with [`actions/create-github-app-token`](https://github.com/actions/create-github-app-token), then hands it to the pull-request step with `sign-commits: true` and `signoff: true`. The token pushes the workflow changes; the commit is signed by the app's bot (so it is **Verified**) and carries a matching `Signed-off-by` (so **DCO** passes). One hour later the token expires on its own. - -## ๐Ÿ”’ The private key: safety - -- **Treat the `.pem` like a password.** Anyone holding it can mint tokens with the app's permissions on the installed repositories. Put it only into the secret; delete the local `.pem` afterwards, or keep it somewhere safe and offline. -- **If it leaks,** revoke it on the app's page (**Private keys** โ†’ delete the key), generate a new one, and update the secret. -- **Nothing to renew.** The app itself does not expire, and only the per-run tokens are short-lived, so unlike a PAT there is no expiry date to keep an eye on. - ---- - -Background on tokens, `GITHUB_TOKEN` and secrets: [GitHub tokens and PAT](GITHUB_PAT.md). This app is used by the [workflow updater](AUTOUPDATE_WORKFLOW.md). diff --git a/doc/GITHUB_PAT.md b/doc/GITHUB_PAT.md deleted file mode 100644 index 145d3ff..0000000 --- a/doc/GITHUB_PAT.md +++ /dev/null @@ -1,89 +0,0 @@ - -# ๐Ÿ”‘ GitHub tokens and Personal Access Tokens (PAT) - -GitHub automation acts on your repository on your behalf, and it authenticates with **tokens** rather than a username and password. This page explains the kinds of token you meet on GitHub, and walks through creating a Personal Access Token for the times a piece of automation needs one of its own. - -> [!NOTE] -> The [workflow updater](AUTOUPDATE_WORKFLOW.md) authenticates with a [GitHub App](GITHUB_APP.md), not a PAT. This page is the general token reference the App guide builds on, and the how-to for when you do need a PAT. - -> [!TIP] -> **TL;DR** โ€” A **Personal Access Token (PAT)** is a token you create yourself and store as a secret when a workflow or the `gh` CLI needs to act as you. Prefer **fine-grained** tokens, scoped to specific repositories and permissions, with an expiry. The automatic `GITHUB_TOKEN` cannot change workflow files, which is one reason automation sometimes needs a PAT or a GitHub App instead. - -- [What a token is](#-what-a-token-is) -- [GITHUB_TOKEN: the automatic one](#-github_token-the-automatic-one) -- [Personal Access Token: the one you create](#-personal-access-token-the-one-you-create) -- [What a secret is](#-what-a-secret-is) -- [Creating a fine-grained PAT](#๏ธ-creating-a-fine-grained-pat) -- [Storing it as a repository secret](#-storing-it-as-a-repository-secret) -- [Keeping it working: expiry and safety](#-keeping-it-working-expiry-and-safety) - -## ๐Ÿงฉ What a token is - -A token is a key that proves "I am allowed to do this". People sign in with a password; automation (workflows, scripts, the `gh` CLI) uses a token instead. A token carries two things: *who* is acting (an identity) and *what* is allowed (a set of permissions). - -## ๐Ÿค– GITHUB_TOKEN: the automatic one - -You never create this one. At the start of **every** workflow run, GitHub mints a fresh, temporary token and hands it to the run as `secrets.GITHUB_TOKEN`. - -- **Throwaway:** valid only for that one run and that one repository, and it expires the moment the run finishes. Even if it appears in a log it is worthless afterwards. -- **Acts as `github-actions[bot]`:** commits or pull requests made with it show up as that bot, not as you. -- **Its rights** come from the `permissions:` block in the workflow (for example `contents: write`). - -It has two deliberate limits worth knowing: - -1. **It cannot change workflow files.** GitHub blocks any push that modifies files under `.github/workflows/` when the push is made with the `GITHUB_TOKEN`, so that a workflow cannot rewrite workflows. -2. **Its actions do not start new workflows.** A pull request opened with it does **not** trigger the repository's CI checks. This is a loop guard. - -Both limits are why automation that touches workflow files (like the workflow updater) needs a PAT or a GitHub App instead of the `GITHUB_TOKEN`. - -## ๐Ÿ‘ค Personal Access Token: the one you create - -A PAT is a key you create by hand in your GitHub account settings, once. - -- **You choose** its permissions and an expiry date, then store it (as a [secret](#-what-a-secret-is)). -- **Acts as you** (a real user), so unlike the bot token its actions *do* trigger workflows, and it *can* change workflow files once you grant it the Workflows permission. -- Two flavours exist: **classic** (broad, account-wide scopes) and **fine-grained** (limited to chosen repositories and specific permissions, with an expiry date). Prefer **fine-grained**. - -## ๐Ÿ”’ What a secret is - -A secret is an encrypted value you store in a repository's settings (*Settings โ†’ Secrets and variables โ†’ Actions*). Workflows read it as `secrets.NAME`, and GitHub masks it in the logs. This is where a PAT goes, under a name your workflow reads. - -## ๐Ÿ› ๏ธ Creating a fine-grained PAT - -1. Avatar (top right) โ†’ **Settings** โ†’ **Developer settings** (bottom of the left menu) โ†’ **Personal access tokens** โ†’ **Fine-grained tokens** โ†’ **Generate new token**. -2. **Token name:** something descriptive. -3. **Expiration:** pick a date (for example 90 days). Note it down; an expired token silently stops whatever uses it. -4. **Resource owner:** your account. -5. **Repository access:** **Only select repositories**, and choose the repositories the automation needs. -6. **Permissions โ†’ Repository permissions:** grant only what the automation needs. For example, automation that touches workflow files needs these, each **Read and write**: - - | Permission | Purpose | - |---|---| - | **Contents** | push branches and files | - | **Pull requests** | open pull requests | - | **Workflows** | change files under `.github/workflows/` | - - (*Metadata: Read* is selected automatically and is mandatory.) -7. **Generate token**, then **copy the value now**. It is shown only once. - -## ๐Ÿ“ฅ Storing it as a repository secret - -Store it in **each** repository that needs it (personal repositories cannot share an organization secret): - -Repo โ†’ **Settings** โ†’ **Secrets and variables** โ†’ **Actions** โ†’ **New repository secret** - -- **Name:** the name your workflow reads (for example `MY_TOKEN`) -- **Value:** the PAT you copied - -## โฐ Keeping it working: expiry and safety - -- **Renew before it expires.** A fine-grained PAT has an expiry date; once it passes, whatever uses it can no longer authenticate and its runs fail. Set a reminder, or choose a long expiry. -- **Treat it like a password.** Anyone holding the value can act on the selected repositories with the granted permissions. If it leaks, revoke it under *Fine-grained tokens* and generate a new one. -- **Scope it tightly.** Only the repositories and permissions needed, nothing more. - ---- - -For the workflow updater specifically, use a [GitHub App](GITHUB_APP.md) rather than a PAT. This page is the general reference for GitHub tokens. diff --git a/doc/INSTALL.md b/doc/INSTALL.md deleted file mode 100644 index eb8156d..0000000 --- a/doc/INSTALL.md +++ /dev/null @@ -1,218 +0,0 @@ - - -# ๐Ÿ“ฆ Installing a Nextcloud app built with ncmake - -This app is built and packaged with [ncmake](https://github.com/ernolf/ncmake), the shared build tool for Nextcloud apps. Installation is therefore the same for every ncmake app: this one guide covers all of it. - -## ๐Ÿ”ค Placeholders and example values - -Two placeholders stand for values specific to the app or the release. Substitute them wherever they appear: - -| Placeholder | Meaning | Example | -|---|---|---| -| `APP_ID` | the app id โ€” the value of `` in the app's `appinfo/info.xml` | `twofactor_oath` | -| `` | the release version you install | `1.2.0` | - -The commands also use two concrete example values. Adapt both to your setup: - -- **`/var/www/nextcloud`** โ€” your Nextcloud installation directory (the one that contains `occ` and `apps/`). -- **`www-data`** โ€” the web server user. This is the Debian/Ubuntu name; other distributions differ (see [ownership](#-the-app-directory-and-ownership)). - -> [!TIP] -> **TL;DR** โ€” Published in the App Store? Install it from there. Otherwise: download a [release tarball](#-method-1--install-a-release-tarball), extract it into `apps/`, `chown` to your web server user, `occ app:enable APP_ID`. Building from source or deploying straight into an instance needs only **podman or docker** โ€” no PHP, no Node. - -## ๐Ÿงญ Which method should I use? - -| Situation | Method | -|---|---| -| The app is in the App Store | Install it from the app management UI or with `occ app:enable APP_ID` โ€” nothing here needed. | -| You have a release tarball and just want it installed | [Method 1: release tarball](#-method-1--install-a-release-tarball) โ€” no build toolchain required. | -| There is no release for your version, or you want to build from a branch | [Method 2: build from source](#-method-2--build-the-tarball-from-source) | -| The Nextcloud instance is on a host you can reach (locally or over SSH) | [Method 3: `make rsync`](#-method-3--deploy-with-make-rsync) โ€” deploy straight into `apps/`. | -| The instance runs inside a container (e.g. Nextcloud All-in-One) | [Method 4: `make cp`](#-method-4--deploy-into-a-running-container-with-make-cp) | - -Methods 2โ€“4 build in throwaway containers, so the host needs **no PHP and no Node** โ€” only a container runtime (see [Container runtime](#-container-runtime)). - -## ๐Ÿ“ The app directory and ownership - -A Nextcloud app lives in an `apps/` directory as `/var/www/nextcloud/apps/APP_ID/`. After copying files in, ownership must match the web server user, then the app is enabled through `occ`. - -> [!IMPORTANT] -> The directory name **must** be the app id (`APP_ID`), not the name of your checkout or the tarball. `occ app:enable` looks the app up by that directory name. - -The web server user depends on your distribution: - -| Distribution / setup | Web server user | -|---|---| -| Debian / Ubuntu | `www-data` | -| RHEL / CentOS / Fedora (Apache) | `apache` | -| RHEL / CentOS / Fedora (nginx) | `nginx` | -| Arch Linux | `http` | - -`occ` is normally run as the web server user (`sudo -u www-data php /var/www/nextcloud/occ โ€ฆ`). - -## ๐Ÿ“ฅ Method 1 โ€” Install a release tarball - -> [!TIP] -> **TL;DR** โ€” extract into `apps/`, `chown`, enable. No toolchain needed. - -Download the latest `APP_ID-.tar.gz` from the project's **Releases** page, then: - -```sh -tar -xzf APP_ID-.tar.gz -C /var/www/nextcloud/apps/ -sudo chown -R www-data:www-data /var/www/nextcloud/apps/APP_ID -sudo -u www-data php /var/www/nextcloud/occ app:enable APP_ID -``` - -## ๐Ÿ”จ Method 2 โ€” Build the tarball from source - -> [!TIP] -> **TL;DR** โ€” `git clone`, `make build && make dist`, then install the tarball like Method 1. - -You need `git`, `make` and a container runtime (podman or docker). Clone the app and build: - -```sh -git clone -cd APP_ID -make build && make dist -``` - -- `make build` installs the runtime PHP dependencies and builds the frontend, each in a throwaway container โ€” nothing is installed on your host. -- `make dist` assembles the runtime file set and writes the tarball to: - - ```text - build/artifacts/dist/APP_ID-.tar.gz - ``` - -Install that tarball exactly as in [Method 1](#-method-1--install-a-release-tarball). - -## ๐Ÿš€ Method 3 โ€” Deploy with `make rsync` - -> [!TIP] -> **TL;DR** โ€” `make build && make rsync TARGET=/var/www/nextcloud/apps OCC=1` copies the app straight into a reachable instance and enables it. - -If the Nextcloud instance is reachable from where you build โ€” the same machine or over SSH โ€” `make rsync` copies the runtime file set straight into `apps/`, without producing a tarball. - -```sh -make build -make rsync TARGET=/var/www/nextcloud/apps -``` - -- `TARGET` is the `apps/` **parent** directory; ncmake appends `/APP_ID` automatically. -- The sync uses `rsync --delete`, so files removed between versions disappear from the instance too. -- `TARGET` may be remote, in `user@host:` form โ€” every step then runs over SSH: - - ```sh - make rsync TARGET=deploy@server:/var/www/nextcloud/apps - ``` - -### One-shot deploy with `OCC=1` - -Add `OCC=1` to wrap the sync into the full refresh cycle, so a single command replaces the whole sequence: - -```sh -make build && make rsync TARGET=/var/www/nextcloud/apps OCC=1 -``` - -With `OCC=1`, ncmake runs, in order: - -1. `occ app:disable APP_ID` (tolerated to fail on a first deploy) -2. the rsync -3. `chown -R www-data: /var/www/nextcloud/apps/APP_ID` -4. `occ app:enable APP_ID` - -The disable/enable cycle makes Nextcloud re-read `info.xml` and run any pending database migrations. - -> [!IMPORTANT] -> `occ` is expected at `/../occ` and runs as the web server user (`web_user`, default `www-data`; via `sudo` unless you already are that user). For a **remote** `TARGET`, the SSH user needs the rights to `sudo` and `chown`. - -Change the web server user with `web_user=`: - -```sh -make rsync TARGET=/var/www/nextcloud/apps OCC=1 web_user=apache -``` - -Without `OCC=1`, only the rsync happens and the finishing commands are printed for copy and paste. - -> [!NOTE] -> Updating is the same command again โ€” `--delete` keeps the installation identical to the current source. - -## ๐Ÿณ Method 4 โ€” Deploy into a running container with `make cp` - -> [!TIP] -> **TL;DR** โ€” for AIO and other dockerized instances: `make build && make cp TARGET=: OCC=1`. - -When the instance's filesystem cannot be reached from outside โ€” a containerized setup such as **Nextcloud All-in-One** โ€” `make cp` copies the runtime file set into the running container instead: - -```sh -make build && make cp TARGET=nextcloud-aio-nextcloud:/var/www/html/custom_apps OCC=1 -``` - -- `TARGET` uses `docker cp` syntax: `:`. The `/APP_ID` subdirectory is appended automatically and replaced as a whole, so removed files disappear too. -- `ENGINE=docker|podman` selects the container CLI. It is deliberately **independent** of `RUNTIME`: the build may use podman while the instance runs under docker, which is why docker is preferred here when both are installed. -- `OCC=1` runs the same disable โ†’ copy โ†’ chown โ†’ enable cycle as Method 3, entirely inside the container, invoking `occ` in the form the All-in-One documentation uses: - - ```sh - docker exec --user www-data -it nextcloud-aio-nextcloud php occ - ``` - -## ๐Ÿงฐ Container runtime - -`make build`, `make dist`, `make rsync` and `make cp` run their package managers in throwaway containers, so your host needs no PHP and no Node. The runtime is auto-detected (podman preferred, then docker) and can be chosen per call with `RUNTIME=`: - -| `RUNTIME=` | What it is | Notes | -|---|---|---| -| `podman` | rootless podman (default when present) | daemonless, files owned by you | -| `docker` | standard rootful docker | ncmake maps your uid/gid in, so no root-owned files appear | -| `docker-rootless` | rootless docker | | -| `bare` | no container | `composer` and `npm` must be on the `PATH` | - -```sh -make build RUNTIME=docker -``` - -> [!CAUTION] -> On SELinux hosts (Fedora, RHEL) bind mounts may need a `:z` label. If you hit permission errors during a build, use `RUNTIME=bare` or adjust your container policy. - -## ๐Ÿ”„ Updating - -- **Deployed with `make rsync` / `make cp`:** run the same command again after a `git pull`. The `--delete` (rsync) or whole-directory replacement (cp) removes files that no longer exist in the new version. - -- **Installed from a tarball:** remove the app first, then reinstall: - - ```sh - sudo -u www-data php /var/www/nextcloud/occ app:remove APP_ID - # then Method 1 or 2 again - ``` - -> [!IMPORTANT] -> Extracting a new tarball over an old directory can leave **stale files** from the previous version behind. Removing the app first (or deleting the directory) avoids that. The `make` deploy methods do not have this problem โ€” they replace the directory as a whole. - -## ๐Ÿงน Uninstalling - -```sh -sudo -u www-data php /var/www/nextcloud/occ app:remove APP_ID -``` - -> [!WARNING] -> `app:remove` disables the app and **deletes its directory**. Data the app stored in the database or in the data directory is handled by the app's own removal migrations โ€” check the app's documentation if you need to preserve it. - -## โ“ FAQ - -**Can I just `git clone` the app straight into `apps/`?** -You can, and it will run once you build it (`cd APP_ID && make build`). But a repository carries a lot that has no place in a running install โ€” tests, CI configuration, dev tooling, frontend sources, screenshots. Every method above ships only the runtime file set; ncmake's [keep model](https://github.com/ernolf/ncmake#-packaging-the-shipped-file-set) strips the rest, so you get a lean install and updates never leave stale dev files behind. - -**Do I need PHP or Node on the machine?** -No. `make build`, `make dist`, `make rsync` and `make cp` run their tools in throwaway containers (podman or docker); only `RUNTIME=bare` expects `composer` and `npm` on the `PATH`. Installing a release tarball (Method 1) needs neither โ€” just `tar` and `occ`. - -**Which method gives the "cleanest" install?** -All of them ship the identical runtime file set, so none is cleaner than another. Choose by access: a release tarball when you have no toolchain, `make rsync` / `make cp` when you can reach the instance and want a one-command deploy. - -## ๐Ÿ“š More - -- `make help` lists every target with the detected app, version and certificate status. -- `make help-` prints extended help for one target (options and examples), e.g. `make help-rsync`. -- Full tool documentation: the [ncmake README](https://github.com/ernolf/ncmake#readme). diff --git a/doc/WORKFLOWS.md b/doc/WORKFLOWS.md deleted file mode 100644 index 748eea2..0000000 --- a/doc/WORKFLOWS.md +++ /dev/null @@ -1,177 +0,0 @@ - -# ๐Ÿค– CI workflows for ncmake apps - -This guide covers everything about GitHub Actions workflows in an ncmake app: the workflow manager that installs and updates them, and the release workflow ncmake ships itself. - -> [!TIP] -> **TL;DR** โ€” `make dev-init` once per machine, then `make workflows-list` shows every available workflow with its status, `make workflows-install W="reuse lint-php"` installs the ones you pick, and a `make workflows-update` from time to time keeps them current. Both take an optional `COMMIT=1` (commit on a branch) or `PR=1` (commit and open the pull request). Commit `.github/workflows/` including the `.ncmake-workflows.json` lock file. - -- [The developer module](#-the-developer-module) -- [Where the workflows come from](#-where-the-workflows-come-from) -- [Listing, installing, updating](#-listing-installing-updating) -- [Committing and opening a PR: `COMMIT` and `PR`](#-committing-and-opening-a-pr-commit-and-pr) -- [Keeping them up to date automatically](#-keeping-them-up-to-date-automatically) -- [The lock file](#-the-lock-file) -- [Placeholders and runner labels](#-placeholders-and-runner-labels) -- [The ncmake release workflow](#-the-ncmake-release-workflow) -- [FAQ](#-faq) - -## ๐Ÿงฉ The developer module - -The workflow manager is not part of the core Makefile: it is a **developer module**. Someone who clones your app to build and install it never needs it, so it stays out of their `make help`. As the maintainer you fetch the modules once per machine: - -```sh -make dev-init -``` - -This discovers all modules the ncmake repo ships (`mk/*.mk`), caches them next to the core Makefile in `~/.cache/ncmake/` and loads them on every subsequent `make` run โ€” in every ncmake app on this machine. The cache keeps itself current with the same TTL/ETag mechanism as the core Makefile, and `NCMAKE_REF` pins the version, as everywhere. `make dev-clean` removes the modules again; `make help` then shows the plain user target set. - -> [!IMPORTANT] -> Like the core Makefile, modules never land in your repository. The only workflow-related file your repo carries is what you deliberately install into `.github/workflows/` โ€” plus its lock file, see below. - -## ๐ŸŒ Where the workflows come from - -The manager knows two sources, and looks them up in this order: - -| Source | Content | -|---|---| -| `ncmake` | the workflows ncmake ships itself, currently the [release workflow](#-the-ncmake-release-workflow) | -| `nextcloud` | the official [nextcloud/.github workflow templates](https://github.com/nextcloud/.github/tree/master/workflow-templates): lint-php, lint-info-xml, psalm-matrix, phpunit, reuse, block-unconventional-commits and many more | - -Discovery is live: the manager lists the sources via the GitHub API at run time, so a workflow added upstream appears in `make workflows-list` immediately โ€” no ncmake update involved. On a name collision the `ncmake` source wins. - -The Nextcloud templates are worth a special note: they are largely **self-configuring**. `lint-php.yml`, for example, computes its PHP version matrix at CI run time from your `appinfo/info.xml` (via the `nextcloud-version-matrix` action), so the very same file works in every app โ€” the same DRY principle the whole of ncmake is built on. Most templates install without any per-app patching. - -## ๐Ÿ“‹ Listing, installing, updating - -```sh -make workflows-list -``` - -prints every workflow of every source with its status: - -| Status | Meaning | -|---|---| -| *(empty)* | available, not installed | -| `installed` | installed and identical to the recorded state, upstream unchanged | -| `update available` | upstream changed since the install โ€” `workflows-update` picks it up | -| `modified` | you edited the local file โ€” never overwritten by `workflows-update` | -| `missing` | in the lock file but deleted locally โ€” reinstalled by `workflows-update` | -| `unmanaged` | present in `.github/workflows/` but not installed through ncmake | -| `gone upstream` | managed, but the source no longer offers it | - -```sh -make workflows-install W="reuse lint-php lint-info-xml" -``` - -fetches the named workflows (the `.yml` suffix is optional), substitutes [placeholders](#-placeholders), writes them into `.github/workflows/` and records them in the lock file. Installing over an existing file overwrites it โ€” that is also how you adopt an `unmanaged` file into management, or reset a `modified` one back to upstream. - -```sh -make workflows-update -``` - -brings every managed workflow to the current upstream state in one go: outdated and missing files are reinstalled, locally modified ones are skipped with a note, current ones are left alone. After installing or updating, review the diff and commit: - -```sh -git add .github/workflows/ -``` - -## ๐ŸŒฟ Committing and opening a PR: `COMMIT` and `PR` - -By default both `workflows-install` and `workflows-update` do exactly one thing: they rewrite the files in `.github/workflows/` and leave the commit to you. That is deliberate โ€” it is what the [automatic updater](AUTOUPDATE_WORKFLOW.md) drives, so the plain form must stay side-effect-free. When you want the full move in one step, two flags take you the rest of the way. They work identically on **both** targets: - -| You run | Refresh files | Branch + commit | Push | Open PR | -|---|:---:|:---:|:---:|:---:| -| `make workflows-update` *(default)* | โœ… | | | | -| `make workflows-update COMMIT=1` | โœ… | โœ… | | | -| `make workflows-update PR=1` | โœ… | โœ… | โœ… | โœ… | -| | | | | | -| `make workflows-install W="โ€ฆ"` *(default)* | โœ… | | | | -| `make workflows-install W="โ€ฆ" COMMIT=1` | โœ… | โœ… | | | -| `make workflows-install W="โ€ฆ" PR=1` | โœ… | โœ… | โœ… | โœ… | - -**`COMMIT=1` โ€” commit on a branch, stop before pushing.** Starting from `main`, it creates the branch (`ncmake/ci/workflow-update` for update, `ncmake/ci/workflows-install` for install), refreshes or installs there, and commits with a `Signed-off-by`. The commit message is a fixed first line plus **one bullet per file that actually changed**, read straight from the staged diff: - -```text -ci: update managed CI workflows from upstream - -- reuse.yml -- lint-php.yml -``` - -Then it **prints the exact `git push` command and stops** โ€” nothing leaves your machine. This is the whole point of the separate flag: you can inspect the commit, `git commit --amend` to add or adjust something, and push when you are ready. If nothing changed, the branch is discarded and `main` is left untouched. - -Before it starts, it checks that the branch does not already exist โ€” locally or on `origin`. If it does, it **stops immediately**, without refreshing, committing or pushing, and tells you how to proceed: a branch on `origin` may carry an open pull request, or be a merged one that was never deleted. Either merge or close its pull request, or delete the branch (`git push origin --delete `), then run the target again. It never deletes a branch for you, since an existing one may well be intentional. - -**`PR=1` โ€” commit, push, open the pull request.** It implies `COMMIT=1` (so everything above happens), then pushes the branch and opens the PR with the [GitHub CLI](GITHUB_PAT.md). The commit's first line becomes the PR title; the body is the same bullet list followed by a short description of what ran. `PR=1` needs `gh` installed (`make gh-install`) and authenticated (`gh auth login`); if either is missing you get a clear message and the commit still waiting on its branch with the push command shown, so nothing is lost. After you merge the pull request its `ncmake/ci/โ€ฆ` branch stays around unless you have GitHub remove merged branches for you โ€” see [Automatically deleting merged branches](DELETE_MERGED_BRANCHES.md). - -> [!IMPORTANT] -> The commit is signed from **your** git config (`commit.gpgsign = true`), exactly like `make version`. If your branch protection requires verified signatures, set that up once (`git config --global commit.gpgsign true` and a configured signing key) or the pushed commit cannot be merged. - -For the unattended equivalent of `PR=1` โ€” a signed, verified PR opened for you on a schedule with no local `gh` โ€” see the automatic updater next. - -## ๐Ÿ”„ Keeping them up to date automatically - -Running `make workflows-update` by hand works, but ncmake also ships a workflow that does it for you. The [workflow updater](AUTOUPDATE_WORKFLOW.md) runs `make workflows-update` on a daily schedule and opens a pull request whenever a managed workflow has changed upstream. It is what replaces Dependabot for `.github/workflows/`. See **[The workflow updater](AUTOUPDATE_WORKFLOW.md)** for installing it and the one-time token it needs. - -## ๐Ÿ”’ The lock file - -`.github/workflows/.ncmake-workflows.json` records, per managed workflow, the source it came from, the upstream blob sha at install time and the hash of the installed content. Those two fingerprints are what makes the status column possible: a differing upstream sha means `update available`, a differing content hash means `modified`. - -The recorded hash is the hash of the file **as written** โ€” after placeholder substitution and the runner rewrite below. Those transforms are deterministic, so a re-install of an unchanged upstream produces the identical bytes and the status stays `installed`; only a real upstream change or a hand edit moves it. - -Next to the lock the manager writes `.ncmake-workflows.json.license`, a [REUSE sidecar](https://reuse.software/spec/) that licenses the generated JSON (which cannot carry an SPDX header of its own). It defaults to `CC0-1.0` โ€” the license Nextcloud apps put on generated files โ€” with a copyright line whose year is read from the clock at generation time, so a file regenerated in a later year updates on its own. Override `wf_lock_license` and `wf_lock_copyright` in `ncmake.mk`. This keeps `make reuse` green without any `REUSE.toml` edit. - -**Commit the lock file and its `.license` sidecar.** They contain no secrets, and with them in the repository every machine โ€” and every co-maintainer โ€” sees the same status and can run `workflows-update`. Without the lock the manager would consider all workflows `unmanaged`. - -## ๐Ÿ”ค Placeholders and runner labels - -GitHub's workflow templates may contain placeholders in the form `$default-branch` (lowercase, hyphenated). The manager substitutes the ones it knows at install time: - -| Placeholder | Replaced with | -|---|---| -| `$default-branch` | the default branch of your origin remote (falls back to `main`) | - -Unknown placeholders of that form are left as-is and reported with a warning, so a new upstream placeholder never breaks the install โ€” you edit the file manually and the manager treats your edit as `modified` from then on. Everything else in the files โ€” `${{ ... }}` expressions, shell variables in `run:` blocks โ€” is none of the manager's business and passes through untouched. - -**Runner labels.** The Nextcloud templates run on the org's own runner pool, with labels like `ubuntu-latest-low`. These are configured at the organization level and are therefore *org-scoped*: a repo outside that org has no such runner, so a job on that label would queue forever and never turn green. On install the manager rewrites them to their GitHub-hosted equivalent: - -| Rewritten from | to | -|---|---| -| `ubuntu-latest-low` | `ubuntu-latest` | - -**Whether to rewrite is decided automatically** from the owner of your `origin` remote. A repo *inside* the org that owns those runners keeps the labels (it really has them); every other repo gets the rewrite. So a `nextcloud/โ€ฆ` repo and your own `you/โ€ฆ` repo both do the right thing with no configuration. The org is `wf_runner_org` (default `nextcloud`); the rules are `wf_runner_rewrite` (space-separated `old=new` pairs). Override either in `ncmake.mk`: - -```make -wf_runner_org = my-org # keep the org labels for repos under my-org -wf_runner_rewrite = # or: never rewrite, regardless of owner -wf_runner_rewrite = big=small a=b # or: your own rewrite rules -``` - -The rewrite is applied before the file is hashed, so it is invisible to the status model โ€” a rewritten file is still `installed`, and `workflows-update` keeps it so. - -> [!TIP] -> This is why the templates are installed through the manager rather than copied by hand: the same fetch that finds updates also localizes the org-specific bits (branch name, runner labels) and keeps the file REUSE-compliant. - -## ๐Ÿšข The ncmake release workflow - -`release.yml` โ€” the one workflow the `ncmake` source itself provides โ€” builds the release tarball in CI. It carries no app-specific data: the shipped file set comes entirely from ncmake (keep model + `.nextcloudignore`), so the file is byte-identical across all ncmake apps. - -The workflow triggers on `release: published` (attaches `-.tar.gz` to the release) and on `workflow_dispatch` (produces the same tarball as a downloadable artifact for inspection, without publishing). The whole build is `make build && make dist`; the tarball is located by glob, so nothing in the file names the app. - -`make` runs on the runner host, not in a job container: `ubuntu-latest` already carries podman (which ncmake uses for the build containers) plus git, curl, rsync, tar and python3, so the workflow needs no `setup-*` steps and no toolchain of its own. `contents: write` is the only permission, for the release upload; no secrets beyond the automatic `GITHUB_TOKEN`. - -Like the bootstrap stub, the file carries ncmake's MIT header and is committed verbatim; the `LICENSES/MIT.txt` you already have for the stub covers it for REUSE. - -## โ“ FAQ - -**Which workflows should a typical ncmake app install?** The set the ncmake reference apps use: `release` (ncmake), plus `reuse`, `lint-php`, `lint-info-xml`, `lint-php-cs`, `psalm-matrix` and `block-unconventional-commits` from the Nextcloud templates. Apps with a frontend add `lint-eslint` and their node workflow; apps with PHPUnit tests add a phpunit template. - -**Can I keep hand-written workflows next to the managed ones?** Yes. Files the manager did not install show up as `unmanaged` in the list and are never touched by `workflows-update`. - -**How do I pin the workflow versions?** The lock file already pins what is installed โ€” nothing changes until you run `workflows-update` or `workflows-install`. The `ncmake` source additionally follows `NCMAKE_REF`, so a tag there pins the release workflow's origin too. - -**What about private repos or API rate limits?** The listing uses the anonymous GitHub API (60 requests per hour per IP), which is plenty for the occasional list/install. The downloads themselves come from raw.githubusercontent.com and are not rate-limited in that way. diff --git a/img/README.md b/img/README.md index be2c974..411abfc 100644 --- a/img/README.md +++ b/img/README.md @@ -40,7 +40,7 @@ From another repo, serve it through a CDN that sends the correct SVG content typ [![built with ncmake](https://cdn.jsdelivr.net/gh/ernolf/ncmake@main/img/ncmake-badge.svg)](https://github.com/ernolf/ncmake) ``` -**Inline mark in a sentence** (for example in `doc/INSTALL.md`) โ€” either the bare chevron next to the word, or the whole `ncmake` mark as one image: +**Inline mark in a sentence** (for example in an app's README) โ€” either the bare chevron next to the word, or the whole `ncmake` mark as one image: ```markdown built and packaged with ncmake **ncmake** diff --git a/mk/workflows.mk b/mk/workflows.mk index ebb3c15..d8efbcb 100644 --- a/mk/workflows.mk +++ b/mk/workflows.mk @@ -402,7 +402,7 @@ make workflows (alias for workflows-list) The CI workflow manager: installs GitHub Actions workflows from their upstream sources (nextcloud/.github workflow templates plus ncmake's own) and keeps them updatable. See make help-workflows-list, -install, -update and -doc/WORKFLOWS.md in the ncmake repo for the full picture. +https://github.com/ernolf/ncmake/wiki/Workflows for the full picture. endef define help_workflows-list @@ -434,7 +434,8 @@ Without flags it only writes the files and reminds you to commit. COMMIT=1 runs from main, commits the result on branch ncmake/ci/workflows-install (one bullet per installed workflow) and prints the push command without pushing - amend it first if you like. PR=1 implies COMMIT=1 and additionally pushes and opens the -pull request via gh. See doc/WORKFLOWS.md. +pull request via gh. See +https://github.com/ernolf/ncmake/wiki/Workflows. make workflows-install W=reuse make workflows-install W="lint-php,lint-info-xml,psalm-matrix" @@ -455,7 +456,7 @@ the automatic updater relies on. COMMIT=1 runs from main, commits the result on branch ncmake/ci/workflow-update (one bullet per updated workflow) and prints the push command without pushing. PR=1 implies COMMIT=1 and additionally pushes and opens the pull request via gh, the same thing the updater does for you. See -doc/WORKFLOWS.md. +https://github.com/ernolf/ncmake/wiki/Workflows. make workflows-update COMMIT=1 make workflows-update PR=1 diff --git a/workflows/branch-cleanup.yml b/workflows/branch-cleanup.yml index 5fee38e..0a5957b 100644 --- a/workflows/branch-cleanup.yml +++ b/workflows/branch-cleanup.yml @@ -4,7 +4,8 @@ # ncmake branch cleanup: when a pull request is merged, delete its head branch # if it is still there. This is the workflow-shipped equivalent of the # repository "Automatically delete head branches" setting (see -# doc/DELETE_MERGED_BRANCHES.md): unlike that per-repository toggle it travels +# https://github.com/ernolf/ncmake/wiki/Deleting-merged-branches): unlike that +# per-repository toggle it travels # with the repository through `make workflows-install`, so a repository gets the # cleanup even when no admin has flipped the setting. Running both is harmless: # whichever removes the branch first wins and the other is a quiet no-op. diff --git a/workflows/workflow-updater.yml b/workflows/workflow-updater.yml index f44a906..d81ef33 100644 --- a/workflows/workflow-updater.yml +++ b/workflows/workflow-updater.yml @@ -12,8 +12,9 @@ # GITHUB_TOKEN may not push workflow files, and it is preferred over a PAT # because the same token both pushes the files and produces verified, signed # commits. Store its credentials as the NCMAKE_UPDATER_CLIENT_ID and -# NCMAKE_UPDATER_PRIVATE_KEY secrets. See doc/AUTOUPDATE_WORKFLOW.md and -# doc/GITHUB_APP.md. +# NCMAKE_UPDATER_PRIVATE_KEY secrets. See +# https://github.com/ernolf/ncmake/wiki/Workflow-updater and +# https://github.com/ernolf/ncmake/wiki/GitHub-App. name: ncmake workflow update @@ -21,7 +22,8 @@ on: schedule: # Daily at 05:30 UTC. GitHub starts scheduled runs on a best-effort basis and # delays them under load, so expect this a few hours late, not on the minute - # (see doc/AUTOUPDATE_WORKFLOW.md). Trigger workflow_dispatch to run without + # (see https://github.com/ernolf/ncmake/wiki/Workflow-updater). Trigger + # workflow_dispatch to run without # waiting for the schedule. - cron: '30 5 * * *' workflow_dispatch: @@ -87,7 +89,8 @@ jobs: # App as the update job, so it always has Contents: write regardless of the # repository's default token permissions. This only ever touches # ncmake/ci/workflow-update, and is unrelated to the repository-wide - # "Automatically delete head branches" setting (see doc/DELETE_MERGED_BRANCHES.md). + # "Automatically delete head branches" setting + # (see https://github.com/ernolf/ncmake/wiki/Deleting-merged-branches). if: >- github.event_name == 'pull_request' && github.event.pull_request.merged == true