Project idea: reader feedback system for lecture sites (errata, bugs, suggestions)

[mmcky]

Motivation

As more of our content production involves systems with a stochastic nature (AI-assisted translation, and code generation), reader feedback becomes more than community engagement — it is a quality-assurance sensor. Readers are the widest-coverage review pass our lectures get, and right now the path from "I spotted an error" to "an issue a maintainer can act on" has high friction: the reader must know about GitHub, find the right repo, open an issue, and describe where on the page the problem is.

This project proposes building a low-friction, location-aware feedback mechanism into our lecture themes, so every lecture site gets it on a theme bump.

Implementation target: quantecon-theme.mystmd first. As the future build system, the new MyST theme is where this should land initially — and it is also the easier place to build it: the theme is a React (Remix) app on @myst-theme/*, with an existing toolbar component family (GitHubButton, LaunchButton, etc.) that a FeedbackButton + slide-out panel slots into naturally, and MyST page metadata already carries per-page source/GitHub information. A backport to quantecon-book-theme (Jinja template + vanilla JS in the existing qe-toolbar) can follow for sites still on the jupyter-book pipeline, if demand warrants.

Sketch of the reader experience

• A "Report / feedback" button in the RHS toolbar (alongside the existing GitHub / download / launch buttons) opens a small slide-out panel.
• The reader picks a category — errata / bug / suggestion — and writes a short comment. If they selected text on the page first, the selection is captured as a quote.
• The submission is automatically linked to the document location: page URL, nearest section anchor (plus a #:~:text= fragment), and the page's source file (both themes already know the repository URL and source path per page).

Where feedback lands

Proposed: issues in the lecture repo the page came from, arriving pre-labelled (e.g. bug / feedback + lecture label, consistent with the consolidated label scheme). Maintainers triage where they already work, and no routing layer is needed. Cross-repo views could later come from qe CLI tooling.

Options for the submission mechanism

Option A — Prefilled GitHub issue (no backend).
"Submit" opens github.com/<repo>/issues/new with title, labels, and a body prefilled with the captured location metadata and the reader's comment.

• ✅ Zero infrastructure; spam handled by GitHub auth; ships as a pure theme feature.
• ❌ Requires a GitHub account, which filters out a large share of actual readers (students, policy audiences).

Option B — Anonymous submission via a small backend.
Same panel, but "Submit" POSTs to a lightweight service (e.g. Cloudflare Worker or GitHub App) that files the issue under a bot identity.

• ✅ Lowest reader friction — the true "crowd-sourced" version; richer UX possible (screenshots, ratings).
• ❌ We own spam/abuse mitigation (e.g. Turnstile), rate limiting, and a piece of infrastructure shared across ~30 lecture sites.

Option C — Off-the-shelf annotation/comment layers (Hypothesis, giscus/utterances).
Jupyter-book supports both natively; Hypothesis gives true text-anchored annotation, giscus gives GitHub-Discussions-backed page comments.

• ✅ Nearly free to enable; could serve as a cheap experiment to gauge demand.
• ❌ Feedback lives outside our GitHub triage workflow (Hypothesis) or is page-level only (giscus); moderation story is weaker.

A and B are not mutually exclusive: the panel UX and location capture are the same work, and only the submit target differs — so A could serve as an MVP with B as a designed-for second phase, if the team prefers a phased route.

Future direction: a triage agent

Once feedback is flowing, the volume (and variable quality) of crowd-sourced reports is itself a good fit for an agent. A triage agent could monitor incoming reader reports and organise them — deduplicating against existing issues, verifying errata against the lecture source, applying/correcting labels, summarising recurring themes per lecture or series, and escalating high-confidence errors to maintainers.

Two shapes are possible, mirroring the destination question above:

• Distributed — the agent operates on the GitHub issues in each lecture repo (e.g. a scheduled workflow or app installed org-wide), keeping triage where maintainers already work.
• Central — the agent monitors all feedback from one vantage point and maintains a cross-repo dashboard/digest, routing items out to lecture repos as needed.

This is deliberately out of scope for the initial build, but the choice of submission mechanism should not preclude it — structured metadata in reports (category, location, source file) is exactly what makes automated triage tractable later.

Open questions

1. Which option (or sequence of options) do we want to pursue?
2. Anonymous feedback: is the reach worth owning spam mitigation and a small service?
3. Should categories map directly onto our standard labels, and do we want a dedicated reader-feedback label for filtering/metrics?
4. Do we want aggregate metrics (reports per lecture / per series) as an explicit goal, e.g. surfaced via qe CLI or the status dashboards?
5. Triage agent: central or distributed (per-repo) — and what report metadata should we capture now to keep both doors open?
6. Process: since this touches the theme plus every lecture repo's triage workflow, should this graduate to a QEP (see Proposal: QuantEcon Enhancement Proposals (QEPs) — a lightweight process for cross-repo decisions #325) once a direction is chosen?