[Doc] Fix graph doc issues

docs/source/user_guide/index.md

@@ -70,6 +70,7 @@ algorithms
 :maxdepth: 1
 :titlesonly:
 
+performance
 fastcache
 graph
 streams

docs/source/user_guide/parallelization.md

@@ -3,7 +3,7 @@
 Each top-level for-loop will be parallelized, within a kernel. Under the hood, each top-level for-loop will be launched as a separate GPU kernel.
 
 A top-level for-loop can be encapsulated in one or more of the following, and still be parallelized:
-- `if` statements where the conditional is `qd.static`
+- `if` statements where the conditional is [`qd.static`](static.md)
 - inline functions (`@qd.func`)
 
 Note that adding a non-static `if` over the top of a for-loop will lead to the for-loop NOT being parallelized.
@@ -88,6 +88,8 @@ Reducing the number and complexity kernel parameters reduces the kernel launch l
 - field args incur less launch latency than ndarray args
 - global fields incur no parameter-related launch latency
 
+For the underlying execution model - what a launch actually involves, where the latency comes from, and when reducing it helps - see [Performance](performance.md).
+
 ## Global memory
 
 In CUDA, there are 3 main types of memory:

docs/source/user_guide/performance.md

@@ -0,0 +1,39 @@
+# Performance
+
+This page covers the execution model that the other performance guides build on: what happens when you launch a GPU kernel, where the time goes, and when reducing that time actually speeds up your program. The other pages in this section - graphs, streams, fastcache, and performance dispatch - are performance tools built on this execution model.
+
+## What a GPU kernel launch is
+
+When you call a `@qd.kernel` from Python, the numerical work does not run inline in the calling thread. Instead Quadrants *launches* the work onto the GPU: it hands a compiled kernel plus its arguments to the GPU driver, the driver enqueues it onto a stream (an ordered queue of GPU work), and the launch call returns to the host before the GPU has finished - usually before it has even started. Host and GPU run asynchronously.
+
+Each top-level `for`-loop in a kernel is a separate *offloaded task*, and each offloaded task becomes one GPU kernel. So a kernel with three top-level for-loops launches three GPU kernels every time you call it. Every launch carries a fixed overhead that is independent of how much data the kernel processes - that per-launch overhead is what we call *kernel launch latency*. When a kernel crunches a lot of data the overhead is negligible; when it does little work, or when you relaunch small kernels many times in a loop, launch latency can dominate the total runtime.
+
+## Where kernel launch latency comes from
+
+It helps to split the latency of a single launch into three parts:
+
+- **Python side.** Everything that happens in the Python interpreter before control reaches Quadrants' C++ runtime. This is the most expensive of the three per call, and it is paid once per *Python* call - so a Python `while`/`for` loop that calls a kernel N times pays it N times.
+- **C++ side.** Work inside the Quadrants runtime (C++) once control has crossed the boundary. Cheaper than the Python side, but non-zero, and paid per offloaded task.
+- **GPU / driver side.** The GPU driver's own cost to place the kernel on a stream, plus the fixed scheduling latency on the GPU itself before the kernel begins running on the hardware.
+
+## Latency hiding
+
+Because host and GPU run asynchronously, kernel launches overlap with kernel execution: the host can be issuing the next launch while earlier kernels are still running on the GPU. As long as the host issues launches at least as fast as the GPU drains them, the queue never empties, the GPU stays busy, and the launch latency is fully *hidden* behind execution.
+
+So reducing launch latency does not always improve overall throughput. When each kernel runs long enough that its execution time exceeds the launch overhead, the GPU is the bottleneck and shaving launch latency changes nothing - the launches were already hidden behind execution.
+
+Kernel launch latency matters when kernels are relatively small. Then the GPU can finish a kernel before the host has issued the next one, so the GPU sits idle waiting for the next launch and the launch latency is *exposed* on the critical path. In that regime, cutting launch latency can directly increase throughput.
+
+## Reducing launch latency
+
+[Graphs](graph.md) are the main tool for reducing launch latency. On backends with hardware graph support, the kernel's whole sequence of GPU tasks is issued as a single graph launch rather than one launch per task, collapsing the per-task C++ and GPU/driver launch overhead. The per-call Python overhead is still paid on every call; to cut that too, move the loop itself into the graph with `graph_do_while` so that many iterations become a single call.
+
+Independently of graphs, you can also shrink the launch latency itself by reducing the number and complexity of kernel parameters - field arguments are cheaper than ndarray arguments, and global fields incur no parameter-related launch latency. See [Parallelization](parallelization.md#does-gpu-kernel-launch-latency-matter) for these launch-tuning guidelines.
+
+## Other performance tools
+
+The rest of this section covers tools that improve performance in other ways, not by reducing launch latency:
+
+- [Streams](streams.md) let independent top-level for-loops run concurrently instead of serializing on the default stream.
+- [Fastcache](fastcache.md) reduces the time to load cached kernels when a new Python process starts.
+- [Performance dispatch](perf_dispatch.md) benchmarks multiple implementations of an operation at runtime and picks the fastest.

docs/source/user_guide/streams.md

@@ -133,5 +133,5 @@ with qd.create_stream() as s:
 
 ## Limitations
 
-- **Not compatible with graphs.** Do not pass `qd_stream` to a kernel decorated with `graph=True` (if you do, a `RuntimeError` will be raised).
-- **Not compatible with autodiff.** Do not pass `qd_stream` to a kernel that uses reverse-mode or forward-mode differentiation, or inside a `qd.ad.Tape` context (if you do, a `RuntimeError` will be raised).
+- **Not compatible with [graphs](graph.md).** Streams cannot be combined with [`graph=True`](graph.md) kernels: passing `qd_stream` to a graph kernel raises a `RuntimeError`, and using `qd.stream_parallel()` inside a graph kernel raises a `QuadrantsSyntaxError`.
+- **Not compatible with [autodiff](autodiff.md).** Do not pass `qd_stream` to a kernel that uses reverse-mode or forward-mode differentiation, or inside a [`qd.ad.Tape`](autodiff.md) context (if you do, a `RuntimeError` will be raised).

python/quadrants/lang/ast/ast_transformer.py

@@ -1621,6 +1621,12 @@ def build_With(ctx: ASTTransformerFuncContext, node: ast.With) -> None:
             )
         if not ctx.is_kernel:
             raise QuadrantsSyntaxError("qd.stream_parallel() can only be used inside @qd.kernel, not @qd.func")
+        kernel = ctx.global_context.current_kernel
+        if kernel is not None and kernel.use_graph:
+            raise QuadrantsSyntaxError(
+                "qd.stream_parallel() is not compatible with graph=True kernels. Streams and graphs cannot be "
+                "combined; remove graph=True or the qd.stream_parallel() block."
+            )
         ctx.ast_builder.begin_stream_parallel()
         build_stmts(ctx, node.body)
         ctx.ast_builder.end_stream_parallel()

tests/python/test_streams.py

@@ -495,6 +495,29 @@ def bad_kernel():
         bad_kernel()
 
 
+@test_utils.test()
+def test_stream_parallel_rejects_graph():
+    """qd.stream_parallel() is incompatible with graph=True kernels."""
+    from quadrants.lang.exception import QuadrantsSyntaxError
+
+    N = 64
+    a = qd.field(qd.f32, shape=(N,))
+    b = qd.field(qd.f32, shape=(N,))
+
+    with pytest.raises(QuadrantsSyntaxError, match="not compatible with graph=True"):
+
+        @qd.kernel(graph=True)
+        def bad_kernel():
+            with qd.stream_parallel():
+                for i in range(N):
+                    a[i] = 1.0
+            with qd.stream_parallel():
+                for j in range(N):
+                    b[j] = 2.0
+
+        bad_kernel()
+
+
 @test_utils.test()
 def test_stream_with_ndarray():
     N = 1024