Skip to content

coro: implement the stackless preemptive runtime and managed call core - #45

Closed
cpunion wants to merge 339 commits into
mainfrom
llvm-coro
Closed

coro: implement the stackless preemptive runtime and managed call core#45
cpunion wants to merge 339 commits into
mainfrom
llvm-coro

Conversation

@cpunion

@cpunion cpunion commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Summary

This is the integrated compiler/runtime implementation for xgo-dev#1546. It preserves ordinary Go synchronous source and standard-library call style, infers suspension transitively, and lowers only the selected functions to LLVM stackless coroutines.

A source function has one primary body. An additional callable descriptor/dispatch representation is emitted only when a value crosses an open dynamic boundary such as any, an interface, a function value, reflection, or foreign code.

What this adds

  • A whole-program effect/demand plan with canonical function identity, frozen per-site lowering facts, cache digests, archive validation, and fail-closed dynamic-call handling.
  • A structured LLVM coroutine builder and physical ABI for LLVM 19--22, including frame allocation/root retention, child await, preemption safepoints, explicit panic outcomes, defer/recover cleanup, and dynamic callable dispatch.
  • One runtime scheduling model built around G, P, executor ownership, operation IDs, wait sets, result leases, select winner claims, cancellation, detach, and producer quiescence.
  • A shared event-source model for timers, poll, workers, channels, semaphores, and notifications, including lossless idle admission and bounded run slices.
  • A native executor fleet with routed wakeups, a bounded worker pool for blocking syscalls/files, poll/deadline integration for sockets, and shutdown/join semantics.
  • Target-neutral host-pull adapters and cross-target compilation coverage for JS/WASM, WASI, embedded, and bare-metal environments without requiring libuv or BDWGC in the scheduler core.
  • Go 1.26 synchronous-source acceptance probes for time.Sleep, timers/tickers, direct file syscalls, os.File, and loopback TCP with deadlines.

Managed callable and reflection boundary

The branch now completes the native equivalent of ffi_call_llgo without forking libffi or adding a new exported libffi symbol:

  • runtime/internal/ffi.CallLLGo enters the compiler intrinsic llgo.coroFFICall.
  • Stock ffi_call invokes an ordinary ABI descriptor thunk.
  • The final thunk transfers the function value's env/ctxt into the target body's LLVM nest or swiftself parameter.
  • For an asynchronous target, libffi runs only the coroutine ramp until it returns the initially suspended child handle. The compiler awaits that handle after the libffi frame has returned, so no C/native stack survives suspension.
  • Function values remain a fixed two-word carrier, {function-or-descriptor, environment}; no variable-sized Go runtime funcval is introduced.

This path covers reflect.Value.Call, CallSlice, methods, method values, interface methods, Type.Method.Func, and native reflect.MakeFunc. MakeFunc's raw libffi callback is bounded: it copies arguments and starts a managed coroutine ramp, while the user callback runs under the LLGo scheduler rather than on the libffi callback stack.

The ABI preserves aggregates, zero-sized/high-alignment results, variadic logical signatures, named function types, and the target-specific closure context convention. x86/ARM/RISC-V use nest; affected AArch64 targets use swiftself where required by the supported LLVM version; unsupported targets keep an explicit-context fallback.

Architectural cutover

The core has completed the planned hard cutover:

  • no staged EnableCoro* feature surface;
  • no legacy logical wait registration or timer/poll dual modes;
  • one static source dispatcher and one native fleet path;
  • one frozen physical plan for helper discovery, preflight, emission, and cache identity;
  • architecture gates prevent removed protocols from returning.

The design and invariants are documented in:

  • doc/llvm-coro-runtime-design.md
  • doc/coro-ir-design.md
  • doc/coro-async-core-contract.md
  • doc/coro-callable-contract.md

Validation

The previous integrated head db12c497b completed all nine pull-request workflows successfully: Build Cache, Coroutine, Docs, Format Check, Go, LLGo, Release Build, Stdlib Coverage, and Targets. The coroutine matrix passed LLVM 19--22 and the LLGo workflow passed all 18 jobs.

Additional local validation for the managed-call head includes:

  • LLVM 19 and LLVM 22 execution of the expanded reflectmakefunccoro scalar/aggregate/type matrix;
  • reflect.Value.Call and typed invocation of named reflect.MakeFunc values that suspend on timers;
  • methods, method values, interface methods, variadics, many results, zero-sized and high-alignment results;
  • the closureall LIT contract and LLGo end-to-end test;
  • exact SSA, C ABI, Go ABI, compiler, runtime FFI, build identity, and architecture-cutover gates.

Current CI is running for 5bd941ef8. The first run found two gofmt-only test differences; that feedback is fixed in this head and both affected LLGo tests pass locally.

Scope boundaries

This PR lands the reusable core and proven vertical slices; it does not claim that every target or every GOROOT test is complete. Remaining work includes broader GOROOT compatibility, dynamic P sizing/work stealing, full timer GC/synctest/asynctimerchan semantics, broader FD/process/signal coverage, ordinary JS/WASI reactor pumping, RTOS/bare-metal HAL adapters, precise moving-GC integration, and target-native/AOT reflect.MakeFunc trampolines where executable libffi closures are unavailable.

Native reflect.MakeFunc currently depends on libffi executable closures. WASM and bare-metal retain the explicit-context ABI and compile path, but need bounded AOT trampoline allocation before claiming the same dynamic callback coverage.

LLVM dependency

The module currently requires github.com/xgo-dev/llvm v0.9.5 and temporarily replaces it with github.com/cpunion/llvm v0.9.4-0.20260724133556-6dccb7285597. That cpunion branch supplies the missing LLVM 19--22 attribute inspection needed to preserve/remap nest and swiftself across C ABI and large-aggregate lowering.

Tracks xgo-dev#1546. This implementation remains in cpunion/llgo for fork validation and is not proposed for xgo-dev/llgo merge at this stage.

cpunion added 30 commits July 16, 2026 23:37
feat(coro): add runnable stackless scheduler and panic prototype
runtime(coro): add stable wait registration and cancellation
runtime(coro): add stable executor request gate
runtime(coro): bind executor driver to scheduler
runtime(coro): add handle-free terminal executor close
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@cpunion cpunion changed the title coro: add the stackless preemptive runtime core coro: implement the stackless preemptive runtime and managed call core Jul 24, 2026
@cpunion

cpunion commented Jul 25, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #46, which contains the completed ProgramIR/HostOp hard cutover, current native and WebAssembly event/stdlib validation, and a focused bounded CI matrix. The old run completed against obsolete code and its monolithic integration process accumulated LLVM residency until timeout.

@cpunion cpunion closed this Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant