status: preserve the clean-status preview DAG - #17
Merged
Conversation
ttaylorr-oai
force-pushed
the
tb/codex/status-preview-unstable
branch
from
August 5, 2026 03:19
1694383 to
33e23cc
Compare
ttaylorr-oai
marked this pull request as draft
August 5, 2026 04:17
valid_cached_dir() used match_stat_data_racy(), whose treatment of ctime and other fields follows core.trustCtime and core.checkStat. Tracked entries can correct a false stat match with a later content comparison, but cached directories have no equivalent check. Renaming a child and restoring its parent's mtime can therefore hide untracked paths under weak stat settings. Compare every field persisted in directory stat_data regardless of those tracked-file settings, and retain the existing racy-timestamp check. The untracked-cache status test renames a child, restores the directory mtime, and verifies that cached and uncached status agree. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Comparing only a pathname, device, and inode does not establish that two observations still describe the same unchanged filesystem object. Users that reopen a path need a reusable comparison covering the represented metadata fields. Represent an object's stat identity as a zero-initialized, fixed-width array containing device, inode, mode, link count, ownership, size, and modification and change timestamps. Include nanoseconds where available and add birth time and generation on Apple platforms. Provide comparison helpers and register both their library source and Clar unit suite in the Makefile and Meson builds. The unit tests check identity equality and reject changes to each represented stat field. No exclude-file validation or production caller is introduced here. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
valid_cached_dir() performs a synchronous lstat() whenever an untracked-cache directory cannot rely on fsmonitor. A separate validation pass cannot remove that duplicated work unless traversal knows whether a saved result was checked and matched. Add transient stat_checked and stat_matches bits to each cached directory and let traversal consume them only when its caller marks the cache preloaded. Clear that marker after traversal and on the symlink-leading-path exit. Existing callers leave the marker clear, so ordinary lstat() validation and the fsmonitor path remain unchanged. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Tracked-index refresh can invalidate the mutable untracked-cache tree. A concurrent directory-validation worker therefore cannot discover nodes or read their validation inputs directly from that live tree. Capture each node pointer, copied pathname, saved stat data, and prior validity before concurrent work begins. The opaque preload object also retains the cache, root, index timestamp, repository, and directory flags needed to recognize its original context. Expose construction and release as a complete ownership boundary. This preparatory change allocates one snapshot per cached directory but has no production caller and does not start workers or publish results. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A captured untracked-cache directory must not be marked reusable from a stale snapshot: tracked-index refresh may invalidate its live node, replace the cache, or change the traversal's directory flags. Run lstat() against each saved pathname and compare its immutable stat snapshot using the strict, racy-aware directory comparison. Publish the checked result only if the current cache, root, and directory flags still match. Preserve any invalidation that happened after capture; failed or changed stats leave ordinary traversal responsible for rescan. Validation remains synchronous, and no status caller invokes the new finish operation at this boundary. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Synchronous validation still places every cached-directory lstat() on one execution path. Independent, already-captured directory snapshots can instead be divided among bounded workers without reading the live cache from those workers. Partition the snapshots using approximately 1,000 directories per worker, cap the worker count at six and at three times the available CPU count, and permit a bounded test override. Workers retain results in their own snapshot ranges; the existing finish operation joins them before publishing anything. Record worker count, thread-creation failures, directory count, and elapsed worker time through the threads, thread_failure, dirs, and wall_us Trace2 keys. Publication validity and applied-result counters remain in the earlier finishing boundary. Run the work synchronously for a single worker or without pthreads. If thread creation stops partway through, join started workers and process every unstarted range synchronously. No status caller enables the preload at this boundary. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Directory validation and tracked-index refresh inspect different snapshots, but running them consecutively leaves both operations on the status command's critical path. Start the cached-directory preload after reading the index and before refresh_index(). Join its workers after configuring excludes and before collecting untracked paths, then pass their results into directory traversal. Release any unfinished preload when status buffers are freed. Keep activation behind GIT_TEST_UNTRACKED_CACHE_AUTO_PRELOAD until production eligibility is defined. The untracked-cache status test checks unchanged and modified directories, preserved output, and the worker count selected by the running build's pthread support. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Clearing CE_FSMONITOR_VALID is not enough to make a provider event authoritative. With core.trustctime disabled, core.checkStat set to minimal, and a restored modification time, stat matching can still accept changed file contents. The same stale match can affect diff, apply, checkout, and unpack-trees. Mark a reported entry with the in-memory CE_CONTENT_CHECK_REQUIRED flag, clear CE_UPTODATE, and discard its cached stat data. Route diff, apply, checkout, and unpack-trees comparisons through ie_match_stat_with_content_check(), which calls ie_modified() only for marked non-gitlinks. Other direct ie_match_stat() callers retain their existing paths. Marking an entry up to date clears the transient flag. Ordinary entries, gitlinks, and unmarked zero-stat entries retain their existing stat behavior. Add hook regressions for restored timestamps, diff and status, indexed apply, checkout, case-insensitive unpacking, unchanged reset, and ordinary zero-stat behavior in t/t7519-status-fsmonitor.sh. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Starting directory-validation workers for a small cache, restricted pathspec, incompatible traversal, or fsmonitor-managed cache adds work without providing a safe whole-worktree reuse opportunity. Enable automatic preload only when the existing untracked cache and its root are valid, fsmonitor is disabled, traversal flags agree, and a bounded count finds at least 2,000 cached directories. Reject pathspecs, disabled untracked output, ignored-output modes, and incompatible -uall cache settings. Keep the test override for focused small-cache coverage. Status tests exercise both sides of the directory threshold and verify that restricted pathspecs and incompatible -uall requests retain the ordinary traversal path. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A filesystem-monitor provider can know that its event history is incomplete without being able to identify every affected path. Treating such a response as an ordinary path leaves tracked entries, cached attributes, and untracked-cache state falsely valid. Reserve // as a provider-only global invalidation record. It cannot collide with a worktree-relative path. When the client receives it, discard cached attribute stacks and untracked-cache state, invalidate every tracked entry, and mark the fsmonitor extension changed. Recognize the existing trivial response only when a complete record consists of a single slash and NUL, newline, or carriage-return terminators. This prevents the new double-slash record from being discarded as a trivial response while preserving existing hook forms. Add a hook regression in t/t7519-status-fsmonitor.sh that changes a tracked file, restores its timestamp, emits the global marker, and requires status to report the change. Global invalidation intentionally scans the tracked index. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
An implicitly started fsmonitor daemon inherits its caller's repository environment and current directory. In a linked worktree, inherited Git directory, worktree, common-directory, prefix, and index settings can make the child discover a different repository than the worktree whose status requested the daemon. Resolve the requested worktree to its canonical path, start the child from that directory, and remove repository-addressing variables from its environment. Keep the existing daemon start command and return an error if the worktree cannot be resolved. Add a macOS regression that implicitly starts fsmonitor from a linked worktree and checks the daemon child's working directory in Trace2. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
S06 and S10 need S01/P08 without S01/P01-P07. Merge the shared standalone commit after P07 and before P09-P11, so all three branches use the same P08 commit.
A pathname monitor cannot establish that every name for a multiply-linked regular file lies inside its watch cone. Persisting CE_FSMONITOR_VALID after checking the tracked name can therefore hide a later write through an unmonitored hardlink. Use fsmonitor_stat_can_be_valid() to exclude regular files with more than one link from persistent fsmonitor validity when the platform reports real link counts. Apply that decision where index refresh, threaded preload, and diff-files first consume an actual stat. Preserve CE_UPTODATE for the current process and retain existing persistent validity for single-link and nonregular entries. Windows and Cygwin synthesize their link counts, so preserve their existing fsmonitor behavior without claiming the hardlink guarantee there. Add a hardlink regression in t/t7519-status-fsmonitor.sh on platforms with trustworthy stat metadata. It keeps a tracked hardlink outside the fsmonitor-valid bitmap and checks that a write through an alias outside the worktree appears in status. The deliberate cost is another stat in a subsequent process. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Implicit fsmonitor startup resolves a Git command through the execution path and invokes its start subcommand. An overridden execution path can therefore select a different Git than the dispatcher that initiated the query, while adding another launcher between the client and daemon. Retain the absolute executable path during dispatcher initialization and expose it only for a real Git dispatcher. Start that executable directly with fsmonitor--daemon run --detach, then wait until its IPC socket is listening before accepting startup. Respect the configured startup timeout, defaulting to 60 seconds, and retain Git-command lookup when an authoritative dispatcher path is unavailable. The canonical worktree and sanitized environment established by S03/P01 remain in place. Update existing startup Trace2 checks for the direct invocation and add a macOS regression with a fake Git on the execution path to verify that the original executable is used. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Matching directory metadata alone cannot prove that its cached ignore rules are unchanged. A rewritten .gitignore with restored timestamps can otherwise leave preload results valid while changing which untracked paths should be visible. Snapshot each cached exclude object ID and validate its per-directory file on the existing preload workers. Open regular files with open_nofollow(), reject files larger than 1 MiB, and compare their raw or trailing-LF blob hash with the cached object ID. Verify the open file's stat identity before and after reading, then reopen its pathname and require the same identity through S01/P08. Publish directory results only when both stat and exclude checks match; otherwise invalidate the cached ignore state and fall back to ordinary traversal. A status test rewrites .gitignore, restores its mtime, and checks the result against uncached status. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Darwin FSEvents identifies the pathname associated with a hardlink event, not every name referring to the same inode. Invalidating only that pathname can leave another tracked hardlink trusted after its contents change. Classify the event's absolute path before handling its hardlink flags. For worktree events, enqueue the provider-wide marker introduced by S04/P02 so clients content-check the tracked set. Leave gitdir events in the existing cookie and gitdir handling; otherwise reads of hardlinked object files could repeatedly trigger global invalidation. Add a MACOS,HARDLINKS daemon regression that rejects a marker for a gitdir hardlink, then verifies the marker and correct status for a changed worktree hardlink with its timestamp restored. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
The fsmonitor.startTimeout setting controls how long a client waits for daemon startup; the daemon's run subcommand does not consume it. Nevertheless, daemon configuration parsing validates that setting for every subcommand. A malformed value can consequently kill an implicitly started daemon before it opens its IPC socket. Pass a run-specific configuration flag into the callback and skip startup-timeout parsing only for run. Continue parsing other daemon settings normally, and preserve strict timeout validation for the explicit start subcommand. Add a macOS regression that verifies implicit status still starts the daemon with a malformed timeout while explicit daemon start rejects the same configuration. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
In collapsed-directory mode, an untracked-cache parent may represent an entire directory by one descendant witness. If that witness becomes invalid or disappears, removing it without inspecting the directory can also hide another unvisited child that remains untracked. Compute cached validity from descendants upward after preload and invalidate collapsed ancestors when a required child proof fails. Before removing a stale collapsed witness, rescan its directory and retain the parent as untracked whenever another child survives. A focused untracked-cache test removes the cached witness while leaving a sibling present and verifies that status still reports the collapsed directory. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Changing a .gitattributes file can change how tracked content is converted without changing the tracked file's stat data. Invalidating the attribute-file path alone therefore leaves cached conversion state and affected fsmonitor-valid tracked entries falsely reusable. Recognize an exact .gitattributes basename in the refresh callback. Discard cached attribute stacks globally and strongly invalidate only tracked entries beneath that file's parent directory. A root attribute file invalidates all tracked entries; tracked entries in sibling directories remain valid after a nested attribute-file event. Mark the fsmonitor extension changed only when an entry is invalidated. Add Clar unit coverage for unrelated paths, nested-directory scope, root-directory scope, cleared validity, zeroed stat data, and the content-check marker. Register u-fsmonitor-attributes in both Makefile and t/meson.build so the suite is included in both build systems. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
An fsmonitor socket is selected through the Git directory, so separate worktree paths can reach the same daemon when they share that directory. A client in the second worktree can then consume change history from a daemon that watches the first, incorrectly treating changed files in its own worktree as clean. Hash the canonical worktree path together with its device and inode, plus birth time and generation on Apple platforms. Cache the resulting 64-character SHA-256 identity in the daemon and attach it to every client query. Check the identity before interpreting the requested token; reject missing or mismatched bindings with a cookie-synchronized trivial response that forces the ordinary refresh path. The protocol change must also tolerate a daemon left running by an older Git. Such a daemon treats a bound query as an opaque token and can return a plausible trivial response. After that exact response, query an unbound capability command. If the daemon does not advertise query-v1, serialize replacement through a per-socket restart lock, stop it, and start the invoking Git executable before retrying the bound query. Keep quit, flush, and capability control commands unbound. Bound daemon lifecycle retries, and fail the query instead of trusting history when the root cannot be identified or an incompatible daemon cannot be replaced. Regression tests cover shared-gitdir worktree aliases, replacement of a legacy daemon, and acceptance of a daemon that advertises a capability superset. The replacement test also verifies that the next status neither refreshes tracked entries nor starts another daemon. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Even after every directory and ignore input has been validated, collapsed-directory traversal still reopens cached subtrees that are known to contain no untracked paths. That walk repeats work the successful preload has already established. Record recursive validation and whether each cached subtree contains untracked output. In collapsed-directory mode, skip reopening a subtree only when its directory, descendants, check-only mode, and ignore inputs remain valid and no cached untracked entry exists. Clear the recursive proof when directory or ignore state is invalidated. The untracked-cache status test verifies that an unchanged empty subtree visits no directories and that a changed descendant still falls back to traversal and reports the new untracked path. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A provider can report a directory move or modification without naming a changed .gitattributes file beneath it. Existing directory handling invalidates tracked entries in the reported cone but can leave cached attribute stacks describing the old conversion rules. Discard cached attribute stacks only after directory handling matches at least one tracked index entry. Record semantic/attributes-cone with the number of matched entries. An unmatched directory keeps its existing case-correction and untracked-path fallback without speculatively flushing attribute state. Extend t/helper/test-read-cache.c to cache an old attribute, process a directory event, and require the new attribute value. Add hook regressions in t/t7519-status-fsmonitor.sh for both an indexed cone and an unmatched directory, including their distinct Trace2 behavior. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Ordinary index preload assigns existing paths directly to workers. A physical directory walk instead discovers new tasks while it runs, so an unbounded queue can exhaust descriptors or strand tasks when worker creation fails. Add a directory-task queue that retains parent and child identities, budgets descriptors against RLIMIT_NOFILE, and tracks queued as well as in-flight work. Reserve at most 128 task descriptors, leave up to 16 for the rest of the process, and run a worker synchronously when extra threads cannot start. Register the common queue for Darwin in Make, CMake, and Meson. No bulk scan is invoked from preload_index(), so existing behavior is unchanged. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A bulk directory worker must locate each observed tracked path and decide whether a directory has tracked descendants. Plain pathname ordering cannot answer either question correctly for sparse indexes. Add sparse-aware entry and descendant lookups with unseen, clean, content-check, and fallback states. Compare observed metadata with ie_match_stat(), and make duplicate observations fall back through an atomic compare-and-exchange or the existing queue mutex. Skip staged, intent-to-add, skip-worktree, removed, and otherwise ineligible entries. Register the index classifier in Make, CMake, and Meson without introducing deletion outcomes or content proofs. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A pathname-based directory walk can cross into a replacement worktree or another mount after the scan begins. Metadata from that namespace cannot safely certify entries from the original worktree. Open the worktree directory with O_NOFOLLOW, accept only a local APFS root, and capture its device, filesystem identity, and stat data. Reopen the configured worktree at completion and compare its identity using the namespace helper supplied by S01/P08. Register the Darwin root helpers in Make, CMake, and Meson. Their presence does not yet activate bulk preload. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
S05 depends on S04, all of S01 through P11, and S03. Join those histories once before applying S05. S05's final patch combines S01/P11's recursively valid empty UNTR subtrees with the paired FSUC token closed by S05. S03 and S04 add tests at the same location in t/t7527-builtin-fsmonitor.sh; keep S03's startup and worktree-binding tests before S04's invalidation tests. # Conflicts: # t/t7527-builtin-fsmonitor.sh
precompose_string_if_needed() lazily reads core.precomposeUnicode on its first non-ASCII input. Concurrent directory workers must not race while initializing repository configuration. Add repo_precompose_utf8_prepare() to resolve that policy before workers start, and add repo_precompose_string_if_needed() for conversion against an explicit repository. Preserve precompose_string_if_needed() as the existing one-argument wrapper. Existing callers retain their behavior; only a caller that opts into explicit preparation separates configuration from parallel conversion. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
S06 depends on S04 and standalone S01/P08. Join those histories once before applying S06. Their trees merge without a content resolution.
A Linux directory scan cannot substitute its results for lstat() when file identity, timestamps, or mount membership are missing. Depending on libc's statx declarations would also tie the implementation to the age of the installed Linux headers. Define the required statx syscall ABI locally and request complete basic statistics and a mount identifier. Reject invalid nanosecond fields, foreign mounts, and device, inode, link-count, owner, size, or timestamp values that cannot be represented in struct stat. Register the metadata module in the Make, CMake, and Meson Linux builds. The native Linux boundary build compiles it with DEVELOPER=1, but this patch does not select a backend or change the fallback. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A directory name observed during enumeration may resolve outside the original worktree after a rename, symlink replacement, magic-link traversal, or mount change. Path-based reopening would then inspect an unverified namespace. Introduce descriptor-relative Linux directory-open helpers. Prefer openat2() with beneath-root resolution and reject symlinks, magic links, and mount crossings when that syscall is available. Otherwise reject empty, absolute, dot-dot, and malformed paths. Open root-relative paths one component at a time and verify each mount. Keep direct child opens descriptor-relative; directory scanning verifies their mounts before enumeration. Register the opening module with Make, CMake, and Meson. The native Linux boundary build compiles it with DEVELOPER=1, but backend selection remains unchanged. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A getdents64 record supplies a type hint, not proof that a path is a regular file or directory. Trusting that hint can hide a tracked replacement, misapply ignore rules, or report the wrong visible untracked shape. Parse record lengths and names within a bounded 1 MiB worker buffer. Require statx metadata and matching mount identity for tracked paths. When collecting untracked paths, obtain authoritative metadata before classifying a wholly untracked file or directory. Use directory hints only to schedule paths with tracked descendants. Preserve per-entry fallback for special and multiply linked tracked files. Stop an untracked subtree once its normal-status witness is visible, and invalidate uncertain untracked results. Register the module in all three Linux builds; the native DEVELOPER=1 boundary build compiles it without selecting the backend. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Holding a directory descriptor establishes what workers read, but does not prove that the original directory stayed in the worktree. A child can also move under a different parent while queued. Publishing observations from either replacement could hide worktree changes. Capture the complete directory statx observation and converted stat identity before enumeration. Verify the descriptor mount and recheck both identities afterward. For queued children, resolve the parent through the held child descriptor and compare it with the recorded parent identity. Add the mount identifier to the shared directory identity and register the Linux scan module with Make, CMake, and Meson. The native DEVELOPER=1 boundary build compiles it, while recorded directory changes prevent the completed scan from being accepted. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Individually anchored descriptors do not establish that the mount namespace or named worktree root stayed unchanged throughout a scan. A mount replacement can invalidate otherwise consistent directory observations. Accept only ext-family and XFS filesystems with complete root statx and mount-identity data. Capture /proc/self/mountinfo before the scan, compare it at completion, and freshly reopen the named worktree root with O_NOFOLLOW to verify its original complete identity. Probe openat2() without requiring it. Register the topology module in all three Linux builds. The native DEVELOPER=1 boundary build compiles it. Missing namespace proof, unsupported filesystems, changed mount tables, or replaced roots reject the result; the retained mount snapshot adds memory and can reject unrelated namespace changes. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
The separately registered Linux metadata, anchored-open, enumeration, directory-validation, and topology modules cannot safely publish a physical scan by themselves. They must share the existing bulk backend lifecycle so every closing check runs before results are accepted. Assemble those modules into the Linux backend and register the shared and platform objects with Make, CMake, and Meson. Retain the existing requirements that core.preloadIndex and core.preloadIndexBulk are enabled and fsmonitor is disabled. Preserve ordinary preload when a required syscall, filesystem, mount proof, or closing validation is unavailable. Cap Linux scans at 16 workers. Each worker can allocate a 1 MiB directory buffer; mount snapshots and retained scan results add further memory. Document ext-family and XFS support and keep directory-type injection confined to the documented test environment. Add and register t7532-preload-index-linux.sh with 12 Linux-only cases. Native Linux validation passes 12/12 in the threaded build and 12/12 in a separate NO_PTHREADS build; CMake and Meson link Git. The suite compares ordinary status for tracked changes, visible and ignored paths, false type hints, fallback shapes, and a synchronized child replacement. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
The bulk preloader rejected an active fsmonitor provider, so status verified ambiguous tracked entries through a separate semantic scan. Publishing bulk observations or refreshed stat data before the closing provider query would permit a concurrent change to invalidate a clean result. Pass held parent descriptors, basenames, and observed metadata from both platform walkers to semantic_verify_file_at(). Borrow the captured provider proof epoch, hash eligible raw-safe files during the bulk walk, and retain clean states and stat updates provisionally. After the closing provider query confirms the same epoch, validate all pending positions before publishing clean states, refreshed stat data, and fsmonitor-valid bits. Clear provisional state on provider failure, epoch mismatch, or invalid updates, and retain the existing complete-refresh fallback. Choose the provider-backed bulk path from its actual safety conditions, not from whether semantic history is awaiting adoption. This lets a trivial daemon response or daemon restart rebuild and close an ordinary bulk proof, including for a skipHash index, while retaining the complete proof epoch and closing query. Require both preload settings, an expanded index, a pending built-in IPC token, and an eligible whole-worktree request. Keep APFS and Linux within their platform and filesystem limits. Allocate a bounded hash buffer and attribute check per content-verification worker, and retain tracked states and stat updates only until closure. Extend the APFS and Linux tests with same-size, restored-mtime content changes. Cover accepted closure, provider failure, dirty status, daemon token reset with a null-checksum index, and Trace2 evidence of hashing, deferred publication, and token acceptance or rejection. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A live exclude-source proof uses filesystem identity to keep one observation coherent. That identity cannot compare equivalent ignore sources captured by separate status processes: replacing a file with the same contents changes its identity without changing ignore semantics. Hash the existing, validated observations in first-observation order. Frame the digest with its version, source object format, unique source count, path, lookup policy, presence, and content identity. Exclude transient stat identity so an equivalent replacement retains the same semantic digest. Extend the existing exclude-proof unit tests to capture independent proofs across a same-content replacement and repeated observation. The digest is independently testable without issuing a sidecar or changing normal exclude-source validation. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A later status invocation cannot safely reuse an empty result unless its persistent record identifies the exact index and semantic inputs that the original scan proved. Accepting truncated, ambiguous, or forward-versioned records would turn a cache miss into a false clean result. Define the version-one CSTS encoding and serialize index identity in fixed-width network-byte-order fields. Bind the index format, entry count, checksum, HEAD tree, configuration and repository hashes, one exclude digest, and a bounded builtin-provider token. Protect the complete record with the repository's object-format checksum. Reject unknown flags, unsupported index formats, null required object IDs, invalid token bounds or prefixes, bad checksums, truncation, and trailing payload. Add fixed-width identity and sidecar unit coverage for both SHA-1 and SHA-256. Register the new source and unit suite in both Make and Meson. This patch defines and tests the format; it neither writes a sidecar nor changes status dispatch. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A valid sidecar encoding is not sufficient if its named index can be replaced between proof capture and publication. Publishing that record would let a later reader associate one clean result with another index. Expose the existing index-snapshot open and named-path revalidation helpers at their first store consumer. Require a durable index identity on local APFS, a matching index format, entry count, and checksum, and agreement between the held descriptor and the named index. Encode the sidecar under its own lockfile and repeat the index checks before committing that lock. Register the store unit suite with Make and Meson. Its local-APFS tests cover successful installation for SHA-1 and SHA-256 and rejection when the source index is replaced after pinning. Other filesystems fail closed. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A clean provider response does not establish that every index entry can be represented by an empty status result. Conflicted entries, submodules, sparse entries, intent-to-add entries, and independently trusted stat state can all require ordinary index processing. Introduce a single conservative certifiability check. Require a non-null index checksum and provider-valid ordinary entries. Reject gitlinks, nonzero stages, intent-to-add, skip-worktree, CE_VALID, and unrecognized entry flags while allowing the explicitly supported in-memory flags. Extend the existing index unit suite to exercise accepted ordinary entries and each unsupported entry shape. The classifier does not issue a proof or change status behavior by itself. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
An empty status result cannot certify the next invocation unless its tracked and untracked observations, ignore sources, provider token, configuration, repository, HEAD, and named index belong to one completed scan. Publishing a digest before provider-token closure, or reusing cached replacement-ref state, could issue a false clean proof. Retain the standard-exclude digest produced by the complete bulk scan. Keep provider-originated digest state pending until token closure accepts it, and preserve the accepted digest when consuming single-use tracked results. Inspect a fresh, uncached ref store and reject effective replacement refs. Then fingerprint the held local-APFS index and worktree, repository paths, locale, and external attribute state. Issue a sidecar only for the literal, top-level, empty porcelain-v2 command after persistent semantic history, an eligible expanded index, a complete untracked scan, and the HEAD cache tree all agree. Install against the pinned index before rolling back its held index lock; otherwise retain ordinary index-update behavior. Add the focused sidecar integration suite and register its source and production code with the relevant Make and Meson builds. Cover prior semantic history, unchanged index contents, exact command shape, and rejection of external attributes, untracked-cache results, and alternate indexes. No early status answer is introduced here. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
An installed sidecar cannot be inspected safely by opening an untrusted adjacent path without bounds. A symbolic link, named pipe, oversized record, or growing file could redirect the read, block status, or consume unbounded memory. Open the named sidecar without following symbolic links and request a nonblocking descriptor. Accept only a regular file of at most 8192 bytes, read exactly its recorded size, reject an additional byte, and parse its checksummed contents into caller-owned storage. Clear failed records and release storage explicitly. Platforms without nonblocking support fail closed. Extend the registered store unit suite to cover owned token storage under both object formats, symbolic links, FIFOs, and an oversized 8193-byte record. The loader is testable at this boundary; it does not yet bypass index deserialization. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Validating a sidecar must recapture standard excludes before status can trust an empty result. Opening an exclude source that has become a named pipe may otherwise block the supposedly cheap validation. Add an explicit nonblocking flag to exclude-source proof creation and carry it into the existing anchored source-open operation. Reject unknown flags, request nonblocking captures for sidecar issuance, and update the existing bulk-scan and unit-test callers to pass zero, preserving their current blocking and symbolic-link policies. Add a focused FIFO unit test showing that an opted-in proof captures and validates an empty pipe without waiting. The later early-status consumer can reuse nonblocking capture without changing ordinary exclude handling. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
An issued clean-status sidecar has no latency benefit while status still deserializes the index before checking it. Moving the check earlier is safe only if the recorded proof is revalidated around an empty builtin-fsmonitor delta. Attempt the sidecar only for the literal top-level porcelain-v2 command on an eligible main worktree. Load the bounded record, pin the named local-APFS index, recapture excludes without blocking, and check configuration, attributes, repository identity, HEAD, and provider mode. Query the builtin provider directly from the stored token. Keep the attribute and exclude proofs alive across that query. Recheck configuration, HEAD, fresh replacement-ref and repository state, attribute contents and namespace, exclude-source identity, and both the held and named index before accepting an empty delta. Return without deserializing index entries only when every check succeeds; otherwise continue through ordinary status. Unsupported anchored-open platforms take that ordinary path. Register the fast-path source with Make and Meson. Extend the existing sidecar integration suite for read-only hits, dirty worktree shapes, loose, packed, and custom replacement refs, sidecar and exclude FIFOs, changed configuration, attributes, HEAD, null-checksum indexes, and post-query replacement or exclude races. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
A clean-status sidecar is a narrowly scoped proof, not an alternate index or a general cache. Documenting only its serialized bytes would hide the full-scan issuance requirement and the revalidation needed before an empty provider response can answer status. Document the adjacent sidecar path, local-APFS and main-worktree eligibility, fixed-width version-one CSTS fields, a checksum using the repository object-format hash, the separate repository-identity hash, a bounded builtin-provider token, and the 8192-byte read limit. Explain why the source index, configuration, repository, HEAD, attributes, and standard excludes must remain coherent. Describe completed-scan issuance, persistent provider history, held index locks, the post-query race fence, nonblocking source opens, and read-only hits. State that every missing, unsupported, stale, malformed, or raced proof falls back to ordinary status. Register the technical document in both the documentation Makefile and Meson. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
The complete stack has three maximal leaves: S17, S15, and S02. Join them once at the integration tip; S01 is now an ancestor of S05 so that paired FSUC state can safely reuse its recursive UNTR proof. Retain both independent status bitfields in wt-status.h and every test block at its validated position in t/t7519-status-fsmonitor.sh. The result contains all 105 patches.
ttaylorr-oai
force-pushed
the
codex-unstable
branch
from
August 5, 2026 20:02
85dde89 to
d88434c
Compare
ttaylorr-oai
force-pushed
the
tb/codex/status-preview-unstable
branch
from
August 5, 2026 20:45
33e23cc to
f2b146d
Compare
ttaylorr-oai
changed the title
ttaylorr-oai
marked this pull request as ready for review
August 5, 2026 20:48
ttaylorr-oai
force-pushed
the
tb/codex/status-preview-unstable
branch
from
August 5, 2026 23:13
f2b146d to
1d292bf
Compare
dreynaud-oai
approved these changes
Aug 5, 2026
ttaylorr-oai
force-pushed
the
tb/codex/status-preview-unstable
branch
4 times, most recently
from
August 6, 2026 02:13
5d48aef to
e025eba
Compare
dreynaud-oai
approved these changes
Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Restack the current clean-status preview DAG onto the generated
codextip and preserve its reviewed internal merge topology.This is the current
status-part-18-unstablegraph, not the oldaggregate squash:
inherited dual-lane release workflow from the new
codexbase.The existing
status-part-*refs remain unchanged as checkpoint refs.Their shared history with this restack is already contained in the
published
codexbase, so they do not become Codex prerequisites.Validation:
git diff --checkt7063-status-untracked-cache.sh(64/64)t7519-status-fsmonitor.sh(68/68)t7527-builtin-fsmonitor.sh(90/90)