Enable HIP Graph support #15
Draft
rtmadduri wants to merge 1 commit into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enable HIP/ROCm Graph-Capture Support
Enables Warp's existing graph-capture infrastructure on HIP/ROCm and validates it on an AMD Instinct MI325X (gfx942, ROCm/HIP 7.2). Phase 1 makes core capture/replay, graph allocations, APIC save, examples, and unit tests work on HIP.
1. Central software gate disabled HIP graph capture
Device.supports_graph_capturereturnedFalsefor HIP, which cascaded throughcapture_begin()(returned early withgraph=None), the test device helpers, and example fallbacks — disabling all graph capture on HIP even though the core native capture path was already device-agnostic.Device.supports_graph_capturenow returnsTrueon HIP (warp/_src/context.py); docstring updated. Thecapture_launchgraph is Noneguard is kept as a safety net.test_graph12/12 (incl.test_graph_allocmempool pointer-remapping), and graph-using suites pass (test_map,test_reload,test_sparse,test_array,test_grad,fem/test_fem_integrate,tile/test_tile_shared_memory,geometry/test_mesh).2.
capture_pause/capture_resumemiscategorized as conditional-only"must be CUDA Toolkit 12.4+".wp_cuda_graph_pause_capture/wp_cuda_graph_resume_capturewere defined inside the#if !defined(__HIP_PLATFORM_AMD__) && CUDA_VERSION >= 12040conditional-node block, with no-op HIP stubs that raised. Pause/resume is actually used to suspend capture while JIT-compiling a module mid-capture — unrelated to conditional (if/while) subgraphs.#ifblock so they compile unconditionally (they only usehipStreamGetCaptureInfo/cudaStreamEndCapture/cudaStreamBeginCaptureToGraph, all already aliased inhip_util.h); removed the HIP stubs (warp/native/warp.cu).3. Linear solvers required conditional graph nodes
check_every=0) inoptim/linear.pyusedwp.capture_while(conditional graph nodes), raisingRuntimeError: Conditional graph nodes are not supported on HIP/ROCmfor cg/cr/bicgstab/gmres (hit byexample_taylor_green)._run_capturable_loopnow checkswp.is_conditional_graph_supported()and, when false (HIP), runs a fixed-count loop ofmaxitercycles instead ofcapture_while. This is numerically correct because the CG/CR/BiCGSTAB/GMRES update kernels zero outalpha/betaonce the residual falls below tolerance, so iterating past convergence is a no-op.use_cuda_graph=FalseandTrue.4. BVH rebuild-in-capture not graph-capturable on HIP
test_capture_bvh_rebuild(and grouped) rely on the LBVH builder's rocPRIM/hipcubDeviceRadixSort, which is not graph-capturable during HIP capture (works fine outside capture), leaving BVH with no graph-capture coverage on HIP.test_capture_bvh_query(tests/geometry/test_bvh.py) andtest_grouped_capture_bvh_query(tests/geometry/test_grouped_bvh.py): build the BVH outside the capture, then capture and replay the query launches against an eager reference. Rebuild-in-capture tests are skipped on HIP.5. Test suite and example audit for HIP graph capture
not d.is_hipgraph guards or stale fallbacks that bypassed the central capability, plus genuine ROCm limitations that would surface as spurious failures.supports_graph_capture/get_graph_capture_test_devices(tests/unittest_utils.py), and added targeted, documented skips for genuine ROCm 7.2 limitations:.wrpsave/load (capture_save): format encodes a CUDAsminteger arch/device type; HIPgfxstrings aren't representable —capture_saveraises a clear error on HIP andtest_apic/test_apic_meshexclude HIP.test_event_elapsed_time_graph) and external-event cross-graph sync (test_event_external) skipped intests/cuda/test_streams.py.tests/cuda/test_async.py.test_fixedarray.pyconditional-graph check switched towp.is_conditional_graph_supported()so it skips correctly on HIP.cuda/test_streams24 OK / 4 skipped,cuda/test_async1213 OK,cuda/test_conditional_captures32 OK with 16 self-skipping viais_conditional_graph_supported().Validation
All results validated on AMD Instinct MI325X (gfx942), ROCm/HIP 7.2, native lib rebuilt for gfx942.