Skip to content

Allreduce implementation, various smaller extra changes#3

Open
keshprad wants to merge 2 commits into
hpcgroup:developfrom
keshprad:dev/keshprad
Open

Allreduce implementation, various smaller extra changes#3
keshprad wants to merge 2 commits into
hpcgroup:developfrom
keshprad:dev/keshprad

Conversation

@keshprad
Copy link
Copy Markdown

@keshprad keshprad commented Oct 19, 2025

Summary of changes:

  • New collective for AllReduce: implemented in both Python (pccl/all_reduce.py) and C++ (csrc/all_reduce.cpp/h).
    • Recursive and ring algorithms available through a unified _all_reduce / all_reduce_2D Python interface and a all_reduce_mpi C++ pybind11 binding.
  • Added missing: Ring AllGather
  • int → int64_t type fixes: Updates element count and block size parameters across all_gather, reduce_scatter, and vector add kernel to prevent overflow when operating on large tensors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant