Core¶

Data Structures¶

Communicators¶

template<CommunicationSpace Comm = DefaultCommunicationSpace, KokkosExecutionSpace Exec = Kokkos::DefaultExecutionSpace> class Communicator¶

Template class for communicator wrappers of different communication space types.

Communicators wrap a communication library-specific communicator (e.g. MPI_Comm) and a Kokkos execution space, tightly coupling the two.

Communicator objects are constructed via factory member functions. The parameterized constructor is private, and no default constructor is defined. They are move-only objects: copy construction and copy assignment are explicitly deleted. Use the duplicate member functions to create equivalent “copies” of communicators. There is always exactly one owner of a Communicator.

Template Parameters:

Comm – The communication space (transport backend) to use. Defaults to DefaultCommunicationSpace.
Exec – The Kokkos execution space to use. Defaults to Kokkos::DefaultExecutionSpace.

using execution_space = Exec ¶

using communication_space = Comm ¶

using communicator_type = Comm::communicator_type¶

using size_type = Comm::size_type¶

using rank_type = Comm::rank_type¶

Common interfaces¶

Both specializations share the following interface:

~Communicator() noexcept¶: Destructor.

Communicator(const Communicator&) = delete¶: Copy constructor is deleted because a Communicator cannot be implicitly copied. Use duplicate instead.

auto operator=(const Communicator&) -> Communicator& = delete¶: Copy assignment operator is deleted because a Communicator cannot be implicitly copied. Use duplicate instead.

Communicator(Communicator&&) noexcept¶: Move-constructs a Communicator.

auto operator=(Communicator&&) noexcept -> Communicator&¶: Move-assigns a Communicator.

[[nodiscard]] constexpr auto size() const noexcept -> size_type¶

Returns:: The size (i.e., number of processes) in the communicator.

[[nodiscard]] constexpr auto rank() const noexcept -> rank_type¶

Returns:: The rank that identifies the calling process within the communicator.

[[nodiscard]] auto split(int color, int key) noexcept -> std::optional<Communicator<Comm, Exec>>¶

Splits a Communicator.

Given a color and a key, creates as many new communicators as distinct values of color are given, ordering processes according to the value of key. All processes with the same color join the same communicator.

Parameters:

color – A value controlling in which split communicator the calling process should be in.
key – A value ordering the calling process within the split communicator.

Returns:

A communicator if the calling process is part of one of the split communicators, std::nullopt if the color is a special value excluding the process at this rank or on error.

[[nodiscard]] auto duplicate() noexcept -> std::optional<Communicator<Comm, Exec>>¶

Duplicates a Communicator.

Returns:: A communicator on success, std::nullopt on error.

MPI specialization¶

template<KokkosExecutionSpace Exec> class Communicator<MpiSpace, Exec>¶

Communicator specialization for the MpiSpace communication space. Wraps an MPI_Comm handle.

using execution_space = Exec ¶

using communication_space = MpiSpace ¶

using communicator_type = MPI_Comm¶

using size_type = int¶

using rank_type = int¶

[[nodiscard]] static auto from_raw(MPI_Comm comm, const Exec &exec = Exec{}) noexcept -> Communicator<MpiSpace, Exec>¶

Constructs a Communicator from a raw MPI_Comm handle and a Kokkos execution space instance. Defaults exec to Exec. The passed handle must be a valid handle and must not be an inter-communicator parent handle. The returned communicator does not own the underlying handle, and the user is responsible for destroying it.

Parameters:

comm – A valid communicator handle.
exec – A Kokkos execution space instance. Defaults to Kokkos::DefaultExecutionSpace.

Returns:

A communicator on success, std::nullopt if the passed handle was MPI_COMM_NULL.

[[nodiscard]] static auto split_from_raw(const MPI_Comm comm, int color, int key, const Exec &exec = Exec{}) noexcept -> std::optional<Communicator<MpiSpace, Exec>>¶

Splits from a raw MPI communicator and associates it to a Kokkos execution space instance. Defaults exec to Exec.

Creates as many new communicators as distinct values of color are given, and orders processes according to the value of key. All processes with the same value of color join the same communicator. A process that passes MPI_UNDEFINED as color will not join a new communicator.

Parameters:

comm – A valid communicator handle.
color – A value controlling in which split communicator the calling process should be in.
key – A value ordering the calling process within the split communicator.
exec – A Kokkos execution space instance. Defaults to Kokkos::DefaultExecutionSpace.

Returns:

A split communicator on success, std::nullopt if the passed color was MPI_UNDEFINED or on error.

[[nodiscard]] static auto duplicate_from_raw(const MPI_Comm comm, const Exec &exec = Exec{}) noexcept -> std::optional<Communicator<MpiSpace, Exec>>¶

Duplicates from a raw MPI communicator.

Parameters:

comm – A valid communicator handle.
exec – A Kokkos execution space instance. Defaults to Kokkos::DefaultExecutionSpace.

Returns:

A communicator on success, std::nullopt on error.

[[nodiscard]] auto comm() noexcept -> MPI_Comm&¶

[[nodiscard]] auto comm() const noexcept -> const MPI_Comm&¶

Returns:: A reference to the underlying MPI_Comm object.

[[nodiscard]] auto exec() const noexcept -> const execution_space&¶

Returns:: A const reference to the associated execution space instance.

NCCL specialization¶

template<> class Communicator<Experimental::NcclSpace, Kokkos::Cuda>¶

Communicator specialization for the Experimental::NcclSpace communication space. Wraps an ncclComm_t handle.

using execution_space = Kokkos::Cuda¶

using communication_space = Experimental::NcclSpace ¶

using communicator_type = ncclComm_t¶

using size_type = int¶

using rank_type = int¶

[[nodiscard]] static auto from_raw(ncclComm_t comm, const Kokkos::Cuda &exec = Kokkos::Cuda{}) noexcept -> Communicator<Experimental::NcclSpace, Kokkos::Cuda>¶

Constructs a Communicator from a raw ncclComm_t handle and a Kokkos CUDA execution space instance. Defaults exec to Kokkos::Cuda. The returned communicator does not own the underlying handle, and the user is responsible for destroying it.

Parameters:

comm – A valid communicator handle.
exec – A Kokkos CUDA execution space instance. Defaults to Kokkos::Cuda.

Returns:

A communicator on success, std::nullopt if the passed handle was nullptr.

[[nodiscard]] static auto split_from_raw(const ncclComm_t comm, int color, int key, const Kokkos::Cuda &exec = Kokkos::Cuda{}) noexcept -> std::optional<Communicator<Experimental::NcclSpace, Kokkos::Cuda>>¶

Splits from a raw NCCL communicator and associates it to a Kokkos CUDA execution space instance. Defaults exec to Kokkos::Cuda.

Creates as many new communicators as distinct values of color are given, and orders processes according to the value of key. All processes with the same value of color join the same communicator. A process that passes NCCL_SPLIT_NOCOLOR as color will not join a new communicator.

Parameters:

comm – A valid communicator handle.
color – A value controlling in which split communicator the calling process should be in.
key – A value ordering the calling process within the split communicator.
exec – A Kokkos CUDA execution space instance. Defaults to Kokkos::Cuda.

Returns:

A split communicator on success, std::nullopt if the passed color was NCCL_SPLIT_NOCOLOR or on error.

[[nodiscard]] static auto duplicate_from_raw(const ncclComm_t comm, const Kokkos::Cuda &exec = Kokkos::Cuda{}) noexcept -> std::optional<Communicator<Experimental::NcclSpace, Kokkos::Cuda>>¶

Duplicates from a raw NCCL communicator.

Parameters:

comm – A valid communicator handle.
exec – A Kokkos CUDA execution space instance. Defaults to Kokkos::Cuda.

Returns:

A communicator on success, std::nullopt on error.

[[nodiscard]] auto comm() noexcept -> ncclComm_t&¶

[[nodiscard]] auto comm() const noexcept -> const ncclComm_t&¶

Returns:: A reference to the underlying ncclComm_t object.

[[nodiscard]] auto exec() const noexcept -> const Kokkos::Cuda&¶

Returns:: A const reference to the associated Kokkos::Cuda execution space instance.

Requests¶

template<CommunicationSpace C = DefaultCommunicationSpace> class Request¶

Template class for request wrappers of different communication space types.

Request objects are move-only: copy construction and copy assignment are explicitly deleted. Moves transfer the native handle and completion callbacks together, leaving the source null and empty. Requests must be explicitly completed before destruction. Destruction never waits, cancels, frees, or completes an operation; a non-null handle or pending callbacks trigger a fatal precondition diagnostic.

Native access through request() and request_ptr() is borrowed. Applications must not complete, free, replace, or retain an owning copy of the handle: doing so bypasses callback and buffer lifetime management. Backend implementations may initialize inactive requests through these accessors.

Template Parameters:: C – The communication backend to use. Defaults to DefaultCommunicationSpace.

using communication_space = C ¶

using request_type = C::request_type¶

using rank_type = C::rank_type¶

Common interfaces¶

Both specializations share the following interface:

Request(const Request&) = delete¶: Copy constructor is deleted because a Request can only be moved.

auto operator=(const Request&) -> Request& = delete¶: Copy assignment operator is deleted because a Request can only be moved.

Request(Request &&other) noexcept¶: Transfers the native handle and callbacks. The source becomes inactive with no pending callbacks.

auto operator=(Request &&other) noexcept -> Request&¶: Transfers the native handle and callbacks. The destination must have a null handle and no pending callbacks; otherwise a fatal precondition diagnostic is issued. Self-move is a no-op. No operation is implicitly completed.

template<KokkosView V> auto extend_view_lifetime(const V &view) -> void¶

Captures a Kokkos View to extend its lifetime until the request completes. Retains allocations owned by managed Views until completion. Has no effect on unmanaged Views: Kokkos Comm cannot retain their user-owned allocations, which the application must keep valid until completion.

Template Parameters:: V – A Kokkos View type.
Parameters:: view – The view whose lifetime should be extended.

auto add_callback(std::function<void()> &&cb) -> void¶

Implementation hook; applications must not register callbacks directly. Callbacks execute in registration order after backend completion, with captured resources retained until all callbacks finish.

Warning

Callbacks must be nonempty and must not throw exceptions. Kokkos Comm does not use exceptions and assumes that callbacks cannot throw; it does not validate callbacks or catch exceptions. Callbacks must not mutate, reactivate, destroy, or re-enter their owning request, including through wait, test, or batch waits. Violating these requirements is unsupported.

Parameters:: cb – The callback function to register.

auto wait() -> void¶: Blocks until the associated operation completes and executes all registered callbacks, including unpacking work. On return, the native handle is null and callbacks have been released. Calling this on an empty request is harmless.

auto test() -> bool¶

Non-blocking query for the completion of the associated operation. Executes all registered callbacks if the operation has completed, then leaves a null handle and no callbacks. An incomplete operation retains its handle and callbacks. Empty requests return true. Repeated individual completion calls do not repeat callbacks.

Returns:: true if the request has completed, false otherwise.

auto wait(Request &request) -> void¶

Free function overload. Waits on request until the associated operation completes.

Parameters:: request – A reference to the request to wait on.

auto wait(Request &&request) -> void¶

Free function overload. Waits on an r-value request, consuming it upon completion.

Parameters:: request – An r-value reference to the request to wait on.

auto wait_all(std::span<Request> requests) -> void¶

Waits for every request that is active on entry. All native handles become null before any completion callbacks execute. Then runs and releases every request’s callbacks, including callbacks attached to inactive requests. An empty span is harmless, and repeated waits do not repeat callbacks.

Parameters:: requests – The list of requests to complete.

auto wait_any(std::span<Request> requests) -> std::optional<rank_type>¶

Waits for one active request to complete. The selected request’s native handle becomes null, then its callbacks execute and are released. Every other request retains its handle and callbacks, even if its operation has already completed in the backend. Inactive requests are skipped, including those with pending callbacks; use wait or wait_all to drain their callbacks. Repeated calls select each active request at most once.

Parameters:: requests – The list of requests to poll.
Returns:: The index within the original span of the selected request, or std::nullopt if the span is empty or all native handles are null. Selection order is unspecified when multiple requests are ready.

auto test(Request &request) -> bool¶

Free function overload. Queries request for completion of the associated operation.

Parameters:: request – A reference to the request to query.
Returns:: true if the request has completed, false otherwise.

MPI specialization¶

template<> class Request<MpiSpace>¶

Request specialization for the MpiSpace communication space. Wraps an MPI_Request handle.

using communication_space = MpiSpace ¶

using request_type = MpiSpace::request_type ¶

using rank_type = MpiSpace::rank_type ¶

Request() noexcept¶: Constructs an inactive request with MPI_REQUEST_NULL.

explicit Request(request_type &&request) noexcept¶

Adopts a non-persistent MPI request and sets the source to MPI_REQUEST_NULL. A named raw handle must be passed with std::move(raw_request). Persistent requests are unsupported; use Channel<MpiSpace> instead. Persistence cannot be inspected portably, so non-persistence is a caller precondition.

Adoption transfers only the request handle. The caller must keep every associated buffer and other operation resource valid until completion, including buffers backed by managed Views: adoption supplies no View metadata.

Parameters:: request – The non-persistent MPI_Request whose ownership is transferred.

auto request() noexcept -> request_type&¶

auto request() const noexcept -> const request_type&¶

Returns:: A reference to the underlying MPI_Request object.

auto request_ptr() noexcept -> request_type*¶

auto request_ptr() const noexcept -> const request_type*¶

Returns:: A pointer to the underlying MPI_Request object.

Note

Both wait_all and wait_any temporarily transfer non-persistent MPI handles into contiguous storage for MPI_Waitall and MPI_Waitany, respectively, incurring an allocation. wait_any restores non-selected handles before callbacks run. MPI statuses are ignored; completion errors are fatal.

NCCL specialization¶

template<> class Request<Experimental::NcclSpace>¶

Request specialization for the Experimental::NcclSpace communication space. Wraps a cudaEvent_t handle to track the completion of CUDA stream operations.

using communication_space = Experimental::NcclSpace ¶

using request_type = Experimental::NcclSpace::request_type ¶

using rank_type = Experimental::NcclSpace::rank_type ¶

explicit Request() noexcept¶: Constructs an empty Request with a null event handle.

~Request() noexcept¶: Requires a null event and no pending callbacks. Violations trigger a fatal diagnostic; no CUDA work is waited for and no active event is destroyed.

auto capture_stream_state(cudaStream_t stream) noexcept -> void¶

Implementation hook; applications and completion callbacks must not call it. Records a CUDA event on stream for completion tracking. An already-owned event is a fatal precondition violation; complete the request explicitly before reuse. Callbacks registered during operation setup may precede this capture.

The backend uses one CUDA device per process. The calling thread must have the stream’s owning device current. Event creation and recording failures are fatal. Successful wait or test destroys and nulls the event before executing completion callbacks.

Parameters:: stream – The CUDA stream whose state to capture.

auto request() noexcept -> request_type&¶

auto request() const noexcept -> const request_type&¶

Returns:: A reference to the underlying cudaEvent_t object.

auto request_ptr() noexcept -> request_type*¶

auto request_ptr() const noexcept -> const request_type*¶

Returns:: A pointer to the underlying cudaEvent_t object.

Note

Both wait_all and wait_any poll only active events and destroy and null each selected event exactly once. wait_all retires every active event before running callbacks. Active polling increases CPU utilization.

Communication Primitives¶

Point-to-point¶

Send¶

Warning

This is not a blocking operation despite being named like MPI_Send.

template<KokkosView SendView, KokkosExecutionSpace ExecSpace = Kokkos::DefaultExecutionSpace, CommunicationSpace CommSpace = DefaultCommunicationSpace> auto send(Communicator<CommSpace, ExecSpace> &h, SendView &sv, int dest) -> Request<CommSpace>¶

Initiates a non-blocking send operation.

Template Parameters:

SendView – The type of the Kokkos view to send.
ExecSpace – The execution space to use. Defaults to Kokkos::DefaultExecutionSpace.
CommSpace – The communication backend to use. Defaults to DefaultCommunicationSpace.

Parameters:

h – A handle to the execution space and transport mechanism.
sv – The Kokkos view to send.
dest – The destination rank.

Returns:

A request object of type Request<CommSpace> representing the non-blocking send operation.

Example usage:

#include <Kokkos_Core.hpp>
#include <KokkosComm/KokkosComm.hpp>

// Create an execution space instance
auto exec = Kokkos::DefaultExecutionSpace();
// Create a communicator
auto comm = KokkosComm::Communicator<>::duplicate_from_raw(raw_comm_handle, exec).value();

// Create a Kokkos view
Kokkos::View<double*> data("send_data", 100);

// Fill the view with some data
Kokkos::parallel_for(
    "fill_data", Kokkos::RangePolicy(exec, 0, 100), KOKKOS_LAMBDA(int i) { data(i) = static_cast<double>(i); }
);
exec.fence();

// Destination rank
int dst_rank = 1;

// Initiate a non-blocking send with a handle
auto req1 = KokkosComm::send(comm, data, dst_rank);

// Simulate a blocking send by waiting immediately
KokkosComm::send(comm, data, dst_rank).wait();

// Wait for a request to complete
KokkosComm::wait(req1);

Receive¶

Warning

This is not a blocking operation despite being named like MPI_Recv.

template<KokkosView RecvView, KokkosExecutionSpace ExecSpace = Kokkos::DefaultExecutionSpace, CommunicationSpace CommSpace = DefaultCommunicationSpace> auto recv(Communicator<CommSpace, ExecSpace> &h, RecvView &sv, int dest) -> Request<CommSpace>¶

Initiates a non-blocking receive operation.

Template Parameters:

RecvView – The type of the Kokkos view for receiving data.
ExecSpace – The execution space where the operation will be performed. Defaults to Kokkos::DefaultExecutionSpace.
CommSpace – The communication backend to use. Defaults to DefaultCommunicationSpace.

Parameters:

h – A handle to the execution space and transport mechanism.
rv – The Kokkos view where the received data will be stored.
src – The source rank from which to receive data.

Returns:

A request object of type Request<CommSpace> representing the non-blocking receive operation.

This function initiates a non-blocking receive operation using the specified execution space and transport mechanism. The data will be received into the provided view from the specified source rank and message tag. The function returns a request object that can be used to check the status of the receive operation or to wait for its completion.

Example usage:

#include <Kokkos_Core.hpp>
#include <KokkosComm/KokkosComm.hpp>

// Create an execution space instance
auto exec = Kokkos::DefaultExecutionSpace();
// Create a communicator
auto comm = KokkosComm::Communicator<>::duplicate_from_raw(raw_comm_handle, exec).value();

// Allocate a view to receive the data
Kokkos::View<double*> data("recv_view", 100);

// Source rank
int src_rank = 1;

// Initiate a non-blocking receive with a handle
auto req1 = KokkosComm::recv(comm, data, src_rank);

// Simulate a blocking receive by waiting immediately
KokkosComm::recv(comm, data, src_rank).wait();

// Wait for a requests to complete
KokkosComm::wait(req1);

Collectives¶

Important

Collective operations act element-wise on the input Views. Multi-dimensional Views are treated as a logically flattened sequence of values, and the reduction is applied over that sequence. All participating Views must have identical extents; mismatched shapes result in undefined behavior.

The reduction operator must be associative, but ordering of partial combinations is not guaranteed, and the operation is not required to be commutative.

Broadcast¶

template<KokkosView View, KokkosExecutionSpace ExecSpace = Kokkos::DefaultExecutionSpace, CommunicationSpace CommSpace = DefaultCommunicationSpace> auto KokkosComm::Experimental::broadcast(Communicator<CommSpace, ExecSpace> &h, View v, int root) -> Request<CommSpace>¶

Broadcasts the v view from the root rank to all ranks’ v view.

Template Parameters:

View – The type of the Kokkos view to broadcast.
ExecSpace – The execution space to use. Defaults to Kokkos::DefaultExecutionSpace.
CommSpace – The communication backend to use. Defaults to DefaultCommunicationSpace.

Parameters:

h – A handle to the execution space and transport mechanism.
v – The Kokkos view to broadcast. Must be valid on all ranks; its data on root is broadcast to all ranks.
root – The rank that holds the source data.

Returns:

A request object of type Request<CommSpace> representing the non-blocking broadcast operation.

Allgather¶

template<KokkosView SendView, KokkosView RecvView, KokkosExecutionSpace ExecSpace = Kokkos::DefaultExecutionSpace, CommunicationSpace CommSpace = DefaultCommunicationSpace> auto KokkosComm::Experimental::allgather(Communicator<CommSpace, ExecSpace> &h, const SendView sv, RecvView rv) -> Request<CommSpace>¶

Gathers the sv view from each rank into the rv view on all ranks.

Data received from rank i is placed at offset i * KokkosComm::span(sv) in rv. The rv view must be large enough to hold h.size() * KokkosComm::span(sv) elements.

Template Parameters:

SendView – The type of the Kokkos view to send from each rank.
RecvView – The type of the Kokkos view to receive the gathered data into.
ExecSpace – The execution space to use. Defaults to Kokkos::DefaultExecutionSpace.
CommSpace – The communication backend to use. Defaults to DefaultCommunicationSpace.

Parameters:

h – A handle to the execution space and transport mechanism.
sv – The Kokkos view to send from this rank.
rv – The Kokkos view where the gathered data from all ranks is stored.

Returns:

A request object of type Request<CommSpace> representing the non-blocking all-gather operation.

Alltoall¶

template<KokkosView SendView, KokkosView RecvView, KokkosExecutionSpace ExecSpace = Kokkos::DefaultExecutionSpace, CommunicationSpace CommSpace = DefaultCommunicationSpace> auto KokkosComm::Experimental::alltoall(Communicator<CommSpace, ExecSpace> &h, const SendView sv, RecvView rv, int count) -> Request<CommSpace>¶

Sends count elements to every other rank and receives count elements from every other rank.

Data to send to destination rank i is taken from sv at offset i * count. Data received from source rank j is placed into rv at offset j * count. Both sv and rv must span at least h.size() * count elements.

Template Parameters:

SendView – The type of the Kokkos view holding the data to send.
RecvView – The type of the Kokkos view where the received data will be stored.
ExecSpace – The execution space to use. Defaults to Kokkos::DefaultExecutionSpace.
CommSpace – The communication backend to use. Defaults to DefaultCommunicationSpace.

Parameters:

h – A handle to the execution space and transport mechanism.
sv – The Kokkos view containing the data to send to all ranks.
rv – The Kokkos view where the received data from all ranks will be stored.
count – The number of elements sent to, and received from, each rank.

Returns:

A request object of type Request<CommSpace> representing the non-blocking all-to-all operation.

Allreduce¶

template<KokkosView SendView, KokkosView RecvView, ReductionOperator RedOp, KokkosExecutionSpace ExecSpace = Kokkos::DefaultExecutionSpace, CommunicationSpace CommSpace = DefaultCommunicationSpace> auto KokkosComm::Experimental::allreduce(Communicator<CommSpace, ExecSpace> &h, const SendView sv, RecvView rv, RedOp) -> Request<CommSpace>¶

Reduces the sv view using RedOp and stores the result in all ranks’ rv view.

Template Parameters:

SendView – The type of the Kokkos view to reduce.
RecvView – The type of the Kokkos view to store the reduced result into.
RedOp – A type satisfying the ReductionOperator concept.
ExecSpace – The execution space to use. Defaults to Kokkos::DefaultExecutionSpace.
CommSpace – The communication backend to use. Defaults to DefaultCommunicationSpace.

Parameters:

h – A handle to the execution space and transport mechanism.
sv – The Kokkos view containing the input data to reduce.
rv – The Kokkos view where the result of the reduction is stored on all ranks.

Returns:

A request object of type Request<CommSpace> representing the non-blocking all-reduce operation.

Reduce¶

template<KokkosView SendView, KokkosView RecvView, ReductionOperator RedOp, KokkosExecutionSpace ExecSpace = Kokkos::DefaultExecutionSpace, CommunicationSpace CommSpace = DefaultCommunicationSpace> auto KokkosComm::Experimental::reduce(Communicator<CommSpace, ExecSpace> &h, const SendView sv, RecvView rv, int root, RedOp) -> Request<CommSpace>¶

Reduces the sv view using RedOp and stores the result in the root rank’s rv view.

The rv view is only used on the root rank and is ignored on all other ranks.

Template Parameters:

SendView – The type of the Kokkos view to reduce.
RecvView – The type of the Kokkos view to store the reduced result into on root.
RedOp – A type satisfying the ReductionOperator concept.
ExecSpace – The execution space to use. Defaults to Kokkos::DefaultExecutionSpace.
CommSpace – The communication backend to use. Defaults to DefaultCommunicationSpace.

Parameters:

h – A handle to the execution space and transport mechanism.
sv – The Kokkos view containing the input data to reduce.
rv – The Kokkos view where the result of the reduction is stored on root.
root – The rank that receives the reduced result.

Returns:

A request object of type Request<CommSpace> representing the non-blocking reduce operation.

Utilities¶

Warning

Non-system data types (i.e. the data types not natively supported by the communication space) are not convertible. This notably includes user-defined types.

template<CommunicationSpace C, typename T> auto datatype() -> C::datatype_type¶

Converts a type T to its communication space C equivalent representation.

When C is:

MpiSpace, returns the corresponding MPI_Datatype type.
NcclSpace, returns the corresponding ncclDataType_t type.

Template Parameters:

C – The target communication space backend to use for data type conversion.
T – The C++-native data type to convert from.

Returns:

The communication space representation of the C++-native data type.

template<CommunicationSpace C, KokkosView V> auto datatype_for([[maybe_unused]] const V &view) -> C::datatype_type¶

Template Parameters:

C – The target communication space backend to use for data type conversion.
V – A Kokkos View type.

Parameters:

view – The Kokkos View to convert the value type from.

Returns:

The communication space representation of the Kokkos View value type.

template<CommunicationSpace C, KokkosView V> auto datatype_for([[maybe_unused]] C &&comm, [[maybe_unused]] const V &view) -> C::datatype_type¶

Template Parameters:

C – The target communication space backend to use for data type conversion.
V – A Kokkos View type.

Parameters:

comm – A communication space object, immediately consumed.
view – The Kokkos View to convert the value type from.

Returns:

The communication space representation of the Kokkos View value type.

template<CommunicationSpace CS, ReductionOperator RO> [[nodiscard]] constexpr auto reduction_op() -> CS::reduction_op_type¶

Converts a Kokkos Comm ReductionOperator type to its communication-space-native equivalent.

When CS is:

MpiSpace, returns the corresponding MPI_Op.
Experimental::NcclSpace, returns the corresponding ncclRedOp_t.

Template Parameters:

CS – The target communication space backend.
RO – A type satisfying the ReductionOperator concept.

Returns:

The communication space representation of the reduction operator.

template<CommunicationSpace CS, ReductionOperator RO> [[nodiscard]] constexpr auto reduction_op_for(RO &&op) -> CS::reduction_op_type¶

Instance-based counterpart of reduction_op().

Template Parameters:

CS – The target communication space backend.
RO – A type satisfying the ReductionOperator concept.

Parameters:

op – An instance of a reduction operator tag.

Returns:

The communication space representation of the reduction operator.