MPI

MPI API Support

MPI	KokkosComm::mpi::	Kokkos::View
MPI_Send	send or send<CommMode::Standard>	✓
MPI_Rsend	send<CommMode::Ready>	✓
MPI_Recv	recv	✓
MPI_Ssend	send<CommMode::Synchronous>	✓
MPI_Isend	isend or isend<CommMode::Standard>	✓
MPI_Irsend	isend<CommMode::Ready>	✓
MPI_Issend	isend<CommMode::Synchronous>	✓
MPI_Reduce	reduce	✓

Point-to-point

template<CommMode SendMode, KokkosExecutionSpace ExecSpace, KokkosView SendView>
Req<Mpi> isend(Handle<ExecSpace, Mpi> &h, const SendView &sv, int dest, int tag)

Initiates a non-blocking send operation.

Template Parameters:

• SendMode – The communication mode.

• ExecSpace – The execution space.

• SendView – The type of the view to be sent.

Parameters:

• h – The handle for the execution space and MPI.

• sv – The view to be sent.

• dest – The destination rank.

• tag – The message tag.

Returns:

A request object for the non-blocking send operation.

template<KokkosView RecvView>
void irecv(const RecvView &rv, int src, int tag, MPI_Comm comm, MPI_Request &req)

Initiates a non-blocking receive operation.

Template Parameters:

RecvView – The type of the view to be received.

Parameters:

• rv – The view to be received.

• src – The source rank.

• tag – The message tag.

• comm – The MPI communicator.

• req – The MPI request object for the non-blocking receive operation.

Throws:

std::runtime_error – If the view is not contiguous.

template<KokkosView SendView>
void send(const SendView &sv, int dest, int tag, MPI_Comm comm)

Initiates a blocking send operation.

Template Parameters:

SendView – The type of the view to be sent.

Parameters:

• sv – The view to be sent.

• dest – The destination rank.

• tag – The message tag.

• comm – The MPI communicator.

template<CommMode SendMode = CommMode::Default, KokkosExecutionSpace ExecSpace, KokkosView SendView>
void send(const ExecSpace &space, const SendView &sv, int dest, int tag, MPI_Comm comm)

Initiates a blocking send operation with a specified execution space and communication mode.

Template Parameters:

• SendMode – The communication mode (default is CommMode::Default).

• ExecSpace – The execution space.

• SendView – The type of the view to be sent.

Parameters:

• space – The execution space.

• sv – The view to be sent.

• dest – The destination rank.

• tag – The message tag.

• comm – The MPI communicator.

template<KokkosView RecvView>
void recv(const RecvView &rv, int src, int tag, MPI_Comm comm, MPI_Status *status)

Initiates a blocking receive operation.

Template Parameters:

RecvView – The type of the view to be received.

Parameters:

• rv – The view to be received.

• src – The source rank.

• tag – The message tag.

• comm – The MPI communicator.

• status – The MPI status object for the blocking receive operation.

template<KokkosExecutionSpace ExecSpace, KokkosView RecvView>
void recv(const ExecSpace &space, RecvView &rv, int src, int tag, MPI_Comm comm)

Initiates a blocking receive operation with a specified execution space.

Template Parameters:

• ExecSpace – The execution space.

• RecvView – The type of the view to be received.

Parameters:

• space – The execution space.

• rv – The view to be received.

• src – The source rank.

• tag – The message tag.

• comm – The MPI communicator.

Collective

template<KokkosExecutionSpace ExecSpace, KokkosView SendView, KokkosView RecvView>
void KokkosComm::reduce(const ExecSpace &space, const SendView &sv, const RecvView &rv, MPI_Op op, int root, MPI_Comm comm)

MPI_Reduce wrapper

Parameters:

• space – The execution space to operate in

• sv – The data to send

• rv – The view to receive into

• op – The MPI_Op to use in the reduction

• root – The root rank for the reduction

• comm – the MPI communicator

Template Parameters:

• SendView – A Kokkos::View to send

• RecvView – A Kokkos::View to recv

• ExecSpace – A Kokkos execution space to operate in

template<KokkosView SendView, KokkosView RecvView>
void allgather(const SendView &sv, const RecvView &rv, MPI_Comm comm)

Performs an allgather operation, gathering data from all processes and distributing it to all processes.

Template Parameters:

• SendView – The type of the view to be sent.

• RecvView – The type of the view to be received.

Parameters:

• sv – The view to be sent.

• rv – The view to be received.

• comm – The MPI communicator.

If sv is a rank-0 view, the value from the jth rank will be placed in index j of rv.

template<KokkosView RecvView>
void allgather(const RecvView &rv, MPI_Comm comm)

Performs an in-place allgather operation, gathering data from all processes and distributing it to all processes.

Template Parameters:

RecvView – The type of the view to be received.

Parameters:

• rv – The view to be received.

• comm – The MPI communicator.

template<KokkosExecutionSpace ExecSpace, KokkosView SendView, KokkosView RecvView>
void allgather(const ExecSpace &space, const SendView &sv, const RecvView &rv, MPI_Comm comm)

Performs an allgather operation with a specified execution space, gathering data from all processes and distributing it to all processes.

Template Parameters:

• ExecSpace – The execution space.

• SendView – The type of the view to be sent.

• RecvView – The type of the view to be received.

Parameters:

• space – The execution space.

• sv – The view to be sent.

• rv – The view to be received.

• comm – The MPI communicator.

inline void barrier(MPI_Comm comm)

Blocks until all processes in the communicator have reached this routine.

Parameters:

comm – The MPI communicator.

template<KokkosView SendView, KokkosView RecvView>
void reduce(const SendView &sv, const RecvView &rv, MPI_Op op, int root, MPI_Comm comm)

Performs a reduction operation, combining data from all processes and distributing the result to the root process.

Template Parameters:

• SendView – The type of the view to be sent.

• RecvView – The type of the view to be received.

Parameters:

• sv – The view to be sent.

• rv – The view to be received.

• op – The MPI operation to be applied.

• root – The rank of the root process.

• comm – The MPI communicator.

template<KokkosExecutionSpace ExecSpace, KokkosView SendView, KokkosView RecvView>
void reduce(const ExecSpace &space, const SendView &sv, const RecvView &rv, MPI_Op op, int root, MPI_Comm comm)

Performs a reduction operation with a specified execution space, combining data from all processes and distributing the result to the root process.

Template Parameters:

• ExecSpace – The execution space.

• SendView – The type of the view to be sent.

• RecvView – The type of the view to be received.

Parameters:

• space – The execution space.

• sv – The view to be sent.

• rv – The view to be received.

• op – The MPI operation to be applied.

• root – The rank of the root process.

• comm – The MPI communicator.

Related Types

enum class CommMode

A scoped enum to specify the mode of an operation. Buffered mode is not supported.

enumerator Standard

Standard mode: the MPI implementation decides whether outgoing messages will be buffered. Send operations can be started whether or not a matching receive has been started. They may complete before a matching receive is started. Standard mode is non-local: successful completion of the send operation may depend on the occurrence of a matching receive.

enumerator Ready

Ready mode: Send operations may be started only if the matching receive is already started.

enumerator Synchronous

Synchronous mode: Send operations complete successfully only if a matching receive is started, and the receive operation has started to receive the message sent.

enumerator Default

Default mode is an alias for Standard mode, but lets users override the behavior of operations at compile-time using the KOKKOSCOMM_FORCE_SYNCHRONOUS_MODE pre-processor define. This forces Synchronous mode for all “default-mode” operations, which can be useful for debugging purposes, e.g., for asserting that the communication scheme is correct.