Low-level MPI interfaces

MPI API Support

MPI routines	KokkosComm::mpi:: namespace	Kokkos::View support
MPI_Send	send or send<CommMode::Standard>	✓
MPI_Rsend	send<CommMode::Ready>	✓
MPI_Ssend	send<CommMode::Synchronous>	✓
MPI_Isend	isend or isend<CommMode::Standard>	✓
MPI_Irsend	isend<CommMode::Ready>	✓
MPI_Issend	isend<CommMode::Synchronous>	✓
MPI_Recv	recv	✓
MPI_Allgather	allgather	✓
MPI_Allgather (in-place)	allgather	✓
MPI_Reduce	reduce	✓
MPI_Barrier	barrier	✓

Point-to-point

template<KokkosView SendView>
auto send(const SendView &sv, int dest, int tag, MPI_Comm comm) -> void

Initiates a blocking send operation.

Template Parameters:

SendView – The type of the view to be sent.

Parameters:

• sv – The view to be sent.

• dest – The destination rank.

• tag – The message tag.

• comm – The MPI communicator.

template<CommMode SendMode = CommMode::Default, KokkosExecutionSpace ExecSpace, KokkosView SendView>
auto send(const ExecSpace &space, const SendView &sv, int dest, int tag, MPI_Comm comm) -> void

Initiates a blocking send operation with a specified execution space and communication mode.

Template Parameters:

• SendMode – The communication mode (default is CommMode::Default).

• ExecSpace – The execution space.

• SendView – The type of the view to be sent.

Parameters:

• space – The execution space.

• sv – The view to be sent.

• dest – The destination rank.

• tag – The message tag.

• comm – The MPI communicator.

template<CommMode SendMode, KokkosExecutionSpace ExecSpace, KokkosView SendView>
auto isend(Handle<ExecSpace, Mpi> &h, const SendView &sv, int dest, int tag) -> Req<Mpi>

Initiates a non-blocking send operation.

Template Parameters:

• SendMode – The communication mode.

• ExecSpace – The execution space.

• SendView – The type of the view to be sent.

Parameters:

• h – The handle for the execution space and MPI.

• sv – The view to be sent.

• dest – The destination rank.

• tag – The message tag.

Returns:

A request object for the non-blocking send operation.

template<KokkosView RecvView>
auto recv(const RecvView &rv, int src, int tag, MPI_Comm comm, MPI_Status *status) -> void

Initiates a blocking receive operation.

Template Parameters:

RecvView – The type of the view to be received.

Parameters:

• rv – The view to be received.

• src – The source rank.

• tag – The message tag.

• comm – The MPI communicator.

• status – The MPI status object for the blocking receive operation.

template<KokkosExecutionSpace ExecSpace, KokkosView RecvView>
auto recv(const ExecSpace &space, RecvView &rv, int src, int tag, MPI_Comm comm) -> void

Initiates a blocking receive operation with a specified execution space.

Template Parameters:

• ExecSpace – The execution space.

• RecvView – The type of the view to be received.

Parameters:

• space – The execution space.

• rv – The view to be received.

• src – The source rank.

• tag – The message tag.

• comm – The MPI communicator.

template<KokkosView RecvView>
auto irecv(const RecvView &rv, int src, int tag, MPI_Comm comm, MPI_Request &req) -> void

Initiates a non-blocking receive operation.

Template Parameters:

RecvView – The type of the view to be received.

Parameters:

• rv – The view to be received.

• src – The source rank.

• tag – The message tag.

• comm – The MPI communicator.

• req – The MPI request object for the non-blocking receive operation.

Throws:

std::runtime_error – If the view is not contiguous.

Collectives

template<KokkosView SendView, KokkosView RecvView>
auto allgather(const SendView &sv, const RecvView &rv, MPI_Comm comm) -> void

Performs an allgather operation, gathering data from all processes and distributing it to all processes.

Template Parameters:

• SendView – The type of the view to be sent.

• RecvView – The type of the view to be received.

Parameters:

• sv – The view to be sent.

• rv – The view to be received.

• comm – The MPI communicator.

template<KokkosView RecvView>
auto allgather(const RecvView &rv, MPI_Comm comm) -> void

Performs an in-place allgather operation, gathering data from all processes and distributing it to all processes.

Template Parameters:

RecvView – The type of the view to be received.

Parameters:

• rv – The view to be received.

• comm – The MPI communicator.

template<KokkosExecutionSpace ExecSpace, KokkosView SendView, KokkosView RecvView>
auto allgather(const ExecSpace &space, const SendView &sv, const RecvView &rv, MPI_Comm comm) -> void

Performs an allgather operation with a specified execution space, gathering data from all processes and distributing it to all processes.

Template Parameters:

• ExecSpace – The execution space.

• SendView – The type of the view to be sent.

• RecvView – The type of the view to be received.

Parameters:

• space – The execution space.

• sv – The view to be sent.

• rv – The view to be received.

• comm – The MPI communicator.

template<KokkosView SendView, KokkosView RecvView>
auto reduce(const SendView &sv, const RecvView &rv, MPI_Op op, int root, MPI_Comm comm) -> void

Performs a reduction operation, combining data from all processes and distributing the result to the root process.

Template Parameters:

• SendView – The type of the view to be sent.

• RecvView – The type of the view to be received.

Parameters:

• sv – The view to be sent.

• rv – The view to be received.

• op – The MPI operation to be applied.

• root – The rank of the root process.

• comm – The MPI communicator.

template<KokkosExecutionSpace ExecSpace, KokkosView SendView, KokkosView RecvView>
auto reduce(const ExecSpace &space, const SendView &sv, const RecvView &rv, MPI_Op op, int root, MPI_Comm comm) -> void

Performs a reduction operation with a specified execution space, combining data from all processes and distributing the result to the root process.

Template Parameters:

• ExecSpace – The execution space.

• SendView – The type of the view to be sent.

• RecvView – The type of the view to be received.

Parameters:

• space – The execution space.

• sv – The view to be sent.

• rv – The view to be received.

• op – The MPI operation to be applied.

• root – The rank of the root process.

• comm – The MPI communicator.

inline auto barrier(MPI_Comm comm) -> void

Blocks until all processes in the communicator have reached this routine.

Parameters:

comm – The MPI communicator.

Related Types

enum class CommMode

A scoped enum to specify the mode of an operation. Buffered mode is not supported.

enumerator Standard

The MPI implementation decides whether outgoing messages will be buffered. Send operations can be started whether or not a matching receive has been started. They may complete before a matching receive is started. Standard mode is non-local: successful completion of the send operation may depend on the occurrence of a matching receive.

enumerator Ready

Send operations may be started only if the matching receive is already started.

enumerator Synchronous

Synchronous mode: Send operations complete successfully only if a matching receive is started, and the receive operation has started to receive the message sent.

enumerator Default

Default mode is an alias for Standard mode, but lets users override the behavior of operations at compile-time using the KOKKOSCOMM_FORCE_SYNCHRONOUS_MODE pre-processor define. This forces Synchronous mode for all “default-mode” operations, which can be useful for debugging purposes, e.g. for asserting that the communication scheme is correct.