vt::collective::CollectiveScope struct
#include <src/vt/collective/collective_scope.h>

A distinct scope for enqueuing ordered collective operations.

Each collective scope builds an individual sequence of collective operations that get orchestrated across nodes using a consensus algorithm so all nodes agree on a particular operation to execute in the sequence.

An example use case is running a blocking MPI collective (like a * MPI_Allreduce) inside VT handlers without causing progress to halt due to asynchrony in VT.

Constructors, destructors, conversion operators

CollectiveScope(CollectiveScope&& other)
CollectiveScope(CollectiveScope const&) deleted
~CollectiveScope()

Public functions

auto operator=(CollectiveScope&&) -> CollectiveScope& deleted
auto operator=(CollectiveScope const&) -> CollectiveScope& deleted
auto mpiCollectiveAsync(ActionType action) -> TagType
Enqueue a lambda with an embedded closed set of MPI operations (including collectives) to execute in the future. Returns immediately, enqueuing the action for the future.
auto isCollectiveDone(TagType tag) -> bool
Query whether an enqueued MPI operation set is complete.
void waitCollective(TagType tag)
Wait on an MPI operation set to complete.
void mpiCollectiveWait(ActionType action)
Enqueue a lambda with an embedded closed set of MPI operations (including collectives). Spin in the VT scheduler until it terminates.
auto isUserTag() const -> bool
Get whether this scope is a user-tagged scope.
auto getScopeBits() const -> TagType
Get the scope tag bits.
template<typename Serializer>
void serialize(Serializer& s)

Function documentation

TagType vt::collective::CollectiveScope::mpiCollectiveAsync(ActionType action)

Enqueue a lambda with an embedded closed set of MPI operations (including collectives) to execute in the future. Returns immediately, enqueuing the action for the future.

Parameters
action in the action containing a closed set of MPI operations
Returns tag representing the operation set

The set of operations specified in the lambda must be closed, meaning that MPI requests must not escape the lambda. After the lambda finishes, the set of MPI collective calls should be complete.

Any buffers captured in the lambda to use with the MPI operations are in use until isCollectiveDone returns true or waitCollective returns on the returned TagType

The operation is counted as activity in the active termination detection epoch

bool vt::collective::CollectiveScope::isCollectiveDone(TagType tag)

Query whether an enqueued MPI operation set is complete.

Parameters
tag in MPI operation set identifier
Returns whether it has finished or not

void vt::collective::CollectiveScope::waitCollective(TagType tag)

Wait on an MPI operation set to complete.

Parameters
tag in MPI collective set identifier

void vt::collective::CollectiveScope::mpiCollectiveWait(ActionType action)

Enqueue a lambda with an embedded closed set of MPI operations (including collectives). Spin in the VT scheduler until it terminates.

Parameters
action in the action containing a set of MPI operations

bool vt::collective::CollectiveScope::isUserTag() const

Get whether this scope is a user-tagged scope.

Returns whether this scope is a user tag

TagType vt::collective::CollectiveScope::getScopeBits() const

Get the scope tag bits.

Returns the scope bits