IntelĀ® Machine Learning Scaling Library  2018
A library providing an efficient implementation of communication patterns used in deep learning.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros
Public Member Functions | List of all members
MLSL::Distribution Class Reference

A class to hold information about the parallelism scheme being used. More...

#include <mlsl.hpp>

Public Member Functions

size_t GetProcessIdx (GroupType groupType)
 
size_t GetProcessCount (GroupType groupType)
 
CommReqBcast (void *buffer, size_t count, DataType dataType, size_t rootIdx, GroupType groupType)
 
CommReqReduce (void *sendBuffer, void *recvBuffer, size_t count, DataType dataType, ReductionType redType, size_t rootIdx, GroupType groupType)
 
CommReqAllReduce (void *sendBuffer, void *recvBuffer, size_t count, DataType dataType, ReductionType redType, GroupType groupType)
 
CommReqAlltoAll (void *sendBuffer, size_t sendCount, void *recvBuffer, DataType dataType, GroupType groupType)
 
CommReqAlltoAllv (void *sendBuffer, size_t *sendCounts, size_t *sendOffsets, void *recvBuffer, size_t *recvCounts, size_t *recvOffsets, DataType dataType, GroupType groupType)
 
CommReqGather (void *sendBuffer, size_t sendCount, void *recvBuffer, DataType dataType, size_t rootIdx, GroupType groupType)
 
CommReqAllGather (void *sendBuffer, size_t sendCount, void *recvBuffer, DataType dataType, GroupType groupType)
 
CommReqScatter (void *sendBuffer, void *recvBuffer, size_t recvCount, DataType dataType, size_t rootIdx, GroupType groupType)
 
CommReqReduceScatter (void *sendBuffer, void *recvBuffer, size_t recvCount, DataType dataType, ReductionType redType, GroupType groupType)
 
void Barrier (GroupType groupType)
 

Detailed Description

A class to hold information about the parallelism scheme being used.

Holds information about the parallelism scheme currently in use (data/model/hybrid parallelism) and provides the low-level API for collectives over data/model/global groups of processes.
These collectives can be used, for example, to broadcast initial parameters, and to collect the snapshot in model/hybrid parallelism.

Member Function Documentation

CommReq* MLSL::Distribution::AllGather ( void *  sendBuffer,
size_t  sendCount,
void *  recvBuffer,
DataType  dataType,
GroupType  groupType 
)

Gathers values from all processes of the group and distribute the combined values to all processes of the group.

Parameters
sendBufferthe address of the send buffer
sendCountthe number of elements in the send buffer
recvBufferthe address of the receive buffer, the receive buffer should be large enough to hold messages from all processes of the group
dataTypethe data type of buffers' elements
groupTypethe group type
Returns
A communication request.
CommReq* MLSL::Distribution::AllReduce ( void *  sendBuffer,
void *  recvBuffer,
size_t  count,
DataType  dataType,
ReductionType  redType,
GroupType  groupType 
)

Reduces values from all processes of the group and distributes the result back to all the processes.

Parameters
sendBufferthe address of the send buffer
recvBufferthe address of the receive buffer, meaningful only at the root process
countthe number of elements in send/receive buffers
dataTypethe data type of buffer's elements
redTypethe reduction operation type
groupTypethe group type
Returns
A communication request.
CommReq* MLSL::Distribution::AlltoAll ( void *  sendBuffer,
size_t  sendCount,
void *  recvBuffer,
DataType  dataType,
GroupType  groupType 
)

Sends values from all processes of the group to all processes of the group.

Parameters
sendBufferthe address of the send buffer
sendCountthe number of elements in the send group buffer, the send buffer should be large enough to hold messages for all processes of the group (at least (sendCount * GetProcessCount(groupType)) elements)
recvBufferthe address of the receive buffer, the receive buffer should be large enough to hold messages from all processes of the group
dataTypethe data type of buffers' elements
groupTypethe group type
Returns
A communication request.
CommReq* MLSL::Distribution::AlltoAllv ( void *  sendBuffer,
size_t *  sendCounts,
size_t *  sendOffsets,
void *  recvBuffer,
size_t *  recvCounts,
size_t *  recvOffsets,
DataType  dataType,
GroupType  groupType 
)

Gathers data from and scatters data to all members of a group.

Parameters
sendBufferthe address of the send buffer
sendCountsthe number of data elements that the process sends in the buffer that is specified in the sendbuf parameter
sendOffsetsthe location, relative to the sendbuf parameter, of the data for each communicator process
recvBufferthe address of receive buffer the receive buffer should be large enough to hold messages from all processes of the group
recvCountsthe number of data elements from each communicator process in the receive buffer
recvOffsetsthe location, relative to the recvbuf parameter, of the data from each communicator process
dataTypethe data type of buffers' elements
groupTypethe group type
Returns
A communication request.
void MLSL::Distribution::Barrier ( GroupType  groupType)

Sets a barrier for all the processes within a group.

Parameters
groupTypethe group type
CommReq* MLSL::Distribution::Bcast ( void *  buffer,
size_t  count,
DataType  dataType,
size_t  rootIdx,
GroupType  groupType 
)

Broadcasts a values from the root process to all other processes of the group.

Parameters
bufferthe starting address of the buffer
countthe number of elements in the buffer
dataTypethe data type of the buffer elements
rootIdxthe index of the root process within a group
groupTypegroup type
Returns
A communication request.
CommReq* MLSL::Distribution::Gather ( void *  sendBuffer,
size_t  sendCount,
void *  recvBuffer,
DataType  dataType,
size_t  rootIdx,
GroupType  groupType 
)

Gathers values from all processes of the group to the root process.

Parameters
sendBufferthe address of the send buffer
sendCountthe number of elements in the send buffer
recvBufferthe address of the receive buffer, meaningful only at the root process, the receive buffer should be large enough to hold messages from all processes of the group
dataTypethe data type of buffers' elements
rootIdxthe index of the root process within a group
groupTypethe group type
Returns
A communication request.
size_t MLSL::Distribution::GetProcessCount ( GroupType  groupType)

Returns the number of processes within a group.

Parameters
groupTypethe group type
Returns
The number of processes.
size_t MLSL::Distribution::GetProcessIdx ( GroupType  groupType)

Returns the process index within a group.

Parameters
groupTypethe group type
Returns
The process index.
CommReq* MLSL::Distribution::Reduce ( void *  sendBuffer,
void *  recvBuffer,
size_t  count,
DataType  dataType,
ReductionType  redType,
size_t  rootIdx,
GroupType  groupType 
)

Reduces values on all processes within a group.

Parameters
sendBufferthe address of the send buffer
recvBufferthe address of the receive buffer, meaningful only at the root process
countthe number of elements in send/receive buffers
dataTypedata the type of buffers' elements
redTypethe reduction operation type
rootIdxthe index of the root process within a group
groupTypethe group type
Returns
A communication request.
CommReq* MLSL::Distribution::ReduceScatter ( void *  sendBuffer,
void *  recvBuffer,
size_t  recvCount,
DataType  dataType,
ReductionType  redType,
GroupType  groupType 
)

Combines values from all processes of the group and scatters the results in blocks back to all the processes.

Parameters
sendBufferthe address of the send buffer
recvBufferthe address of the receive buffer
recvCountthe number of elements in the receive buffer (the number of elements in a block)
dataTypethe data type of buffers' elements
redTypethe reduction operation type
groupTypethe group type
Returns
A communication request.
CommReq* MLSL::Distribution::Scatter ( void *  sendBuffer,
void *  recvBuffer,
size_t  recvCount,
DataType  dataType,
size_t  rootIdx,
GroupType  groupType 
)

Sends values from the root process to all processes of the group.

Parameters
sendBufferthe address of the send buffer, meaningful only at the root process, the send buffer should be large enough to hold messages for all processes of the group
recvBufferthe address of the receive buffer
recvCountthe number of elements in the receive buffer
dataTypethe data type of buffers' elements
rootIdxthe index of the root process within a group
groupTypethe group type
Returns
A communication request.

The documentation for this class was generated from the following file: