Intel® Machine Learning Scaling Library  2018
A library providing an efficient implementation of communication patterns used in deep learning.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros
Classes | Typedefs | Enumerations
MLSL Namespace Reference

The namespace containing all Intel MLSL classes and types. More...

Classes

struct  QuantParams
 A struct to hold quantization parameters. More...
 
class  CommBlockInfo
 A class to hold block information for activations packing/unpacking. More...
 
class  Activation
 A wrapper class for operation input and output activations. More...
 
class  ParameterSet
 A wrapper class for operation parameters. More...
 
class  Distribution
 A class to hold information about the parallelism scheme being used. More...
 
class  OperationRegInfo
 A class to hold Operation registration information. More...
 
class  Operation
 A class to hold information about learnable parameters (parameter sets) and activations corresponding to a certain operation of the computational graph. More...
 
class  Statistics
 A class to measure and store performance statistics of communication among processes that perform computation in the computational graph. More...
 
class  Session
 A class to represent a collection of Operation objects with the same global mini-batch size. More...
 
class  Environment
 A singleton object that holds global Intel MLSL functions. More...
 

Typedefs

typedef int CommReq
 

Enumerations

enum  DataType { DT_FLOAT = 0, DT_DOUBLE = 1, DT_BYTE = 2 }
 
enum  PhaseType { PT_TRAIN = 0, PT_TEST = 1 }
 
enum  GroupType { GT_DATA = 0, GT_MODEL = 1, GT_GLOBAL = 2 }
 Groups of processes supported by Intel MLSL. More...
 
enum  ReductionType { RT_SUM = 0, RT_MIN = 1, RT_MAX = 2 }
 
enum  OpType {
  OT_CC = 0, OT_BIAS = 1, OT_ACT = 2, OT_POOL = 3,
  OT_SPLIT = 4, OT_CONCAT = 5, OT_BCAST = 6, OT_REDUCE = 7,
  OT_DATA = 8, OT_EVAL = 9
}
 Compute operation types. More...
 
enum  CompressionType { CT_NONE = 0, CT_QUANTIZATION = 1 }
 

Detailed Description

The namespace containing all Intel MLSL classes and types.

Typedef Documentation

typedef int MLSL::CommReq

Type definition for a communication request type.

Enumeration Type Documentation

Compression type.

Enumerator
CT_NONE 

Do not use compression

CT_QUANTIZATION 

Use quantization

The data type used for representing input/output activations and parameters (weights or biases).

Enumerator
DT_FLOAT 
DT_DOUBLE 
DT_BYTE 

Groups of processes supported by Intel MLSL.

A group represents a subset of processes handling the model.
The data group contains processes working on the same part of parameters for different input batches (i.e. doing data parallelism) and communicating to exchange gradients with respect to parameters.
The model group contains processes working on different parts of parameters for the same input batch (i.e. doing model parallelism) and communicating to exchange activations and gradients with respect to activations.
The global group contains all Intel MLSL processes.

Enumerator
GT_DATA 

data group

GT_MODEL 

model group

GT_GLOBAL 

global group

Compute operation types.

Each operation type defines specific relationship between the input and output activations and associated parameters (weights or biases).
(abbreviations: IA – input activation, OA – output activation)

Enumerator
OT_CC 

Cross-correlation - IA and OA independent and has parameters

OT_BIAS 

Bias - the same IA and OA (dependent) but has parameters

OT_ACT 

Activation operation - Same IA and OA and no parameters

OT_POOL 

Activation operation - Same IA and OA and no parameters

OT_SPLIT 

OA depends on IA (=OA1+OA2...) and no parameters

OT_CONCAT 

OA depends on IA1+IA2+... and no parameters

OT_BCAST 

OA1=IA, OA2=IA, ... and no parameters

OT_REDUCE 

OA=IA1+IA2+... and no parameters

OT_DATA 

Only OA, no IA

OT_EVAL 

Only IA, no OA

Phases supported by Intel MLSL.

Enumerator
PT_TRAIN 

training phase

PT_TEST 

testing phase

Reduction operations for Distribution::Reduce(), Distribution::AllReduce() and Distribution::ReduceScatter().

Enumerator
RT_SUM 
RT_MIN 
RT_MAX