Neuron Glossary
Contents
This document is relevant for: Inf1
, Inf2
, Trn1
, Trn1n
Neuron Glossary#
Table of contents
Terms#
Neuron Devices (Accelerated Machine Learning chips)#
Term |
Description |
---|---|
|
AWS first generation accelerated machine learning chip supporting inference only |
|
AWS second generation accelerated machine learning chip supporting training and inference |
|
Accelerated machine learning chip (e.g. Inferentia or Trainium) |
Neuron powered Instances#
Term |
Description |
---|---|
|
Inferentia powered accelerated compute EC2 instance |
|
Trainium powered accelerated compute EC2 instance |
NeuronCore terms#
Term |
Description |
---|---|
|
The machine learning compute cores within Inferentia/Trainium |
|
Neuron Core withing Inferentia |
|
Neuron Core withing Trainium |
|
2D systolic array (within the NeuronCore), used for matrix computations |
|
A scalar-engine within each NeuronCore, which can accelerate element-wise operations (e.g. GELU, ReLU, reciprocal, etc) |
|
A vector-engine with each NeuronCore, which can accelerate spatial operations (e.g. layerNorm, TopK, pooling, etc) |
|
Embedded General Purpose SIMD cores, within each NeuronCore, to accelerate custom-operators |
|
The SP engine, which is integrated inside NeuronCore. Used for synchronization and DMA triggering. |
|
Dedicated engine for collective communication, allows for overlapping computation and communication |
|
Interconnect between NeuronCores |
|
Interconnect between NeuronCores in Inferentia device |
|
Interconnect between NeuronCores in Trainium device |
Abbreviations#
Abbreviation |
Description |
---|---|
|
Neuron Core |
|
Neuron Core |
|
Neuron Device |
|
Neuron Device |
|
Tensor Engine |
|
Scalar Engine |
|
Vector Engine |
|
Sync Engine |
|
Collective Communication Engine |
|
Float32 |
|
TensorFloat32 |
|
Float16 |
|
Bfloat16 |
|
Configurable Float8 |
|
Round Nearest Even |
|
Stochastic Rounding |
|
Custom Operators |
|
Neuron Runtime |
|
Data Parallel |
|
Data Parallel degree |
|
Tensor Parallel |
|
Tensor Parallel degree |
|
Pipeline Parallel |
|
Pipeline Parallel degree |
This document is relevant for: Inf1
, Inf2
, Trn1
, Trn1n