This document is relevant for: Trn2, Trn3
Parallelism#
Design documentation for parallelism strategies. For configuration guidance, see Tensor, data, and expert parallelism in the features guide.
Topic |
Description |
|---|---|
Data parallelism overview |
|
Expert parallelism for MoE |
|
Tensor parallelism overview |
|
Independent TP/DP for vision encoders |
This document is relevant for: Trn2, Trn3