This document is relevant for: Trn2, Trn3

Parallelism

Design documentation for parallelism strategies. For configuration guidance, see Tensor, data, and expert parallelism in the features guide.

Topic	Description
Attention DP	Sharding Q/O weights across DP groups
Component DP sharding	Per-component independent sharding
Data parallelism	Data parallelism overview
Decode Context Parallelism	KV cache sequence sharding for long contexts
Expert parallelism	Expert parallelism for MoE
Tensor parallelism	Tensor parallelism overview
Vision encoder parallelism	Independent TP/DP for vision encoders

This document is relevant for: Trn2, Trn3