This document is relevant for: Trn2, Trn3

Parallelism

Design documentation for parallelism strategies. For configuration guidance, see Tensor, data, and expert parallelism in the features guide.

Topic	Description
Attention DP	Sharding Q/O weights across DP groups
Component DP sharding	Per-component independent sharding
Data parallelism	Data parallelism overview
Decode Context Parallelism	KV cache sequence sharding for long contexts
Encoder parallelism	Independent TP/DP for multimodal encoders
Expert parallelism	Expert parallelism for MoE
Tensor parallelism	Tensor parallelism overview

This document is relevant for: Trn2, Trn3