This document is relevant for: Trn2, Trn3

Multimodal#

Design documentation for multimodal inference. For configuration guidance, see Multimodal support in the features guide.

Topic

Description

Block Packing Vision Attention

FFD block packing for multi-image attention efficiency

On-Device Encoder Cache

Block-based on-device cache for vision encoder outputs

M-RoPE

Spatial position embeddings for VLMs

This document is relevant for: Trn2, Trn3