This document is relevant for: Trn2, Trn3

Qwen3-VL (Dense) Model Recipe#

Introduction#

Qwen3-VL is a multimodal language model developed by the Qwen team. It supports image-text understanding, visual reasoning, OCR, document analysis, and multi-image comparison. The -Instruct variant is instruction-tuned for chat and task-following, while the -Thinking variant adds extended chain-of-thought reasoning.

Qwen3-VL is now supported for inference serving with vLLM using the Neuron SDK on AWS Trainium2 (trn2) and Trainium3 (trn3) hardware.

Compatible model checkpoints:

Model	HuggingFace
Qwen3-VL-32B-Instruct	Qwen/Qwen3-VL-32B-Instruct
Qwen3-VL-32B-Thinking	Qwen/Qwen3-VL-32B-Thinking
Qwen3-VL-8B-Instruct	Qwen/Qwen3-VL-8B-Instruct
Qwen3-VL-8B-Thinking	Qwen/Qwen3-VL-8B-Thinking

The -Thinking checkpoint shares the same architecture and serving path as -Instruct and runs with the identical configuration. The accuracy numbers below were measured on the -Thinking variant.

Features#

Per-model feature availability for Qwen3-VL. See the features guide for configuration details.

Category	Feature	Status
Multimodal Inputs	Text	✅
	Single image + text	✅
	Multi-image + text	✅
	Video + text	✅
Quantization	BF16	✅
Parallelism	Tensor parallelism (TP)	✅
	Vision encoder parallelism	✅
	Pipeline parallelism (PP)	❌
	Context parallelism (CP)	❌
Performance	Segmented prefill	✅
	On-device sampling (greedy, top-k, top-p)	✅
	Chunked prefill (mixed batching)	❌
Compilation	torch.compile (XLA backend)	✅
	CPU mode (testing)	✅

Status legend:

✅ Supported: integrated and tested for Qwen3-VL
❌ Not supported: may be considered for future releases

Tensor parallelism is recommended at tensor_parallel_size=16 on a trn2.48xlarge (16 NeuronCores), matching the configuration used throughout the tutorial.

Multi-image input is validated up to 30 images at 512x512 resolution.

Vision encoder parallelism shards the encoder by the num_blocks dimension of the block-packed images (unlike text DP, which shards by request batch), configurable via vision_neuron_config.

Accuracy Evaluation#

Benchmark: POPE (Polling-based Object Probing Evaluation): 5,000 binary Yes/No questions about object presence in images across three difficulty subsets: popular, adversarial, random. Results are from the open-source VLMEvalKit harness.

Metric	32B-Thinking, Neuron Trn2 BF16
Overall F1	85.94
Adversarial F1	84.86
Popular F1	85.63
Random F1	87.38

Reproduce: Serve the -Thinking checkpoint following the tutorial and add --served-model-name Qwen3-VL-32B-Thinking to vllm serve, then run VLMEvalKit’s run.py against the running server over its OpenAI-compatible endpoint:

git clone https://github.com/open-compass/VLMEvalKit.git
cd VLMEvalKit && git checkout fd3884b && pip install . && cd ..
python VLMEvalKit/run.py --data POPE --model Qwen3-VL-32B-Thinking \
    --base-url http://localhost:8000/v1 --max-tokens 256

Tutorials#

Tutorial: Deploy Qwen3-VL-32B with vLLM Neuron