This document is relevant for: Inf1, Inf2, Trn1, Trn1n

Neuron Documentation Release Notes#

Neuron 2.19.0#

Date: 07/03/2024

Neuron 2.18.0#

Date: 04/01/2024

Neuron 2.16.0#

Date: 12/21/2023

Neuron 2.15.0#

Date: 10/26/2023

Known Issues and Limitations#

Following tutorials are currently not working. These tutorials will be updated once there is a fix.

Neuron 2.14.0#

Date: 09/15/2023

  • Neuron Calculator now supports multiple model configurations for Tensor Parallel Degree computation. See Neuron Calculator

  • Announcement to deprecate --model-type=transformer-inference flag. See Announcing deprecation for --model-type=transformer-inference compiler flag

  • Updated HF ViT benchmarking script to use --model-type=transformer flag. See [script]

  • Updated torch_neuronx.analyze API documentation. See PyTorch NeuronX Analyze API for Inference

  • Updated Performance benchmarking numbers for models on Inf1,Inf2 and Trn1 instances with 2.14 release bits. See _benchmark

  • New tutorial for Training Llama2 7B with Tensor Parallelism and ZeRO-1 Optimizer using neuronx-distributed Training Llama2 7B and Llama3 8B with Tensor Parallelism and ZeRO-1 Optimizer (neuronx-distributed )

  • New tutorial for T5-3B model inference using neuronx-distributed (tutorial)

  • Updated Neuron Persistent Cache documentation regarding clarification of flags parsed by neuron_cc_wrapper tool which is a wrapper over Neuron Compiler CLI. See Neuron Persistent Cache

  • Added tokenizers_parallelism=true in various notebook scripts to supress tokenizer warnings making errors easier to detect

  • Updated Neuron device plugin and scheduler YAMLs to point to latest images. See yaml configs

  • Added notebook script to fine-tune deepmind/language-perceiver model using torch-neuronx. See sample script

  • Added notebook script to fine-tune clip-large model using torch-neuronx. See sample script

  • Added SD XL Base+Refiner inference sample script using torch-neuronx. See sample script

  • Upgraded default diffusers library from 0.14.0 to latest 0.20.2 in Stable Diffusion 1.5 and Stable Diffusion 2.1 inference scripts. See sample scripts

  • Added Llama-2-13B model training script using neuronx-nemo-megatron ( tutorial )

Neuron 2.13.0#

Date: 08/28/2023

Neuron 2.12.0#

Date: 07/19/2023

Neuron 2.11.0#

Date: 06/14/2023

This document is relevant for: Inf1, Inf2, Trn1, Trn1n