All Posts

Neuron no longer supports jax_neuronx.nki_call API in jax-neuronx starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Release 2.23 no longer supports jax_neuronx.nki_call API in jax-neuronx package.

Read more ...

Neuron no longer supports base Ubuntu 20 operating system starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Release 2.23 no longer includes support for base Ubuntu 20.04 operating system.

Read more ...

Neuron no longer supports Tensorflow 2.8 and 2.9 starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.23, Neuron no longer supports for TensorFlow-Neuron 2.8 and 2.9 versions.

Read more ...

Neuron no longer supports NeMo Megatron starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.23, Neuron no longer supports NeMo Megatron.

Read more ...

Announcing migration of NxD Core inference examples from NxD Core repository to NxD Inference repository starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.23, the following models and modules in NxD Core inference examples are now only available through NxD Inference package:

Read more ...

Announcing end of support for torch_neuronx.nki_jit API in torch-neuronx starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Release 2.23 will be the last release to include support for torch_neuronx.nki_jit API in torch-neuronx package.

Read more ...

Announcing end of support for mllama 3.2 Meta Checkpoint API starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Release 2.23 will be the last release to include support for the mllama 3.2 Meta checkpoint API. In the next release (Neuron 2.24), Neuron will end support.

Read more ...

Announcing end of support for XLA_USE_BF16 and XLA_DOWNCAST_BF16 starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.23, Neuron will begin phasing out support for the XLA_DOWNCAST_BF16 and XLA_USE_BF16 environment variables. In this release, usage of these variables will trigger warnings. Neuron will end support in a subsequent release, aligned with the torch-xla maintenance schedule.

Read more ...

Announcing end of support for NxDT Megatron Models starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.23 will be the last release to include support for NxDT Megatron Models.

Read more ...

Announcing end of support for NKI block dimension starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.23 will be the last release to include support for the NKI block dimension in NKI tensor creation routines. Starting with this release, using the block dimension will generate EOS warnings. In the next release (Neuron Release 2.24), these warnings will be upgraded to errors.

Read more ...

Neuron no longer supports PyTorch 2.1 starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.22, Neuron no longer includes support for PyTorch 2.1, its associated Deep Learning Containers (DLCs), and Deep Learning AMIs (DLAMIS).

Read more ...

Neuron no longer supports PyTorch 1.13 starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.22, Neuron no longer supports PyTorch 1.13, its associated Deep Learning Containers (DLCs), and Deep Learning AMIs (DLAMIS) for Trn1, Trn2, and Inf2 instances.

Read more ...

Neuron no longer includes support for Ubuntu20 DLCs and DLAMIs starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.22, Neuron no longer includes offerings for Ubuntu20 Deep Learning Containers (DLCs) and Deep Learning AMIs (DLAMIs).

Read more ...

Neuron no longer includes support for Neuron DET tool starting with this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.22, Neuron no longer supports the Neuron Distributed Event Tracing (NDET/neuron-det) tool (Neuron Distributed Event Tracing (NDET) User Guide).

Read more ...

Neuron no longer includes Python 3.8 support starting this release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.22, Neuron no longer includes support for Python 3.8 as it has its reached end-of-life status.

Read more ...

Announcing end of support for jax_neuronx.nki_call API in jax-neuronx from starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.23, Neuron will end support for jax_neuronx.nki_call API in jax-neuronx package.

Read more ...

Announcing end of support for Tensorflow 2.8 and 2.9 starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.23, Neuron will end support for TensorFlow 2.8 and 2.9. Future Neuron releases will not include support for Tensorflow-Neuron 2.8 and 2.9 versions.

Read more ...

Announcing end of support for Neuron support for NeMo Megatron starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.23, Neuron will end support for NeMo Megatron.

Read more ...

PyTorch Neuron versions 1.9 and 1.10 no longer supported

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.21, Neuron no longer supports torch-neuron 1.9 and 1.10 versions. Current users of torch-neuron 1.9 and 1.10 versions are advised to migrate to the latest torch-neuron supported version. Please see Release Content.

Read more ...

Announcing migration of NxD Core examples from NxD Core repository to NxD Inference repository in next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Release 2.21 will be the last release to include NxD Core repository inference examples under the NxD Core repository: aws-neuron/neuronx-distributed. Starting with Neuron Release 2.21, the models and modules in NxD Core inference examples are now available through NxD Inference package. We recommend customers to update their applications to use examples from the NxD Inference repository. See NxD Inference Overview

Read more ...

Announcing end of support for Ubuntu20 DLCs and DLAMIs

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron Release 2.21, AWS Neuron will begin phasing out support for Ubuntu20 Deep Learning Containers (DLCs) and Deep Learning AMIs (DLAMIs). Neuron 2.21 will be the last release to provide bug fixes, and by Neuron 2.22, these offerings will no longer be available.

Read more ...

Announcing end of support for Python 3.8 in future releases

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Due to Python 3.8 reaching its end-of-life status, future Neuron releases will no longer include support for this version.

Read more ...

Announcing end of support for PyTorch 2.1 starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Release 2.21 is the last release to support PyTorch 2.1, its associated Deep Learning Containers (DLCs), and Deep Learning AMIs (DLAMIS).

Read more ...

Announcing end of support for PyTorch 1.13 starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Release 2.21 is the last release to support PyTorch 1.13, its associated Deep Learning Containers (DLCs), and Deep Learning AMIs (DLAMIS) for Trn1, Trn2, and Inf2 instances.

Read more ...

Announcing end of support for Neuron DET tool starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Release 2.21 will be the last release to support the Neuron Distributed Event Tracing (NDET/neuron-det) tool (Neuron Distributed Event Tracing (NDET) User Guide).

Read more ...

‘neurondevice’ resource name in Neuron Device K8s plugin no longer supported

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.20, Neuron no longer supports resource name ‘neurondevice’.

Read more ...

‘neuron-device-version’ field in neuron-monitor no longer supported

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.20, Neuron no longer supports the field ‘neuron-device-version’ in neuron-monitor.

Read more ...

Tensorflow-Neuron 1.x no longer supported

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.20, Neuron no longer supports Tensorflow-Neuron 1.x. Current users of those versions are advised to migrate to latest tensorflow-neuron version 2.10.1. Please see TensorFlow Neuron for more details.

Read more ...

Neuron Runtime no longer supports Amazon Linux 2 (AL2)

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting from Neuron release 2.20, the Neuron Runtime (aws-neuronx-runtime-lib) will no longer support Amazon Linux 2 (AL2). The Neuron Driver (aws-neuronx-dkms) is now the only Neuron package that supports Amazon Linux 2.

Read more ...

Announcing maintenance mode for torch-neuron 1.9 and 1.10 versions

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.20, torch-neuron 1.9 and 1.10 versions will enter maintenance mode. Future Neuron releases will not include support for torch-neuron 1.9 and 1.10 versions. Current users of torch-neuron 1.9 and 1.10 versions are advised to migrate to torch-neuron 1.13.

Read more ...

Announcing Name Change for Neuron Component

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.20, the name of the following Neuron component will change as follows:

Read more ...

Neuron support for MxNet enters maintenance mode

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.19, Neuron support for MxNet (mxnet-neuron) is entering maintenance mode.

Read more ...

Announcing end of support for ‘neurondevice’ resource name in Neuron Device K8s plugin

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.19 will be the last release to include resource name ‘neurondevice’.

Read more ...

Announcing end of support for ‘neuron-device-version’ field in neuron-monitor

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.19 will be the last release to include the field ‘neuron-device-version’ in neuron-monitor.

Read more ...

Announcing end of support for Tensorflow-Neuron 1.x

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.19 will be the last release to support Tensorflow-Neuron 1.x. Future Neuron releases will not include support for Tensorflow-Neuron 1.x versions. Current users of those versions are advised to migrate to latest tensorflow-neuron version 2.10.1.

Read more ...

Announcing end of support for Probuf versions <= 3.19 for PyTorch NeuronX, NeuronX Distributed, and Transformers NeuronX libraries

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.19 will be the last release that will include Probuf <= 3.19 support for PyTorch NeuronX, NeuronX Distributed, and Transformers NeuronX libraries. Future Neuron releases will not include Probuf <= 3.19 support for PyTorch NeuronX.

Read more ...

Announcing end of support for Neuron Runtime support of Amazon Linux 2 (AL2)

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.19 will be the last release that will include Neuron Runtime support for Amazon Linux 2 . Future Neuron releases will not include Neuron Runtime support for Amazon Linux 2.

Read more ...

Announcing maintenance mode for Transformers NeuronX library starting next release

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting from Neuron release 2.24, Transformers NeuronX library is entering maintenance mode. Future releases of transformers-neuronx will address critical security issues only and we will gradually end support.

Read more ...

Announcing end of support for Neuron virtual environments in AWS Deep Learning AMI (Amazon Linux 2)

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.18.2 will be the last release that will include support for the following virtual environments in AWS Deep Learning AMI (Amazon Linux 2):

Read more ...

Announcing end of support for Neuron Release 2.18.0 Deep Learning AMIs

This document is relevant for: Inf1, Inf2, Trn1, Trn2

We are announcing end of support for Neuron release 2.18.0 Deep Learning AMIs. DLAMIs released between March 26,2024 (2024-03-26) and April 10, 2024 (2024-04-10) were shipped without the audit package. The following are the affected DLAMIs:

Read more ...

Tensorflow-Neuron 1.x enters maintenance mode

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.18, Tensorflow-Neuron 1.x is entering maintenance mode. Future releases of Tensorflow-Neuron 1.x will address critical security issues only and we will gradually end support. Current users of those versions are advised to migrate to latest tensorflow-neuron version 2.10.1.

Read more ...

PyTorch NeuronX version 2.0 (Beta) no longer supported

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.17 no longer supports PyTorch NeuronX version 2.0 (Beta).

Read more ...

Announcing Name Change for Neuron Components

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.16, the name of the following Neuron components will change as follows:

Read more ...

Announcing deprecation for NeuronX Distributed Training Samples in Neuron Samples Repository

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.16 will be the last release to include support for NeuronX Distributed Training Samples (Llama-2, GPT-NeoX 20B, and GPT-NeoX 6.9B) under the AWS Neuron Samples Github repository.

Read more ...

Announcing deprecation for Neuron Containers with Runtime 1.x

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.3 was the last release to support Neuron Runtime 1.x (neuron-rtd). Current users of Neuron DLC/DLAMI with Neuron Runtime 1.x are required to update their image to support latest Neuron Runtime versions. For instructions, see the Setup Guide.

Read more ...

Announcing End of Support for PyTorch NeuronX version 2.0 (beta)

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.16 will be the last release that will include beta support for PyTorch NeuronX version 2.0 (beta). Future Neuron releases will not include support for PyTorch NeuronX version 2.0.

Read more ...

Announcing End of Support for PyTorch Neuron version 1.10

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.16 will be the last release that will include support for PyTorch Neuron version 1.10. Future Neuron releases will not include support for PyTorch Neuron version 1.10.

Read more ...

Announcing End of Support for nemo option-argument

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.15 will be the last release that will include support for nemo option-argument in the existing –distribution_strategy compiler option. Future releases will not include Neuron support for nemo option-argument. Users are advised to migrate to the new llm-training option-argument.

Read more ...

Announcing End Of Support for OPT example in Transformers NeuronX

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.15 will be the last release that will include OPT example in Transformers NeuronX.

Read more ...

Announcing deprecation for --model-type=transformer-inference compiler flag

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.14, the --model-type=transformer-inference compiler flag is deprecated.

Read more ...

Announcing end of support for torch-neuron version 1.9

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Title underline too short.

Read more ...

AWS Neuron reference for Megatron-LM no longer supported

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.13 no longer includes support for AWS Neuron reference for Megatron-LM.

Read more ...

Announcing end of support for AWS Neuron reference for Megatron-LM

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.12 will be the last release that will include support for AWS Neuron reference for Megatron-LM. Future releases will not include Neuron support for Megatron-LM.

Read more ...

AWS Deep Learning AMIs now available with Neuron 2.12 version

This document is relevant for: Inf1, Inf2, Trn1, Trn2

We are happy to announce that the following Deep Learning AMIs are now available with latest Neuron Version 2.12.

Read more ...

Announcing end of support for Python 3.7

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Title underline too short.

Read more ...

Announcing end of support for Ubuntu 18

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.12 will be the last release that will include support for Ubuntu 18 . Future Neuron releases will not include support for Ubuntu 18

Read more ...

AWS Deep Learning AMIs now available with Neuron 2.10 version

This document is relevant for: Inf1, Inf2, Trn1, Trn2

We are happy to announce that the following Deep Learning AMIs are now available with latest Neuron Version 2.10. These DLAMIs now support all the Neuron EC2 instances including Inf1, Inf2, Trn1/Trn1n.

Read more ...

Announcing end of support for tensorflow-neuron versions 2.7

This document is relevant for: Inf1

Neuron release 2.10 will be the last release that will include tensorflow-neuron versions 2.7. Future Neuron releases will not include tensorflow-neuron versions 2.7

Read more ...

Announcing end of support for mxnet-neuron versions 1.5

This document is relevant for: Inf1

Neuron release 2.10 will be the last release that will include mxnet-neuron versions 1.5. Future Neuron releases will not include mxnet-neuron versions 1.5

Read more ...

PyTorch 1.13 Deep Learning Container for Inf2 & Trn1/Trn1n now available for SageMaker

This document is relevant for: Inf2, Trn1, Trn2

We are happy to announce that an updated Deep Learning Container that supports PyTorch 1.13 and Neuron 2.9.1 versions is now available for Sagemaker Training.

Read more ...

Introducing Neuron packaging and installation changes for Inf1 customers

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.5, Neuron introduces changes in Neuron packages and installation instructions for Inf1, the following Neuron packages will change names:

Read more ...

Announcing end of support for tensorflow-neuron versions 2.5 and 2.6

This document is relevant for: Inf1

Neuron release 2.5 will be the last release that will include tensorflow-neuron versions 2.5 and 2.6. Future Neuron releases will not include tensorflow-neuron versions 2.5 and 2.6.

Read more ...

Announcing end of support for torch-neuron versions 1.7 and 1.8

This document is relevant for: Inf1

Neuron release 2.5 will be the last release that will include torch-neuron versions 1.7 and 1.8. Future Neuron releases will not include torch-neuron versions 1.7 and 1.8.

Read more ...

Neuron GPG key for Ubuntu installation has expired

This document is relevant for: Inf1, Inf2, Trn1, Trn2

GPG, or GNU Privacy Guard, is a public key cryptography implementation. This allows for the secure transmission of information between parties and can be used to verify that the origin of a message is genuine.

Read more ...

Amazon SageMaker now supports Trn1 training jobs

This document is relevant for: Inf2, Trn1, Trn2

We are happy to announce that Amazon SageMaker now supports running training jobs on ml.trn1 instance types.

Read more ...

Introducing AWS Deep Learning AMI Neuron PyTorch

This document is relevant for: Inf2, Trn1, Trn2

We are happy to announce that Deep Learning AMI (DLAMI) with pre-installed PyTorch Neuron (torch-neuronx) is now available, for more information see:

Read more ...

Introducing the first release of Neuron 2.x enabling EC2 Trn1 General Availability (GA)

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron release 2.3 is the first release of Neuron 2.x that enables GA of the new EC2 Trn1 instances. Neuron release 2.3 extends the latest release of Neuron 1.x (Neuron 1.19.2), adding support for Deep Learning training on the AWS Trainium chips.

Read more ...

Introducing Packaging and installation changes

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.3, Neuron introduces changes in Neuron packages and installation instructions.

Read more ...

Introducing New Neuron GitHub Repositories

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.3, Neuron Github repositories will be migrated to the new AWS Neuron GitHub Organization.

Read more ...

Announcing Neuron Runtime 1.x (neuron-rtd) end-of-support

This document is relevant for: Inf1, Inf2, Trn1, Trn2

Starting with Neuron release 2.3, Neuron components like Neuron System Tools and Neuron Driver will no longer support Neuron Runtime 1.x.

Read more ...

End of support for torch-neuron version 1.5

This document is relevant for: Inf1

Starting with Neuron 1.19.0 release, torch-neuron 1.5 will no longer be supported, and no further releases of torch-neuron version 1.5 will be issued. Current users of torch-neuron version 1.5 are advised to migrate to latest torch-neuron version.

Read more ...

End of support for tensorflow-neuron versions 2.1, 2.2, 2.3 and 2.4

This document is relevant for: Inf1

Starting with Neuron 1.18.0 release, tensorflow-neuron versions 2.1, 2.2, 2.3 and 2.4 will no longer be supported, and no further releases of tensorflow-neuron versions 2.1, 2.2, 2.3 and 2.4 will be issued. Current users of those versions are advised to migrate to latest tensorflow-neuron version.

Read more ...

Announcing end of support for torch-neuron version 1.5 starting with Neuron 1.19.0 release

This document is relevant for: Inf1

Starting with Neuron 1.19.0 release, torch-neuron version 1.5 will no longer be supported. Last release of torch-neuron version 1.5 will be issued as part of Neuron 1.18.0 release. Current users of those versions are advised to migrate to latest torch-neuron version.

Read more ...

Announcing end of support for NEURONCORE_GROUP_SIZES starting with Neuron 1.20.0 release

This document is relevant for: Inf1

Starting with Neuron SDK 1.20.0, NEURONCORE_GROUP_SIZES environment variable will no longer be supported. Setting NEURONCORE_GROUP_SIZES environment variable will no longer affect applications behavior. Current customers using NEURONCORE_GROUP_SIZES environment variable are advised to use NEURON_RT_VISIBLE_CORES environment variable or NEURON_RT_NUM_CORES environment variable instead.

Read more ...

Previous Announcements

This document is relevant for: Inf1

02/17/2022 - tensorflow-neuron versions 2.1, 2.2, 2.3 and 2.4 enter maintenance mode

Read more ...