This document is relevant for: Inf1, Inf2, Trn1, Trn2, Trn3

Component Release Notes for Neuron Agentic Development#

Latest version (in 2.33.0): 1.4

The release notes for Neuron Agentic Development, the open-source suite of AI agents and skills that author, debug, profile, and analyze NKI kernels, port HuggingFace models to NxD Inference, validate numerical equivalence of ported models, and plan and analyze LLM inference performance on AWS Trainium from inside agentic IDEs such as Claude Code and Kiro. Read these notes for the changes, improvements, and bug fixes in each AWS Neuron SDK release. For an introduction to the feature, see Neuron Agentic Development.

Source, installation instructions, and the current catalog of agents and skills are maintained in the open-source repository: aws-neuron/neuron-agentic-development on GitHub.

Neuron Agentic Development (Neuron 2.33.0 Release)#

Improvements#

  • NKI 0.7.0 skill support. The NKI agentic skills are updated for compatibility with NKI 0.7.0.

New skills#

  • neuron-parallelism-planner — Plans tensor, context, and data parallelism (and expert parallelism for MoE models) and batch size for transformer LLM inference on Trainium before you run on hardware. Takes a run-spec (the model’s HuggingFace config.json, hardware constants, and decode ITL and prefill TTFT SLAs), validates its analytic entitlement model against a meta-device FLOP-counter oracle, sweeps every config and context bucket, and selects the config with the highest per-node throughput that meets both SLAs. Reports predicted ITL, TTFT, throughput, per-device HBM, and the binding constraint, and models KV-cache replication, FP8 weights and KV cache, and speculative decoding. Supports decoder-only softmax-attention transformers (MHA, GQA, MQA, MLA, and mixed sliding-window and global attention) with dense or MoE FFNs. Linear-attention, recurrent (for example, Gated DeltaNet), and state-space layers are not supported. Sweeps tensor, context, and data parallelism, and full expert parallelism for MoE models; pipeline parallelism is not supported. Plans single-node, colocated serving only. (Source)

  • neuron-hardware-entitlement — Computes per-operator compute and HBM floors for one chosen parallelism config and identifies which operator binds on the tensor engine, weight reads, or KV-cache reads. Rolls operator floors into serialized attention, FFN, and lm_head component floors that reconcile exactly with the Parallelism Planner, evaluates a config across a whole bucket regime, and attributes a deployed graph’s resident HBM back to operator floors to rank overages. Shares the Parallelism Planner’s run-spec, engine, and oracle, and supports the same architectures. (Source)

  • neuron-framework-profiling-vllm-neuron — Captures device and system profiles from a running vLLM Neuron server on Trainium, isolating the prefill pass and steady-state decode into separate traces so each can be attributed on its own. Arms the profiler on a scheduler event rather than a timer, falls back to a decode-only server when a timed trigger cannot fire, manages the server lifecycle so a capture does not orphan device-holding workers, and records a run specification and manifest that make the capture reproducible. (Source)

  • neuron-framework-profile-analysis-vllm-neuron — Turns a capture into a bottleneck localization. Ingests the device and system traces into queryable parquet, establishes the session workload and step time from the live dispatch records, computes memory and compute roofline floors from the capture’s own device constants, partitions each step at its collectives, and rolls instructions up to model components so attention, MLP, and lm_head time and HBM traffic are separable. Pairs with neuron-hardware-entitlement to cross-validate a profiled step against an analytical floor. (Source)

New agents#

  • neuron-parallelism-planner-agent — Reconciles a joint latency and throughput SLA into a single-node DP × TP config, reports the full TP sweep and the binding constraint, and hands off to the profiling skills for on-hardware validation.

  • neuron-hardware-entitlement-agent — Computes per-operator floors for a config handed off from the planner, flags likely collective-bound configs, and routes optimization effort to the operator with the most headroom.

New and updated documentation#

Neuron Agentic Development (Neuron 2.32.0 Release)#

Improvements#

  • New Autoport skill for vLLM Neuron. The new neuron-framework-autoport-vllm-neuron skill ports GPU-compliant transformer models to the vLLM Neuron backend on AWS Trainium. The agent researches the model architecture, sizes tensor and expert parallelism, generates the model implementation, registers it in the vLLM model registry, and validates accuracy — so you can bring a model to vLLM Neuron serving without writing the port by hand. See the Autoport vLLM Neuron deep dive.

  • Model Equivalence now validates vLLM Neuron ports. The Deep dive: Validate model ports with the Equivalence skill skill adds a vLLM Neuron adapter so you can verify numerical equivalence for ports that target the vLLM Neuron backend, not just NxD Inference. Select it with target_stack: vllm_neuron, or let the skill auto-detect it from the modeling file’s imports. See the vLLM Neuron section of the Equivalence deep dive.

  • NKI 0.6.0 skill support. The NKI agentic skills are updated for compatibility with NKI 0.6.0. This includes migration guidance for the deprecated nl.dynamic_range and while reg: loop constructs — the agent can walk you through converting them to the new fori_loop() and while_loop() APIs.

New and updated documentation#

Neuron Agentic Development (Neuron 2.31.0 Release)#

Improvements#

  • NKI 0.5.0 skill support. The NKI agentic skills are updated for compatibility with NKI 0.5.0.

Neuron Agentic Development (Neuron 2.30.0 Release)#

New skills#

  • neuron-framework-autoport — Ports HuggingFace transformer models to NxD Inference with full compilation and accuracy validation. The agent handles architecture analysis, implementation, compilation, inference testing, and greedy-token-match validation end to end. (Source)

  • neuron-framework-equivalence — Validates numerical equivalence between a HuggingFace reference model and its NxD Inference port. Uses progressive 3-tensor R-ratio analysis, component-level testing, fault localization, and end-to-end accuracy verification. (Source)

New and updated documentation#

  • Added Getting Started guide covering installation, environment setup, and first agent invocation.

  • Added Autoport developer guide — deep dive on how the Autoport skill works internally, including workflow stages, parameters, and environment setup.

  • Added Equivalence developer guide — deep dive on the Equivalence skill’s 8-stage validation workflow and R-ratio methodology.

  • Added Autoport tutorial — step-by-step walkthrough of porting a HuggingFace model using the Autoport agent.

  • Added Equivalence tutorial — step-by-step walkthrough of validating a ported model using the Equivalence agent.

This document is relevant for: Inf1, Inf2, Trn1, Trn2, Trn3