This document is relevant for: Inf2, Trn1

nosearch:

Neuron Custom C++ Operators [Beta] (archived)

Warning

Neuron Custom C++ Operators are in maintenance mode starting with Neuron 2.33. The aws-neuronx-gpsimd-customop-lib library receives critical security and bug fixes only, and this documentation is archived and no longer updated. The feature runs only on NeuronCore-v2 (Inf2 and Trn1) through the PyTorch/XLA path. To implement new custom operators, use the Neuron Kernel Interface (NKI). See aws-neuronx-gpsimd-customop-lib is now in maintenance mode starting with Neuron 2.33.0.

Neuron Custom C++ Operators enable developers to write C++ Custom Operators (“CustomOps”) that run on NeuronCores. This enables developers to extend operator support beyond what is officially supported by Neuron.

Developers can use standard PyTorch custom operators programming interfaces to leverage Neuron Custom C++ Operators feature. This makes it easy to migrate CPU Custom Operators to Neuron, and implement new beta operators, all without any intimate knowledge of the NeuronCore hardware.

Note

Neuron Custom C++ Operators feature is currently supported on NeuronCore-v2 architecture only, which is found in Trainium (Trn1) and second-generation Inferentia (Inf2) chips.

API Reference Guide

• Custom Operators API Reference Guide [Beta] (archived)

Developer Guide

• Neuron Custom C++ Operators Developer Guide [Beta] (archived)

Tutorials

• Neuron Custom C++ Operators in MLP Training (archived)

• Neuron Custom C++ Operators Performance Optimization (archived)

Misc

• Neuron Custom C++ Tools Release Notes

• Neuron Custom C++ Library Release Notes

This document is relevant for: Inf2, Trn1