This document is relevant for: Inf2, Trn1, Trn2

Neuron Custom C++ Operators [Beta]

Neuron Custom C++ Operators enable developers to write C++ Custom Operators (“CustomOps”) that run on NeuronCores. This enables developers to extend operator support beyond what is officially supported by Neuron.

Developers can use standard PyTorch custom operators programming interfaces to leverage Neuron Custom C++ Operators feature. This makes it easy to migrate CPU Custom Operators to Neuron, and implement new beta operators, all without any intimate knowledge of the NeuronCore hardware.

Note

Neuron Custom C++ Operators feature is currently supported on NeuronCore-v2 architecture only, which is found in Trainium (Trn1) and second-generation Inferentia (Inf2) chips.

API Reference Guide

• Custom Operators API Reference Guide [Beta]

Developer Guide

• Neuron Custom C++ Operators Developer Guide [Beta]

Tutorials

• Neuron Custom C++ Operators in MLP Training

• Neuron Custom C++ Operators Performance Optimization

Misc

• Neuron Custom C++ Tools Release Notes

• Neuron Custom C++ Library Release Notes

This document is relevant for: Inf2, Trn1, Trn2