Posts tagged nki

NKI Library renames the kv_parallel_segmented_prefill kernel starting with Neuron 2.31.0

Starting with Neuron SDK 2.31.0, the NKI Library kv_parallel_segmented_prefill kernel has been renamed to attention_kv_parallel_segmented_cte. The source file moves from core/attention/kv_parallel_segmented_prefill.py to core/attention/attention_kv_parallel_segmented_cte.py.

Callers must update both their import path and the function name. For the kernel reference, see Attention KV-Parallel Segmented CTE.

Read more ...

Announcing upcoming deprecation of the SPMD launch grid requirement for LNC2 NKI kernels

Launching an LNC2 NKI kernel currently relies on an SPMD launch grid whose dimension matches the LNC degree. This requirement will be deprecated in a future release. The behavior does not change in NKI 0.5.0 (Neuron 2.31.0).

When this change takes effect:

Read more ...