NxD Training# NxD Training is a PyTorch library for end-to-end distributed training. NxD Training Overview & Setup Overview Setup API Reference Guide YAML Configuration Settings Developer Guide Integrating a New Model Integrating a new dataset/dataloader Registering an optimizer and LR scheduler Migrating from Neuron-NeMo-Megatron to Neuronx Distributed Training NxD Training Compatibility with NeMo Tutorials Megatron GPT Pretraining HuggingFace Llama3-8B Pretraining HuggingFace Llama3-8B Supervised Fine-tuning Checkpoint Conversion HuggingFace Llama3-8B Direct Preference Optimization (DPO) based Fine-tuning HuggingFace Llama3.1/Llama3-70B Pretraining Checkpoint Conversion App Notes Introducing NxD Training Tensor Parallelism Overview Pipeline Parallelism Overview Activation Memory Reduction Misc Known Issues and Workarounds
