PyTorch - HuggingFace Pretrained BERT Tutorial¶
In this tutorial we will compile and deploy HuggingFace Pretrained BERT model on an Inf1 instance. To enable faster enviroment setup, you will run the tutorial on an inf1.6xlarge instance to enable both compilation and deployment (inference) on the same instance.
Model compilation can be executed on an inf1 instance. Follow the same EC2 Developer Flow Setup using other instance families and leverage Amazon Simple Storage Service (S3) to share the compiled models between different instances.
If you already have Inf1 environment ready, you can skip to Run The Tutorial.
Launch Inf1 instance by following the below steps, please make sure to choose an inf1.6xlarge instance.
Please follow the instructions at launch an Amazon EC2 Instance to Launch an Inf1 instance, when choosing the instance type at the EC2 console. Please make sure to select the correct instance type. To get more information about Inf1 instances sizes and pricing see Inf1 web page.
When choosing an Amazon Machine Image (AMI) make sure to select Deep Learning AMI with Conda Options. Please note that Neuron Conda packages are supported only in Ubuntu 18 DLAMI and Amazon Linux2 DLAMI, Neuron Conda packages are not supported in Amazon Linux DLAMI.
After launching the instance, follow the instructions in Connect to your instance to connect to the instance
You can also launch the instance from AWS CLI, please see AWS CLI commands to launch inf1 instances.
After connecting to the instance from the terminal, clone the Neuron Github repository to the EC2 instance and then change the working directory to the tutorial directory:
git clone https://github.com/aws/aws-neuron-sdk.git cd aws-neuron-sdk/src/examples/pytorch/bert_tutorial
The Jupyter notebook is available as a file with the name tutorial_pretrained_bert.ipynb, you can either run the Jupyter notebook from a browser or run it as a script from terminal:
Running tutorial from browser
First setup and launch the Jupyter notebook on your local browser by following instructions at Jupyter Notebook QuickStart
Open the Jupyter notebook from the menu and follow the instructions
You can also view the Jupyter notebook at: