Use MoXing to Develop Training Scripts for Handwritten Digit Recognition
This section describes how to use MoXing to recognize handwritten digits and images from an MNIST dataset on the ModelArts platform.
The following figure shows the process of identifying handwritten digits and images using MoXing.
- Preparing Data: Obtain the MNIST dataset and upload it to OBS.
- Training a Model: Use the MoXing framework to compile the model training script and create a training job for model training.
- Deploying the Model: After obtaining the trained model file, create a prediction job to deploy the model as a real-time prediction service.
- Verifying the Model: Initiate a prediction request and obtain the prediction result.
ModelArts provides a sample MNIST dataset named Mnist-Data-Set. This example uses this dataset to build a model. Perform the following operations to upload the dataset to the OBS directory test-modelarts/dataset-mnist created in preparation.
- Download the Mnist-Data-Set dataset to the local PC.
- Decompress the Mnist-Data-Set.zip file to the Mnist-Data-Set directory on the local PC.
- Upload all files in the Mnist-Data-Set directory to the test-modelarts/dataset-mnist directory on OBS. For details about how to upload files, see Uploading a File.
In the imported MNIST dataset, .gz files are compressed files with the same name and are not used this time. In this example, only the files that are not compressed are used. The used files are as follows:
- t10k-images-idx3-ubyte: validation set, which contains 10,000 samples
- t10k-labels-idx1-ubyte: labels of the validation set, which contains the labels of the 10,000 samples
- train-images-idx3-ubyte: training set, which contains 60,000 samples
- train-labels-idx1-ubyte: labels of the training set, which contains the labels of the 60,000 samples
Training a Model
After the data preparation is completed, use the MoXing API to compile the training script code. ModelArts provides a code sample, train_mnist.py. The following uses this sample to train the model.
- Download the ModelArts-Lab project from GitHub and obtain training script train_mnist.py from the \ModelArts-Lab-master\official_examples\Using_MoXing_to_Create_a_MNIST_Dataset_Recognition_Application\codes directory of the project.
- Upload the train_mnist.py file to OBS, for example, test-modelarts/mnist-MoXing-code.
- On the ModelArts management console, choose Training Jobs > Create.
- On the Create Training Job page, set required parameters based on Figure 1 and Figure 2, and click Next.
Data Source: Select the data storage location, and then select the OBS path for saving the dataset.
- On the Confirm tab page, check the parameters of the training job and click Submit.
- On the Training Jobs page, when the training job status changes to Running Success, the model training is completed. If any exception occurs, click the job name to go to the job details page and view the training logs.
The training job may take more than 10 minutes to complete. If the training time exceeds a certain period (for example, one hour), manually stop it to release resources. Otherwise, the account balance may be insufficient, especially for the models that are trained using GPUs.
- (Optional) During or after model training, you can create a TensorBoard job to view parameter statistics.
Set Log Path to the value of Training Output Path for the training job, for example, /test-modelarts/mnist-model/. Complete the TensorBoard job creation as prompted.
Deploying the Model
After the model training is completed, deploy the model as a real-time prediction service. ModelArts provides the compiled inference code customize_service.py and configuration file config.json.
- Download the ModelArts-Lab project from GitHub and obtain inference code customize_service.py and configuration file config.json from the \ModelArts-Lab-master\official_examples\Using_MoXing_to_Create_a_MNIST_Dataset_Recognition_Application\codes directory of the project.
- Upload the customize_service.py and config.json files to OBS. The files must be stored in the path for saving the model generated for the training job, for example, test-modelarts/mnist-model/model.
- The training job creates a model folder in the path specified by Training Output Path to store the generated model.
- The inference code and configuration file must be uploaded to the model folder.
- On the ModelArts management console, choose Model Management > Models in the left navigation pane. The Models page is displayed. Click Import in the upper left corner.
- On the Import Model page, set required parameters as shown in Figure 3 and click Next.
- On the Models page, if the model status changes to Normal, the model has been imported successfully. Choose Deploy > Real-Time Services in the Operation column to deploy the model as a real-time service.
- On the Deploy page, set parameters by referring to Figure 4 and click Next.
- On the Confirm tab page, check the configurations and click Submit to create a real-time service.
- After the real-time service is created, the Service Deployment > Real-Time Services page is displayed. The service deployment takes some time. When the service status changes to Running, the service is successfully deployed.
Verifying the Model
After the real-time service is deployed, access the service to send a prediction request for test.
- On the Real-Time Services page, click the name of the real-time service. The real-time service details page is displayed.
- On the real-time service details page, click the Prediction tab.
- Click Upload next to Image File to upload an image with a white handwritten digit on a black background and click Predict.
After the prediction is completed, the prediction result is displayed in the Test Result pane. According to the prediction result, the digit on the image is 4.NOTE:
Figure 5 Prediction result
- As specified in the inference code and configuration files, the size of the image used for prediction must be 28 x 28 pixels, and the image must be in the JPG format and must contain white handwritten digits on a black background.
- You are advised not to use the images provided by the dataset. You can use the drawing tool provided by the Windows operating system to draw an image for prediction.
- If a single-channel image that is not in the required format is used, the prediction result may be inaccurate.