Help Center> ModelArts> FAQs> Training Jobs> Compiling the Training Code> How Do I Create a Training Job When a Dependency Package Is Referenced by the Model to Be Trained?
Updated on 2023-11-22 GMT+08:00

How Do I Create a Training Job When a Dependency Package Is Referenced by the Model to Be Trained?

Store the pip-requirements.txt file in the training code directory.

Any one of the following file names can be used. This section uses pip-requirements.txt as an example.

  • pip-requirement.txt
  • pip-requirements.txt
  • requirement.txt
  • requirements.txt

Before the training boot file is executed, the system automatically runs the following command to install the specified Python packages:

pip install -r pip-requirements.txt

Storing the Installation File in the Code Directory

ModelArts allows you to install third-party dependency packages during model training in either of the following ways:

Installation File Specifications

The installation file varies depending on the dependency package type.

  • Open-source installation packages

    Installation using the source code from GitHub is not supported.

    Create a file named pip-requirements.txt in the code directory, and specify the name and version number of the dependency package in the file. The format is [Package name]==[Version].

    Take for example, an OBS path specified by Code Dir that contains model files and the pip-requirements.txt file. The code directory structure would be as follows:

    |---OBS path to the model boot file
         |---model.py               #Model boot file
         |---pip-requirements.txt   #Defined configuration file, which specifies the name and version of the dependency package

    The following shows the content of the pip-requirements.txt file:

    alembic==0.8.6
    bleach==1.4.3
    click==6.6
  • WHL packages

    If the training background does not support the download of open source installation packages or use of user-compiled WHL packages, the system cannot automatically download and install the package. In this case, place the WHL package in the code directory, create a file named pip-requirements.txt, and specify the name of the WHL package in the file. The dependency package must be a .whl file.

    Take for example, an OBS path specified by Code Dir that contains model files, the .whl file, and the pip-requirements.txt file. The code directory structure would be as follows:

    |---OBS path to the model boot file
         |---model.py               #Model boot file
         |---XXX.whl                #Dependency package. If multiple dependencies are required, place multiple dependency packages here.
         |---pip-requirements.txt   #Defined configuration file, which specifies the name of the dependency package

    The following shows the content of the pip-requirements.txt file:

    numpy-1.15.4-cp36-cp36m-manylinux1_x86_64.whl
    tensorflow-1.8.0-cp36-cp36m-manylinux1_x86_64.whl

Compiling the Training Code FAQs

more