Deploying the Development Environment in Physical Machines

  • Currently, the development environment can be installed on a physical machine in Ubuntu 20.04 and CentOS 7.
  • Do not modify the code of other files except the run.sh file in the compilation directory.
  1. Install the CANN package and Ascend-adapted TensorFlow plugin package by referring to the CANN Software Installation Guide.
  2. Configure environment variables.

    The CANN software provides a process-level environment variable setting script to automatically set environment variables. The environment variables automatically become invalid after the processes end.

    You can run the following commands in the shell script for starting programs to set CANN-related environment variables or run the following commands through the CLI. (The default installation path /usr/local/Ascend of the root user is used as an example.)

    source /usr/local/Ascend/cann/set_env.sh
  3. Run the following command to decompress the software package. If no path is specified, the software package is extracted in the current directory by default. --no-same-owner needs to be added to the decompression command.
    # Extract the package to the current directory.
    tar -xf Ascend-mindxsdk-mxrec-{version}_linux-{arch}.tar.gz --no-same-owner
    # Extract the package to a specified directory.
    tar -xf Ascend-mindxsdk-mxrec-{version}_linux-{arch}.tar.gz --no-same-owner -C Specific_path
  4. Check the directory structure after the decompression.
    1
    2
    3
    4
    5
    6
    mindxsdk-mxrec/
    |-- tf1_whl
    |   `-- mx_rec-{version}-py3-none-linux_{arch}.whl    # .wheel package of the mx_rec framework for TensorFlow 1.15.0
    |-- tf2_whl
    |   `-- mx_rec-{version}-py3-none-linux_{arch}.whl    # .wheel package of the mx_rec framework for TensorFlow 2.6.5
    `-- version.info
    
  5. Install the .wheel package. (Select a .wheel package matching the TensorFlow version based on your requirements.)
    pip3 install mx_rec-{version}-py3-none-linux_{arch}.whl 
    By default, the .wheel package is installed in the site-packages directory of Python. If you use the --target parameter to specify a directory, you need to add the Rec SDK TensorFlow path to the PYTHONPATH environment variable after the installation.
    export PYTHONPATH={rec_install_path}:{rec_install_path}/mx_rec:$PYTHONPATH
  6. Install dependencies. If no image is built and development is performed on a PM, the following Python dependencies must be installed:
    1
    pip3.7 install numpy decorator sympy==1.4 cffi==1.12.3 pyyaml pathlib2 grpcio grpcio-tools protobuf==3.20.0 scipy requests mpi4py easydict scikit-learn==0.20.0 attrs
    
    Configure HOROVOD_WITH_MPI and HOROVOD_WITH_TENSORFLOW before installing Horovod. The installation command is as follows:
    HOROVOD_WITH_MPI=1 HOROVOD_WITH_TENSORFLOW=1 pip3.7 install horovod --no-cache-dir
  7. Optional: To use the dynamic capacity expansion function, compile and install the corresponding operator package by referring to (Optional) Installing the Dynamic Capacity Expansion Operator Package of the on-chip memory.
  8. Optional: To use the Hadoop distributed file system, deploy the environment and set up a cluster by referring to Hadoop official documents. Hadoop 2.7.5 is recommended.