Build and Deployment

Custom Operator Build and Packaging Project

The HCCL code repository provides a custom operator build and packaging project. You can develop your project by referring to the sample of the custom Send/Receive communication operators (here). The directory structure is as follows:

├── build.sh                                # Entry of the build project in the root directory of the HCCL code repository
├── CMakeLists.txt                          # Build configuration file in the root directory of the HCCL code repository
├── cmake/                                  # CMake function
│   ├── config.cmake
│   ├── func.cmake
│   ├── package.cmake
│   └── makeself_custom.cmake
├── scripts/
│   ├── custom/
│   │   └── install.sh                     # Installation script of the custom operator package
│   └── sign/
│       └── add_header_sign.py              # Signature script of the AICPU operator package
└── examples/04_custom_ops_p2p               # Custom operator project directory
    ├── CMakeLists.txt                       # Build configuration file of the custom operator
    ├── op_host/
    │   ├── send.cc                         # Source file for implementing the HcclSendCustom operator
    │   ├── recv.cc                         # Source file for implementing the HcclRecvCustom operator
    │   ├── load_kernel.cc                  # Logic for loading the AICPU kernel on the host
    │   └── launch_kernel.cc                # Logic for launching the AICPU kernel on the host
    ├── op_kernel_aicpu/
    │   ├── libp2p_aicpu_kernel.json        # Description file of the AICPU kernel operator
    │   ├── aicpu_kernel.cc                 # AICPU kernel implementation logic
    │   └── exec_op.cc                      # AICPU operator orchestration logic
    ├── inc/
    │   ├── hccl_custom_p2p.h               # API header file of the custom Send/Receive operators
    │   ├── common.h                        # Common header file
    │   └── log.h                           # Log macro definition
    ├── scripts/
    │   └── hccl_custom_p2p_check_cfg.xml   # Signature configuration file
    └── testcase/
        ├── main.cc                          # Test sample source file
        └── Makefile                         # Build configuration file

You are advised to store code files based on the preceding directory structure. Pay attention to the following:

1. Place the header file of the custom operator in the inc folder.
2. Place the implementation code on the host in the op_host folder, and the implementation code on the AICPU in the op_kernel_aicpu folder.
3. Adjust the content of the CMakeLists.txt file as required.

Building the Custom Operator Package

1. Set the CANN environment variable.

   source /usr/local/Ascend/cann/set_env.sh

   /usr/local/Ascend indicates the default CANN installation path for the root user. For a regular user or to a custom path, replace it with the actual path.

2. Download the HCCL code repository.

   git clone https://gitcode.com/cann/hccl.git

3. Build the custom operator package.

   bash build.sh --vendor=<vendor> --ops=<ops> --custom_ops_path=<ops_project_path>

   Where:

   • <vendor>: custom operator package identification information, which must be unique.
   • <ops>: custom operator name, which must be unique.
   • <ops_project_path>: root directory of the custom operator project, for example, ./examples/04_custom_ops_p2p in Custom Operator Build and Packaging Project. It can be set to an absolute path or a relative path.
   After the build is complete, the custom operator installation package cann-hccl_custom_<ops>_linux-<arch>.run is generated in the build_out directory of the current directory.

   • <ops>: operator name specified using the --ops option when the custom operator package is built.
   • <arch>: system architecture of the current build environment.

Deploying the Custom Operator Package

Run the following command to perform installation:

./build_out/cann-hccl_custom_<ops>_linux-<arch>.run --install

The installation information of the custom operator package is as follows:

• Header file: ${ASCEND_HOME_PATH}/opp/vendors/<vendor>/include
• Dynamic library: ${ASCEND_HOME_PATH}/opp/vendors/<vendor>/lib64
• AICPU operator information library file: ${ASCEND_HOME_PATH}/opp/vendors/<vendor>/aicpu/config
• AICPU operator package: ${ASCEND_HOME_PATH}/opp/vendors/<vendor>/aicpu/kernel
• Installation script: ${ASCEND_HOME_PATH}/opp/vendors/<vendor>/scripts/install.sh

<vendor> indicates the operator identification information specified using the --vendor option when the custom operator package is built.