ClusterInfo and IpInfo
ClusterInfo and IpInfo are defined in ${INSTALL_DIR}/include/ge/llm_engine_types.h. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann. The details are as follows:
1 2 3 4 5 6 7 8 9 10 11 | struct ClusterInfo { uint64_t remote_cluster_id = 0U; // cluster_id of the peer LLM Engine. int32_t remote_role_type = 0; // role_type of the peer LLM Engine. The value 0 indicates the full LLM Engine, while the value 1 indicates the incremental LLM Engine. std::vector<IpInfo> local_ip_infos; // IP address of the local LLM Engine device. std::vector<IpInfo> remote_ip_infos; // IP address of the peer LLM Engine device. } struct IpInfo { uint32_t ip = 0U; // uint32_t value converted from the little-endian IP address. uint16_t port = 0U; // port number, which does not need to be specified on the client side but needs to be specified on the server (HCCL specifies a fixed port for users to fill in). } |
Parent topic: APIs to Be Deprecated