Prometheus Metrics Interface
Provides a Metrics interface for Prometheus to call and integrate.
GET http://ip:port/metrics
For security reasons, NPU Exporter enables the container-level port (default 8082) by default. The request IP is the Kubernetes container IP. When the K8s network plugin is Calico, the network policy is set to allow access from applications with the label app=prometheus.
None
Data is returned in the Prometheus-specific format. The relevant data information is shown below for reference only, and the actual echo shall prevail. For detailed description of the data information, see the following text or Data Information Description.xlsx. The data information that comes with Prometheus does not need attention and is not shown here. Some data information is only supported by certain product forms, and the actually reported data information shall prevail.
...
# HELP machine_npu_nums Amount of npu installed on the machine.
# TYPE machine_npu_nums gauge
machine_npu_nums 8
# HELP machine_card_nums Amount of card installed on the machine.
# TYPE machine_card_nums gauge
machine_card_nums 8
# HELP npu_chip_info_aicore_current_freq the npu ai core current frequency, unit is 'MHz'
# TYPE npu_chip_info_aicore_current_freq gauge
npu_chip_info_aicore_current_freq{container_name="",id="0",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:61:00.0",pod_name="",vdie_id="185011D4-21104518-A0C4ED94-14CC040A-56102003"} 1000 1723621883587
npu_chip_info_aicore_current_freq{container_name="",id="1",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:DB:00.0",pod_name="",vdie_id="185011D4-21E04718-93B2ED94-14CC040A-BF102003"} 1000 1723621883932
npu_chip_info_aicore_current_freq{container_name="",id="2",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:B2:00.0",pod_name="",vdie_id="185011D4-20C02418-59D4ED94-14CC040A-F9102003"} 1000 1723621884277
npu_chip_info_aicore_current_freq{container_name="",id="3",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:3E:00.0",pod_name="",vdie_id="185011D4-21502C18-0464ED94-14CC040A-6E102003"} 1000 1723621884682
npu_chip_info_aicore_current_freq{container_name="",id="4",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:60:00.0",pod_name="",vdie_id="185011D4-21A02418-64946D94-14CC040A-F8102003"} 1000 1723621885026
npu_chip_info_aicore_current_freq{container_name="",id="5",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:DA:00.0",pod_name="",vdie_id="185011DC-21F02B18-C4B66D94-14CC040A-57102003"} 1000 1723621885385
npu_chip_info_aicore_current_freq{container_name="",id="6",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:B1:00.0",pod_name="",vdie_id="185011D4-20602118-14646D94-14CC040A-8A102003"} 1000 1723621885784
npu_chip_info_aicore_current_freq{container_name="",id="7",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:3D:00.0",pod_name="",vdie_id="185011D4-21504C18-10656D94-14CC040A-6B102003"} 1000 1723621886131
# HELP npu_chip_info_bandwidth_rx the npu interface receive speed, unit is 'MB/s'
# TYPE npu_chip_info_bandwidth_rx gauge
npu_chip_info_bandwidth_rx{container_name="",id="0",model_name="910A-Ascend-V1",namespace="",pcie_bus_info="0000:61:00.0",pod_name="",vdie_id="185011D4-21104518-A0C4ED94-14CC040A-56102003"} 0 1723621883587
...This interface supports querying default metric groups and custom metric groups. For details on customizing metric groups, see Custom Metric Development. The default metric groups include the following sections. The collection and reporting of metric groups are controlled by switches in the configuration file. If a switch is configured as enabled, the corresponding metric group will be collected and reported; if a switch is configured as disabled, the corresponding metric group will not be collected and reported.
- Version Data Information
- Basic Node Information
- NPU Data Information
- Utilization Data Information
- vNPU Data Information
- Network Data Information
- DDR Data Information
- On-Chip Memory Data Information
- HCCS Data Information
- PCIe Data Information
- RoCE Data Information
- SIO Data Information
- Optical Module Data Information
- UB Data Information
To understand the description of labels in the above data information, see Data Label Information.
- If the process runs on the host and the Pod does not use an NPU, the values of
pod_name,container_name, andnamespacewill be empty. - NPU Exporter obtains the corresponding information by calling the underlying HDK interface. For the HDK interfaces called for data information, refer to HDK Interface Call.
- If NPU Exporter does not support the product form or fails to call the HDK interface when querying a certain data information item, this data information will not be reported.
Table 1 Version data information
|
Table 2 Basic node information
| Category | Data Information Name | Data Information Description | Data Label | Field Type | Unit | Supported Product Form |
|---|---|---|---|---|---|---|
| nodeBase | node_base_info | Basic node Information |
| String | 1: Placeholder character, having no actual meaning. |
|
Table 3 NPU data information
Table 4 Utilization data information
| Category | Data Information Name | Data Information Description | Data Label | Unit | Supported Product Form |
|---|---|---|---|---|---|
| utilization | npu_chip_info_utilization | Ascend AI processor's AICore utilization | Label 1 | % |
|
| utilization | container_npu_utilization | AICore utilization of NPU with container information Note: Telegraf does not support reporting this metric | Label 1 | % |
|
| utilization | npu_chip_info_vector_utilization | Ascend AI processor's AIVector utilization | Label 1 | % |
|
| utilization | npu_chip_info_cube_utilization | Ascend AI processor's AICube utilization | Label 1 | % |
|
| utilization | npu_chip_info_overall_utilization | Overall utilization of the Ascend AI Processor | Label 1 | % |
|
Table 5 vNPU data information
Table 6 Network data information
Table 7 DDR data information
|
|||||
Table 8 On-chip memory data information
Table 9 HCCS data information
Table 10 PCIe data information
Table 11 RoCE data information
Table 12 SIO data information
|
|||||
Table 13 Optical module data information
Table 14 UB data information
The data labels used in the above tables are described as follows.
Table 15 Data label information
NPU Exporter obtains corresponding information by calling the underlying HDK interfaces. For more details, refer to HDK Interfaces Called by NPU Exporter.xlsx. To find the HDK interface corresponding to the data information, refer to the following steps.
Log in to the Ascend Computing Documentation center, select and click the corresponding product name to enter the documentation page. For example, to search for the Atlas 800I A2 Inference Server documentation, click "Atlas 800I A2".
Find "Secondary Development" in the left navigation bar, and select the corresponding document based on the interface type.
- For DCMIs, select "API Reference" and click DCMI API Reference.
- For HCCN Tool, select "API Reference" and click Atlas A2 Center Inference and Training Hardware HCCN Tool API Reference.
In the search bar on the document homepage, directly search for the corresponding interface name or keyword to obtain relevant information about the interface.
Table 16 Status codes
| Status Code | Meaning |
|---|---|
| 200 | Normal status |
| 307 | Temporary redirection |
| 500 | Internal server error |