Service Metric API (JSON Format)

Function

Obtains the dynamic average values of TTFT and TBT requests (average value of nearly 1000 requests by default), number of requests that are being executed, number of requests that are waiting, and number of remaining NPU blocks during the inference service.

Format

Operation type: GET

URL: https://{ip}:{port}/metrics-json

• The {ip} field preferentially reads the environment variable MIES_CONTAINER_MANAGEMENT_IP. If the environment variable does not exist, managementIpAddress in the configuration file is used. If managementIpAddress does not exist in the configuration file, ipAddress in the configuration file is used.
• The {port} field preferentially reads the managementPort parameter in the configuration file. If the configuration file does not contain the managementPort parameter, the port parameter in the configuration file is used.

Request Parameters

None

Usage Example

Request example:

GET https://{ip}:{port}/metrics-json

Response example:

{
    "resultType": "vector",
    "result": [
        {
            "metric": [
                {
                    "__name__": "TTFT",
                    "job": "node",
                    "instance": "127.0.0.2:1026"
                }
            ],
            "value": "0"
        },
        {
            "metric": [
                {
                    "__name__": "TBT",
                    "job": "node",
                    "instance": "127.0.0.2:1026"
                }
            ],
            "value": "0"
        },
        {
            "metric": [
                {
                    "__name__": "waitingInferRequestNum",
                    "job": "node",
                    "instance": "127.0.0.2:1026"
                }
            ],
            "value": "0"
        },
        {
            "metric": [
                {
                    "__name__": "processingInferRequestNum",
                    "job": "node",
                    "instance": "127.0.0.2:1026"
                }
            ],
            "value": "0"
        },
        {
            "metric": [
                {
                    "__name__": "remainBlocks",
                    "job": "node",
                    "instance": "127.0.0.2:1026"
                }
            ],
            "value": "1024"
        }
    ]
}

Response status code: 200

Output Description