使用样例

限制与约束

Atlas 800I A2 推理服务器和Atlas 300I Duo 推理卡硬件支持此特性。
当前ChatGLM3-6B、Qwen2.5系列模型支持此特性。
使用Function Call特性无需配置额外参数。
当前仅支持OpenAI接口，且不与其他特性叠加使用。
Function Call特性暂不支持流式推理。

操作步骤

本节以使用ChatGLM3-6B为例，介绍Function Call如何使用，使用Function Call时无需配置额外参数。

进入mindie-service安装目录并启动服务。

cd {MindIE安装目录}/latest/mindie-service/
./bin/mindieservice_daemon

向服务发送请求，参数说明见推理接口章节。

请求样例：

curl -H "Accept: application/json" -H "Content-type: application/json" --cacert ca.pem --cert client.pem  --key client.key.pem -X POST -d '{
    "model": "chatglm3-6b",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful customer support assistant. Use the supplied tools to assist the user."
        },
        {
            "role": "user",
            "content": "Hi, can you tell me the delivery date for my order?  my order number is 999888"
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_delivery_date",
                "description": "Get the delivery date for a customer\u0027s order. Call this whenever you need to know the delivery date, for example when a customer asks \u0027Where is my package\u0027",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "order_id": {
                            "type": "string",
                            "description": "The customer\u0027s order ID."
                        }
                    },
                    "required": [
                        "order_id"
                    ],
                    "additionalProperties": false
                }
            }
        }
    ],
    "tool_choice": "required",
    "stream": false
}'  https://127.0.0.1:1025/v1/chat/completions

响应样例：

{
    "id": "chatcmpl-123",
    "object": "chat.completion",
    "created": 1677652288,
    "model": "chatglm3-6b",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "",
                "tool_calls": [
                    {
                        "function": {
                            "arguments": "{\"order_id\": \"999888\"}",
                            "name": "get_delivery_date"
                        },
                        "id": "call_JwmTNF3O",
                        "type": "function"
                    }
                ]
            },
            "finish_reason": "tool_calls"
        }
    ],
    "usage": {
        "prompt_tokens": 226,
        "completion_tokens": 122,
        "total_tokens": 348
    },
    "prefill_time": 200,
    "decode_time_arr": [56, 28, 28, 28, 28, ..., 28, 32, 28, 28, 41, 28, 25, 28]
}

根据模型返回的tool_calls调用相关的本地工具，使用assistant角色关联2中接口返回的tool_calls和id，并使用tool角色关联工具执行的结果和2中接口返回的id，向大模型发送请求。

curl -H "Accept: application/json" -H "Content-type: application/json" --cacert ca.pem --cert client.pem  --key client.key.pem -X POST -d '{
    "model": "chatglm3-6b",
    "messages": [
        {
            "role": "user",
            "content": "Hi, can you tell me the delivery date for my order?  my order number is 999888"
        },
        {
            "role": "assistant",
            "content": "",
            "tool_calls": [
                {
                    "function": {
                        "arguments": "{\"order_id\": \"999888\"}",
                        "name": "get_delivery_date"
                    },
                    "id": "call_JwmTNF3O",
                    "type": "function"
                }
            ]
        },
        {
            "role": "tool",
            "content": "the delivery date is 2024.09.10.",
            "tool_call_id": "call_JwmTNF3O"
        }
    ],
    "stream": false,
    "max_tokens": 4096
}' https://127.0.0.1:1025/v1/chat/completions

父主题： Function Call