[object Object]

The function call capability of an LLM, also known as the tool use capability, allows the model to call external tools, thereby extending the application scope of the model. Specifically, it allows a model to directly call external functions or APIs to obtain the capability of executing specific tasks, obtaining real-time data, or enhancing decision-making. This feature not only broadens the application scope of models to tackle more complex and specific problems, but also enhances the practicality and interactivity of models. It establishes an efficient connection between LLMs and external systems, offering users richer and more personalized services.

The following uses tool use to represent the function call feature.

Figure 1 Tool use process[object Object][object Object]

[object Object]

1. The upper-layer application sends system prompts and user inputs to an LLM, and provides the tool set available for model execution.
2. Based on system prompts and user inputs, the model determines whether to directly return an answer or select one or more functions from the tool set provided by the application. If a tool is selected, return the tool name and tool parameters to the upper-layer application.
3. The upper-layer application parses the response from the inference engine, extracts the information about the selected tool, and executes the function selected by the model to obtain the tool use result.
4. The upper-layer application uses the tool use result to construct a prompt for generating an answer, and sends the prompt to the model again to request to generate a final answer.
5. The model summarizes information based on the tool use result, generates an answer, and returns the answer.

[object Object]

• The Atlas 800I A2 inference server, Atlas 800I A3 SuperPoD server, and Atlas 300I Duo inference card support this feature.
• Currently, the ChatGLM3-6B, Qwen3-32B, Qwen3-235B-A22B, Qwen3-30B-A3B, DeepSeek-R1-0528, Qwen2.5-Instruct, and DeepSeek-V3.1 series models support this feature.
• To use the function call feature, parameters listed in must be configured for the DeepSeek-V3.1 series models. For other models, these parameters do not need to be configured.
• Currently, only the OpenAI chat API is supported.
• The function call feature can be used with the quantization, long sequence, multi-server inference, prefill-decode disaggregation, MoE, Multi-LoRA, SplitFuse, parallel decoding, expert parallelism, MTP, prefix cache, reasoning analysis (excluding DeepSeek-V3.1, which does not support enabling both function call and reasoning analysis in a single request), tensor parallelism, and MLA features. However, the SplitFuse, parallel decoding, and MTP features cannot be used with the function call feature in streaming inference.
• Currently, the function call feature does not support the postprocessing parameters [object Object], [object Object], [object Object], [object Object], [object Object], and [object Object]. If the temperature is set to a large value (which will increase sampling randomness), the stability of function call triggering may be affected.
• The function call feature supports non-streaming inference. Only the function call feature of the Qwen3-32B, Qwen3-235B-A22B, Qwen3-30B-A3B, and DeepSeek-R1-0528 models supports streaming inference.
• When a request packet contains the function call feature, the current version supports a maximum JSON nesting depth of 10 levels.

[object Object]

lists the parameters that can be configured when the function call feature is used.

Table 1 Supplementary parameters for the function call feature: models parameter in ModelConfig [object Object][object Object]

[object Object]undefined

Table 2 Registered ToolsCallProcessor [object Object][object Object]

[object Object]undefined[object Object]

The following uses DeepSeek-V3.1 as an example to describe how to use the function call feature.

1. Open the [object Object] file of the server.

   • Installation using the [object Object] package:
   [object Object]
   • Installation using the [object Object] package:
   [object Object]

2. Set serving parameters.

   Add the [object Object] and [object Object] fields to the [object Object] file of the server based on . For details about the serving parameters, see . The following is an example of parameter configuration:

   [object Object]

   [object Object]

3. Start the service.

   • Installation using the [object Object] package:
   [object Object]
   • Installation using the [object Object] package:
   [object Object]

4. [object Object][object Object]Send a request to the service. For details about the parameters, see "Serving APIs" > "RESTful APIs on the EndPoint Service Plane" > "Compatible with OpenAI APIs" > "Inference API" in MindIE Motor Development Guide

   Request example:

   [object Object]

   Response example:

   [object Object]

   Call the related local tool based on [object Object] returned by the model, use the assistant role to associate [object Object] with the ID returned by the API in , use the tool role to associate the tool execution result with the ID returned by the API in , and send a request to the LLM.

   [object Object]