Service Startup

Setting Up the Environment

Install and deploy the environment by referring to MindIE Installation Guide and set parameters as required by referring to "Core Concepts and Configurations" > "Configuration Parameters (Serving)" in MindIE LLM Development Guide.

Procedure

Server supports service applications compatible with third-party framework APIs such as Triton, OpenAI, TGI, and vLLM. You are advised to enable HTTPS communication and configure the service certificate, private key, and other certificate files required for HTTPS communication. For details, see "MindIE Configuration" > "Server Configuration" > "Single-Node Inference" in MindIE Installation Guide.

The default IP address and port number for Server startup are https://127.0.0.1:1025. You can modify ipAddress and port in the config.json file to configure the startup IP address and port number.
Server can implement functions such as service status query, model information query, and text/streaming inference.

HTTP lacks the necessary security mechanisms and is vulnerable to data leakage, data tampering, and man-in-the-middle attacks. Exercise caution when using HTTP.

Start the service in either of the following ways.
The startup command must be run in the /{MindIE installation directory}/latest/mindie-service directory.

Before starting the service, you are advised to use the pre-check tool of MindStudio to verify the fields in the configuration file and check the validity of the configuration. For details, see Link.
- Method 1 (recommended): Start the service using a background process. After the service is started in background process mode, the process is retained when the window is closed.
```
nohup ./bin/mindieservice_daemon > output.log 2>&1 &
```
  If the following information is printed in the file captured by the standard output stream, the startup is successful:
```
Daemon start success!
```
- Method 2: Directly start the service.
```
./bin/mindieservice_daemon
```
  If the following information is displayed, the service is started successfully.
  1
  Daemon start success!
- According to security requirements, the permission on the bin directory is 550, and the directory does not have the write permission. However, during inference, the operator generates the kernel_meta folder in the current directory, which requires the write permission. Therefore, mindieservice_daemon cannot be directly started in the bin directory.
- Ascend-CANN-Toolkit generates the kernel_meta_temp_xxxx directory in the directory where the service is started. This directory stores the CCE file of the operator. Therefore, you need to start the inference service in the directory on which the current user has the write permission (for example, Ascend-mindie-server_{version}_linux-{arch} or a temporary directory in Ascend-mindie-server_{version}_linux-{arch}).
- To switch to another user, run the rm -f /dev/shm/* command to delete the shared files created by the previous user. This prevents inference failure in case the new user does not have the read and write permissions on the shared files created by the previous user.
- The output.log file captured by the standard output stream supports user-defined files and paths.
You can use an HTTPS client (Linux curl command or Postman tool) to send HTTPS requests. The following uses Linux curl command as an example.
Open a new window and run the following command to send a request, for example, to display the current model list:
```
curl -H "Accept: application/json" -H "Content-type: application/json" --cacert ca.pem --cert client.pem  --key client.key.pem -X GET https://127.0.0.1:1025/v1/models
```
- --cacert: path of the signature verification certificate.
- ca.pem: signature verification certificate or root certificate of the Server server certificate.
- --cert: path of the client certificate file.
- client.pem: client certificate.
- --key: path of the client private key file.
- client.key.pem: private key of the client certificate. (The private key is not encrypted. You are advised to use an encrypted key.)
You need to modify the parameters according to the actual situation.

Parent topic: Quick Start