Configuration Description

allow_all_zero_ip_listening

• true: All-zero monitoring is allowed.
• false: All-zero monitoring is disabled.

Mandatory. The default value is false.

Whether to allow all-zero monitoring. If the value is false, the program fails to run and an error is reported when all-zero monitoring is used.

deploy_mode

• pd_separate or pd_disaggregation: multi-node prefill-decode disaggregation deployment
• pd_disaggregation_single_container: single-node prefill-decode disaggregation deployment
• single_node: no-identity deployment

Mandatory. The default value is pd_separate.

Deployment mode.

initial_dist_server_port

[1024, 65535]

Start port number of the D node server instance configured in the DeepSeek large-scale EP distributed parallel scenario. This parameter does not take effect in other scenarios.

cluster_port

-

Optional. The default value is 8899.

Port of the MindCluster ClusterD gRPC server.

process_manager: process status backup function

to_file

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable the process status backup function.

If this function is enabled, the process status backup file is automatically loaded when the program is started. During the running of the program, the process status backup file is automatically generated. For details about the backup file, see Table 2.

file_path

-

This parameter is mandatory when to_file is enabled.

Path of the process status backup file. The path must exist and be readable. Ensure that the path is secure and reliable. The process status backup file is automatically created in the path during running. If process files are stored in a temporary path in a container, the files will be lost after the Controller container is restarted. In this case, you need to restart the entire cluster to restore services.

If the identity decision-making configuration is modified, delete the process status backup file and restart the entire cluster. Otherwise, the historical identity decision-making result is inherited.

cluster_status: cluster service status export function

to_file

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable the cluster service status export function.

If this function is enabled, the cluster status is output to a file during program running. For details about the cluster service status output file, see Table 3.

file_path

-

This parameter is mandatory when to_file is enabled.

Path of the cluster status export file. The path must exist and be readable. Ensure that the path is secure and reliable. The file is automatically created in the path during running.

ccae: end-to-end O&M management function

By default, the ms_controller.json configuration file does not contain CCAE-related parameters. If you need to enable the CCAE-related parameters, add them as required. The following is an example:

"ccae" : {
    "ip": "",
    "port": 0,
    "kafka_port" : 26329
  },

ip

IPv4 or IPv6

Mandatory when CCAE is enabled.

IP address of the CCAE server.

port

[1024, 65535]

Mandatory when CCAE is enabled.

CCAE server port.

kafka_port

[1024, 65535]

Mandatory when CCAE is enabled.

Port through which logs are reported to CCAE

EnableDynamicAdjustTimeoutConfig: dynamic timeout delay configuration

By default, the ms_controller.json configuration file does not contain this parameter. If you need to enable this parameter, add it as required. The following is an example:

"EnableDynamicAdjustTimeoutConfig" : true,

controller_backup_cfg: configuring master/slave switchover of the Controller

function_sw

• true: enabled
• false: disabled

Whether to enable master/slave switchover of the Controller. The default value is false.

database_server_dns

-

Domain name of the etcd server. The default value is etcd.default.svc.cluster.local.

database_server_port

-

Port number of the etcd server. The default value is 2379.

fault_recovery_func_dict: cluster fault recovery function

lingqu_link

true: enabled

false: disabled

In the Atlas 800I A3 SuperPoD Server environment, this parameter specifies whether to quickly recover inference services after the LingQu network L1-L2 1520 port becomes faulty. The default value is false.

global_rank_table_file_path

-

Mandatory.

Path of the global cluster information table. The file must exist and be readable. Ensure that the path is secure and reliable.

If the environment variable GLOBAL_RANK_TABLE_FILE_PATH is set, the value of the environment variable is read first.

cluster_synchronization_seconds

[1, 65535], in seconds.

Mandatory. The default value is 1.

Interval for synchronizing cluster information.

rank_table_detecting_seconds

[1, 65535], in seconds.

Mandatory. The default value is 5.

Interval for reading the cluster information table.

disappeared_server_waiting_seconds

[1, 65535], in seconds.

Mandatory. The default value is 7200.

Waiting time for deleting a Server from the global cluster information table. When the waiting time arrives, the Server will be deleted.

default_p_rate

[0, 15]

Mandatory. The default value is 0.

Proportion of prefill nodes in prefill-decode disaggregation deployment mode.

• 0: The optimal ratio is automatically determined. The value of default_d_rate must be 0 at the same time.
• Other values: specified ratio of P. The value of default_d_rate must be the same value at the same time, and the sum of default_p_rate and default_d_rate must be less than or equal to 16.

If the environment variable MINDIE_MS_P_RATE is set, the value of the environment variable is read first.

default_d_rate

[0, 15]

Mandatory. The default value is 0.

Proportion of decode nodes in prefill-decode disaggregation deployment mode.

• 0: The optimal ratio is automatically determined. The value of default_p_rate must be 0 at the same time.
• Other values: specified ratio of D. The value of default_p_rate must be the same value at the same time, and the sum of default_d_rate and default_p_rate must be less than or equal to 16.

If the environment variable MINDIE_MS_D_RATE is set, the value of the environment variable is read first.

is_heterogeneous

• false: homogeneous scenario. That is, the Server randomly specifies the PD identity on different hardware devices.

• true: heterogeneous scenario. That is, Atlas 800I A2 inference server (32 GB) is specified as a P node, and Atlas 800I A2 inference server (64 GB) is specified as a D node.

Mandatory. The default value is false.

In the heterogeneous scenario, the hardware_type field must be added to the cluster information table.

server_online_attempt_times

[1, 65535]

Mandatory. The default value is 4.

Number of times that the online status of Server nodes is checked during cluster initialization.

server_online_wait_seconds

[1, 65535], in seconds.

Mandatory. The default value is 5.

Interval for checking the online status of Server nodes during cluster initialization.

init_role_attempt_times

[1, 65535]

Mandatory. The default value is 2.

Number of retries during Server identity initialization. Identity re-delivery is triggered only when the identity is detected as Unknown.

check_role_attempt_times

[1, 65535]

Mandatory. The default value is 300.

Number of checks for Server identity initialization.

check_role_wait_seconds

[1, 65535], in seconds.

Mandatory. The default value is 5.

Interval for checking Server identity initialization.

mindie_server_control_port

[1024, 65535]

Mandatory. The default value is 1026.

Management port of the Server.

mindie_server_port

[1024, 65535]

Mandatory. The default value is 1025.

Data port of the Server.

mindie_server_metric_port

[1024, 65535]

Mandatory. The default value is 1027.

Metrics port of the Server.

controller_alarm_port

[1024, 65535]

Mandatory. The default value is 1027.

Port used when the Controller functions as the alarm server.

mindie_ms_coordinator_port

[1024, 65535]

Mandatory. The default value is 1026.

Management port of the MindIE Coordinator.

node_manager_port

[1024, 65535]

Mandatory. The default value is 1028.

Management port of the MindIE node_manager.

mindie_ms_coordinator_external_port

[1024, 65535]

Mandatory. The default value is 1028.

External port of the MindIE Coordinator.

http_timeout_seconds

[1, 65535], in seconds.

Mandatory. The default value is 5.

Communication timeout interval.

http_retries

[0, 65535]

Mandatory. The default value is 3.

Number of retries upon a communication exception.

role_decision_methods

digs

Mandatory. The default value is digs.

PD identity decision-making algorithm. Currently, only digs is supported.

digs_request_summary_input_length

[1, 65535]

Mandatory. The default value is 3000.

Average input length of an inference request.

After the model and device configurations are determined, this parameter and digs_request_summary_output_length are the main parameters that affect the optimal PD ratio.

• When an inference request has a long input and a short output, the number of prefill instances is greater than that of decode instances.
• When an inference request has a short input and a long output, the number of decode instances is greater than that of prefill instances.

digs_request_summary_output_length

[1, 65535]

Mandatory. The default value is 200.

Average output length of an inference request.

After the model and device configuration are determined, this parameter and digs_request_summary_input_length are the main parameters that affect the optimal PD ratio.

• When an inference request has a long input and a short output, the number of prefill instances is greater than that of decode instances.
• When an inference request has a short input and a long output, the number of decode instances is greater than that of prefill instances.

digs_model_config_path

-

Mandatory. The default value is the configuration file path of the Llama3-70B model.

Model parameter information required by the identity decision-making algorithm. The file must exist and be readable. For details about the format, see the description in the model configuration file.

digs_machine_config_path

-

Mandatory. The default value is the hardware parameter file path of the Atlas 800I A2 inference server.

Machine parameter information required by the identity decision-making algorithm. The file must exist and be readable. For details about the format, see the hardware device file description.

digs_prefill_slo

[1, 65535]

Mandatory. The default value is 1000.

Prefill rate.

digs_decode_slo

[1, 65535]

Mandatory. The default value is 50.

Decode rate.

multi_node_infer_config

multi_node_infer_enable

• false: disabled

• true: enabled

Optional. The default value is false.

This parameter takes effect only in DeepSeek large-scale expert parallelism scenarios. It specifies whether to enable multi-node inference.

p_node_config

-

Configures the P node.

d_node_config

-

Configures the D node.

enable_dist_dp_server

• false: disabled

• true: enabled

Optional. The default value is false.

This parameter takes effect only in DeepSeek large-scale expert parallelism scenarios. It specifies whether to enable the distributed parallel policy.

node_machine_num

[1, 768]

Optional. The default value is 2.

This parameter takes effect only in DeepSeek large-scale expert parallelism scenarios. It specifies the number of prefill/decode instances.

tp_size

[1, 16]

Optional. The default value is 4.

This parameter takes effect only in DeepSeek large-scale expert parallelism scenarios. It specifies the tp data parallel size.

dp_size

[1, 512]

Optional. The default value is 4.

This parameter takes effect only in DeepSeek large-scale expert parallelism scenarios. It specifies the dp data parallel size.

sp_size

[1, 16]

Optional. The default value is 1.

This parameter takes effect only in DeepSeek large-scale expert parallelism scenarios. It specifies the sp data parallel size.

model_type

-

Mandatory. The default value is llama3-70B.

Name of an inference model.

By default, the name of the model reported to CCAE is the value of this parameter. If the name of the model to be reported is the name of the model in use, change the value of this parameter.

transfer_type

D2DTransfer

Mandatory. The default value is D2DTransfer.

Transfer type.

digs_pp

[1, 65535]

Mandatory. The default value is 1.

Number of concurrent jobs.

digs_periodic_role_decision

auto_pd_role_switching_enable

• false: disabled

• true: enabled

Mandatory. The default value is true.

Whether to enable PD node identity switchover.

role_decision_time_period

[3600, 1296000]

Mandatory. The default value is 86400.

Interval for switching the PD node identity, in seconds.

tasks_end_wait_seconds

[1, 65535]

Mandatory. The default value is 300.

Waiting duration after task execution in the PD node identity switchover scenario, in seconds.

http_server: Controller HTTPS server configuration

ip

IPv4 or IPv6

Mandatory. The default value is 127.0.0.1.

IP address of the Controller server, which is used for health check.

If the environment variable POD_IP is set, the value of the environment variable is read first.

port

[1024, 65535]

Mandatory. The default value is 1026.

Controller server port, which is used for health check.

tls_config: certificate configuration

request_coordinator_tls_enable

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable TLS security authentication for the communication API with the Coordinator. You are advised to enable this function to ensure secure communication with the Coordinator. If this function is disabled, high network security risks exist.

request_coordinator_tls_items: Coordinator HTTP client certificate configuration

ca_cert

-

Mandatory when TLS is enabled.

Path of the CA root certificate file of the Coordinator HTTPS client. The file must exist and be readable.

tls_cert

-

Mandatory when TLS is enabled.

Path of the TLS certificate file of the Coordinator HTTPS client. The file must exist and be readable.

tls_key

-

Mandatory when TLS is enabled.

Path of the TLS private key file of the Coordinator HTTPS client. The file must exist and be readable.

tls_passwd

-

Mandatory when TLS is enabled.

Path of the file of the private key password encrypted by KMC on the Coordinator HTTPS client. The file must exist and be readable.

kmc_ksf_master

-

Mandatory when TLS is enabled.

KMC keystore file of the Coordinator HTTPS client encryption password. The file must exist and be readable.

kmc_ksf_standby

-

Mandatory when TLS is enabled.

Standby KMC keystore file of the Coordinator HTTPS client encryption password. The file must exist and be readable.

tls_crl

-

Mandatory when TLS is enabled.

Path of the CRL file for Coordinator HTTPS client verification. The file must exist and be readable. If the value is empty, revocation verification is not performed.

request_server_tls_enable

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable TLS security authentication for the communication API with the Server. You are advised to enable this function to ensure secure communication with the Server. If this function is disabled, high network security risks exist.

request_server_tls_items: Server HTTP client certificate configuration

ca_cert

-

Mandatory when TLS is enabled.

Path of the CA root certificate file of the Server HTTPS client. The file must exist and be readable.

tls_cert

-

Mandatory when TLS is enabled.

Path of the TLS certificate file of the Server HTTPS client. The file must exist and be readable.

tls_key

-

Mandatory when TLS is enabled.

Path of the TLS private key file of the Server HTTPS client. The file must exist and be readable.

tls_passwd

-

Mandatory when TLS is enabled.

Path of the file of the private key password encrypted by KMC on the Server HTTPS client. The file must exist and be readable.

kmc_ksf_master

-

Mandatory when TLS is enabled.

KMC keystore file of the Server HTTPS client encryption password. The file must exist and be readable.

kmc_ksf_standby

-

Mandatory when TLS is enabled.

Standby KMC keystore file of the Server HTTPS client encryption password. The file must exist and be readable.

tls_crl

-

Mandatory when TLS is enabled.

Path of the CRL file for Server HTTPS client verification. The file must exist and be readable. If the value is empty, revocation verification is not performed.

http_server_tls_enable

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable TLS on the Controller HTTPS server. You are advised to enable this function to ensure communication security between the Controller and the client. If this function is disabled, high network security risks exist.

http_server_tls_items attributes: certificate configuration of the Controller HTTP server

ca_cert

-

Mandatory when TLS is enabled.

Path of the CA root certificate file of the Controller HTTPS server. The file must exist and be readable.

tls_cert

-

Mandatory when TLS is enabled.

Path of the TLS certificate file of the Controller HTTPS server. The file must exist and be readable.

tls_key

-

Mandatory when TLS is enabled.

Path of the TLS private key file of the Controller HTTPS server. The file must exist and be readable.

tls_passwd

-

Mandatory when TLS is enabled.

Path of the file of the private key password encrypted by KMC on the Controller HTTPS server. The file must exist and be readable.

kmc_ksf_master

-

Mandatory when TLS is enabled.

KMC keystore file of the Controller HTTPS server encryption password. The file must exist and be readable.

kmc_ksf_standby

-

Mandatory when TLS is enabled.

Standby KMC keystore file of the Controller HTTPS server encryption password. The file must exist and be readable.

tls_crl

-

Mandatory when TLS is enabled.

Path of the CRL file for the Controller HTTPS server to verify the client. The file must exist and be readable. If the value is empty, revocation verification is not performed.

cluster_tls_enable

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable TLS security authentication for the communication API with the MindCluster ClusterD. You are advised to enable this function to ensure secure communication with the MindCluster ClusterD. If this function is disabled, high network security risks exist.

cluster_tls_items attributes: MindCluster ClusterD gRPC server certificate configuration

ca_cert

-

Mandatory when TLS is enabled.

Path of the CA root certificate file of the MindCluster ClusterD gRPC server. The file must exist and be readable.

tls_cert

-

Mandatory when TLS is enabled.

Path of the TLS certificate file of the MindCluster ClusterD gRPC server. The file must exist and be readable.

tls_key

-

Mandatory when TLS is enabled.

Path of the TLS private key file of the MindCluster ClusterD gRPC server. The file must exist and be readable.

tls_passwd

-

Mandatory when TLS is enabled.

Path of the file of the private key password encrypted by KMC on the MindCluster ClusterD gRPC server. The file must exist and be readable.

kmc_ksf_master

-

Mandatory when TLS is enabled.

KMC keystore file of the MindCluster ClusterD gRPC server encryption password. The file must exist and be readable.

kmc_ksf_standby

-

Mandatory when TLS is enabled.

Standby KMC keystore file of the MindCluster ClusterD gRPC server encryption password. The file must exist and be readable.

tls_crl

-

Mandatory when TLS is enabled.

Path of the CRL file for the MindCluster ClusterD gRPC server to verify the client. The file must exist and be readable. If the value is empty, revocation verification is not performed.

etcd_server_tls_enable

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable TLS security authentication for the communication API with the etcd. You are advised to enable this function to ensure secure communication with the etcd. If this function is disabled, high network security risks exist.

etcd_server_tls_items attributes: querying communication certificate configuration of the etcd status information

ca_cert

Prefill-decode disaggregation

Mandatory when TLS is enabled.

Path of the CA root certificate file of the etcd server. The file must exist and be readable.

tls_cert

Prefill-decode disaggregation

Mandatory when TLS is enabled.

Path of the TLS certificate file of the etcd server. The file must exist and be readable.

tls_key

Prefill-decode disaggregation

Mandatory when TLS is enabled.

Path of the TLS private key file of the etcd server. The file must exist and be readable.

tls_passwd

Prefill-decode disaggregation

Mandatory when TLS is enabled.

Path of the file of the private key password encrypted by KMC on the etcd server. The file must exist and be readable.

kmcKsfMaster

Prefill-decode disaggregation

Mandatory when TLS is enabled.

KMC keystore file of the etcd server encryption password. The file must exist and be readable.

kmcKsfStandby

Prefill-decode disaggregation

Mandatory when TLS is enabled.

Standby KMC keystore file of the etcd server encryption password. The file must exist and be readable.

tls_crl

Prefill-decode disaggregation

Mandatory when TLS is enabled.

Path of the CRL file for the etcd server to verify the client. The file must exist and be readable. If the value is empty, revocation verification is not performed.

ccae_tls_enable

• true: enabled
• false: disabled

Optional. The default value is true.

Whether to enable TLS on the CCAE HTTPS server. This parameter is mandatory when CCAE is enabled.

ccae_tls_items attributes: certificate configuration of the CCAE HTTPS server

By default, the ms_controller.json configuration file does not contain CCAE-related parameters. If you need to enable the CCAE-related parameters in the tls_config field, add them as required. The following is an example:

    "ccae_tls_enable": true,
    "ccae_tls_items": {
      "ca_cert" : "",
      "tls_cert": "",
      "tls_key": "",
      "tls_passwd": "",
      "kmc_ksf_master": "",
      "kmc_ksf_standby": "",
      "tls_crl": ""
    }

ca_cert

-

Mandatory when CCAE is enabled.

Path of the CA root certificate file of the CCAE HTTPS server. The file must exist and be readable.

tls_cert

-

Mandatory when CCAE is enabled.

Path of the TLS certificate file for verifying the CCAE HTTPS server. The file must exist and be readable.

tls_key

-

Mandatory when CCAE is enabled.

Path of the TLS private key file for verifying the CCAE HTTPS server. The file must exist and be readable

tls_passwd

-

Mandatory when CCAE is enabled.

Path of the file of the private key password encrypted by KMC for verifying the CCAE HTTPS server. The file must exist and be readable.

kmc_ksf_master

-

Mandatory when CCAE is enabled.

KMC keystore file for verifying the CCAE HTTPS server encryption password. The file must exist and be readable.

kmc_ksf_standby

-

Mandatory when CCAE is enabled.

Standby KMC keystore file for verifying the CCAE HTTPS server encryption password. The file must exist and be readable.

tls_crl

-

Mandatory when CCAE is enabled.

Path of the CRL file for the CCAE HTTPS server to verify the client. The file must exist and be readable. If the value is empty, revocation verification is not performed.

alarm_tls_enable

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable TLS security authentication of the API for communication with the Coordinator alarms. You are advised to enable this function to ensure secure communication with the Coordinator alarms. If this function is disabled, high network security risks exist.

alarm_tls_items attributes: certificate configuration of the Controller functioning as the alarm server

ca_cert

-

Mandatory when TLS is enabled.

Path of the CA root certificate file of the Controller alarm server. The file must exist and be readable.

tls_cert

-

Mandatory when TLS is enabled.

Path of the TLS certificate file of the Controller alarm server. The file must exist and be readable.

tls_key

-

Mandatory when TLS is enabled.

Path of the TLS private key file of the Controller alarm server. The file must exist and be readable.

tls_passwd

-

Mandatory when TLS is enabled.

Path of the file of the private key password encrypted by KMC on the Controller alarm server. The file must exist and be readable.

kmc_ksf_master

-

Mandatory when TLS is enabled.

KMC keystore file of the Controller alarm server encryption password. The file must exist and be readable.

kmc_ksf_standby

-

Mandatory when TLS is enabled.

Standby KMC keystore file of the Controller alarm server encryption password. The file must exist and be readable.

tls_crl

-

Mandatory when TLS is enabled.

Path of the CRL file for the Controller alarm server to verify the client. The file must exist and be readable. If the value is empty, revocation verification is not performed.

external_coordinator_tls_enable

• true: enabled
• false: disabled

Mandatory. The default value is true.

Whether to enable TLS security authentication for the Coordinator external port accessed by the Controller functioning as the client. You are advised to enable this function to ensure secure communication with the Coordinator external port. If this function is disabled, high network security risks exist.

external_coordinator_tls_items attributes: certificate-related configurations of the Coordinator external port accessed by the Controller functioning as the client

ca_cert

-

Mandatory when TLS is enabled.

Path of the CA root certificate file for accessing the Coordinator external port. The file must exist and be readable.

tls_cert

-

Mandatory when TLS is enabled.

Path of the TLS certificate file for accessing the Coordinator external port. The file must exist and be readable.

tls_key

-

Mandatory when TLS is enabled.

Path of the TLS private key file for accessing the Coordinator external port. The file must exist and be readable.

tls_passwd

-

Mandatory when TLS is enabled.

Path of the file of the private key password encrypted by KMC for accessing the Coordinator external port. The file must exist and be readable.

kmc_ksf_master

-

Mandatory when TLS is enabled.

KMC keystore file of the encryption password for accessing the Coordinator external port. The file must exist and be readable.

kmc_ksf_standby

-

Mandatory when TLS is enabled.

Standby KMC keystore file of the encryption password for accessing the Coordinator external port. The file must exist and be readable.

tls_crl

-

Mandatory when TLS is enabled.

Path of the CRL file of client verification for accessing the Coordinator external port. The file must exist and be readable. If the value is empty, revocation verification is not performed.

log_info: log function You are advised to use environment variables to configure logs. For details, see Log Configuration.

log_level

• CRITICAL
• ERROR
• WARN
• INFO
• DEBUG

Optional. The default value is INFO.

Sets the log level.

If the environment variable MINDIEMS_LOG_LEVEL or MINDIE_LOG_LEVEL is set, the value of the environment variable is read first.

to_file

• true: Output logs to a file.
• false: Do not output logs to a file.

Optional. The default value is true.

Whether to output logs to a file.

If the environment variable MINDIE_LOG_TO_FILE is set, the value of the environment variable is read first.

to_stdout

• true: Output logs to the standard output stream.
• false: Do not output logs to the standard output stream.

Optional. The default value is false.

Whether to output logs to the standard output stream.

If the MINDIE_LOG_TO_STDOUT environment variable is set, the value of the environment variable is read first.

run_log_path

-

Optional. The default path is the same as that of the MINDIE_LOG_PATH environment variable. For details, see Log Configuration. If the MINDIE_LOG_PATH environment variable is set, the value of the environment variable is read first. This parameter takes effect when to_file is set to true.

Run log path. The path must exist and be readable. The file is automatically created during running.

operation_log_path

-

Optional. The default path is the same as that of the MINDIE_LOG_PATH environment variable. For details, see Log Configuration. If the MINDIE_LOG_PATH environment variable is set, the value of the environment variable is read first. This parameter takes effect when to_file is set to true.

Audit log path. The path must exist and be readable. The file is automatically created during running.

max_log_str_size

[128, 65535]

Optional. The default value is 4096.

Maximum length of a log.

max_log_file_size

[1, 500], in MB.

Optional. The default value is 20. This parameter takes effect when to_file is set to true.

Maximum size of a log file. If the MINDIE_LOG_ROTATE environment variable is set, the value of the environment variable is read first.

max_log_file_num

[1, 64]

Optional. The default value is 10. This parameter takes effect when to_file is set to true.

Maximum number of log files that can be stored. If the MINDIE_LOG_ROTATE environment variable is set, the value of the environment variable is read first.

MINDIE_MS_CONTROLLER_CONFIG_FILE_PATH

Path for reading the ms_controller configuration file.

GLOBAL_RANK_TABLE_FILE_PATH

Path for reading the cluster information file. For details about the cluster information file, see the global_ranktable.json file in 8.

The priority of this environment variable is higher than that of the global_rank_table_file_path attribute in the ms_controller configuration file.

POD_IP

IP address of the pod where ms_controller is located.

The priority of this environment variable is higher than that of the http_server.ip attribute in the ms_controller configuration file.

MINDIE_MS_P_RATE

Proportion of prefill nodes in prefill-decode disaggregation deployment mode.

• 0: The optimal ratio is automatically determined. The ratio of D must also be 0.
• Other values: specified ratio of P, and the ratio of D must be non-zero at the same time.

The priority of this environment variable is higher than that of the default_p_rate attribute in the ms_controller configuration file.

MINDIE_MS_D_RATE

Proportion of decode nodes in prefill-decode disaggregation deployment mode.

• 0: The optimal ratio is automatically determined. The ratio of P must also be 0.
• Other values: specified ratio of D, and the ratio of P must be non-zero at the same time.

The priority of this environment variable is higher than that of the default_d_rate attribute in the ms_controller configuration file.

MINDIEMS_LOG_LEVEL

You can dynamically set the log level output by the Controller.

The default value is empty. The priority of this environment variable is higher than that of the log_level parameter in Table 1. The log levels are as follows:

• CRITICAL
• ERROR
• WARN
• INFO
• DEBUG

MINDIE_CHECK_INPUTFILES_PERMISSION

You can set whether to check external mounted files, including ms_controller.json, rank_table.json and certificate-related files. The default value is empty, indicating that permission verification is required.

• 0: Permission verification is not performed on external mounted files.
• Other values: Permission verification is performed on external mounted files.

When you use MINDIE_MS_CONTROLLER_CONFIG_FILE_PATH to set the configuration file path, ms_controller.json is used as an external mounted file.

MODEL_ID

Unique ID of the deployed model.

Note: For details about log-related environment variables, see Log Configuration.

hidden_size

Feature dimension of a hidden layer.

• String
• Integer
• double

8129

[0.000001, 2147483647.000000]

initializer_range

Range of the weight parameter initialization.

• String
• Integer
• double

0.02

[0.000001, 2147483647.000000]

intermediate_size

Size of the FFN hidden layer.

• String
• Integer
• double

28672

[0.000001, 2147483647.000000]

max_position_embeddings

Maximum number of positions to be embedded.

• String
• Integer
• double

4096

[0.000001, 2147483647.000000]

num_attention_heads

Number of attention heads.

• String
• Integer
• double

64

[0.000001, 2147483647.000000]

num_hidden_layers

Number of hidden layers.

• String
• Integer
• double

80

[0.000001, 2147483647.000000]

num_key_value_heads

Number of KV heads.

• String
• Integer
• double

8

[0.000001, 2147483647.000000]

torch_dtype

Data type used by PyTorch.

String

float16

• float16
• bfloat16

BW_GB

Inter-card communication bandwidth on a node.

• String
• Integer
• double

392

[0.000001, 2147483647.000000]

BW_RDMA_Gb

Inter-node communication bandwidth.

• String
• Integer
• double

200

[0.000001, 2147483647.000000]

BWeff

Intra-node communication bandwidth efficiency.

• String
• Integer
• double

0.5

[0.000001, 1.000000]

TFLOPS

Computing power of a single card.

• String
• Integer
• double

246

[0.000001, 2147483647.000000]

TFLOPSeff

Computing efficiency.

• String
• Integer
• double

0.5

[0.000001, 1.000000]

MBW_TB

NPU memory access bandwidth.

• String
• Integer
• double

0.8

[0.000001, 2147483647.000000]

MBW_TBeff

NPU memory access bandwidth efficiency.

• String
• Integer
• double

0.3

[0.000001, 1.000000]

alpha

Collective communication start delay.

• String
• Integer
• double

2

[0.000001, 2147483647.000000]

MEMCapacity

Graphics memory size (GB).

• String
• Integer
• double

32

[0.000001, 2147483647.000000]

eta_OOM

OOM threshold of the graphics memory.

• String
• Integer
• double

0.9

[0.000001, 1.000000]

staticTransferDelay

Static delay of inter-node communication.

• String
• Integer
• double

0.00001

[0.000001, 2147483647.000000]