RESTful响应状态码

200

success

OK

-

400

Bad Request

{

"type": "Bad Request",

"error": error message，详见错误信息列表

}

• "Force P release link failed."
• "DMI inference requests can be sent only to the prefill node."
• "DMI request must have req-type, req-id, d-target headers."
• "The length of req-id cannot exceed 1024."
• "req-type must be prefill but got {reqType}."
• "d-target must be an IPv4 address."
• "d-target should not be itself."
• "is-recompute is an optional parameter. When it is set, it must be true or false."
• "Non DMI does not support dresult request."
• "Only the Decode node supports dresult but you send to " + {pdRole}."

404

NotFoundError

{

"type": "NotFoundError",

"error": error message，详见错误信息列表

}

• "Stop request id {stopReqId} not found."
• "Max wait time is {CV_WAIT_TIME}, input is invalid or too long."
• "Wait time should be in range of [1, {CV_WAIT_TIME}], input is not valid."
• "Model {modelName} not found."

422`

Input Validation Error

{

"type": "Input Validation Error",

"error": error message，详见错误信息列表

}

• "Failed to parse context to infer param."
• "Failed to parse request context to json"
• "Request contains not model or model null."
• "Request param model must be string."
• "Request param model must be not empty."
• "Model not found."
• "Tools not array."
• "Tool param invalid, tool type not exist."
• "Tool param invalid tool type."
• "Tool param invalid tool string type."
• "Tool param not function type."
• "Tool param invalid, tool function not exist."
• "Tool param invalid function object."
• "Function object invalid, function name not exist."
• "Function object invalid name type."
• "The name of function must be a-z, A-Z, 0-9, underscores and dashs within max length of 64"
• "tool_choice string not in [none, auto, required]."
• "tool_choice not string or object."
• "Messages MUST be a string for recompute."
• "Request param contains not messages or messages null."
• "Request param messages not arr or arr is empty."
• "Request param contains not role or role null."
• "Request param contains role must be system, assistant, user or tool."
• "Request param contains not tool_call_id or tool_call_id null while role is tool."
• "Request param tool_call_id len should not be 0 while role is tool."
• "The type of tool_call_id is abnormal."
• "Tool call param contains no id."
• "The type of the tool. Currently, only function is supported."
• "Tool call param contains no function that the model called."
• "Tool call param contains no function name."
• "Tool call param function name expected to be a string."
• "Tool call param contains no function arguments."
• "Tool call param function arguments expected to be a string."
• "Request param contains not content or content null."
• "Request param content len should not be 0."
• "Request param contains no tool_call_id."
• "Tools is empty"
• "Messages parsed not is a array"
• "Messages len not in (0, {MAX_INPUTS_NUM}], but the length of inputs is {utf16.length()}"
• "Request param contains not messages or messages null"
• "Request param messages not arr or arr is empty"
• "Request param contains not role or role null"
• "Request param contains role must be system, assistant, user or tool"
• "Request param contains not content or content null"
• "Request param content len should not be 0"
• "Inputs arr is empty."
• "Request param image_url null."
• "Request param audio_url null."
• "Request param video_url null."
• "The number of multimodal url. should be no more than {MAX_MULTIMODAL_URL_NUM}."
• "The type of inputs is abnormal."
• "Failed to get token from input: {msg}."
• "Unknown error."
• "Invalid input prompt length {prompt.length()}"
• "Cant find pid memory index {pid}."
• "Cant find share memory in encode"
• "Encode cast buffer header failed."
• "Tokenizer encode wait sub process timeout."
• "Invalid output token length " + std::to_string(tokenIdSize)
• "Encode cast buffer to int64 failed."
• "Encode memset_s failed."
• "[InferTokenizer::DownloadUrl] download fail: {e.what()}."
• "[InferTokenizer::DownloadUrl] get unknown error."
• "Messages token length must be in(0, {MAX_TOKENS_NUM)], but got {reqTokens.size()}"
• "req should contain 'inputs' and the type should be array."
• "Inputs count must be 1."
• "Cannot find name in inputs or its type is not string."
• "The length of name set in inputs exceeds 256."
• "Cannot find shape in inputs or its type is not array"
• "Shape element must be integer type."
• "Shape must be at most two-dimensional, and the last element's value must be in (0, {MAX_TOKENS_NUM}]."
• "Can not find datatype in inputs or its type is not string."
• "Unsupported datatype, got {typeName}."
• "Cannot find data in inputs or its type is not array."
• "Data element must be no more than max token id length="MAX_TOKENS_NUM"."
• "Data size should be {inputShape[0][1]}, got {body["data"].size()}."
• "Data element must be integer type."
• "The id can contain only digits, letters, underscores(_), hyphens(-) and no more than {MAX_INPUT_ID_LENGTH} words in length."
• "Req should contain 'outputs' and the type should be array."
• "Outputs size should be {inputDataType.size()}, got {outputs.size()}."
• "Outputs name must not be null."
• "Input conflict."
• "Inputs must not be null."
• "The type of inputs is abnormal."
• "Inputs must be necessary and data type must be string and length in (0, {MAX_INPUTS_NUM}], but the length of inputs is {utf16.length()}."
• "Req should contain 'inputs' and the type should be string."
• "The text_input not found."
• "text_input arr is empty."
• "The type of text_input is abnormal for id = {logId}."
• "text_input must be necessary and data type must be string and length in (0, {MAX_INPUTS_NUM}], but the length of inputs is {utf16.length()}"
• "text_input token length must be in (0, {MAX_TOKENS_NUM}], but got {reqTokens.size()}."
• "Contains not prompt or prompt null."
• "Prompt arr is empty."
• "The type of prompt is abnormal."
• "Prompt must be necessary and data type must be string and length in (0, {MAX_INPUTS_NUM}], but the length of inputs is {utf16.length()}."
• "Param stream not boolean."
• "repetition_penalty must be in (0.0, 2.0], got {jsonObj[key]}."
• "Input validation error: `stop` item count more than {MAX_STOP_STRING_NUM}, but got {stopStrings.size()}"
• "Input validation error: `stop` must be list[string] if list, and item length in [1, {MAX_STOP_STRING_LEN}] with total <= {MAX_TOTAL_STOP}."
• "Input validation error: length of `stop` must be in [1, {MAX_STOP_STRING_LEN} but got {GetU16Str(stopStrings).length()}."
• "Param stop must be string or list[string]."
• "presence_penalty not float number."
• "presence_penalty not in [-2.0, 2.0]."
• "frequency_penalty not number."
• "frequency_penalty not in [-2.0, 2.0]."
• "Request param include_stop_str_in_output is not support in dmi mode."
• "max_tokens not number_integer."
• "max_tokens must be (0, 2147483647], got {jsonObj[key]}."
• "temperature must be in [0.0,2.0], got {inputNum}."
• "top_k must be in [0,2147483647], got {jsonObj[key]}."
• "stop_token_ids not array."
• "stop_token_ids not list[int]."
• "top must be float."
• "top_p must be in (0.0, 1.0], got {jsonObj[key]}."
• "Check open ai req parameter error"
• "typical_p must be in (0.0,1.0], got {jsonObj[key]}."
• "Check self develop req parameter error."
• "truncate must be in (0,2147483647], got {body[key]}."
• "The decoder_input_details must not be true when stream is true."
• "The format of adapter_id is invalid."
• "Check tgi req parameter error."
• "firstTokenCost exceeds the max value of size_t type."
• "The decodeTime length is invalid."
• "The decodeTime exceeds the max value of size_t type."
• "Check triton text req parameter error"
• "repetition_penalty must be strictly positive, got {jsonObj[key]}."
• "seed must be in (0, 18446744073709551615], got {jsonObj[key]}."
• "temperature must be in (0.0, {MAX_FLOAT_VALUE}], got {jsonObj[key]}."
• "top_k must be in [0,2147483647], got {jsonObj[key]}."
• "top_p must be in (0.0,1.0], got {jsonObj[key]}."
• "batch_size must be in (0,{MAX_INT32_VALUE}], got {jsonObj[key]}."
• "max_new_tokens must be in (0, 2147483647], got {jsonObj[key]}."
• "priority must be in [1,5], got {jsonObj[key]}."
• "timeout must be strictly positive and small than 1 hour, got {jsonObj[key]}."
• "Check triton token req parameter error."
• "seed is not number unsigned."
• "Input validation error: param stop must be string or list[string]."
• "model must be string type."
• "repetition_penalty must be in (0.0,2.0], got {jsonObj[key]}."
• "Check vllm req parameter error."
• "Failed to parse context to json body."
• "Not found {key}."
• "key must not be null."
• "key must be {type} type."
• "The id no more than {MAX_INPUT_ID_LENGTH} words in length."
• "Stop request id = {stopReqId} is invalid."
• "Req body converts to json fail. Reset to previous node status."
• "Error: `inputs` or `prompt` must be necessary and data type must be string. Additionally, the request body must be valid json."
• "reqType does not match type InferReqType."
• "stream must be boolean type."
• "Req body converts to json fail."
• "Inputs in request body is invalid."
• "Inputs must be necessary and data type must be string and length in [0, {MAX_INPUTS_NUM}], but the length of inputs is {utf16.length()}."
• "TikToken process fail."
• "Failed to check the input text. Can't convert string to UTF-16."

424

Generation Error

{

"type": "Generation Error",

"error": error message，详见错误信息列表

}

• "Failed to generate infer request."
• "Failed to enqueue inferRequest: remainBlocks is nullptr."
• "Failed to enqueue inferRequest: remainPrefillSlots is nullptr."
• "Failed to enqueue inferRequest: remainPrefill is nullptr."

500

Incomplete Generation

{

"type": "Incomplete Generation",

"error": error message，详见错误信息列表

}

• "Failed to generate simulate infer request."
• "Stop request failed, internal failed."
• "Failed to enqueue inferRequest: backend manager has not been initialized.";
• "Failed to enqueue inferRequest: engine has not been initialized."
• "Failed to enqueue inferRequest: Model instance has been finalized or not initialized."
• "Failed to enqueue inferRequest: llmInferEngine is not initialized!"
• "Failed forward in for infer engine."
• "Engine callback timeout."
• "Failed to get engine response."
• "Health status changed during health detector."
• "[P Node] Failed to get tokenIds from inferResponse."
• "[P Node] Failed to decode tokenIds to respStr."
• "[P Node] Send D request failed!"
• "[P Node] Unknown transfer flag."
• "Role status is invalid."
• "Environment variable MIES_SERVICE_MONITOR_MODE is not set."
• "Failed to get health status."
• "No contact node detected."

503

Service Unavailable

{

"type": "Service Unavailable",

"error": error message，详见错误信息列表

}

• "The service has been stopped."
• "Parse req json failed. Reset to previous node status."
• "The server cannot process the inference request due to an unknown status."