Token Calculation API

Function

Converts text into a token.

Format

Operation type: POST

URL: https://{ip}:{port}/v1/tokenizer

• Preferably set {ip} to {predict_ip} in Starting Coordinator. If the parameter is not configured, set {ip} to the value of the predict_ip parameter in the ms_coordinator.json configuration file.
• Preferably set {port} to {predict_port} in Starting Coordinator. If the parameter is not configured, set {port} to the value of the predict_port parameter in the ms_coordinator.json configuration file.

Request Parameters

Usage Example

Request example:

POST https://{ip}:{port}/v1/tokenizer

Request body:

{
    "inputs": "what is your name?"
}

Response example:

{
    "token_number": 6,
    "tokens": ["what", "is", "your", "name", "?"]
}

Output Description