Class Introduction

Function

Uses Transformers to start a model locally and provides the text-to-vector sparse embedding function. Weights of models in the BertModel class supported by Transformers are required. The class inherits the langchain_core.embeddings.Embeddings API. Currently, only the BAAI/bge-m3 model is supported.

If the configured model weight is not in the safetensors format, convert the model weight to the safetensors format before using it. This prevents security problems caused by insecure model weight formats such as CKPT and BIN.

Prototype

from mx_rag.embedding.local import SparseEmbedding
SparseEmbedding(model_path, dev_id, use_fp16)

Parameters

Parameter	Data Type	Required/Optional	Description
model_path	String	Required	Model weight file directory. The path length cannot exceed 1024 characters. The path cannot be a soft link or a relative path. The size of each file in the directory cannot exceed 10 GB, the level cannot exceed 64, and the total number of files cannot exceed 512. The running user group and non-running users cannot have the write permission on the files in the directory. All files within the directory, as well as the parent directory itself, must have their group ownership set to the running user. The storage path cannot be in the path list: ["/etc", "/usr/bin", "/usr/lib", "/usr/lib64", "/sys/", "/dev/", "/sbin", "/tmp"].
dev_id	Integer	Optional	ID of the model's running NPU. You can use npu-smi info to query the available ID. The value range is [0, 63]. The default value is 0.
use_fp16	Bool	Optional	Whether to use FP16. The default value is True.

Parameter

Data Type

Required/Optional

Description

model_path

String

Required

Model weight file directory. The path length cannot exceed 1024 characters. The path cannot be a soft link or a relative path.

The size of each file in the directory cannot exceed 10 GB, the level cannot exceed 64, and the total number of files cannot exceed 512.
The running user group and non-running users cannot have the write permission on the files in the directory.
All files within the directory, as well as the parent directory itself, must have their group ownership set to the running user.
The storage path cannot be in the path list: ["/etc", "/usr/bin", "/usr/lib", "/usr/lib64", "/sys/", "/dev/", "/sbin", "/tmp"].

dev_id

Integer

Optional

ID of the model's running NPU. You can use npu-smi info to query the available ID. The value range is [0, 63]. The default value is 0.

use_fp16

Bool

Optional

Whether to use FP16. The default value is True.

Example

from paddle.base import libpaddle
from mx_rag.embedding.local import SparseEmbedding
# Same as embed = SparseEmbedding("/path/to/model", 1).
embed = SparseEmbedding.create(model_path="/path/to/model", dev_id=1)
print(embed.embed_documents(['abc', 'bcd']))
print(embed.embed_query('abc'))

Parent topic: SparseEmbedding