sparse_lookup

Function

Queries the sparse feature table in the Rec SDK TensorFlow training framework.

Currently, one table can be queried for one or multiple times. If a table is queried for multiple times, the maximum number of query times is 128.

Currently, the tf.SparseTensor data type is not supported. If the data type is tf.SparseTensor, convert it to tf.Tensor. The sample code is as follows:

# Sample code
sparse_ids = tf.SparseTensor(indices=[[0, 0], [1, 2]], values=[1, 2], dense_shape=[3, 4])
dense_ids = tf.sparse.to_dense(sparse_ids, default_value=0)
embedding = sparse_lookup(sparse_hashtable, dense_ids)

Prototype

def sparse_lookup(hashtable, ids, send_count, is_train=True, name=None, modify_graph=False, batch=None, access_and_evict_config=None, is_grad=True, serving_default_value, **kwargs)

Parameters

Parameter	Type	Mandatory/Optional	Description
hashtable	BaseSparseEmbedding	Mandatory	Sparse table to be queried.
ids	FeatureSpec/tf.Tensor	Mandatory	Queried keyword (key). The parameter type varies according to the function mode. In non-automatic graph modification mode, ids corresponds to FeatureSpec. In automatic graph modification mode, ids corresponds to tf.Tensor.
send_count	int	Optional. It is mandatory when static shape is enabled.	All2All communication technology. Value range: [1, 2147483647]. When the dynamic shape function is enabled, this parameter does not need to be passed or can be set to None. The default value is None.
is_train	bool	Mandatory	Whether the training mode is enabled. The default value is True. Value: True: training mode False: evaluation or prediction mode
name	str	Optional	Name of the query operation. The value is a string of 1 to 255 characters. The default value is None.
modify_graph	bool	Optional	Whether to enable the automatic graph modification. This function modifies and optimizes the original model graph before creating a session instance. The default value is False. Value: True: automatic graph modification enabled False: automatic graph modification disabled
batch	dict{str:tf.Tensor}	Optional	Dataset iterator. When ids is of the FeatureSpec type and dynamic shape is used, batch must be passed. The key is the feature name, and the value is the result after get_next () is executed using the corresponding tf.Dataset. The default value is None.
access_and_evict_config	dict{str:int}	Optional	This parameter is used when feature access and eviction are enabled in automatic graph modification mode. The dict consists of three key-value pairs. The key values are access_threshold, eviction_threshold, and faae_coefficient. The default values of access_threshold and eviction_threshold are None, and the default value of faae_coefficient is 1. value of access_threshold and eviction_threshold indicates the corresponding threshold. value of faae_coefficient indicates the feature access coefficient.
is_grad	bool	Optional	Whether gradient update is required for a query. The default value is True. Value: True: gradient updated False: gradient not updated
serving_default_value	tf.Tensor	Optional	Default embedding value of a feature that is filtered during training or a new feature during prediction. If this parameter is not specified, the default value None is used.

**kwargs Parameters

Parameter	Type	Mandatory/Optional	Description
feature_spec_name_ids_dict	dict	Optional	Dictionary structure. key is the FeatureSpec name, and value is the parameter ID of the public API sparse_lookup(). There is no default value.
multi_lookup	bool	Optional	Whether one table is queried for multiple times. There is no default value. Value: True: table queried for multiple times False: table not queried for multiple times
lookup_ids	FeatureSpec/tf.Tensor	Optional	Queried keyword (key). The parameter type varies according to the function mode. There is no default value. In non-automatic graph modification mode, ids corresponds to FeatureSpec. In automatic graph modification mode, ids corresponds to tf.Tensor.

Parameter

Type

Mandatory/Optional

Description

feature_spec_name_ids_dict

dict

Optional

Dictionary structure. key is the FeatureSpec name, and value is the parameter ID of the public API sparse_lookup(). There is no default value.

multi_lookup

bool

Optional

Whether one table is queried for multiple times. There is no default value.

Value:

True: table queried for multiple times
False: table not queried for multiple times

lookup_ids

FeatureSpec/tf.Tensor

Optional

Queried keyword (key). The parameter type varies according to the function mode. There is no default value.

In non-automatic graph modification mode, ids corresponds to FeatureSpec.
In automatic graph modification mode, ids corresponds to tf.Tensor.

feature_spec_name_ids_dict, multi_lookup, and lookup_ids of **kwargs are for internal use. You are not advised to pass these three parameters through kwargs.
If kwargs is used to pass other parameters that are not described, Rec SDK TensorFlow does not use these parameters.

Return Value

Success: The queried tensor result is returned.
Failure: An exception is thrown.

Example

from mx_rec.core.embedding import sparse_lookup
from mx_rec.core.asc.feature_spec import FeatureSpec
feature_spec = FeatureSpec("sparse_feature", table_name="sparse_embeddings_table",
                                batch_size=1)
embedding = sparse_lookup(sparse_hashtable,
                          feature_spec,
                          send_count=6000,
                          is_train=True,
                          name="sparse_embeddings")

sparse_lookup

Function

Prototype

Parameters

**kwargs Parameters

Return Value

Example

See Also