UDF API List

This section describes the external APIs provided by the UDF module. You can call these APIs to develop custom processing functions, and then execute such functions on the CPU through DataFlow graph construction.

You can view the API implementation in python/site-packages/dataflow/flow_func/flow_func.py in CANN component directory. The APIs are as follows.

Class FlowMsg

This class is used to process the input and output operations of FlowFunc.

Table 1 FlowMsg APIs

API

Description

FlowMsg Constructor

Constructs an object of class FlowMsg.

get_msg_type

Obtains the message type of FlowMsg.

get_tensor

Obtains the tensor object of FlowMsg.

set_ret_code

Sets the error code in the FlowMsg message.

get_ret_code

Obtains the error code in the input FlowMsg message.

set_start_time

Sets the start timestamp in the FlowMsg message header.

get_start_time

Obtains the start timestamp in the FlowMsg message.

set_end_time

Sets the end timestamp in the FlowMsg message header.

get_end_time

Obtains the end timestamp in the FlowMsg message.

set_flow_flags

Sets flags in the FlowMsg message header.

get_flow_flags

Obtains flags in the FlowMsg message header.

set_route_label

Sets route labels.

get_transaction_id

Obtains the transaction ID in the FlowMsg message. The transaction ID starts from 1, and is incremented by 1 each time a batch of data is fed. This function can be used to identify the ID of the data batch.

set_msg_type

Sets the message type of FlowMsg.

get_raw_data

Obtains data of the rawdata type.

set_transaction_id

Sets the transaction ID used for DataFlow data transmission.

Tensor Class

This class is used to perform tensor-related operations. The obtained tensor is dataflow.Tensor.

Table 2 Tensor APIs

API

Description

Tensor Constructor

Constructs an object of class Tensor.

get_shape

Obtains the shape of a tensor.

get_data_type

Obtains the data type of a tensor.

get_data_size

Obtains the data size of a tensor.

get_element_cnt

Obtains the number of elements in a tensor.

reshape

Reshapes a tensor without changing the tensor content.

Class MetaParams

This class is used to obtain shared variable information.

Table 3 MetaParams APIs

API

Description

PyMetaParams Constructor

Constructs an object of class PyMetaParams.

get_name

Obtains the instance name of FlowFunc.

get_attr_int

Obtains the attribute value of the int type with the specified name.

get_attr_bool_list

Obtains the attribute value of the bool array with the specified name.

get_attr_int_list

Obtains the attribute value of the int array with the specified name.

get_attr_int_list_list

Obtains the attribute value of the 2D array of the int type with the specified name.

get_attr_bool

Obtains the attribute value of the bool type with the specified name.

get_attr_float_list

Obtains the attribute value of the float array with the specified name.

get_attr_tensor_dtype

Obtains the attribute value of the numpy.dtype type with the specified name.

get_attr_tensor_dtype_list

Obtains the attribute value of the array of the numpy.dtype types with the specified name.

get_attr_str

Obtains the attribute value of the string type with the specified name.

get_attr_str_list

Obtains the attribute value of the string array with the specified name.

get_attr_float

Obtains the attribute value of the float type with the specified name.

get_input_num

Obtains the number of FlowFunc inputs.

get_output_num

Obtains the number of FlowFunc outputs.

get_work_path

Obtains the working path of FlowFunc.

get_running_device_id

Obtains the ID of the running device.

Class MetaRunContext

This class is used to process the context information of the processing function of FlowFunc, such as allocating a tensor, setting the output, and running FlowModel.

Table 4 MetaRunContext APIs

API

Description

MetaRunContext Constructor

Constructs an object of class MetaRunContext.

alloc_tensor_msg

Allocates FlowMsg of the tensor type based on the shape, data type, and alignment size.

set_output

Sets the tensor of the output with a specified index.

set_multi_outputs

Sets the tensors of the outputs with specified indexes in batches.

run_flow_model

Runs a specified model synchronously.

alloc_empty_data_msg

Allocates a message of the MsgType type that carries empty data.

get_user_data

Obtains the user-defined data.

raise_exception

The UDF proactively reports an exception.

get_exception

The UDF obtains the exception reported by other UDF nodes.

alloc_raw_data_msg

Allocates a continuous memory based on the input size to carry the FlowMsg of the raw data type.

to_flow_msg

Converts a dataflow tensor into a FlowMsg.

Class AffinityPolicy

Table 5 AffinityPolicy APIs

API

Description

Class AffinityPolicy

Defines the affinity policy enumeration.

Class BalanceConfig

When balanced distribution is required, you need to configure the output data identifier and weight matrix. The scheduling module can implement balanced distribution among multiple instances based on the configuration.

Table 6 BalanceConfig APIs

API

Description

BalanceConfig Constructor

Constructs an object of class BalanceConfig.

set_data_pos

Sets the position of the output data in the weight matrix.

get_inner_config

Obtains inner configuration objects. This API is called by set_output or set_multi_outputs.

Class FlowMsgQueue

This class serves as the input queue for FlowFunc in streaming input scenarios (the input parameter of FlowFunc is a queue). It dequeues FlowMsg entries, converts each to its corresponding data type based on MsgType, and returns the converted value to the caller.

Table 7 FlowMsgQueue APIs

API

Description

FlowMsgQueue Constructor

Constructs and destructs an object of class FlowMsgQueue.

get

Obtains elements in a queue.

get_nowait

Obtains elements in a queue without waiting. The function is the same as get(block=False).

full

Checks whether the queue is full.

full

Checks whether the queue is empty.

qsize

Obtains the number of current elements in a queue.

UDF Log APIs

Logging APIs are exposed for Python UDFs. You need to import the flow_func module to use these APIs. Use the defined logger object within this module to call its encapsulated logging APIs of different severity levels.

Table 8 UDF log APIs

API

Description

FlowFuncLogger Constructor

Constructs an object of class FlowFuncLogger.

get_log_header

Obtains log extension header information.

is_log_enable

Checks whether logs of the corresponding level and type are enabled.

Error-Level Log Macro of Run Logs

Defines the error-level log macro of run logs.

Info-Level Log Macro of Run Logs

Defines the info-level log macro of run logs.

Error-Level Log Macro of Debug Logs

Defines the error-level log macro of debug logs.

Warn-Level Log Macro of Debug Logs

Defines the warn-level log macro of debug logs.

Info-Level Log Macro of Debug Logs

Defines the info-level log macro of debug logs.

Debug-Level Log Macro of Debug Logs

Defines the debug-level log macro of debug logs.