Tool for Creating Python UDF Projects

This tool generates template directories and files for Python UDF projects. You can develop functions based on the template to avoid manual project creation.

This tool is stored in ${install_path}/cann/python/site-packages/dataflow/tools/create_func_ws.py, where ${install_path} indicates the CANN installation directory with the default value /usr/local/Ascend.

Perform the following operations:

1. Set environment variables.

   After the CANN software is installed, log in to the environment as the CANN operating user and run the source ${INSTALL_DIR}/set_env.sh command to set environment variables. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

2. Create a Python UDF project.

   The following describes how to create a project whose workspace directory is test_sub.

   Run the following command under any directory:

   python3.11 -m dataflow.tools.create_func_ws -f sub:i0:i1:o0 -w ./test_sub -c Sub

   The test_sub folder is generated in the current directory. The directory structure is as follows:

   ├── CMakeLists.txt  // CMakeLists compilation configuration file. Generally, you do not need to modify it. If special processing is required for packaging, you can modify the configuration file. For example, you can add a configuration file for release during packaging.
   ├── func_sub.json // UDF configuration file, which is used to specify UDF inputs, outputs, and other related configurations. Generally, you do not need to modify the file. If the generated workspace directory is different from the actual workspace directory, change the workspace parameter in the file to the actual runtime path.
   ├── src_cpp
   │   └── func_sub.cpp  // UDF registration implementation and the logic for C++ to call the Python UDF. No modification is required.
   └── src_python
       └── func_sub.py  // User implementation function, which needs to be compiled by users. Multiple Python files can be added to the current directory and will be packed to the operating environment.

   The following describes the options in the command:

   • -m: (mandatory) Python common command-line option, which is used as the script runtime library module. In this tool, the option is fixed to dataflow.tools.create_func_ws.
   • -f: (mandatory) input function information, input index, and output index. The format is Function name:ix:ix:ox:ox, where i indicates input and is followed by an input index, and o indicates output and is followed by an output index. If there are multiple functions, use commas (,) to separate them. An example is sub:i0:i1:o0. In this example, the function name is sub, the two inputs are i0 and i1, and the output is o0. (The function name is directly used as the Python function name. You are advised to use lowercase letters and underscores (_). In addition, the generated CPP code combines Proc based on the upper camel case as the CPP function.)
   • -w: (optional) workspace folder. An example is ./test_sub. In this example, the specified workspace folder is test_sub in the current directory. The default value is ./ (the current directory). Note that the generated file will overwrite the file with the same name in the workspace.
   • -c: (optional) function class name. An example is Sub (which can be directly used as the Python and C++ class names. The upper camel case style is recommended). If this option is left empty, the function name in the -f option is used.

To view all options supported by this tool, run the command with the -h argument.