Session to GeSession Migration Guide

Library Link Changes

API Changes

Tensor Type Changes

The input and output of all running APIs are changed from ge::Tensor to gert::Tensor. The details are as follows:

• Namespace changes

  Feature	ge::Tensor	gert::Tensor
  Namespace	ge	gert
  Data structure	std::shared_ptr<TensorImpl> is used to manage internal implementation.	Plain Old Data (POD): all data is stored in inline mode.
  Memory layout	Indirect access through the impl_ pointer	Flat layout, supporting direct memcpy
  Placement support	Setting the placement through TensorDesc	Supporting multiple placement types
  Copy behavior	Shallow copy (shared_ptr semantics)	Shallow copy, pointer sharing
  Performance	General	High performance
  Application scenario	Graph construction	Graph run

• Data structure changes

  Internal Structure of ge::Tensor

  Internal Structure of gert::Tensor

  1 2 3 4 class Tensor { private: std::shared_ptr<TensorImpl> impl_; // Smart pointers for management }; Using shared_ptr to manage TensorImpl Bottom-layer implementation sharing during copy Data described by TensorDesc

  1 2 3 4 5 6 7 8 9 10 class Tensor { private: StorageShape storage_shape_; // Shape information StorageFormat storage_format_; // Format information TensorVersion version_; // Version uint8_t reserved_[3]; // Reserved field ge::DataType data_type_; // Data type TensorData tensor_data_; // Data pointer and placement uint8_t reserved_field_[40]; // Reserved field }; All fields are directly included in the object. It is a standard layout (std::is_standard_layout).

The following describes the lifecycle of gert::Tensor and how to construct it.

• Tensor lifecycle description
  • RunGraph API

    1 2 3 4

    std::vector<gert::Tensor> inputs = ...; std::vector<gert::Tensor> outputs; session->RunGraph(graph_id, inputs, outputs); // Inputs and outputs can be safely destroyed after the call is complete.

  • RunGraphWithStreamAsync API

    1 2 3 4 5 6

    // Note: You do not need to call CompileGraph and LoadGraph first. They will be automatically processed. std::vector<gert::Tensor> inputs = ...; std::vector<gert::Tensor> outputs; session->RunGraphWithStreamAsync(graph_id, stream, inputs, outputs); // Inputs and outputs cannot be destroyed before stream synchronization. // You need to call aclrtSynchronizeStream or other synchronization APIs.

  • RunGraphAsync API (important)

    1 2 3 4 5 6 7 8 9

    using RunAsyncCallbackV2 = std::function<void(Status, std::vector<gert::Tensor>&)>; std::vector<gert::Tensor> inputs = ...; session->RunGraphAsync(graph_id, inputs, [](Status ret, std::vector<gert::Tensor>& outputs) { // Process the output. }); // Important: Inputs cannot be destroyed immediately. // Inputs can be destroyed only after the callback function is called. // This is because the inputs are read during model run, and the callback function is called to ensure that the model run is complete.

• Methods for constructing gert::Tensor
  • (Recommended) Method 1: Using TensorData

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

    #include "exe_graph/runtime/tensor.h" #include "acl_rt.h" // Construct a host tensor. void* host_buf = nullptr; aclError ret = aclrtMallocHost(&host_buf, data_len); // Allocate the host memory. if (ret != ACL_ERROR_NONE) { // Process the error. } // Use TensorData for construction. gert::TensorData td(host_buf, nullptr, data_len, gert::kOnHost); gert::Tensor tensor; tensor.SetData(std::move(td)); // Set the data type (if required). // tensor.SetDataType(ge::DT_FLOAT);

  • Method 2: Using constructors

    1 2 3 4 5 6 7 8 9 10

    // Use shape, format, and dtype for construction. gert::StorageShape shape = {{batch_size, channels, height, width}, {4}}; gert::StorageFormat format = {ge::FORMAT_ND, ge::FORMAT_ND, {}}; gert::Tensor tensor(shape, format, ge::DT_FLOAT); // Allocate memory. void* host_buf = nullptr; aclrtMallocHost(&host_buf, tensor.GetSize()); gert::TensorData td(host_buf, nullptr, tensor.GetSize(), gert::kOnHost); tensor.SetData(std::move(td));

  • Method 3: Constructing a device tensor

    1 2 3 4 5 6 7 8 9 10 11

    // Allocate device memory. void* dev = nullptr; aclError ret = aclrtMalloc(&dev, bytes, ACL_MEM_MALLOC_NORMAL_ONLY); if (ret != ACL_ERROR_NONE) { // Process the error. } // Construct a device tensor. gert::TensorData td(dev, nullptr, bytes, gert::kOnDeviceHbm); gert::Tensor device_tensor; device_tensor.SetData(std::move(td));

• Free the gert::Tensor memory.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

  // Free the host tensor memory. void FreeHostTensor(gert::Tensor &tensor) { if (tensor.GetAddr() != nullptr) { aclrtFreeHost(tensor.GetAddr()); } } // Free the device tensor memory. void FreeDeviceTensor(gert::Tensor &tensor) { if (tensor.GetAddr() != nullptr) { aclrtFree(tensor.GetAddr()); } } // Batch free void FreeTensorVector(std::vector<gert::Tensor> &tensors, bool is_device) { for (auto &t : tensors) { if (t.GetAddr() != nullptr) { if (is_device) { aclrtFree(t.GetAddr()); } else { aclrtFreeHost(t.GetAddr()); } } } }

For details about the aclrtMallocHost, aclrtMalloc, aclrtFreeHost, and aclrtFree APIs, see Memory Management.

Changes in Build and Run APIs

• CompileGraph and LoadGraph are no longer required.

  In GeSession, the RunGraph, RunGraphAsync, and RunGraphWithStreamAsync APIs automatically check whether a graph has been built and loaded. If not built, they build it first. If not loaded, they load it first.

  1 2 3 4 5 6

  // Automatic processing mechanism of GeSession GeSession session(options); session.AddGraph(graph_id, graph); // You do not need to manually call CompileGraph and LoadGraph. session.RunGraph(graph_id, inputs, outputs); // Automatic build and loading

• Mutually exclusive run modes

  GeSession provides three run modes: RunGraph, RunGraphAsync, and RunGraphWithStreamAsync. They are mutually exclusive. Once a run mode is used, the graph must continue to use the same mode.

Build Configuration Changes

Makefile or CMakeLists.txt needs to be updated.

# Old configuration
target_link_libraries(your_app libge_runner.so)

# New configuration
target_link_libraries(your_app libge_runner_v2.so)