aclnnBidirectionLSTMV2

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	×
Atlas A3 training products/Atlas A3 inference products	×
Atlas A2 training products/Atlas A2 inference products	×
Atlas 200I/500 A2 inference products	×
Atlas inference products	√
Atlas training products	×

Function

Description: The LSTM (Long Short-Term Memory) network is a special type of recurrent neural network (RNN) model. It performs LSTM network computation, receives an input sequence and initial states, and returns an output sequence and final states. Compared with aclnnBidirectionLSTM, this API adds support for sequences of unequal lengths.
Formula:
$f_t =sigm(W_f[h_{t-1}, x_t] + b_f)\\ i_t =sigm(W_i[h_{t-1}, x_t] + b_i)\\ o_t =sigm(W_o[h_{t-1}, x_t] + b_o)\\ \tilde{c}_t =tanh(W_c[h_{t-1}, x_t] + b_c)\\ c_t =f_t ⊙ c_{t-1} + i_t ⊙ \tilde{c}_t\\ c_{o}^{t} =tanh(c_t)\\ h_t =o_t ⊙ c_{o}^{t}\\$
- $x_t ∈ R^{d}$ : Input vector of the LSTM unit.
- $f_t ∈ (0, 1)^{h}$ : forget gate activation vector.
- $i_t ∈ (0, 1)^{h}$ : input gate and update gate activation vector.
- $o_t ∈ (0, 1)^{h}$ : output gate activation vector.
- $h_i ∈ (-1, 1)^{h}$ : hidden state vector, also referred to as the output vector of the LSTM cell.
- $\tilde{c}_t ∈ (-1, 1)^{h}$ : cell input activation vector.
- $c_t ∈ R^{h}$ : cell state vector.
- $W ∈ R^{h×d}，(U ∈ R^{h×h})∩(b ∈ R^{h})$ : weight matrices and bias vector parameters learned during training.

Function Prototype

Each operator is divided into a two-phase API. You must first call the "aclnnBidirectionLSTMV2GetWorkspaceSize" API to obtain the input parameters and calculate the required workspace size based on the process, and then call the "aclnnBidirectionLSTMV2" API to perform the computation.

aclnnStatus aclnnBidirectionLSTMV2GetWorkspaceSize(
  const aclTensor *x,
  const aclTensor *initH,
  const aclTensor *initC,
  const aclTensor *wIh,
  const aclTensor *wHh,
  const aclTensor *bIhOptional,
  const aclTensor *bHhOptional,
  const aclTensor *wIhReverseOptional,
  const aclTensor *wHhReverseOptional,
  const aclTensor *bIhReverseOptional,
  const aclTensor *bHhReverseOptional,
  const aclTensor *batchSizeOptional,
  int64_t          numLayers,
  bool             isbias,
  bool             batchFirst,
  bool             bidirection,
  bool             packed,
  const aclTensor *yOut,
  const aclTensor *outputHOut,
  const aclTensor *outputCOut,
  uint64_t        *workspaceSize,
  aclOpExecutor  **executor)

aclnnStatus aclnnBidirectionLSTMV2(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnBidirectionLSTMV2GetWorkspaceSize

Parameters:

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
x (aclTensor*)	Input	Input vector of the LSTM unit, i.e., x in the formula.	Empty tensors are supported. When the parameter packed is set to true, the shape supports three dimensions (time_step, batch_size, input_size). When the parameter packed is set to true, the shape supports two dimensions (total_batch_size, input_size). Here, time_step indicates the time dimension; batch_size indicates the number of batches to be processed at each moment; input_size indicates the number of input features; total_batch_size indicates the total number of batches across all moments when the batch counts at each moment are not aligned.	FLOAT16	ND	3	√
initH (aclTensor*)	Input	Initializes the hidden state, which is h in the formula.	Supports empty tensors. The shape supports three dimensions (num_layers, batch_size, hidden_size) or (2 * num_layers, batch_size, hidden_size) when bidirection is True. Here, `num_layers` corresponds to the parameter `numLayers`, indicating the number of LSTM layers; `hidden_size` indicates the number of features of the hidden state.	FLOAT16	ND	3	√
initC (aclTensor*)	Input	Initial cell state, i.e., c in the formula.	Supports empty tensors. The shape supports three dimensions (num_layers, batch_size, hidden_size) or, when bidirection is True, (2 * num_layers, batch_size, hidden_size).	FLOAT16	ND	3	√
wIh (aclTensor*)	Input	Input-hidden weight, W in the formula.	Supports empty tensors. The shape supports two dimensions (4 * hidden_size, input_size).	FLOAT16	ND	2	√
wHh (aclTensor*)	Input	Hidden-hidden weight, W in the formula.	Supports empty tensor. Shape supports two dimensions (4 * hidden_size, hidden_size).	FLOAT16	ND	2	√
bIhOptional (aclTensor*)	Input	Input-hidden bias, i.e., b in the formula.	Supports empty tensor. Shape supports one dimension (4 * hidden_size).	FLOAT16	ND	1	√
bHhOptional（aclTensor*）	Input	Hidden-hidden bias, which is b in the formula.	Supports empty tensors. The shape supports one dimension (4 * hidden_size).	FLOAT16	ND	1	√
wIhReverseOptional (aclTensor*)	Input	Reverse input-hidden weight, W in the formula.	Supports empty tensors. The shape supports two dimensions (4 * hidden_size, input_size).	FLOAT16	ND	2	√
wHhReverseOptional (aclTensor*)	Input	Reverse hidden-hidden weight, W in the formula.	Supports empty tensors. The shape supports two dimensions (4 * hidden_size, input_size).	FLOAT16	ND	2	√
bIhReverseOptional (aclTensor*)	Input	Reverse input-hidden bias, denoted as b in the formula.	Supports empty tensors. The shape supports one dimension (4 * hidden_size).	FLOAT16	ND	1	√
bHhReverseOptional (aclTensor*)	Input.	Reverse hidden-hidden bias, b in the formula.	Supports empty tensors. The shape supports one dimension (4 * hidden_size).	FLOAT16	ND	1	√
batchSizeOptional (aclTensor*)	Input	Specifies the batch_size that actually participates in the computation in each iteration.	Supports empty tensors. The shape supports one dimension (time_step).	INT	ND	1	√
numLayers (int64_t)	Input	Indicates the number of LSTM layers.	Currently, only 1 is supported.	-	-	-	-
isbias (bool)	Input	Indicates whether bias is included.	-	-	-	-	-
batchFirst (bool)	Input	Indicates whether batch is the first dimension.	Currently only false is supported.	-	-	-	-
bidirection (bool)	Input	Indicates whether it is bidirectional.	-	-	-	-	-
packed (bool)	Input	Indicates whether the input x is packed.	Takes effect only when batchSizeOptional is not empty.	-	-	-	-
yOut (aclTensor*)	Output	Output vector of the LSTM unit.	When bidirection is false and packed is false, the shape supports three dimensions (time_step, batch_size, input_size). When bidirection is false and packed is true, the shape supports two dimensions (total_batch_size, input_size). When bidirection is true and packed is false, the shape supports three dimensions (time_step, batch_size, 2 * input_size). When bidirection is true and packed is true, the shape supports two dimensions (total_batch_size, 2 * input_size).	FLOAT16	ND	3	√
outputHOut (aclTensor*)	Output	Final hidden state, i.e., h in the formula.	The shape supports three dimensions (num_layers, batch_size, hidden_size), or (2 * num_layers, batch_size, hidden_size) when bidirection is True.	FLOAT16	ND	3	√
outputCOut (aclTensor*)	Output	Final cell state, i.e., c in the formula.	The shape supports three-dimensional (num_layers, batch_size, hidden_size) or (2 * num_layers, batch_size, hidden_size) when bidirection is True.	FLOAT16	ND	3	√
workspaceSize (uint64_t*)	Output	Returns the workspace size to be applied for on the Device side.	-	-	-	-	-
executor（aclOpExecutor**）	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

Return value

aclnnStatus: return code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Code	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The input parameter is a required input, output, or required attribute, and is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	The input parameter is of the aclTensor type and its data type is not within the supported range.
ACLNN_ERR_INNER_TILING_ERROR	561002	If the input parameter type is aclTensor and its shape does not match the parameter description above.

aclnnBidirectionLSTMV2

Parameters:

Parameter	Input/Output	Description
workspace	Input	Address of the workspace memory applied for on the Device side.
workspaceSize	Input	Size of the workspace applied for on the Device side, obtained through the first-phase API aclnnBidirectionLSTMV2GetWorkspaceSize.
executor	Input	Operator executor, which contains the operator computation process.
stream	Input	Specifies the stream for task execution.

Return value

aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

Deterministic computation:
- aclnnBidirectionLSTMV2 defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_bidirection_lstmv2.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shapeSize = 1;
  for (auto i : shape) {
    shapeSize *= i;
  }
  return shapeSize;
}

void PrintOutResult(std::vector<int64_t> &shape, void** deviceAddr) {
  auto size = GetShapeSize(shape);
  std::vector<float> resultData(size, 0);
  auto ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]),
                         *deviceAddr, size * sizeof(resultData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("mean result[%ld] is: %f\n", i, resultData[i]);
  }
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: resource initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to apply for device-side memory.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
  // Copy host data to the device memory by calling aclrtMemcpy.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Create an aclTensor by calling the aclCreateTensor API.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
  // Set deviceId based on the actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the input and output. Customize the construction based on the API.
  int time_step = 2;
  int batch_size = 32;
  int input_size = 32;
  int hidden_size = 32;

  int64_t numLayers = 1;
  bool isbias = true;
  bool batchFirst = false;
  bool bidirection = true;
  bool packed = false;

  std::vector<int64_t> selfShape = {time_step, batch_size, input_size};
  std::vector<int64_t> weightHIShape = {4 * hidden_size, input_size};
  std::vector<int64_t> weightHHShape = {4 * hidden_size, hidden_size};
  std::vector<int64_t> initHShape = {2, batch_size, hidden_size};
  std::vector<int64_t> initCShape = {2, batch_size, hidden_size};
  std::vector<int64_t> biasHIShape = {4 * hidden_size};
  std::vector<int64_t> biasHHShape = {4 * hidden_size};
  std::vector<int64_t> outShape = {time_step, batch_size, 2 * hidden_size};
  std::vector<int64_t> outHShape = {2, batch_size, hidden_size};
  std::vector<int64_t> outCShape = {2, batch_size, hidden_size};

  void* selfDeviceAddr = nullptr;
  void* weightHIDeviceAddr = nullptr;
  void* weightHHDeviceAddr = nullptr;
  void* weightHIReverseDeviceAddr = nullptr;
  void* weightHHReverseDeviceAddr = nullptr;
  void* initHDeviceAddr = nullptr;
  void* initCDeviceAddr = nullptr;
  void* biasHIDeviceAddr = nullptr;
  void* biasHHDeviceAddr = nullptr;
  void* biasHIReverseDeviceAddr = nullptr;
  void* biasHHReverseDeviceAddr = nullptr;
  void* outDeviceAddr = nullptr;
  void* outHDeviceAddr = nullptr;
  void* outCDeviceAddr = nullptr;

  aclTensor* self = nullptr;
  aclTensor* weightHI = nullptr;
  aclTensor* weightHH = nullptr;
  aclTensor* weightHIReverse = nullptr;
  aclTensor* weightHHReverse = nullptr;
  aclTensor* biasHI = nullptr;
  aclTensor* biasHH = nullptr;
  aclTensor* biasHIReverse = nullptr;
  aclTensor* biasHHReverse = nullptr;
  aclTensor* batchSize = nullptr;
  aclTensor* initH = nullptr;
  aclTensor* initC = nullptr;
  aclTensor* out = nullptr;
  aclTensor* outH = nullptr;
  aclTensor* outC = nullptr;

  std::vector<uint16_t> selfHostData(GetShapeSize(selfShape));
  std::vector<uint16_t> weightHIHostData(GetShapeSize(weightHIShape));
  std::vector<uint16_t> weightHHHostData(GetShapeSize(weightHHShape));
  std::vector<uint16_t> biasHIHostData(GetShapeSize(biasHIShape));
  std::vector<uint16_t> biasHHHostData(GetShapeSize(biasHHShape));
  std::vector<uint16_t> initHHostData(GetShapeSize(initHShape));
  std::vector<uint16_t> initCHostData(GetShapeSize(initCShape));
  std::vector<uint16_t> outHostData(GetShapeSize(outShape));
  std::vector<uint16_t> outHHostData(GetShapeSize(outHShape));
  std::vector<uint16_t> outCHostData(GetShapeSize(outCShape));

  ret = CreateAclTensor(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_FLOAT16, &self);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(weightHIHostData, weightHIShape, &weightHIDeviceAddr, aclDataType::ACL_FLOAT16, &weightHI);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(weightHHHostData, weightHHShape, &weightHHDeviceAddr, aclDataType::ACL_FLOAT16, &weightHH);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(initHHostData, initHShape, &initHDeviceAddr, aclDataType::ACL_FLOAT16, &initH);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(initCHostData, initCShape, &initCDeviceAddr, aclDataType::ACL_FLOAT16, &initC);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biasHIHostData, biasHIShape, &biasHIDeviceAddr, aclDataType::ACL_FLOAT16, &biasHI);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biasHHHostData, biasHHShape, &biasHHDeviceAddr, aclDataType::ACL_FLOAT16, &biasHH);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(weightHIHostData, weightHIShape, &weightHIReverseDeviceAddr, aclDataType::ACL_FLOAT16, &weightHIReverse);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(weightHHHostData, weightHHShape, &weightHHReverseDeviceAddr, aclDataType::ACL_FLOAT16, &weightHHReverse);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biasHIHostData, biasHIShape, &biasHIReverseDeviceAddr, aclDataType::ACL_FLOAT16, &biasHIReverse);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biasHHHostData, biasHHShape, &biasHHReverseDeviceAddr, aclDataType::ACL_FLOAT16, &biasHHReverse);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_FLOAT16, &out);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(outHHostData, outHShape, &outHDeviceAddr, aclDataType::ACL_FLOAT16, &outH);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(outCHostData, outCShape, &outCDeviceAddr, aclDataType::ACL_FLOAT16, &outC);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // 3. Call the CANN operator library API. Replace with the specific API name.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;

  // Call the first-phase API of aclnnBidirectionLSTMV2.
  ret = aclnnBidirectionLSTMV2GetWorkspaceSize(self, initH, initC, weightHI, weightHH,
                                            biasHI, biasHH, weightHIReverse, weightHHReverse, biasHIReverse, biasHHReverse,
                                            batchSize,
                                            numLayers, isbias, batchFirst, bidirection, packed,
                                            out, outH, outC,
                                            &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnBidirectionLSTMV2GetWorkspaceSize failed. ERROR: %d\n", ret); return ret);

  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }

  // Call the second-phase API of aclnnBidirectionLSTMV2.
  ret = aclnnBidirectionLSTMV2(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnBidirectionLSTMV2 failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Wait for task execution to finish synchronously.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify this based on the specific API definition.
  PrintOutResult(outShape, &outDeviceAddr);
  PrintOutResult(outHShape, &outHDeviceAddr);
  PrintOutResult(outCShape, &outCDeviceAddr);

  // 6. Release aclTensor and aclScalar. Modify this based on the specific API definition.
  aclDestroyTensor(self);
  aclDestroyTensor(weightHI);
  aclDestroyTensor(weightHH);
  aclDestroyTensor(initH);
  aclDestroyTensor(initC);
  aclDestroyTensor(biasHI);
  aclDestroyTensor(biasHH);
  aclDestroyTensor(weightHIReverse);
  aclDestroyTensor(weightHHReverse);
  aclDestroyTensor(biasHIReverse);
  aclDestroyTensor(biasHHReverse);
  aclDestroyTensor(out);
  aclDestroyTensor(outH);
  aclDestroyTensor(outC);

  // 7. Release device resources.
  aclrtFree(selfDeviceAddr);
  aclrtFree(weightHIDeviceAddr);
  aclrtFree(weightHHDeviceAddr);
  aclrtFree(initHDeviceAddr);
  aclrtFree(initCDeviceAddr);
  aclrtFree(biasHIDeviceAddr);
  aclrtFree(biasHHDeviceAddr);
  aclrtFree(weightHIReverseDeviceAddr);
  aclrtFree(weightHHReverseDeviceAddr);
  aclrtFree(biasHIReverseDeviceAddr);
  aclrtFree(biasHHReverseDeviceAddr);
  aclrtFree(outDeviceAddr);
  aclrtFree(outHDeviceAddr);
  aclrtFree(outCDeviceAddr);

  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();

  return 0;
}