aclnnBidirectionLSTM

Applicable Products

Applicable Products	Supported
Ascend 950PR/Ascend 950DT	×
Atlas A3 training products/Atlas A3 inference products	×
Atlas A2 training products/Atlas A2 inference products	×
Atlas 200I/500 A2 inference products	×
Atlas inference products	√
Atlas training products	×

Function

Description: The LSTM (Long Short-Term Memory) network is a special type of recurrent neural network (RNN) model. Performs LSTM network computation, receives an input sequence and initial states, and returns an output sequence and final states.
Formula:
$f_t =sigm(W_f[h_{t-1}, x_t] + b_f)\\ i_t =sigm(W_i[h_{t-1}, x_t] + b_i)\\ o_t =sigm(W_o[h_{t-1}, x_t] + b_o)\\ \tilde{c}_t =tanh(W_c[h_{t-1}, x_t] + b_c)\\ c_t =f_t ⊙ c_{t-1} + i_t ⊙ \tilde{c}_t\\ c_{o}^{t} =tanh(c_t)\\ h_t =o_t ⊙ c_{o}^{t}\\$
- $x_t ∈ R^{d}$ : Input vector of the LSTM cell.
- $f_t ∈ (0, 1)^{h}$ : forget gate activation vector.
- $i_t ∈ (0, 1)^{h}$ : input gate/update gate activation vector.
- $o_t ∈ (0, 1)^{h}$ : output gate activation vector.
- $h_i ∈ (-1, 1)^{h}$ : hidden state vector, also referred to as the output vector of the LSTM cell.
- $\tilde{c}_t ∈ (-1, 1)^{h}$ : cell input activation vector.
- $c_t ∈ R^{h}$ : cell state vector.
- $W ∈ R^{h×d}, (U ∈ R^{h×h})∩(b ∈ R^{h})$ : weight matrix and bias vector parameters to be learned during training.

Function Prototype

Each operator is divided into a two-phase API. You must first call the "aclnnBidirectionLSTMGetWorkspaceSize" API to obtain the input parameters and calculate the required workspace size based on the process, and then call the "aclnnBidirectionLSTM" API to perform the computation.

aclnnStatus aclnnBidirectionLSTMGetWorkspaceSize(
  const aclTensor *x,
  const aclTensor *initH,
  const aclTensor *initC,
  const aclTensor *wIh,
  const aclTensor *wHh,
  const aclTensor *bIhOptional,
  const aclTensor *bHhOptional,
  const aclTensor *wIhReverseOptional,
  const aclTensor *wHhReverseOptional,
  const aclTensor *bIhReverseOptional,
  const aclTensor *bHhReverseOptional,
  int64_t          numLayers,
  bool             isbias,
  bool             batchFirst,
  bool             bidirection,
  const aclTensor *yOut,
  const aclTensor *outputHOut,
  const aclTensor *outputCOut,
  uint64_t        *workspaceSize,
  aclOpExecutor  **executor)

aclnnStatus aclnnBidirectionLSTM(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnBidirectionLSTMGetWorkspaceSize

Parameters:

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (shape)	Non-contiguous Tensor
x (aclTensor*)	Input	Input vector of the LSTM cell, x in the formula.	Supports null Tensor. Shape supports 3D (time_step, batch_size, input_size), where `time_step` represents the time dimension, `batch_size` represents the number of batches processed at each time step, and `input_size` represents the number of input features.	FLOAT16	ND	3	√
initH (aclTensor*)	Input	Initializes the hidden state, h in the formula.	Supports null tensor. shape supports 3D (num_layers, batch_size, hidden_size) or when bidirection is True (2 * num_layers, batch_size, hidden_size). Here, `num_layers` corresponds to the parameter `numLayers`, which represents the number of LSTM layers; `hidden_size` represents the number of hidden state features.	FLOAT16	ND	3	√
initC (aclTensor*)	Input	Initializes the cell state, c in the formula.	Supports null tensor. Shape supports 3D (num_layers, batch_size, hidden_size) or (2 * num_layers, batch_size, hidden_size) when bidirection is True.	FLOAT16	ND	3	√
wIh (aclTensor*)	Input	Input-hidden weight, W in the formula.	Supports null tensor. Supports 2D shape (4 * hidden_size, input_size).	FLOAT16	ND	2	√
wHh (aclTensor*)	Input	Hidden-hidden weight, W in the formula.	Supports null tensor. The shape supports 2D (4 * hidden_size, hidden_size).	FLOAT16	ND	2	√
bIhOptional (aclTensor*)	Input	Input-hidden bias, which is b in the formula.	Supports null tensor. Supports 1D shape (4 * hidden_size).	FLOAT16	ND	1	√
bHhOptional (aclTensor*)	Input	Hidden-hidden bias, i.e., b in the formula.	Supports null tensor. Shape supports 1D (4 * hidden_size).	FLOAT16	ND	1	√
wIhReverseOptional (aclTensor*)	Input	Reverse input-hidden weight, W in the formula.	Supports null tensor. Supports 2D shape (4 * hidden_size, input_size).	FLOAT16	ND	2	√
wHhReverseOptional (aclTensor*)	Input	Reverse hidden-hidden weight, W in the formula.	Supports null tensor. Shape supports 2D (4 * hidden_size, input_size).	FLOAT16	ND	2	√
bIhReverseOptional (aclTensor*)	Input	Reverse input-hidden bias, b in the formula.	Supports null tensor. Shape supports 1D (4 * hidden_size).	FLOAT16	ND	1	√
bHhReverseOptional (aclTensor*)	Input	Reverse hidden-hidden bias, b in the formula.	Supports null tensor. Shape supports 1D (4 * hidden_size).	FLOAT16	ND	1	√
numLayers (int64_t)	Input	Represents the number of LSTM layers.	Currently, only 1 is supported.	-	-	-	-
isbias (bool)	Input	Represents whether bias is present.	-	-	-	-	-
batchFirst (bool)	Input	Indicates whether batch is the first dimension.	Currently only false is supported.	-	-	-	-
bidirection (bool)	Input	Indicates whether it is bidirectional.	-	-	-	-	-
yOut (aclTensor*)	output	Output vector of the LSTM cell.	Shape supports 3D (time_step, batch_size, hidden_size), or (time_step, batch_size, 2 * hidden_size) when bidirection is True.	FLOAT16	ND	3	√
outputHOut (aclTensor*)	Output	Final hidden state, h in the formula.	Supports 3D shape (num_layers, batch_size, hidden_size) or (2 * num_layers, batch_size, hidden_size) when bidirection is True.	FLOAT16	ND	3	√
outputCOut (aclTensor*)	Output	Final cell state, c in the formula.	The shape supports 3D (num_layers, batch_size, hidden_size) or (2 * num_layers, batch_size, hidden_size) when bidirection is True.	FLOAT16	ND	3	√
workspaceSize (uint64_t*)	Output	Returns the workspace size requested by the Device side.	-	-	-	-	-
executor (aclOpExecutor**)	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

Return value

aclnnStatus: return code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Code	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The passed-in parameter is a required input, output, or required attribute, and is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	If the passed-in parameter type is aclTensor and its data type is not within the supported range.
ACLNN_ERR_INNER_TILING_ERROR	561002	If the input parameter type is aclTensor and its shape does not match the parameter description above.

aclnnBidirectionLSTM

Parameters:

Parameter	Input/Output	Description
workspace	Input	Workspace memory address requested on the Device side.
workspaceSize	Input	Workspace size requested on the Device side, obtained from the first-phase API aclnnBidirectionLSTMGetWorkspaceSize.
executor	Input	Operator executor that contains the operator computation flow.
stream	Input	Specifies the stream for executing the task.

Return value

aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

Deterministic computation:
- aclnnBidirectionLSTM defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_bidirection_lstm.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shapeSize = 1;
  for (auto i : shape) {
    shapeSize *= i;
  }
  return shapeSize;
}

void PrintOutResult(std::vector<int64_t> &shape, void** deviceAddr) {
  auto size = GetShapeSize(shape);
  std::vector<float> resultData(size, 0);
  auto ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]),
                         *deviceAddr, size * sizeof(resultData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("mean result[%ld] is: %f\n", i, resultData[i]);
  }
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate for resource initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to request device-side memory.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
  // Copy host-side data to device-side memory using aclrtMemcpy.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. Refer to the ACL API manual.
  // Fill in the deviceId based on your actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the input and output. Custom construction is required based on the API.
  int time_step = 2;
  int batch_size = 32;
  int input_size = 32;
  int hidden_size = 32;

  int64_t numLayers = 1;
  bool isbias = true;
  bool batchFirst = false;
  bool bidirection = true;

  std::vector<int64_t> selfShape = {time_step, batch_size, input_size};
  std::vector<int64_t> weightHIShape = {4 * hidden_size, input_size};
  std::vector<int64_t> weightHHShape = {4 * hidden_size, hidden_size};
  std::vector<int64_t> initHShape = {2, batch_size, hidden_size};
  std::vector<int64_t> initCShape = {2, batch_size, hidden_size};
  std::vector<int64_t> biasHIShape = {4 * hidden_size};
  std::vector<int64_t> biasHHShape = {4 * hidden_size};
  std::vector<int64_t> outShape = {time_step, batch_size, 2 * hidden_size};
  std::vector<int64_t> outHShape = {2, batch_size, hidden_size};
  std::vector<int64_t> outCShape = {2, batch_size, hidden_size};

  void* selfDeviceAddr = nullptr;
  void* weightHIDeviceAddr = nullptr;
  void* weightHHDeviceAddr = nullptr;
  void* weightHIReverseDeviceAddr = nullptr;
  void* weightHHReverseDeviceAddr = nullptr;
  void* initHDeviceAddr = nullptr;
  void* initCDeviceAddr = nullptr;
  void* biasHIDeviceAddr = nullptr;
  void* biasHHDeviceAddr = nullptr;
  void* biasHIReverseDeviceAddr = nullptr;
  void* biasHHReverseDeviceAddr = nullptr;
  void* outDeviceAddr = nullptr;
  void* outHDeviceAddr = nullptr;
  void* outCDeviceAddr = nullptr;

  aclTensor* self = nullptr;
  aclTensor* weightHI = nullptr;
  aclTensor* weightHH = nullptr;
  aclTensor* weightHIReverse = nullptr;
  aclTensor* weightHHReverse = nullptr;
  aclTensor* biasHI = nullptr;
  aclTensor* biasHH = nullptr;
  aclTensor* biasHIReverse = nullptr;
  aclTensor* biasHHReverse = nullptr;
  aclTensor* initH = nullptr;
  aclTensor* initC = nullptr;
  aclTensor* out = nullptr;
  aclTensor* outH = nullptr;
  aclTensor* outC = nullptr;

  std::vector<uint16_t> selfHostData(GetShapeSize(selfShape));
  std::vector<uint16_t> weightHIHostData(GetShapeSize(weightHIShape));
  std::vector<uint16_t> weightHHHostData(GetShapeSize(weightHHShape));
  std::vector<uint16_t> biasHIHostData(GetShapeSize(biasHIShape));
  std::vector<uint16_t> biasHHHostData(GetShapeSize(biasHHShape));
  std::vector<uint16_t> initHHostData(GetShapeSize(initHShape));
  std::vector<uint16_t> initCHostData(GetShapeSize(initCShape));
  std::vector<uint16_t> outHostData(GetShapeSize(outShape));
  std::vector<uint16_t> outHHostData(GetShapeSize(outHShape));
  std::vector<uint16_t> outCHostData(GetShapeSize(outCShape));

  ret = CreateAclTensor(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_FLOAT16, &self);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(weightHIHostData, weightHIShape, &weightHIDeviceAddr, aclDataType::ACL_FLOAT16, &weightHI);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(weightHHHostData, weightHHShape, &weightHHDeviceAddr, aclDataType::ACL_FLOAT16, &weightHH);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(initHHostData, initHShape, &initHDeviceAddr, aclDataType::ACL_FLOAT16, &initH);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(initCHostData, initCShape, &initCDeviceAddr, aclDataType::ACL_FLOAT16, &initC);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biasHIHostData, biasHIShape, &biasHIDeviceAddr, aclDataType::ACL_FLOAT16, &biasHI);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biasHHHostData, biasHHShape, &biasHHDeviceAddr, aclDataType::ACL_FLOAT16, &biasHH);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(weightHIHostData, weightHIShape, &weightHIReverseDeviceAddr, aclDataType::ACL_FLOAT16, &weightHIReverse);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(weightHHHostData, weightHHShape, &weightHHReverseDeviceAddr, aclDataType::ACL_FLOAT16, &weightHHReverse);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biasHIHostData, biasHIShape, &biasHIReverseDeviceAddr, aclDataType::ACL_FLOAT16, &biasHIReverse);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biasHHHostData, biasHHShape, &biasHHReverseDeviceAddr, aclDataType::ACL_FLOAT16, &biasHHReverse);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_FLOAT16, &out);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(outHHostData, outHShape, &outHDeviceAddr, aclDataType::ACL_FLOAT16, &outH);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(outCHostData, outCShape, &outCDeviceAddr, aclDataType::ACL_FLOAT16, &outC);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // 3. Call the CANN operator library API. Modify it to the specific API name.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;

  // Call the first-phase API of aclnnBidirectionLSTM.
  ret = aclnnBidirectionLSTMGetWorkspaceSize(self, initH, initC, weightHI, weightHH,
                                            biasHI, biasHH, weightHIReverse, weightHHReverse, biasHIReverse, biasHHReverse,
                                            numLayers, isbias, batchFirst, bidirection,
                                            out, outH, outC,
                                            &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnBidirectionLSTMGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);

  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }

  // Call the second-phase API of aclnnBidirectionLSTM.
  ret = aclnnBidirectionLSTM(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnBidirectionLSTM failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronously wait for task execution to finish.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify based on the specific API definition.
  PrintOutResult(outShape, &outDeviceAddr);
  PrintOutResult(outHShape, &outHDeviceAddr);
  PrintOutResult(outCShape, &outCDeviceAddr);

  // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
  aclDestroyTensor(self);
  aclDestroyTensor(weightHI);
  aclDestroyTensor(weightHH);
  aclDestroyTensor(initH);
  aclDestroyTensor(initC);
  aclDestroyTensor(biasHI);
  aclDestroyTensor(biasHH);
  aclDestroyTensor(weightHIReverse);
  aclDestroyTensor(weightHHReverse);
  aclDestroyTensor(biasHIReverse);
  aclDestroyTensor(biasHHReverse);
  aclDestroyTensor(out);
  aclDestroyTensor(outH);
  aclDestroyTensor(outC);

  // 7. Release device resources.
  aclrtFree(selfDeviceAddr);
  aclrtFree(weightHIDeviceAddr);
  aclrtFree(weightHHDeviceAddr);
  aclrtFree(initHDeviceAddr);
  aclrtFree(initCDeviceAddr);
  aclrtFree(biasHIDeviceAddr);
  aclrtFree(biasHHDeviceAddr);
  aclrtFree(weightHIReverseDeviceAddr);
  aclrtFree(weightHHReverseDeviceAddr);
  aclrtFree(biasHIReverseDeviceAddr);
  aclrtFree(biasHHReverseDeviceAddr);
  aclrtFree(outDeviceAddr);
  aclrtFree(outHDeviceAddr);
  aclrtFree(outCDeviceAddr);

  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();

  return 0;
}