aclnnLstmBackward

Applicable Products

Function

• Description: Backward propagation of LSTM, which computes the gradient of the forward input, weight params, and initial state hx.

• Formula:

  Single-Layer LSTM Backward Propagation Formula

  Component	Formula
  Input concatenation	$\mathbf{z}_t = \begin{bmatrix} \mathbf{h}_{t-1} \\ \mathbf{x}_t \end{bmatrix}$
  Forget gate	$\mathbf{f}_t = \sigma(\mathbf{W}_f \mathbf{z}_t + \mathbf{b}_f)$
  Input gate	$\mathbf{i}_t = \sigma(\mathbf{W}_i \mathbf{z}_t + \mathbf{b}_i)$
  Candidate state	$\mathbf{g}_t = \tanh(\mathbf{W}_g \mathbf{z}_t + \mathbf{b}_c)$
  Output gate	$\mathbf{o}_t = \sigma(\mathbf{W}_o \mathbf{z}_t + \mathbf{b}_o)$
  Cell state	$\mathbf{c}_t = \mathbf{f}_t \odot \mathbf{c}_{t-1} + \mathbf{i}_t \odot \mathbf{g}_t$
  Hidden state	$\mathbf{h}_t = \mathbf{o}_t \odot \tanh(\mathbf{c}_t)$

  Where:

  • $\sigma$ is the sigmoid function

  • $\odot$ denotes element-wise multiplication (Hadamard product)

  • $W_*$ is a learnable weight matrix

  • $b_*$ is a learnable bias term

  Backward Propagation Variable Definitions

  • Total loss: $L = \sum_{t=1}^{T} L_t$

  • Hidden state gradient: $\delta\mathbf{h}_t = \frac{\partial L}{\partial \mathbf{h}_t}$

  • Cell state gradient: $\delta\mathbf{c}_t = \frac{\partial L}{\partial \mathbf{c}_t}$

  Backward Propagation Algorithm (Time Step t -> t-1)

  • Initialization

  $$\delta\mathbf{h}_{T} = \mathbf{0}, \quad \delta\mathbf{c}_{T} = \mathbf{0}, \quad \mathbf{f}_{T} = \mathbf{0}$$

  • Loop $t = T - 1$ to $0$

    1. Current hidden state gradient
    $$\delta\mathbf{h}_t = \frac{\partial L_t}{\partial \mathbf{h}_t} + \delta\mathbf{h}_{\text{next}}$$
    2. Current cell state gradient
    $$\delta\mathbf{c}_t = \delta\mathbf{h}_t \odot \mathbf{o}_t \odot (1 - \tanh^2(\mathbf{c}_t)) + \delta\mathbf{c}_{\text{next}} \odot \mathbf{f}_{\text{next}}$$
    3. Gate Gradient Computation
    $$\delta\mathbf{o}_t = \delta\mathbf{h}_t \odot \tanh(\mathbf{c}_t) \odot \mathbf{o}_t \odot (1 - \mathbf{o}_t)$$ $$\delta\mathbf{g}_t = \delta\mathbf{c}_t \odot \mathbf{i}_t \odot (1 - \mathbf{g}_t^2)$$ $$\delta\mathbf{i}_t = \delta\mathbf{c}_t \odot \mathbf{g}_t \odot \mathbf{i}_t \odot (1 - \mathbf{i}_t)$$ $$\delta\mathbf{f}_t = \delta\mathbf{c}_t \odot \mathbf{c}_{t-1} \odot \mathbf{f}_t \odot (1 - \mathbf{f}_t)$$
    4. Parameter Gradient Accumulation
    $$\frac{\partial L}{\partial \mathbf{W}_f} \mathrel{+}= \delta\mathbf{f}_t \mathbf{z}_t^\top$$ $$\frac{\partial L}{\partial \mathbf{b}_f} \mathrel{+}= \delta\mathbf{f}_t$$ $$\frac{\partial L}{\partial \mathbf{W}_i} \mathrel{+}= \delta\mathbf{i}_t \mathbf{z}_t^\top$$ $$\frac{\partial L}{\partial \mathbf{b}_i} \mathrel{+}= \delta\mathbf{i}_t$$ $$\frac{\partial L}{\partial \mathbf{W}_g} \mathrel{+}= \delta\mathbf{g}_t \mathbf{z}_t^\top$$ $$\frac{\partial L}{\partial \mathbf{b}_g} \mathrel{+}= \delta\mathbf{g}_t$$ $$\frac{\partial L}{\partial \mathbf{W}_o} \mathrel{+}= \delta\mathbf{o}_t \mathbf{z}_t^\top$$ $$\frac{\partial L}{\partial \mathbf{b}_o} \mathrel{+}= \delta\mathbf{o}_t$$
    5. Propagate to the previous time step
    $$\delta\mathbf{z}_t = \mathbf{W}_f^\top \delta\mathbf{f}_t + \mathbf{W}_i^\top \delta\mathbf{i}_t + \mathbf{W}_g^\top \delta\mathbf{g}_t + \mathbf{W}_o^\top \delta\mathbf{o}_t$$ $$\delta\mathbf{h}_{\text{prev}} = \delta\mathbf{z}_t[1:\dim(\mathbf{h}_{t-1})]$$ $$\delta\mathbf{c}_{\text{prev}} = \delta\mathbf{c}_t \odot \mathbf{f}_t$$
    6. Update propagation variables
    $$\delta\mathbf{h}_{\text{next}} \leftarrow \delta\mathbf{h}_{\text{prev}}$$ $$\delta\mathbf{c}_{\text{next}} \leftarrow \delta\mathbf{c}_{\text{prev}}$$ $$\mathbf{f}_{\text{next}} \leftarrow \mathbf{f}_t$$
  Gradient Computation Principle

  • Cell State Gradient Derivation

    $$\delta\mathbf{c}_t = \frac{\partial L}{\partial \mathbf{h}_t} \frac{\partial \mathbf{h}_t}{\partial \mathbf{c}_t} + \frac{\partial L}{\partial \mathbf{c}_{t+1}} \frac{\partial \mathbf{c}_{t+1}}{\partial \mathbf{c}_t}$$

    Where:

    $$\frac{\partial \mathbf{h}_t}{\partial \mathbf{c}_t} = \mathbf{o}_t \odot (1 - \tanh^2(\mathbf{c}_t))$$ $$\frac{\partial \mathbf{c}_{t+1}}{\partial \mathbf{c}_t} = \mathbf{f}_{t+1}$$

  • Forget Gate Gradient Derivation

    $$\delta\mathbf{f}_t = \frac{\partial L}{\partial \mathbf{a}_f^t} = \delta\mathbf{c}_t \odot \mathbf{c}_{t-1} \odot \mathbf{f}_t \odot (1 - \mathbf{f}_t)$$

  • Parameter Gradient Derivation

    $$\frac{\partial L}{\partial \mathbf{W}_f} = \sum_{t=1}^{T} \delta\mathbf{f}_t \mathbf{z}_t^\top$$

  • LSTM Gradient Flow Characteristics

    Long-range dependency handling

    $$\frac{\partial \mathbf{c}_T}{\partial \mathbf{c}_1} = \prod_{k=2}^{T} \mathbf{f}_k \quad \text{(diagonal matrix)}$$
  Multilayer LSTMBackward backward propagation

  In a multilayer LSTM network, gradient propagation between layers focuses only on the transfer of hidden states (ignoring internal details of a single layer, such as gating mechanisms or cell states). Let:

  • $\mathbf{h}^{(l)}$: hidden state of layer $l$ ($l = 1, 2, \dots, L$, where $L$ is the total number of layers)

  • $L$: loss function

  • $\frac{\partial L}{\partial \mathbf{h}^{(l)}}$: gradient of the loss function with respect to the hidden state of layer $l$

  Core propagation formula

  The gradient propagates from the top layer ($l = L$) to the bottom layer ($l = 1$), and the inter-layer relationship is given by the chain rule:

  $$\frac{\partial L}{\partial \mathbf{h}^{(l-1)}} = \frac{\partial L}{\partial \mathbf{h}^{(l)}} \cdot \frac{\partial \mathbf{h}^{(l)}}{\partial \mathbf{h}^{(l-1)}}$$

  Where:

  • $\frac{\partial L}{\partial \mathbf{h}^{(l)}}$: gradient of the current layer $l$ (obtained from the backward propagation of the previous layer).

  • $\frac{\partial \mathbf{h}^{(l)}}{\partial \mathbf{h}^{(l-1)}}$: Jacobian matrix of the hidden state of layer $l$ with respect to the hidden state of layer $l-1$.

  • $\cdot$: matrix multiplication (gradient propagation is essentially vector-matrix multiplication).

  That is, the gradient dx of the output of each layer is the gradient dy of the input of the previous layer.

Function Prototype

Each operator is divided into a two-phase API. The "aclnnLstmBackwardGetWorkspaceSize" API must be called first to obtain the workspace size required for computation and the executor that encapsulates the operator computation flow. Then, the "aclnnLstmBackward" API is called to perform the computation.

aclnnStatus aclnnLstmBackwardGetWorkspaceSize(
  const aclTensor     *input,
  const aclTensorList *hx,
  const aclTensorList *params,
  const aclTensor     *dy,
  const aclTensor     *dh,
  const aclTensor     *dc,
  const aclTensorList *i,
  const aclTensorList *j,
  const aclTensorList *f,
  const aclTensorList *o,
  const aclTensorList *h,
  const aclTensorList *c,
  const aclTensorList *tanhc,
  const aclTensor     *batchSizesOptional,
  bool                hasBias,
  int64_t             numLayers,
  double              dropout,
  bool                train,
  bool                bidirectional,
  bool                batchFirst,
  const aclBoolArray  *outputMask,
  aclTensor           *dxOut,
  aclTensor           *dhPrevOut,
  aclTensor           *dcPrevOut,
  aclTensorList       *dparamsOut,
  uint64_t            *workspaceSize,
  aclOpExecutor       **executor)

aclnnStatus aclnnLstmBackward(
  void            *workspace,
  uint64_t        workspaceSize,
  aclOpExecutor   *executor,
  aclrtStream     stream)

aclnnLstmBackwardGetWorkspaceSize

• Parameters:

  Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
  input	Input	Fixed-length input sequence of the LSTM, corresponding to x in the formula.	batch_size indicates the number of sequence groups; time_step indicates the time dimension; input_size indicates the number of input features.	FLOAT32, FLOAT16	ND	If a valid batchSizesOptional is passed in, the shape is [time_step * batch_size, input_size]. If a null pointer batchSizesOptional is passed in, the shape is [time_step, batch_size, input_size] or [batch_size, time_step, input_size].	√
  hx	Input	Initial hidden and cell states of each LSTM layer. Corresponds to h(t-1) and c(t-1) at time step 0.	The list length is 2, containing h_0 and c_0. For multilayer bidirectional, each tensor data is arranged along dimension 0 first bidirectional then backward layer by layer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	The shape of each tensor in the list is [D * num_layers, batch_size, hidden_size].	√
  params	Input	List of weight and bias tensors for each layer of the LSTM, corresponding to w and b in the formula.	When bidirection is True, `D = 2`; otherwise, `D = 1`. When hasBiases is True, `B = 2`; otherwise, `B = 1`. The list length is D * B * num_layers. When both bidirection and hasBias are True, the layout is: [weight_ih_0, weight_hh_0, bias_ih_0, bias_hh_0, weight_ih_reverse_0, weight_hh_reverse_0, bias_ih_reverse_0, bias_hh_reverse_0]. When hasBias is False, there is no bias term. When bidirection is False, there is no reverse term. For multiple layers, the layout is arranged layer by layer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	weight_ih: [4*hidden_size, cur_input_size] weight_hh: [4*hidden_size, hidden_size] bias_ih: [4*hidden_size] bias_hh: [4*hidden_size]	√
  dy	Input	Gradient of the hidden state output from the last layer of LSTM forward propagation. Corresponds to ∂L/∂h^(l) in the formula.	For bidirectional mode, data is arranged along the last dimension in forward-backward order. The data type is consistent with input.	FLOAT32, FLOAT16	ND	If a valid batchSizesOptional is passed in, the shape is [time_step * batch_size, hidden_size * D]. If a null pointer batchSizesOptional is passed in, the shape is [time_step, batch_size, hidden_size * D] or [batch_size, time_step, hidden_size * D].	√
  dh	Input	Gradient of the hidden state output by each layer during LSTM forward propagation at time T, passed from the next time step. Corresponds to δh_next.	For multilayer bidirectional, data is arranged along dimension 0 first bidirectional then layer by layer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[numLayers * D, batch_size, hidden_size]	√
  dc	Input	The gradient of the cell output at each LSTM layer at time step T, passed from the next time step. Corresponds to δc_next.	For multilayer bidirectional, data is arranged along dimension 0 first bidirectional then layer by layer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[numLayers * D, batch_size, hidden_size]	√
  i	Input	Activation value of the input gate output by each layer during LSTM forward propagation. Corresponds to i in the formula.	The list length is D * num_layers. For multilayer bidirectional, tensors are arranged first bidirectional then multilayer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[time_step, batch_size, hidden_size]	√
  g	Input	Activation value of the candidate cell state output by each layer in the LSTM forward propagation. Corresponds to g in the formula.	The list length is D * num_layers. For multilayer bidirectional, tensors are arranged first bidirectional then multilayer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[time_step, batch_size, hidden_size]	√
  f	Input.	Activation value of the forget gate at each layer in LSTM forward propagation. Corresponds to f in the formula.	The list length is D * num_layers. For multilayer bidirectional, tensors are arranged first by direction then by layer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[time_step, batch_size, hidden_size]	√
  o	Input	Activation value of the output gate at each layer in the LSTM forward propagation. Corresponds to o in the formula.	The list length is D * num_layers. For multilayer bidirectional, tensors are arranged first bidirectional then multilayer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[time_step, batch_size, hidden_size]	√
  h	Input	Hidden state of each layer in the LSTM forward propagation. Corresponds to h in the formula.	The list length is D * num_layers. For multilayer bidirectional, tensors are arranged first bidirectional then multilayer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[time_step, batch_size, hidden_size]	√
  c	Input	The final cell state of each layer in the LSTM forward propagation. Corresponds to c in the formula.	The list length is D * num_layers. For multilayer bidirectional, tensors are arranged first bidirectional then multilayer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[time_step, batch_size, hidden_size]	√
  tanhc	Input	Output of the final cell state of each layer in LSTM forward propagation after the tanh activation function. Corresponds to tanh(c).	The list length is D * num_layers. For multilayer bidirectional, the tensors are arranged first bidirectional then multilayer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[time_step, batch_size, hidden_size]	√
  batchSizesOptional	Input	Number of valid sequences in the batch at each time step of the variable-length LSTM input sequence.	Supported for variable-length sequences.	INT64	ND	[time_step]	√
  hasBias	Input	Indicates whether bias b exists.	-	BOOL	-	-	-
  numLayers	Input	Indicates the number of LSTM layers.	The value is greater than 0.	INT64	-	-	-
  train	Input	Indicates whether it is a training scenario.	-	BOOL	-	-	-
  bidirection	Input	Indicates whether it is bidirectional.	-	BOOL	-	-	-
  batchFirst	Input	Indicates whether the format of input data input, y, dy, and dxOut has batch in the first dimension.	-	BOOL	-	-	-
  outputMask	Input	Indicates whether to compute four gradients.	The array length is 4. Not supported yet.	ACL_BOOL_ARRAY	-	-	-
  dxOut	Output	Gradient on the input, corresponding to δx in the formula.	The shape is consistent with input. The data type is consistent with input.	FLOAT32, FLOAT16	ND	-	√
  dhPrevOut	Output	Gradient of the initial hidden state of each LSTM layer, corresponding to δh_prev at t = 0.	In the multilayer bidirectional case, data along dimension 0 is arranged first by bidirectional then by layer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[D * num_layers, batch_size, hidden_size]	√
  dcPrevOut	Output	Gradient of the initial cell of each LSTM layer, corresponding to δc_prev at t=0.	For multilayer bidirectional, data is arranged along dimension 0 first bidirectional then layer by layer. The data type is consistent with input.	FLOAT32, FLOAT16	ND	[D * num_layers, batch_size, hidden_size]	√
  dparamsOut	Output	A list of gradient tensors for weights and biases. Corresponds to δw and δb in the formula.	The list length is D * B * num_layers. The layout is consistent with the input params. The data type is consistent with input.	FLOAT32, FLOAT16	ND	dweight_ih: [4*hidden_size, cur_input_size] dweight_hh: [4*hidden_size, hidden_size] dbias: [4*hidden_size]	√
  workspaceSize	Output	Returns the workspace size to be applied on the Device side.	-	-	-	-	-
  executor	Output	Returns the operator executor, which includes the operator computation flow.	-	-	-	-	-

• Return value

  aclnnStatus: return code. For details, see aclnn Return Code.

  The first-phase API performs input parameter validation and returns an error in the following scenarios:

  Return Code	Error Code	Description
  ACLNN_ERR_PARAM_NULLPTR	161001	The passed-in parameter is an aclTensor or aclTensorList, is not batchSizesOptional, and is a null pointer.
  ACLNN_ERR_PARAM_INVALID	161002	If the passed-in parameter is aclTensor or aclTensorList and the data type is not within the supported range.
  		If the passed-in parameter type is aclTensor or aclTensorList and the data types are different.
  		If the passed-in parameter type is aclTensor or aclTensorList and the shape does not meet the corresponding shape requirements.
  		numLayers does not satisfy > 0.

aclnnLstmBackward

• Parameters:

  Parameter	Input/Output	Description
  workspace	Input	Workspace memory address applied on the Device side.
  workspaceSize	Input	Workspace size applied on the Device side, obtained from the first-phase API aclnnLstmBackwardGetWorkspaceSize.
  executor	Input	Operator executor that includes the computation flow of the operator.
  stream	Input	Specifies the stream for task execution.

• Return value

  aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

• Deterministic computation:

  • aclnnLstmBackward defaults to a deterministic implementation.

• Boundary value scenario description:

  • When the input is Inf, the output is NAN.

  • When the input is NaN, the output is NaN.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <cmath>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_lstm_backward.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shapeSize = 1;
  for (auto i : shape) {
    shapeSize *= i;
  }
  return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: AscendCL initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor, aclFormat format=aclFormat::ACL_FORMAT_ND) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to apply for device-side memory.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
  // Call aclrtMemcpy to copy host-side data to device-side memory.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, format,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. See the AscendCL external API list.
  // Fill in the deviceId based on your actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the inputs and outputs. Custom construction is required based on the API interface.
  // Define variables.
  int64_t t = 1;
  int64_t n = 1;
  int64_t inputSize = 8;
  int64_t hiddenSize = 8;

  // Shape definition.
  std::vector<int64_t> xShape = {t, n, inputSize};
  std::vector<int64_t> wiShape = {hiddenSize * 4, inputSize};
  std::vector<int64_t> whShape = {hiddenSize * 4, hiddenSize};
  std::vector<int64_t> bShape = {hiddenSize * 4};
  std::vector<int64_t> yShape = {t, n, hiddenSize};
  std::vector<int64_t> initHShape = {1, n, hiddenSize};
  std::vector<int64_t> initCShape = initHShape;  // Same as initHShape.
  std::vector<int64_t> hShape = yShape;
  std::vector<int64_t> cShape = hShape;
  std::vector<int64_t> dyShape = yShape;
  std::vector<int64_t> dhShape = {1, n, hiddenSize};
  std::vector<int64_t> dcShape = dhShape;
  std::vector<int64_t> iShape = hShape;
  std::vector<int64_t> jShape = hShape;
  std::vector<int64_t> fShape = hShape;
  std::vector<int64_t> oShape = hShape;
  std::vector<int64_t> tanhCtShape = hShape;

  // Backward propagation output tensor shape.
  std::vector<int64_t> dwiShape = wiShape; // Same as wi.
  std::vector<int64_t> dwhShape = wiShape; // Same as wh.
  std::vector<int64_t> dbShape = bShape; // Same as b.
  std::vector<int64_t> dxShape = xShape; // Same as x.
  std::vector<int64_t> dhPrevShape = initHShape; // Same as initH.
  std::vector<int64_t> dcPrevShape = initCShape; // Same as initC.

  // Device address pointer.
  void* xDeviceAddr = nullptr;
  void* wiDeviceAddr = nullptr;
  void* whDeviceAddr = nullptr;
  void* biDeviceAddr = nullptr;
  void* bhDeviceAddr = nullptr;
  void* yDeviceAddr = nullptr;
  void* initHDeviceAddr = nullptr;
  void* initCDeviceAddr = nullptr;
  void* hDeviceAddr = nullptr;
  void* cDeviceAddr = nullptr;
  void* dyDeviceAddr = nullptr;
  void* dhDeviceAddr = nullptr;
  void* dcDeviceAddr = nullptr;
  void* iDeviceAddr = nullptr;
  void* jDeviceAddr = nullptr;
  void* fDeviceAddr = nullptr;
  void* oDeviceAddr = nullptr;
  void* tanhCtDeviceAddr = nullptr;

  // Backward propagation output device address pointer.
  void* dwiDeviceAddr = nullptr;
  void* dwhDeviceAddr = nullptr;
  void* dbiDeviceAddr = nullptr;
  void* dbhDeviceAddr = nullptr;
  void* dxDeviceAddr = nullptr;
  void* dhPrevDeviceAddr = nullptr;
  void* dcPrevDeviceAddr = nullptr;

  // ACL Tensor pointer.
  aclTensor* x = nullptr;
  aclTensor* wi = nullptr;
  aclTensor* wh = nullptr;
  aclTensor* bi = nullptr;
  aclTensor* bh = nullptr;
  aclTensor* y = nullptr;
  aclTensor* initH = nullptr;
  aclTensor* initC = nullptr;
  aclTensor* h = nullptr;
  aclTensor* c = nullptr;
  aclTensor* dy = nullptr;
  aclTensor* dh = nullptr;
  aclTensor* dc = nullptr;
  aclTensor* i = nullptr;
  aclTensor* j = nullptr;
  aclTensor* f = nullptr;
  aclTensor* o = nullptr;
  aclTensor* tanhCt = nullptr;

  // Backward propagation output ACL tensor pointer.
  aclTensor* dwi = nullptr;
  aclTensor* dwh = nullptr;
  aclTensor* dbi = nullptr;
  aclTensor* dbh = nullptr;
  aclTensor* dx = nullptr;
  aclTensor* dhPrev = nullptr;
  aclTensor* dcPrev = nullptr;

  std::vector<float> xHostData(xShape[0] * xShape[1] * xShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> wiHostData(wiShape[0] * wiShape[1], 1.0f); // (8+8)*32 = 16*32 = 512 ones
  std::vector<float> whHostData(whShape[0] * whShape[1], 1.0f); // (8+8)*32 = 16*32 = 512 ones
  std::vector<float> biHostData(bShape[0], 1.0f); // 32 ones
  std::vector<float> bhHostData(bShape[0], 1.0f); // 32 ones
  std::vector<float> yHostData(yShape[0] * yShape[1] * yShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> initHHostData(initHShape[0] * initHShape[1] * initHShape[2], 1.0f); // 1*8 = 8 ones
  std::vector<float> initCHostData(initCShape[0] * initCShape[1] * initCShape[2], 1.0f); // 1*8 = 8 ones
  std::vector<float> hHostData(hShape[0] * hShape[1] * hShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> cHostData(cShape[0] * cShape[1] * cShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> dyHostData(dyShape[0] * dyShape[1] * dyShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> dhHostData(dhShape[0] * dhShape[1] * dhShape[2], 1.0f); // 1*8 = 8 ones
  std::vector<float> dcHostData(dcShape[0] * dcShape[1] * dcShape[2], 1.0f); // 1*8 = 8 ones
  std::vector<float> iHostData(iShape[0] * iShape[1] * iShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> jHostData(jShape[0] * jShape[1] * jShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> fHostData(fShape[0] * fShape[1] * fShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> oHostData(oShape[0] * oShape[1] * oShape[2], 1.0f); // 1*1*8 = 8 ones
  std::vector<float> tanhCtHostData;
  tanhCtHostData.reserve(cHostData.size());
  for (const auto& cVal : cHostData) {
      tanhCtHostData.push_back(std::tanh(cVal)); // Apply the tanh function to each c value.
  }
  // Backward propagation output host data (initialized to 0).
  std::vector<float> dwiHostData(dwiShape[0] * dwiShape[1], 0.0f);
  std::vector<float> dwhHostData(dwhShape[0] * dwhShape[1], 0.0f);
  std::vector<float> dbiHostData(dbShape[0], 0.0f);
  std::vector<float> dbhHostData(dbShape[0], 0.0f);
  std::vector<float> dxHostData(dxShape[0] * dxShape[1] * dxShape[2], 0.0f);
  std::vector<float> dhPrevHostData(dhPrevShape[0] * dhPrevShape[1] * dhPrevShape[2], 0.0f);
  std::vector<float> dcPrevHostData(dcPrevShape[0] * dcPrevShape[1] * dcPrevShape[2], 0.0f);


  // Create x aclTensor.
  ret = CreateAclTensor(xHostData, xShape, &xDeviceAddr, aclDataType::ACL_FLOAT, &x, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create params aclTensorList.
  ret = CreateAclTensor(wiHostData, wiShape, &wiDeviceAddr, aclDataType::ACL_FLOAT, &wi);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(whHostData, whShape, &whDeviceAddr, aclDataType::ACL_FLOAT, &wh);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(biHostData, bShape, &biDeviceAddr, aclDataType::ACL_FLOAT, &bi);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(bhHostData, bShape, &bhDeviceAddr, aclDataType::ACL_FLOAT, &bh);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* paramsArray[] = {wi, wh, bi, bh};
  auto paramsList = aclCreateTensorList(paramsArray, 4);

  // Create y aclTensor.
  ret = CreateAclTensor(yHostData, yShape, &yDeviceAddr, aclDataType::ACL_FLOAT, &y);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create initH aclTensor.
  ret = CreateAclTensor(initHHostData, initHShape, &initHDeviceAddr, aclDataType::ACL_FLOAT, &initH, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create initC aclTensor.
  ret = CreateAclTensor(initCHostData, initCShape, &initCDeviceAddr, aclDataType::ACL_FLOAT, &initC, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* initHcArray[] = {initH, initC};
  auto initHcList = aclCreateTensorList(initHcArray, 2);

  // Create h aclTensor.
  ret = CreateAclTensor(hHostData, hShape, &hDeviceAddr, aclDataType::ACL_FLOAT, &h, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* hArray[] = {h};
  auto hList = aclCreateTensorList(hArray, 1);

  // Create c aclTensor.
  ret = CreateAclTensor(cHostData, cShape, &cDeviceAddr, aclDataType::ACL_FLOAT, &c, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* cArray[] = {c};
  auto cList = aclCreateTensorList(cArray, 1);

  // Create dy aclTensor.
  ret = CreateAclTensor(dyHostData, dyShape, &dyDeviceAddr, aclDataType::ACL_FLOAT, &dy, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create dh aclTensor.
  ret = CreateAclTensor(dhHostData, dhShape, &dhDeviceAddr, aclDataType::ACL_FLOAT, &dh, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create dc aclTensor.
  ret = CreateAclTensor(dcHostData, dcShape, &dcDeviceAddr, aclDataType::ACL_FLOAT, &dc, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create the i aclTensor.
  ret = CreateAclTensor(iHostData, iShape, &iDeviceAddr, aclDataType::ACL_FLOAT, &i, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* iArray[] = {i};
  auto iList = aclCreateTensorList(iArray, 1);

  // Create the j aclTensor.
  ret = CreateAclTensor(jHostData, jShape, &jDeviceAddr, aclDataType::ACL_FLOAT, &j, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* jArray[] = {j};
  auto jList = aclCreateTensorList(jArray, 1);

  // Create the f aclTensor.
  ret = CreateAclTensor(fHostData, fShape, &fDeviceAddr, aclDataType::ACL_FLOAT, &f, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* fArray[] = {f};
  auto fList = aclCreateTensorList(fArray, 1);

  // Create the o aclTensor.
  ret = CreateAclTensor(oHostData, oShape, &oDeviceAddr, aclDataType::ACL_FLOAT, &o, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* oArray[] = {o};
  auto oList = aclCreateTensorList(oArray, 1);

  // Create the tanhCt aclTensor.
  ret = CreateAclTensor(tanhCtHostData, tanhCtShape, &tanhCtDeviceAddr, aclDataType::ACL_FLOAT, &tanhCt, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* tanhCtArray[] = {tanhCt};
  auto tanhCtList = aclCreateTensorList(tanhCtArray, 1);

  // Create backward propagation output tensors.

  // Create the dx aclTensor.
  ret = CreateAclTensor(dxHostData, dxShape, &dxDeviceAddr, aclDataType::ACL_FLOAT, &dx, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create the dhPrev aclTensor.
  ret = CreateAclTensor(dhPrevHostData, dhPrevShape, &dhPrevDeviceAddr, aclDataType::ACL_FLOAT, &dhPrev, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create the dcPrev aclTensor.
  ret = CreateAclTensor(dcPrevHostData, dcPrevShape, &dcPrevDeviceAddr, aclDataType::ACL_FLOAT, &dcPrev, aclFormat::ACL_FORMAT_NCL);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // Create the dparams aclTensorList.
  ret = CreateAclTensor(dwiHostData, dwiShape, &dwiDeviceAddr, aclDataType::ACL_FLOAT, &dwi);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(dwhHostData, dwhShape, &dwhDeviceAddr, aclDataType::ACL_FLOAT, &dwh);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(dbiHostData, bShape, &dbiDeviceAddr, aclDataType::ACL_FLOAT, &dbi);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(dbhHostData, bShape, &dbhDeviceAddr, aclDataType::ACL_FLOAT, &dbh);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  aclTensor* dparamsArray[] = {dwi, dwh, dbi, dbh};
  auto dparamsList = aclCreateTensorList(dparamsArray, 4);

  // 3. Call the CANN operator library API. Replace with the specific API name.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;
  // Call the first-phase API of aclnnLstmBackward.
  ret = aclnnLstmBackwardGetWorkspaceSize(x, initHcList, paramsList, dy, dh, dc, iList, jList, fList,
    oList, hList, cList ,tanhCtList, nullptr, true, 1, 0, true, false, false, nullptr, dx, dhPrev, dcPrev, dparamsList,
    &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnLstmBackwardGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }
  // Call the second-phase API of aclnnLstmBackward.
  ret = aclnnLstmBackward(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnLstmBackward failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronize and wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output values and copy the results from the device side memory to the host side. Modify based on the specific API definition.
  // Print the dparams result.
  auto dwiSize = GetShapeSize(dwiShape);
  std::vector<float> resultDwiData(dwiSize, 0);
  ret = aclrtMemcpy(resultDwiData.data(), resultDwiData.size() * sizeof(resultDwiData[0]), dwiDeviceAddr,
                    dwiSize * sizeof(resultDwiData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy dwi result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < dwiSize; i++) {
    LOG_PRINT("result dwi[%ld] is: %f\n", i, resultDwiData[i]);
  }

  auto dwhSize = GetShapeSize(dwhShape);
  std::vector<float> resultDwhData(dwhSize, 0);
  ret = aclrtMemcpy(resultDwhData.data(), resultDwhData.size() * sizeof(resultDwhData[0]), dwhDeviceAddr,
                    dwhSize * sizeof(resultDwhData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy dwh result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < dwhSize; i++) {
    LOG_PRINT("result dwh[%ld] is: %f\n", i, resultDwhData[i]);
  }

  auto dbiSize = GetShapeSize(bShape);
  std::vector<float> resultDbiData(dbiSize, 0);
  ret = aclrtMemcpy(resultDbiData.data(), resultDbiData.size() * sizeof(resultDbiData[0]), dbiDeviceAddr,
                    dbiSize * sizeof(resultDbiData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy dbi result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < dbiSize; i++) {
    LOG_PRINT("result dbi[%ld] is: %f\n", i, resultDbiData[i]);
  }

  auto dbhSize = GetShapeSize(bShape);
  std::vector<float> resultDbhData(dbhSize, 0);
  ret = aclrtMemcpy(resultDbhData.data(), resultDbhData.size() * sizeof(resultDbhData[0]), dbhDeviceAddr,
                    dbhSize * sizeof(resultDbhData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy dbh result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < dbhSize; i++) {
    LOG_PRINT("result dbh[%ld] is: %f\n", i, resultDbhData[i]);
  }

  // Print the dx result.
  auto dxSize = GetShapeSize(dxShape);
  std::vector<float> resultDxData(dxSize, 0);
  ret = aclrtMemcpy(resultDxData.data(), resultDxData.size() * sizeof(resultDxData[0]), dxDeviceAddr,
                    dxSize * sizeof(resultDxData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy dx result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < dxSize; i++) {
    LOG_PRINT("result dx[%ld] is: %f\n", i, resultDxData[i]);
  }

  // Print the dh_prev result.
  auto dhPrevSize = GetShapeSize(dhPrevShape);
  std::vector<float> resultDhPrevData(dhPrevSize, 0);
  ret = aclrtMemcpy(resultDhPrevData.data(), resultDhPrevData.size() * sizeof(resultDhPrevData[0]), dhPrevDeviceAddr,
                    dhPrevSize * sizeof(resultDhPrevData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy dh_prev result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < dhPrevSize; i++) {
    LOG_PRINT("result dh_prev[%ld] is: %f\n", i, resultDhPrevData[i]);
  }

  // Print the dc_prev result.
  auto dcPrevSize = GetShapeSize(dcPrevShape);
  std::vector<float> resultDcPrevData(dcPrevSize, 0);
  ret = aclrtMemcpy(resultDcPrevData.data(), resultDcPrevData.size() * sizeof(resultDcPrevData[0]), dcPrevDeviceAddr,
                    dcPrevSize * sizeof(resultDcPrevData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy dc_prev result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < dcPrevSize; i++) {
    LOG_PRINT("result dc_prev[%ld] is: %f\n", i, resultDcPrevData[i]);
  }

  // Release the aclTensor.
  aclDestroyTensor(x);
  aclDestroyTensor(wi);
  aclDestroyTensor(wh);
  aclDestroyTensor(bi);
  aclDestroyTensor(bh);
  aclDestroyTensor(y);
  aclDestroyTensor(initH);
  aclDestroyTensor(initC);
  aclDestroyTensor(h);
  aclDestroyTensor(c);
  aclDestroyTensor(dy);
  aclDestroyTensor(dh);
  aclDestroyTensor(dc);
  aclDestroyTensor(i);
  aclDestroyTensor(j);
  aclDestroyTensor(f);
  aclDestroyTensor(o);
  aclDestroyTensor(tanhCt);
  aclDestroyTensor(dwi);
  aclDestroyTensor(dwh);
  aclDestroyTensor(dbi);
  aclDestroyTensor(dbh);
  aclDestroyTensor(dx);
  aclDestroyTensor(dhPrev);
  aclDestroyTensor(dcPrev);

  // Release the tensorList.
  aclDestroyTensorList(paramsList);
  aclDestroyTensorList(initHcList);
  aclDestroyTensorList(hList);
  aclDestroyTensorList(cList);
  aclDestroyTensorList(iList);
  aclDestroyTensorList(jList);
  aclDestroyTensorList(fList);
  aclDestroyTensorList(oList);
  aclDestroyTensorList(tanhCtList);
  aclDestroyTensorList(dparamsList);

  // Release the Device resources.
  aclrtFree(xDeviceAddr);
  aclrtFree(wiDeviceAddr);
  aclrtFree(whDeviceAddr);
  aclrtFree(biDeviceAddr);
  aclrtFree(bhDeviceAddr);
  aclrtFree(yDeviceAddr);
  aclrtFree(initHDeviceAddr);
  aclrtFree(initCDeviceAddr);
  aclrtFree(hDeviceAddr);
  aclrtFree(cDeviceAddr);
  aclrtFree(dyDeviceAddr);
  aclrtFree(dhDeviceAddr);
  aclrtFree(dcDeviceAddr);
  aclrtFree(iDeviceAddr);
  aclrtFree(jDeviceAddr);
  aclrtFree(fDeviceAddr);
  aclrtFree(oDeviceAddr);
  aclrtFree(tanhCtDeviceAddr);
  aclrtFree(dwiDeviceAddr);
  aclrtFree(dwhDeviceAddr);
  aclrtFree(dbiDeviceAddr);
  aclrtFree(dbhDeviceAddr);
  aclrtFree(dxDeviceAddr);
  aclrtFree(dhPrevDeviceAddr);
  aclrtFree(dcPrevDeviceAddr);
  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();

  return 0;
}