aclnnCrossEntropyLoss

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	√
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	×
Atlas inference products	×
Atlas training products	×

Function

Description: Computes the cross-entropy loss of the input.
Formula:

When reductionOptional = mean, the cross-entropy loss is calculated as:
$l_n = -weight_{y_n}*log\frac{exp(x_{n,y_n})}{\sum_{c=1}^Cexp(x_{n,c})}*1\{y_n\ !=\ ignoreIndex \}$ $loss=\begin{cases}\sum_{n=1}^N\frac{1}{\sum_{n=1}^Nweight_{y_n}*1\{y_n\ !=\ ignoreIndex \}}l_n,&\text{if reductionOptional = 'mean'} \\\sum_{n=1}^Nl_n,&\text {if reductionOptional = 'sum' }\\\{l_0,l_1,...,l_n\},&\text{if reductionOptional = 'None' }\end{cases}$
log_prob formula:
$lse_n = log\sum_{c=1}^{C}exp(x_{n,c})$ $logProb_{n,c} = x_{n,c} - lse_n$
The zloss formula is:
$zloss_n = lseSquareScaleForZloss * (lse_n)^2$
Where N is the batch size and C is the number of labels.

Function Prototype

Each operator is divided into a two-phase API. The "aclnnCrossEntropyLossGetWorkspaceSize" API must be called first to obtain the workspace size required for computation and the executor that encapsulates the operator computation flow. Then, the "aclnnCrossEntropyLoss" API is called to perform the computation.

aclnnStatus aclnnCrossEntropyLossGetWorkspaceSize(
    const aclTensor *input,
    const aclTensor *target,
    const aclTensor *weightOptional,
    char            *reductionOptional,
    int64_t          ignoreIndex,
    double           labelSmoothing,
    double           lseSquareScaleForZloss,
    bool             returnZloss,
    const aclTensor *lossOut,
    const aclTensor *logProbOut,
    const aclTensor *zlossOut,
    const aclTensor *lseForZlossOut,
    uint64_t        *workspaceSize,
    aclOpExecutor  **executor)

aclnnStatus aclnnCrossEntropyLoss(
    void          *workspace,
    uint64_t       workspaceSize,
    aclOpExecutor *executor,
    aclrtStream    stream)

aclnnCrossEntropyLossGetWorkspaceSize

Parameters

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Shape	Non-contiguous Tensor
input (aclTensor*)	Input	The input in the formula.	-	FLOAT, FLOAT16, BFLOAT16	ND	(N, C) N is the batch size, and C is the number of labels, which must be greater than 0.	√
target (aclTensor*)	Input	Represents the labels, i.e., y in the formula.	-	INT32, INT64	ND	(N) N equals the zeroth dimension of input. The value range is [0, C). When ignoreIndex is specified, the value of target can also equal ignoreIndex.	√
weightOptional (aclTensor*)	Input	Specifies the scaling weight for each class, i.e., weight in the formula.	If not specified, no weighting is applied to the target.	FLOAT	ND	(C)	√
reductionOptional（char*）	Input	Specifies the reduction method for loss.	Supports ["mean", "sum", "none"].	-	-	-	-
ignoreIndex (int64_t)	Input	Specifies the label value to be ignored.	-	INT64	-	-	-
labelSmoothing (double)	Input	Specifies the smoothing amount when computing the loss.	The value ranges from 0.0 to 1.0.	DOUBLE	-	-	-
lseSquareScaleForZloss (double)	Input	Indicates the scale required for zloss computation. It is lse_square_scale_for_zloss in the formula.	A value in the range [0, 1). Currently not supported.	DOUBLE	-	-	-
returnZloss (bool)	Input	Controls whether to return the zloss output.	Pass in True when zLoss output is required, and False otherwise. Currently not supported.	BOOL	-	-	-
lossOut (aclTensor*)	Output	Represents the output loss.	When reductionOptional is "none", the shape is [N], consistent with the zeroth dimension of input; otherwise, the shape is [1].	Same as input.	ND	-	-
logProbOut (aclTensor*)	Output	Output for backward propagation.	-	Same as input.	ND	(N,C)	-
zlossOut (aclTensor*)	Output	Represents the auxiliary loss.	Currently not supported. It must be passed in during computation but does not participate in the calculation.	Same as input.	ND	Same as lossOut.	-
lseForZlossOut (aclTensor*)	Output	Indicates the tensor output to backward propagation in the zloss scenario. The output is None when lseSquareScaleForZloss is 0.	Currently not supported. It must be passed in during computation but does not participate in the computation.	Same as input.	ND	(N)	-
workspaceSize (uint64_t*)	Output	Returns the workspace size to be requested on the Device side.	-	-	-	-	-
executor (aclOpExecutor**)	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

Return Value

aclnnStatus: return code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Value	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The input, target, lossOut, logProbOut, zlossOut, and lseForZlossOut passed in are null pointers.
ACLNN_ERR_PARAM_INVALID	161002	The data types of input, target, lossOut, logProbOut, zlossOut, and lseForZlossOut passed in are not within the supported range.
ACLNN_ERR_INNER_TILING_ERROR	561002	The shapes of the input, target, and weightOptional are not within the supported range.
ACLNN_ERR_INNER_TILING_ERROR	561002	The values of reductionOptional and labelSmoothing are not within the supported range.

aclnnCrossEntropyLoss

Parameters

Parameter	Input/Output	Description
workspace	Input.	Address of the workspace memory applied for on the Device side.
workspaceSize	Input.	Size of the workspace memory applied for on the Device side, obtained from the first-phase API aclnnCrossEntropyLossGetWorkspaceSize.
executor	Input	Operator executor, which contains the operator computation flow.
stream	Input	Specifies the stream for task execution.

Return Value

aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

target only supports class label indices, not probability inputs.
zloss-related features are currently not supported. Passing related inputs, i.e., lseSquareScaleForZloss and returnZloss, will not take effect.
Deterministic computation:
- aclnnCrossEntropyLoss defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_cross_entropy_loss.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shape_size = 1;
  for (auto i : shape) {
    shape_size *= i;
  }
  return shape_size;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to allocate memory on the device side.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

  // Call aclrtMemcpy to copy data from the host side to the device memory.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
    // 1. (Boilerplate) Initialize the device/stream. Refer to the acl API list.
    // Enter the deviceId based on the actual device.
    int32_t deviceId = 0;
    aclrtStream stream;
    auto ret = Init(deviceId, &stream);
    // Perform the check as needed.
    CHECK_RET(ret == 0, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);
    // 2. Construct the input and output. Custom construction is required based on the API.
    std::vector<int64_t> inputShape = {2, 5};
    std::vector<int64_t> targetShape = {2,};
    std::vector<int64_t> weightShape = {5,};
    std::vector<int64_t> lossOutShape = {1,};
    std::vector<int64_t> logProbOutShape = {2,5};
    std::vector<int64_t> zlossOutShape = {1,};
    std::vector<int64_t> lseForZlossOutShape = {2,};

    void* inputDeviceAddr = nullptr;
    void* targetDeviceAddr = nullptr;
    void* weightDeviceAddr = nullptr;

    void* lossOutDeviceAddr = nullptr;
    void* logProbOutDeviceAddr = nullptr;
    void* zlossDeviceAddr = nullptr;
    void* lseForZlossDeviceAddr = nullptr;
    aclTensor* input = nullptr;
    aclTensor* target = nullptr;
    aclTensor* weight = nullptr;
    aclTensor* lossOut = nullptr;
    aclTensor* logProbOut = nullptr;
    aclTensor* zloss = nullptr;
    aclTensor* lseForZloss = nullptr;

    // data
    std::vector<float> inputHostData = {5, 0, 3, 3, 7,
                                            9, 3, 5, 2, 4};
    std::vector<int64_t> targetHostData = {0, 0};
    std::vector<float> lossOutHostData = {1.0937543};
    std::vector<float> logProbOutHostData = {
        -2.159461, -7.159461, -4.159461, -4.159461, -0.159461,
        -0.0280476, -6.0280476, -4.0280476, -7.0280476, -5.0280476};
    std::vector<float> zlossOutHostData = {0};
    std::vector<float> lseForZlossOutHostData = {0, 0};

    // attr
    char* reduction = "mean";
    int64_t ignoreIndex = -100;
    float labelSmoothing = 0.0;
    float lseSquareScaleForZloss = 0.0;
    bool returnZloss = 0;

    // Create the input aclTensor.
    ret = CreateAclTensor(inputHostData, inputShape, &inputDeviceAddr, aclDataType::ACL_FLOAT, &input);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the target aclTensor.
    ret = CreateAclTensor(targetHostData, targetShape, &targetDeviceAddr, aclDataType::ACL_INT64, &target);
    CHECK_RET(ret == ACL_SUCCESS, return ret);

    // Create the lossOut aclTensor.
    ret = CreateAclTensor(lossOutHostData, lossOutShape, &lossOutDeviceAddr, aclDataType::ACL_FLOAT, &lossOut);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the logProbOut aclTensor.
    ret = CreateAclTensor(logProbOutHostData, logProbOutShape, &logProbOutDeviceAddr, aclDataType::ACL_FLOAT, &logProbOut);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the zloss aclTensor.
    ret = CreateAclTensor(zlossOutHostData, zlossOutShape, &zlossDeviceAddr, aclDataType::ACL_FLOAT, &zloss);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // lseForZloss aclTensor
    ret = CreateAclTensor(lseForZlossOutHostData, lseForZlossOutShape, &lseForZlossDeviceAddr, aclDataType::ACL_FLOAT, &lseForZloss);
    CHECK_RET(ret == ACL_SUCCESS, return ret);

    uint64_t workspaceSize = 0;
    aclOpExecutor* executor;

    // 3. Call the CANN operator library API. Replace with the specific API name.
    // Call the first-phase API of aclnnCrossEntropyLoss.
    ret = aclnnCrossEntropyLossGetWorkspaceSize(input, target, weight, reduction, ignoreIndex, labelSmoothing, lseSquareScaleForZloss, returnZloss, lossOut, logProbOut, zloss, lseForZloss, &workspaceSize, &executor);

    CHECK_RET(
        ret == ACL_SUCCESS,
        LOG_PRINT("aclnnCrossEntropyLossGetWorkspaceSize failed. ERROR: %d\n",
                    ret);
        return ret);

    // Allocate device memory based on the workspaceSize calculated by the first-phase API.
    void *workspaceAddr = nullptr;
    if (workspaceSize > 0) {
        ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
        CHECK_RET(ret == ACL_SUCCESS,
                LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret);
                return ret);
    }

    // Call the second-phase API of aclnnCrossEntropyLoss.
    ret = aclnnCrossEntropyLoss(workspaceAddr, workspaceSize, executor, stream);
    CHECK_RET(ret == ACL_SUCCESS,
                LOG_PRINT("aclnnCrossEntropyLoss failed. ERROR: %d\n", ret);
                return ret);

    // 4. (Boilerplate) Synchronize and wait for task completion.
    ret = aclrtSynchronizeStream(stream);
    CHECK_RET(ret == ACL_SUCCESS,
                LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret);
                return ret);

    // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify based on the specific API definition.

    auto size1 = GetShapeSize(lossOutShape);
    auto size2 = GetShapeSize(logProbOutShape);
    std::vector<float> resultData1(size1, 0);
    std::vector<float> resultData2(size2, 0);
    ret = aclrtMemcpy(resultData1.data(), resultData1.size() * sizeof(resultData1[0]), lossOutDeviceAddr,
                        size1 * sizeof(resultData1[0]), ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy loss result from device to host failed. ERROR: %d\n", ret); return ret);
    LOG_PRINT("loss is: \n[");
    for (int64_t i = 0; i < size1; i++) {
        LOG_PRINT("%f, ", resultData1[i]);
    }
    LOG_PRINT("]\n");

    ret = aclrtMemcpy(resultData2.data(), resultData2.size() * sizeof(resultData2[0]), logProbOutDeviceAddr,
                        size2 * sizeof(resultData2[0]), ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy logProb result from device to host failed. ERROR: %d\n", ret); return ret);
    LOG_PRINT("logprob is: \n [");
    for (int64_t i = 0; i < size2; i++) {
        LOG_PRINT("%f,", i, resultData2[i]);
    }
    LOG_PRINT("]\n");

    // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
    aclDestroyTensor(input);
    aclDestroyTensor(target);
    aclDestroyTensor(lossOut);
    aclDestroyTensor(logProbOut);

    // 7. Release device resources.
    aclrtFree(inputDeviceAddr);
    aclrtFree(targetDeviceAddr);
    aclrtFree(lossOutDeviceAddr);
    aclrtFree(logProbOutDeviceAddr);
    if (workspaceSize > 0) {
        aclrtFree(workspaceAddr);
    }
    aclrtDestroyStream(stream);
    aclrtResetDevice(deviceId);
    aclFinalize();
    return 0;
}