Computation Accuracy Evaluation Indicators

Accuracy comparison uses the GPU-based computation result as the benchmark to compare the operator accuracy differences in different algorithm dimensions, and determines whether the running operators have accuracy issues based on the result of each algorithm dimension.

The following are recommended accuracy evaluation indicators. You need to determine the actual accuracy that your operators need to meet. For details, see Parameters in a Complete Model Comparison Result.

Computation accuracy evaluation indicators:

1. CosineSimilarity: Calculate the cosine values of two vectors to determine their similarity. A value closer to 1 indicates that the two calculated tensors are more similar. The acceptable threshold for similarity is a value greater than 0.99. In this process, NAN may exist because one of the vectors may be 0.
2. RelativeEuclideanDistance: A RelativeEuclideanDistance value closer to 0 indicates higher similarity. The acceptable threshold for similarity is a value smaller than 0.05.
3. KullbackLeiblerDivergence: The smaller the Kullback-Leibler divergence, the closer the approximate distribution is to the true distribution. The acceptable threshold for similarity is a value smaller than 0.005.
4. MeanAbsoluteError and RootMeanSquareError: These two indicators are associated with each other. A larger MeanAbsoluteError value and a RootMeanSquareError value that is equal to or close to the MeanAbsoluteError value suggest that the overall deviation is more centralized. The acceptable threshold for similarity is a value equal to 1.