Documentation

Preparing .npy Data of an ONNX Model

Prerequisites

  • Ensure that each operator node has a name. Otherwise, the file name cannot be generated correctly. The operator node without a name must be named first.
  • In this example, each node has only one output. For multi-output operators, you need to configure the output_index obtaining logic in the file name generation implementation.

Code Example

This version does not support the generation of .npy files of an ONNX model. You need to install the ONNX environment and prepare the source .npy files in advance. This section provides only an ONNX .npy file example that meets the accuracy comparison requirements.

To generate an .npy file that meets the accuracy comparison requirements, add the dump operation to the code as follows after the inference is complete.

Create the onnx_dump directory or customize a directory in the current directory based on the output_dump_path parameter in the code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
import os
import onnx
import onnxruntime
import numpy as np
import time

from skl2onnx.helpers.onnx_helper import enumerate_model_node_outputs
from skl2onnx.helpers.onnx_helper import select_model_inputs_outputs
from skl2onnx.helpers.onnx_helper import save_onnx_model

# Modify the model to add an output node.
model_onnx = onnx.load("./resnet50.onnx")
output = []
for out in enumerate_model_node_outputs(model_onnx):
    output.append(out)

num_onnx = select_model_inputs_outputs(model_onnx,outputs=output)
save_onnx_model(num_onnx, "resnet50_dump.onnx")

# Perform inference to obtain an output. In this example, a random input is used.
input_data = np.random.random((1,3,224,224)).astype(np.float32)
input_data.tofile("test_data.bin")
sess = onnxruntime.InferenceSession("resnet50_dump.onnx")
input_name = sess.get_inputs()[0].name
output_name = [node.name for node in sess.get_outputs()]
res = sess.run(output_name, {input_name: input_data})

# Obtain the output name and ensure that the operator nodes are all named properly.
node_name = [node.name for node in model_onnx.graph.node]

# Save data.
node_output_num = [len(node.output) for node in model_onnx.graph.node]
idx = 0
for num, name in zip(node_output_num, node_name):
    for i in range(num):
        data = res[idx] 
        file_name = name + "." + str(i) + "." + str(round(time.time() * 1000000)) + ".npy"
        output_dump_path = os.path.join("./onnx_dump/", file_name)
        np.save(output_dump_path, data.astype(np.float16))
        idx += 1

The .npy data file is named in the format of {op_name}.{output_index}.{timestamp}.npy. Ensure that the {output_index} field is set to 0. Otherwise, no comparison result is generated. This is because the accuracy comparison starts from the first data piece whose output_index value is 0 by default.