Running AOE Tuning
- If there is a single AOE process, ensure that the available drive space in the home directory of the tuning user is greater than or equal to 20 GB and the available memory is greater than or equal to 32 GB. Note that if your model has large-shape operators, a larger memory may be required. If there are multiple AOE processes, increase the drive space and memory accordingly.
- AOE does not allow different users to use the same device for tuning at the same time.
- Before tuning, disable the Profiling function to avoid affecting the tuning result. For details about how to disable the profiling function, see the Performance Tuning Tool User Guide .
- You are advised to perform subgraph tuning and then operator tuning. The reason is that performing subgraph tuning first can generate the graph partition mode. After subgraph tuning is complete, the operators are partitioned into the final shapes. Operator tuning can then be performed based on the final shapes. If operator tuning is performed first, the shapes of the tuned operators are not the final shapes after operator partitioning. This does not meet the actual application scenarios.
- Use AOE to tune subgraphs.
Command example: xx.xx.xx.xx indicates the IP address of NCS in the operating environment, and 8000 indicates the port number of NCS. Replace them as required.
aoe --framework 0 --model ./resnet18.prototxt --weight ./resnet18.caffemodel --job_type 1 --ip xx.xx.xx.xx --port 8000
- Use AOE to reload and tune subgraphs.
After the current subgraph tuning process is interrupted, if you want to continue tuning from the previous phase, select this mode.
Command example: xx.xx.xx.xx indicates the IP address of NCS in the operating environment, and 8000 indicates the port number of NCS. Replace them as required.
aoe --framework 0 --model ./resnet18.prototxt --weight ./resnet18.caffemodel --job_type 1 --ip xx.xx.xx.xx --port 8000 --reload
This command must be used in the same directory where the tuning command is executed last time. The reason is that the intermediate file of the last tuning process is required for reloading tuning.
- Run AOE to tune operators.
Command example: xx.xx.xx.xx indicates the IP address of NCS in the operating environment, and 8000 indicates the port number of NCS. Replace them as required.
aoe --framework 0 --model ./resnet18.prototxt --weight ./resnet18.caffemodel --job_type 2 --ip xx.xx.xx.xx --port 8000