Overview

Environment Configuration

The dependency libraries are as follows:

NNAE (8.0.0 ≤ version < 8.5.0)
Python (version ≥ 3.9)
torch (version ≥ 2.0.1)
torch_npu (version ≥ 2.0.1.post4)

NumPy (version ≥ 1.26.4)
scikit-learn (version ≥ 1.4.1.post1)
tqdm (version ≥ 4.66.1)

You can run the pip install command to install torch, torch_npu, NumPy, scikit-learn, and tqdm. The following is an example:

pip install numpy tqdm scikit-learn torch_npu torch

For versions earlier than CANN 8.5.0, you need to install NNAE separately. The installation procedure is as follows:

Download the NNAE package.
Run the following command to add the execution permission:
```
chmod u+x ./Ascend-cann-nnae_{version}_linux-{arch}.run
```

Run the following command for installation:

./Ascend-cann-nnae_{version}_linux-{arch}.run --install

Set environment variables as prompted.

source /{nnae_installation_path}/nnae/set_env.sh

Important Notes

If the following error is displayed when you import torch and torch_npu:
1
.../libgomp.so: cannot allocate memory in static TLS block
Execute export LD_PRELOAD=.../libgomp.so (libgomp.so path in the error message).

If the following dependencies cannot be installed using pip during NumPy installation:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behavior is the source of the following dependency conflicts.
auto-tune 0.1.0 requires decorator, which is not installed.
dataflow 0.0.1 requires jinja2, which is not installed.
opc-tool 0.1.0 requires attrs, which is not installed.
opc-tool 0.1.0 requires decorator, which is not installed.
opc-tool 0.1.0 requires psutil, which is not installed.
schedule-search 0.0.1 requires absl-py, which is not installed.
schedule-search 0.0.1 requires decorator, which is not installed.
te 0.4.0 requires attrs, which is not installed.
te 0.4.0 requires cloudpickle, which is not installed.
te 0.4.0 requires decorator, which is not installed.
te 0.4.0 requires ml-dtypes, which is not installed.
te 0.4.0 requires psutil, which is not installed.
te 0.4.0 requires scipy, which is not installed.
te 0.4.0 requires tornado, which is not installed.

Run the following command:

pip install attrs cloudpickle decorator jinja2 ml-dtypes psutil scipy tornado absl-py

If the training codebook encounters the following issues:
1 2
OpenBLAS warning: precompiled NUM_THREADS exceeded, adding auxiliary array for thread metadata. Segmentation fault (core dumped)
Run the following command:
1
export OPENBLAS_NUM_THREADS=1
This environment variable may affect the performance. After the codebook training is complete, you are advised to set this environment variable to the preset value.
--useOfflineCompile description
Online operator compilation typically takes a longer time than offline compilation. --useOfflineCompile then can be used to determine whether to enable offline operator compilation and use the pre-compiled offline operator package for execution. Note that this option requires the single-operator package to be pre-installed. The procedure for installing the operator package is as follows:
1. Download the operator package.
2. Run the following command to add the execution permission:
  - Versions earlier than CANN 8.5.0
```
chmod u+x ./Ascend-cann-kernels-{chip_type}_{version}_linux-{arch}.run
```
  - CANN 8.5.0 and later versions
```
chmod u+x ./Ascend-cann-{chip_type}-ops_{version}_linux-{arch}.run
```
3. Run the following command for installation:
  - Versions earlier than CANN 8.5.0
```
./Ascend-cann-kernels-{chip_type}_{version}_linux-{arch}.run --install
```
  - CANN 8.5.0 and later versions
```
./Ascend-cann-{chip_type}-ops_{version}_linux-{arch}.run --install
```
4. Set environment variables as prompted.
  - Versions earlier than CANN 8.5.0
```
source /{kernels_installation_path}/kernels/set_env.sh
```
  - CANN 8.5.0 and later versions
```
source /usr/local/Ascend/cann/set_env.sh
```

Parent topic: Vstar Codebook Generation