BIOS Configuration

Configuring the High-Performance Mode

Principle: Generally, the Ascend AI Processor provides two power policies: Efficiency and Performance.

• Efficiency indicates the energy-saving mode, where the CPU supports dynamic voltage and frequency scaling (DVFS) to adjust its operating frequency dynamically based on the workload.
• Performance indicates the performance mode, where dynamic frequency scaling is disabled, forcing the CPU to run at a fixed maximum frequency.

Tuning configuration: Set Power Policy to Performance to achieve optimal performance.

Disadvantage: Enabling the high-performance mode results in higher power consumption.

Recommended scenario: It is advised to enable the high-performance mode in inference scenarios. This effectively boosts CPU performance and reduces CPU idle bubbles (stalls), thereby enhancing overall model performance.

The labo installed openEuler 22.03 SP4 on the Atlas 800T A2 and deployed the Llama-7B and Qwen2-7B models to study their inference performance before and after enabling the high-performance mode. The following experimental data were tested based on MindIE. Due to differences in hardware and software environments, the data below are for reference only and do not serve as a performance standard.

Configuration method: Access the BIOS through the BMC and set Power Policy to Performance under Advanced > Performance Config > Power Policy.

Modifying the Memory Refresh Rate

Principle: DRAM memory utilizes capacitors to store data. Due to leakage currents, the electrical charge dissipates over time, preventing long-term data retention. Therefore, continuous recharging is required, a process known as memory refresh. Refresh operations cannot be executed simultaneously with read/write operations, meaning that refreshes negatively impact memory performance. The BIOS provides an Auto option for the memory refresh rate, which automatically adjusts the refresh rate based on the operating temperature. Compared to the default 32 ms configuration, this option can enhance memory performance.

Tuning configuration:Set the memory refresh rate option to Auto.

Recommended scenario: Dynamically adjusting the memory refresh rate can improve memory copy performance to a certain extent, thereby helping to boost overall model performance.

The laboratory installed openEuler 22.03 SP4 on the Atlas 800T A2 and deployed the Llama-7B and Qwen2-7B models to study the impact of different memory refresh rates on the inference performance of the models. The following experimental data were tested based on MindIE. Due to differences in hardware and software environments, the data below are for reference only and do not serve as a performance standard.

Configuration method: Access the BIOS through the BMC and set Custom Refresh Rate to Auto under Advanced > Memory Config > Custom Refresh Rate.

Modifying the CPU Prefetching Configuration

Principle: When the CPU reads data from the memory into its internal high-speed Cache, it leverages the principle of locality. In addition to retrieving the currently requested data, the CPU prefetches surrounding data into the Cache. If this prefetched data matches the data required for the next access, system performance is enhanced.

Tuning configuration: Enable CPU prefetching.

Disadvantage: CPU prefetching yields a high cache hit rate in scenarios where data access is highly concentrated, making it suitable to enable. Conversely, in scenarios with scattered data access, CPU prefetching should be disabled.

Recommended scenario: In inference scenarios, enabling the CPU prefetching feature can effectively improve CPU data-reading performance, thereby boosting overall model performance.

The laboratory installed openEuler 22.03 SP4 on the Atlas 800T A2 and deployed the Llama-7B and Qwen2-7B models to study their inference performance before and after enabling CPU prefetching. The following experimental data were tested based on MindIE. Due to differences in hardware and software environments, the data below are for reference only and do not serve as a performance standard.

Configuration method: Access the BIOS through the BMC and set CPU Prefetching Configuration to Enabled under Advanced > MISC Config > CPU Prefetching Configuration.

Disabling SMMU

Principle: The System Memory Management Unit (SMMU) primarily handles the translation from virtual addresses to physical addresses. However, the SMMU can introduce additional page table lookup overhead and latency, thereby degrading overall system performance.

Tuning configuration: Disable SMMU.

Recommended scenario: Disabling SMMU in non-VM environments (such as BMSs or Docker containers) helps to enhance model performance.

The labo installed openEuler 22.03 SP4 on the Atlas 800T A2 and deployed the Llama-7B and Qwen2-7B models to study their inference performance before and after enabling SMMU. The following experimental data were tested based on MindIE. Due to differences in hardware and software environments, the data below are for reference only and do not serve as a performance standard.

Configuration method: Access the BIOS through the BMC and set Support Smmu to Disabled under Advanced > MISC Config > Support Smmu.