基础环境

cann sdk：7.0.0

问题描述

在A+K上跑模型，发现CPU端runtime执行很慢，速度仅有A+X的30%~50%。

我们写了一个简单的测试程序

import torch
import torch_dipu
import platform
from time import time


lhs = torch.randn((1, 4096), dtype=torch.float32).cuda()
rhs = torch.randn((4096, 4096), dtype=torch.float32).cuda()

print("mm perf start", flush=True)

for i in range(100):
    result = torch.mm(lhs, rhs)


num = 1000
with torch_dipu.profiler.NativeProfile(f"./mm_profile_{platform.machine()}", False):
    start = time()
    for i in range(num):
        result = torch.mm(lhs, rhs)
    duration = time() - start
print(f"average time: {1000 * duration / num}ms", flush=True)

以Runtime@KernelLaunchWithHandle为例，在A+X上耗时19us，而在A+K耗时48us，截图如下：

A+X上lscpu结果

Architecture:        x86_64 

CPU op-mode(s):      32-bit, 64-bit 

Byte Order:          Little Endian 

CPU(s):              128 

On-line CPU(s) list: 0-127 

Thread(s) per core:  2 

Core(s) per socket:  32 

Socket(s):           2 

NUMA node(s):        2 

Vendor ID:           GenuineIntel 

CPU family:          6 

Model:               143 

Model name:          Intel(R) Xeon(R) Platinum 8462Y+ 

Stepping:            8 

CPU MHz:             3947.072 

CPU max MHz:         4100.0000 

CPU min MHz:         800.0000 

BogoMIPS:            5600.00 

Virtualization:      VT-x 

L1d cache:           48K 

L1i cache:           32K 

L2 cache:            2048K 

L3 cache:            61440K 

NUMA node0 CPU(s):   0-31,64-95 

NUMA node1 CPU(s):   32-63,96-127 

Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 invpcid_single intel_ppin cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid bus_lock_detect cldemote movdiri movdir64b enqcmd fsrm md_clear serialize tsxldtrk pconfig arch_lbr amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities

A+K上lscpu结果

a+k: 

Architecture:                    aarch64 

CPU op-mode(s):                  64-bit 

Byte Order:                      Little Endian 

CPU(s):                          192 

On-line CPU(s) list:             0-191 

Thread(s) per core:              1 

Core(s) per socket:              48 

Socket(s):                       4 

NUMA node(s):                    8 

Vendor ID:                       0x48 

Model:                           0 

Stepping:                        0x1 

Frequency boost:                 disabled 

CPU max MHz:                     2600.0000 

CPU min MHz:                     200.0000 

BogoMIPS:                        200.00 

L1d cache:                       12 MiB 

L1i cache:                       12 MiB 

L2 cache:                        96 MiB 

L3 cache:                        192 MiB 

NUMA node0 CPU(s):               0-23 

NUMA node1 CPU(s):               24-47 

NUMA node2 CPU(s):               48-71 

NUMA node3 CPU(s):               72-95 

NUMA node4 CPU(s):               96-119 

NUMA node5 CPU(s):               120-143 

NUMA node6 CPU(s):               144-167 

NUMA node7 CPU(s):               168-191 

Vulnerability Itlb multihit:     Not affected 

Vulnerability L1tf:              Not affected 

Vulnerability Mds:               Not affected 

Vulnerability Meltdown:          Not affected 

Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl 

Vulnerability Spectre v1:        Mitigation; __user pointer sanitization 

Vulnerability Spectre v2:        Not affected 

Vulnerability Srbds:             Not affected 

Vulnerability Tsx async abort:   Not affected 

Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm ssbs

详细的profiler文件和ascend 日志见附件。

请问性能差这么多符合预期吗？有没有A+K上的CPU优化策略，使得差距变小？