启动脚本

docker run -it -d  --net=host --shm-size=8g \

    --privileged \    

    --name deepseek32b \

    --device=/dev/davinci_manager \

    --device=/dev/hisi_hdc \

    --device=/dev/devmm_svm \

    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \

    -v /usr/local/sbin:/usr/local/sbin:ro \

    -v /data/datasets:/data/datasets:ro \

    mindie:2.0.RC2-300I-Duo-py311-openeuler24.03-lts bash

服务器信息

CPU: 2*Kunpeng 920 32C@2.6G;2.内存:128G;

3.GPU算力:Atlas 300l Pro 24G 推理卡*2;

4. 硬盘:1块内置240G SSD+4块4T硬盘;

5.网卡：四口千兆；5.电源：900W*2

报错信息如下

[2025-08-20 17:33:28.020+0800] [1468] [281472887877984] [batchscheduler] [ERROR] [model.py:62] : [Model]    >>> Exception:NPU out of memory. Tried to allocate 272.00 MiB (NPU 1; 21.02 GiB total capacity; 18.88 GiB already allocated; 18.88 GiB current active; 678.39 MiB free; 18.96 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 

Traceback (most recent call last): 

  File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 60, in initialize 

    return self.python_model.initialize(config) 

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 133, in initialize 

    self.generator = Generator( 

                     ^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 189, in __init__ 

    self.generator_backend = get_generator_backend(model_config) 

                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/__init__.py", line 31, in get_generator_backend 

    return generator_cls(model_config) 

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 133, in __init__ 

    super().__init__(model_config) 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 135, in __init__ 

    self.model_wrapper = get_model_wrapper(model_config, backend_type) 

                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/__init__.py", line 15, in get_model_wrapper 

    return wrapper_cls(**model_config) 

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 79, in __init__ 

    self.model_runner.load_weights() 

  File "/usr/local/Ascend/atb-models/atb_llm/runner/model_runner.py", line 232, in load_weights 

    self.model.to(weights.device) 

  File "/usr/local/lib64/python3.11/site-packages/torch_npu/utils/_module.py", line 78, in to 

    return self._apply(convert) 

           ^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 

    module._apply(fn) 

  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 

    module._apply(fn) 

  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 

    module._apply(fn) 

  [Previous line repeated 3 more times] 

  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 833, in _apply 

    param_applied = fn(param) 

                    ^^^^^^^^^ 

  File "/usr/local/lib64/python3.11/site-packages/torch_npu/utils/_module.py", line 76, in convert 

    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) 

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

RuntimeError: NPU out of memory. Tried to allocate 272.00 MiB (NPU 1; 21.02 GiB total capacity; 18.88 GiB already allocated; 18.88 GiB current active; 678.39 MiB free; 18.96 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 

[2025-08-20 17:33:28.020+0800] [1467] [281473449849184] [batchscheduler] [ERROR] [model.py:62] : [Model]    >>> Exception:NPU out of memory. Tried to allocate 272.00 MiB (NPU 0; 21.02 GiB total capacity; 18.88 GiB already allocated; 18.88 GiB current active; 678.70 MiB free; 18.96 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 

Traceback (most recent call last): 

  File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 60, in initialize 

    return self.python_model.initialize(config) 

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 133, in initialize 

    self.generator = Generator( 

                     ^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 189, in __init__ 

    self.generator_backend = get_generator_backend(model_config) 

                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/__init__.py", line 31, in get_generator_backend 

    return generator_cls(model_config) 

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 133, in __init__ 

    super().__init__(model_config) 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 135, in __init__ 

    self.model_wrapper = get_model_wrapper(model_config, backend_type) 

                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/__init__.py", line 15, in get_model_wrapper 

    return wrapper_cls(**model_config) 

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 79, in __init__ 

    self.model_runner.load_weights() 

  File "/usr/local/Ascend/atb-models/atb_llm/runner/model_runner.py", line 232, in load_weights 

    self.model.to(weights.device) 

  File "/usr/local/lib64/python3.11/site-packages/torch_npu/utils/_module.py", line 78, in to 

    return self._apply(convert) 

           ^^^^^^^^^^^^^^^^^^^^ 

  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 

    module._apply(fn) 

  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 

    module._apply(fn) 

  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 

    module._apply(fn) 

  [Previous line repeated 3 more times] 

  File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 833, in _apply 

    param_applied = fn(param) 

                    ^^^^^^^^^ 

  File "/usr/local/lib64/python3.11/site-packages/torch_npu/utils/_module.py", line 76, in convert 

    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)