开源生态推理引擎
vLLM Ascend是用于在昇腾NPU上高效运行vLLM推理框架的硬件适配插件，实现了昇腾NPU与vLLM框架的无缝对接。借助MindIE Turbo昇腾通用加速套件可在昇腾NPU上实现高效大语言模型推理支持，达到更大的吞吐、更低的时延
'%3e%3cpath%20d='M7.42383659,0.128588822%20L13.5544139,3.47415529%20C13.8094807,3.61334983%2013.9034137,3.93296177%2013.7642192,4.18802855%20C13.7158605,4.27664344%2013.6430288,4.34947506%2013.5544139,4.3978338%20L12.5599242,4.939%20L12.5325284,4.91906381%20C12.2339845,4.75614305%2012.0140048,4.63609617%2011.8194256,4.52991077%20L11.7053008,4.46763081%20L11.3739844,4.28682551%20C11.1430671,4.16080978%2010.8862952,4.02068473%2010.5065153,3.81343769%20C10.5504783,3.71797671%2010.5744972,3.61471789%2010.5744972,3.50694214%20C10.5744972,3.02004037%2010.0842738,2.62532849%209.47955268,2.62532849%20C8.87483152,2.62532849%208.38460816,3.02004037%208.38460816,3.50694214%20C8.38460816,3.99384392%208.87483152,4.3885558%209.47955268,4.3885558%20C9.69511315,4.3885558%209.89612489,4.33840151%2010.0670108,4.25272947%20C10.5675512,4.5258834%2010.8697439,4.69079538%2011.143192,4.84002091%20L11.4286656,4.99580898%20C11.5807572,5.07880813%2011.742371,5.16700367%2011.9424418,5.27618591%20L7.42383659,7.74340027%20C7.10966012,7.91485204%206.72987707,7.91485204%206.41570059,7.74340027%20L0.28512324,4.3978338%20C0.0300564618,4.25863927%20-0.0638765514,3.93902733%200.0753179832,3.68396055%20C0.123676725,3.59534566%200.196508349,3.52251403%200.28512324,3.47415529%20L1.29617987,2.9224332%20C1.31643214,2.94351611%201.34008059,2.96182761%201.36684629,2.97643413%20C2.73497221,3.72304508%203.05315507,3.8966832%203.63959006,4.21700588%20C3.59075501,4.31693686%203.56393322,4.42578352%203.56393322,4.5396742%20C3.56393322,5.02657598%204.05415658,5.42128786%204.65887775,5.42128786%20C5.26359891,5.42128786%205.75382227,5.02657598%205.75382227,4.5396742%20C5.75382227,4.05277243%205.26359891,3.65806055%204.65887775,3.65806055%20C4.45187077,3.65806055%204.25828084,3.70431354%204.09350063,3.78527552%20C3.95865782,3.71168938%203.84083034,3.6473888%203.72836999,3.58601716%20L3.39705364,3.40521186%20C3.05893053,3.22069199%202.66537365,3.00592091%201.90691146,2.59201444%20L4.44795081,1.20190437%20C4.47327058,1.23277304%204.50507711,1.25882742%204.54256365,1.27765658%20L5.6997877,1.85914023%20L5.69940708,2.60208751%20C5.46494583,2.69036042%205.30251025,2.88197558%205.30251025,3.10413787%20C5.30251025,3.41006092%205.61052027,3.65806055%205.99046987,3.65806055%20C6.37041948,3.65806055%206.6784295,3.41006092%206.6784295,3.10413787%20C6.6784295,2.88197558%206.51599392,2.69036042%206.28153267,2.60208751%20L6.28113462,1.67106533%20C6.28113462,1.55614394%206.21763024,1.45130418%206.11757807,1.40104886%20L5.05792415,0.869%20L6.41570059,0.128588822%20C6.72987707,-0.0428629406%207.10966012,-0.0428629406%207.42383659,0.128588822%20Z'%20id='形状结合'%20fill='%23FFFFFF'%3e%3c/path%3e%3cpath%20d='M13.2403904,7.21045458%20L7.49279809,10.1124596%20C7.12652193,10.2973953%206.69412962,10.2971334%206.32807781,10.111754%20L0.599146744,7.21045458%20L0.599146744,7.21045458'%20id='路径'%20stroke='%23FFFFFF'%20stroke-width='1.849763'%20stroke-linecap='round'%20stroke-linejoin='round'%3e%3c/path%3e%3cpath%20d='M13.7737723,10.4834119%20L8.1249557,13.3160625%20C7.36650675,13.6963936%206.47303044,13.6963936%205.71458149,13.3160625%20L0.0657648441,10.4834119%20L0.0657648441,10.4834119'%20id='路径'%20stroke='%23FFFFFF'%20stroke-width='1.93579849'%20stroke-linecap='round'%20stroke-linejoin='round'%3e%3c/path%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e){kind=icon}
MindIE
推理引擎
推理服务
推理模型套件
加速套件
MindIE Service 昇腾推理服务
MindIE SD
MindIE LLM
MindIE Turbo 昇腾通用加速套件
可选
第三方加速引擎（vLLM I …）
AI 框架
'%3e%3cg%20id='编组-131备份'%3e%3cg%20id='编组-19备份-2'%20fill='url(%23linearGradient-1)'%3e%3crect%20id='矩形'%20x='0'%20y='0'%20width='24'%20height='24'%20rx='3.6'%3e%3c/rect%3e%3c/g%3e%3cg%20id='编组'%20transform='translate(8.243649,%205.981847)'%3e%3c/g%3e%3c/g%3e%3cg%20id='形状结合-2'%20transform='translate(2.094362,%206.500000)'%20fill='url(%23linearGradient-3)'%20fill-rule='nonzero'%3e%3cpath%20d='M18.1116505,2.16567476%20L18.961165,2.00643397%20C19.0404531,2.44169213%2019.2669903,2.65932122%2019.6407767,2.65932122%20C19.9352751,2.64870516%2020.0881877,2.54254464%2020.0995146,2.34083963%20C20.0881877,2.18159884%2019.986246,2.07013029%2019.7936893,2.00643397%20C19.6351133,1.94273765%2019.4255663,1.88434936%2019.1650485,1.8312691%20C18.5307443,1.69326041%2018.2249191,1.39070291%2018.2475728,0.92359659%20C18.3042071,0.339713688%2018.7289644,0.0318481583%2019.5218447,0%20C20.3373786,0%2020.8131068,0.318481583%2020.9490291,0.955444748%20L20.1165049,1.1465337%20C20.0372168,0.796203957%2019.8446602,0.626347113%2019.538835,0.636963166%20C19.3122977,0.647579218%2019.1877023,0.73250764%2019.1650485,0.891748432%20C19.1423948,1.05098922%2019.3066343,1.16245778%2019.657767,1.22615409%20C19.9749191,1.28985041%2020.2354369,1.37477883%2020.4393204,1.48093936%20C20.8017799,1.67202831%2020.9886731,1.95335371%2021,2.32491555%20C20.9773463,2.95126267%2020.5299353,3.26974425%2019.657767,3.2803603%20C18.7855987,3.30159241%2018.2702265,2.93003056%2018.1116505,2.16567476%20Z%20M0,0.808589352%20L3.14131607,0.808589352%20L3.14131607,2.1674441%20L1.208,2.167%20L1.208,9.641%20L3.14131607,9.64114525%20L3.14131607,11%20L0,11%20L0,0.808589352%20Z%20M17.3980583,11%20L14.2567422,11%20L14.2567422,9.64114525%20L16.189,9.641%20L16.189,2.167%20L14.2567422,2.1674441%20L14.2567422,0.808589352%20L17.3980583,0.808589352%20L17.3980583,11%20Z%20M4.69360949,2.1674441%20L4.03248328,9.18819366%20L5.87281122,9.18819366%20L6.11445092,5.4250719%20L7.8716548,9.18819366%20L9.52640345,9.18819366%20L10.9955728,5.4250719%20L11.525247,9.18819366%20L13.365575,9.18819366%20L12.6406559,2.1674441%20L10.8099935,2.1674441%20L8.72802589,7.28760881%20L6.52427184,2.1674441%20L4.69360949,2.1674441%20Z'%20id='形状结合'%3e%3c/path%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e){kind=icon}
MindSpore 昇思
PyTorch硬件使能
CANN 异构计算架构'%20fill='%23FFFFFF'%3e%3cg%20id='编组-53备份'%20opacity='0'%20stroke='%23FFFFFF'%20stroke-width='0.666666667'%3e%3crect%20id='矩形备份-6'%20x='0.333333333'%20y='0.333333333'%20width='15.3333333'%20height='15.3333333'%3e%3c/rect%3e%3c/g%3e%3cg%20id='合并形状'%20transform='translate(0.000000,%203.500000)'%3e%3cpath%20d='M6.50746269,0%20C7.28327972,0%207.92084446,0.579436445%207.99316758,1.32033116%20L7.997,1.404%20L8,7.88533333%20C8,8.50094673%207.50094673,9%206.88533333,9%20L1.43283582,9%20C0.626894199,8.90317634%200,8.23429162%200,7.42207792%20C0,6.97435417%200.190490753,6.57018298%200.496463468,6.28299015%20L0.594259697,6.19902439%20C0.712003247,6.12067542%200.850920996,6.0781589%200.99647652,6.0781589%20C1.39214321,6.0781589%201.73134328,6.45034576%201.73134328,6.83766234%20C1.73134328,6.385793%201.55552856,5.77947586%200.997897736,5.77947586%20C0.753570923,5.77947586%200.579818936,5.84655453%200.44997702,5.92468848%20C0.315367699,5.67519728%200.23880597,5.38855872%200.23880597,5.08441558%20C0.23880597,4.08384776%201.0674133,3.27272727%202.08955224,3.27272727%20L2.10515269,3.27274455%20C2.60575771,3.27519073%203.06428826,3.53705649%203.39939966,3.88865235%20C3.26011478,3.33890726%202.76042194,2.88316914%202.27294916,2.79157961%20L2.26865672,2.92207792%20C2.26865672,2.01833924%203.01707624,1.28571429%203.94029851,1.28571429%20C4.19795255,1.28571429%204.44199184,1.34277577%204.65994789,1.44469339%20C4.66149236,1.86771473%205.12839997,2.41462951%205.52194391,2.47917604%20C5.2382247,2.21592377%205.01492537,1.87471803%205.01492537,1.48441405%20L5.017,1.517%20L5.01492537,1.46103896%20C5.01492537,0.654129424%205.68315709,0%206.50746269,0%20Z'%3e%3c/path%3e%3cpath%20d='M8.6,7.3%20L10.6,7.3%20L10.6,8%20L8.6,8%20Z%20M11.3,7.3%20L12,7.3%20L12,8%20L11.3,8%20Z%20M15,6.66666667%20C15.5522847,6.66666667%2016,7.11438192%2016,7.66666667%20C16,8.21895142%2015.5522847,8.66666667%2015,8.66666667%20C14.5646009,8.66666667%2014.1941929,8.38840664%2014.0569087,8.00001923%20L12.7,8%20L12.7,7.3%20L14.0694888,7.29967384%20C14.2158174,6.92895912%2014.5772778,6.66666667%2015,6.66666667%20Z%20M14,3%20C14.5522847,3%2015,3.44771525%2015,4%20C15,4.55228475%2014.5522847,5%2014,5%20C13.5904275,5%2013.2383646,4.75377203%2013.0837642,4.40126897%20L12.33,4.4%20L11.2,5.7%20L11.188,5.699%20L10.3,5.7%20L10.3,5%20L10.881,5%20L12.0121181,3.7%20L13.0457341,3.70014682%20C13.1731152,3.29436031%2013.5521818,3%2014,3%20Z%20M9.6,5%20L9.6,5.7%20L8.6,5.7%20L8.6,5%20L9.6,5%20Z%20M11.3,0.7%20C11.8522847,0.7%2012.3,1.14771525%2012.3,1.7%20C12.3,2.25228475%2011.8522847,2.7%2011.3,2.7%20C11.1429779,2.7%2010.9944085,2.66380925%2010.8621783,2.59931404%20L10.1656059,3.4%20L8.6,3.4%20L8.6,2.7%20L9.846,2.7%20L10.3781603,2.08823948%20C10.3278257,1.96886984%2010.3,1.83768443%2010.3,1.7%20C10.3,1.14771525%2010.7477153,0.7%2011.3,0.7%20Z'%3e%3c/path%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e){kind=icon}
MindStudio
全流程工具链
开发资源
安装资源
开源生态推理引擎资源获取
通过Dockfile文件构建镜像，准备具体模型运行所需的基础环境，包括CANN、FrameworkPTAdapter、MindIE Turbo、vLLM和vLLM Ascend，可实现模型快速上手推理，镜像制作可以参考“镜像制作指导”
模型列表
上手开发
拓展资源
ModelZoo
满足商用标准的一站式昇腾大模型服务平台
魔乐社区
提供丰富的资源，涵盖AI模型推理、优化和应用的最佳实践
vLLM Ascend
昇腾通过vLLM Ascend插件适配vLLM开源框架
昇腾学习
打造人工智能学习平台，做你的技术加油站
博客
技术沉淀，经验分享，记录思考与成长