华为计算微信公众号
昇腾AI开发者公众号
华为计算微博
华为计算今日头条
【问题描述】
启动Flask 服务时无异常，模型成功加载，但调用model.execute就会报错：alloc device memory failed, error: 107002
错误代码：
from flask import Flask, request import numpy as np import acl from acllite_resource import AclLiteResource from acllite_model import AclLiteModel from acllite_utils import display_time acl_resource = AclLiteResource() acl_resource.init() model_path = 'model.om' model = AclLiteModel(model_path) app = Flask(__name__) @app.route('/infer', methods=['POST']) def det_infer(): input_data = np.zeros((1,3,244,244)) output = model.execute([input_data]) resp = dict(result=output) return resp if __name__ == '__main__': app.run(host="0.0.0.0", port=8080)
【解决方案】
1.在det_infer()函数中调用acl.rt.create_context接口创建context，加载模型并进行推理
from flask import Flask, request import numpy as np import acl from acllite_resource import AclLiteResource from acllite_model import AclLiteModel from acllite_utils import display_time acl_resource = AclLiteResource() acl_resource.init() model_path = 'model.om' app = Flask(__name__) @app.route('/infer', methods=['POST']) def det_infer(): device_id, ret = acl.rt.get_device() context, ret = acl.rt.create_context(0) model = AclLiteModel(model_path) input_data = np.zeros((1,3,244,244)) output = model.execute([input_data]) resp = dict(result=output) return resp if __name__ == '__main__': app.run(host="0.0.0.0", port=8080)
2.获取模型加载的Context，在det_infer()函数内调用acl.rt.set_context来设置函数中的context
from flask import Flask, request import numpy as np import acl from acllite_resource import AclLiteResource from acllite_model import AclLiteModel from acllite_utils import display_time acl_resource = AclLiteResource() context = acl_resource.init() model_path = 'model.om' model = AclLiteModel(model_path) app = Flask(__name__) @app.route('/infer', methods=['POST']) def det_infer(): ret = acl.rt.set_context(context) input_data = np.zeros((1,3,244,244)) output = model.execute([input_data]) resp = dict(result=output) return resp if __name__ == '__main__': app.run(host="0.0.0.0", port=8080)
本帖最后由 匿名用户 于 2024/06/27 16:38:01 编辑
我要发帖子
点赞
收藏
回复
分享
举报
【问题描述】
启动Flask 服务时无异常，模型成功加载，但调用model.execute就会报错：alloc device memory failed, error: 107002
错误代码：
from flask import Flask, request import numpy as np import acl from acllite_resource import AclLiteResource from acllite_model import AclLiteModel from acllite_utils import display_time acl_resource = AclLiteResource() acl_resource.init() model_path = 'model.om' model = AclLiteModel(model_path) app = Flask(__name__) @app.route('/infer', methods=['POST']) def det_infer(): input_data = np.zeros((1,3,244,244)) output = model.execute([input_data]) resp = dict(result=output) return resp if __name__ == '__main__': app.run(host="0.0.0.0", port=8080)【解决方案】
1.在det_infer()函数中调用acl.rt.create_context接口创建context，加载模型并进行推理
from flask import Flask, request import numpy as np import acl from acllite_resource import AclLiteResource from acllite_model import AclLiteModel from acllite_utils import display_time acl_resource = AclLiteResource() acl_resource.init() model_path = 'model.om' app = Flask(__name__) @app.route('/infer', methods=['POST']) def det_infer(): device_id, ret = acl.rt.get_device() context, ret = acl.rt.create_context(0) model = AclLiteModel(model_path) input_data = np.zeros((1,3,244,244)) output = model.execute([input_data]) resp = dict(result=output) return resp if __name__ == '__main__': app.run(host="0.0.0.0", port=8080)2.获取模型加载的Context，在det_infer()函数内调用acl.rt.set_context来设置函数中的context
from flask import Flask, request import numpy as np import acl from acllite_resource import AclLiteResource from acllite_model import AclLiteModel from acllite_utils import display_time acl_resource = AclLiteResource() context = acl_resource.init() model_path = 'model.om' model = AclLiteModel(model_path) app = Flask(__name__) @app.route('/infer', methods=['POST']) def det_infer(): ret = acl.rt.set_context(context) input_data = np.zeros((1,3,244,244)) output = model.execute([input_data]) resp = dict(result=output) return resp if __name__ == '__main__': app.run(host="0.0.0.0", port=8080)