Repository navigation
PaddleX/latest/pipeline_deploy/serving #3524
giscus[bot]
Bot
Replies: 60 comments 146 replies
|
您好，paddlex --serve --pipeline image_classification这样启动服务之后，关闭服务的命令是什么呀？ |
|
ctrl+c |
|
paddlex --serve --pipeline {产线名称或产线配置文件路径} [{其他命令行选项}] 如果在部署多个产线， 是否要启动多个服务吗？ |
|
您好，需要启动多个服务 |
|
服务化部署支持实时数据的高性能推理？ |
|
暂时没有支持流式返回的服务。不过，对于实时数据，可以考虑多实例+高性能推理增加并发、降低延迟～ |
|
我在测试‘高稳定性服务化部署’，使用通用OCR SDK，本地docker启动成功，client.py测试GRPCInferenceService通过，Metrics Service访问成功，但是HTTPService请求失败，一直报错400 Bad Request。想问一下HTTPService的接口调用文档有没有，请求参数是什么，怎么才能正确访问HTTPService |
|
你好，我们暂时没有提供HTTPService的调用文档，目前只支持gRPC～ |
|
我也遇见了同样的问题使用http构造请求失败，使用java示例请求也失败了 |
|
SDK的下载链接都失效了，麻烦修复下吧 |
|
你好，目前这块正在升级中，文档先更新了，最新的3.0.1 SDK预计今天之内上传～ |
您好，想问下此次更新主要是更新了什么内容呢？ |
|
这次主要是bug修复，去除了一些无效的参数，另外PP-ChatOCR产线也补充了一个 |
|
您好 |
|
前者是部署镜像，用于高稳定性服务化部署，内置了相关环境；后者是开发镜像，主要用来开发调试paddlex代码，如果对镜像体积不关心，也可以用在部署（高性能推理、基础服务化部署等）。 |
|
docker部署的，为什么λ localhost ~/PaddleX paddlex --serve --pipeline OCR |
|
可能需要先确认一下本地是否能跑通 |
|
我使用阿里云或者腾讯云的linux系统就可以，同样的命令，但是使用麒麟系统的这个就不太行 是不是不太支持这个版本呀？ |
|
建议可以提一个issue，我们会跟进这个问题 |
|
高稳定性服务化部署如何通过http方式调用 |
|
暂不支持，后续我们会考虑添加HTTP调用示例～ |
|
文档中说到 手动构造http请求 的方式，好像不能用呀，端口都没通。 |
|
请问具体报什么错呢？ |
|
已解决，是我机器端口号冲突了。把--network host改为-p 7090:8000 -p 7091:8001 -p 7092:8002就可以了。 |
|
感谢你的反馈，后续我们将考虑增加端口占用检查，并优化错误提示 |
|
您好，这个SDK又不能下载了 |
|
不好意思，这几天在更新中，预计今天之内会上传。 |
|
您好，想请教下CUDA版本为12.4的话 怎么获取ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.1-gpu的版本 这个只支持CUDA版本为11.8的吧？ |
|
你好，我用paddlex3.0.0rc1-gpu版本能跑成功了 但是OCR识别的结果还是和cpu一样 需要10秒左右；显卡是Tesla T4 请问还有其它调优方式吗？config_gpu.pbtxt配置是这样： |
|
请使用paddlex和paddle框架最新版本吧 3.0.0rc1应该是很早以前的版本～ |
|
之前只是paddlex3.1-gpu会报如下错误：Traceback (most recent call last): |
|
这个问题看起来是因为我们底层使用的pypdfium2库无法正确解析这个PDF，建议可以到pypdfium2库提一个issue～ |
|
This could also be a threading issue. pypdfium2 only supports single-threaded use, otherwise you have to guard it with a mutex. |
|
服务化部署如何开启高性能推理？ |
|
基础服务化部署在启动服务时指定参数 |
|
高性能推理插件已经安装，但是加上这个参数启动时报错，不加这个参数是可以正常启动的。 paddlex --serve --use_hpip --pipeline OCRCreating model: ('PP-LCNet_x1_0_doc_ori', None) |
|
请确认你使用的是最新版本的paddlex～ |
|
执行 1.1 paddlex --install serving 一直报错啊，os 版本 AlmaLinux9.6， 在linux 上你们强烈建议使用 docker安装paddlex ，这里怎么没有使用docekr 安装的paddlex 执行服务部署的教程呀， 请大佬指教， 谢谢！ Using cached future-1.0.0-py3-none-any.whl (491 kB) [notice] A new release of pip is available: 25.0.1 -> 25.1.1 During handling of the above exception, another exception occurred: Traceback (most recent call last): |
|
好的，谢谢大佬， 对于 2.4.2 可以使用grpc的方式吗， 监听的有端口， 是不是按http 的方式传入json就可以呢， 如果前台要出图片流， 或者 base64 字符， grpc 如何调用呢，谢谢大佬指导 |
|
不好意思，我刚贴的文档是之前的版本的……可以参考这里： grpc的话，建议用Python客户端来调用；HTTP的话，可以手动构造请求。 |
|
谢谢大佬指导，我们的业务环境比较特殊， 我们前台app 是安卓自动循环调用相机每帧去识别的，所以有大量的请求发送到server端，而且每次识别要尽量快，既然容器监听的有grpc的端口， 那么这种场景grpc 应该比http 性能要好很多吧，可以指导一下grpc的方式吗？ |
|
这个版本 ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.0.3-gpu \ 我运行正常，但是运行 docker run E0716 02:02:04.653384 7 model_repository_manager.cc:1890] Poll failed for model directory 'ocr': instance group ocr_0 of model ocr has kind KIND_GPU but server does not support GPUs I0716 02:02:04.653444 7 server.cc:549] I0716 02:02:04.653449 7 server.cc:592] I0716 02:02:04.653482 7 tritonserver.cc:1920] I0716 02:02:04.653493 7 server.cc:252] Waiting for in-flight requests to complete. |
|
因为你使用的是CPU镜像～请使用GPU镜像吧（以 |
|
https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/serving.html#23 调整后的 pipeline_config.yaml pipeline_name: OCR text_type: general use_doc_preprocessor: True SubPipelines: SubModules: |
|
明白了，看起来是因为部分模型使用的后端（比如paddle）不支持这个配置，所以出现了上面的报错，在不手动固定后端的情况下，可能还是需要逐模型配置～ |
|
{ 大佬，ocr识别的话，这个file必须必须是个url地址吗？ |
|
不是，请参考产线使用文档吧，其中有详细的API参数说明。 |
|
启用hpi时，启动服务发生以下错误 Creating model: ('PP-DocLayout_plus-L', None) |
|
如果这个错误不影响正常使用（只是程序卡住打印这些错误），可以忽略～ |
|
对于这种启动服务后还需下载的模型，因为网络原因无法下载怎么办，可以手动挂载吗 I0718 07:56:17.887946 7 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001 The above exception was the direct cause of the following exception: Traceback (most recent call last): The above exception was the direct cause of the following exception: Traceback (most recent call last): |
|
再问一下，字体文件又该挂载到哪个位置？ |
|
解决了 |
|
2.4.2 手动构造 HTTP 请求，构造的请求体格式不对，"data"中的json应该是String类型。 |
|
pipeline_name: OCR text_type: general use_doc_preprocessor: True SubPipelines: SubModules: Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 224, 224], [1, 3, 224, 224], [8, 3, 224, 224]]} pipeline_config.yaml配置修改下模型为PP-OCRv5_server_det, PP-OCRv5_server_rec 启动报以上错误 原来使用的mobile模型， 我就修改了这个地方，起订报错了，这两个模型不可用吗？ |
|
NFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine... [INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine... 字后还有两个报错，ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 4: Could not find any implementation for node Conv.140 + BatchNormalization.86 + Relu.59 due to insufficient workspace. 这个是空间不足吗？ |
|
这些错误如果不影响正常直行的话可以忽略～ |
|
我这边调用报错了，没法调用 如果ocr mobile5模型有些内容的话识别不出来，换成ocr server5的话，识别精度会不会好点呢 |
|
请问是在PaddleX官方镜像中安装和使用高性能推理插件的吗？ server模型的识别精度更高，不过可能需要消耗更多的计算资源。 |
|
已经部署完毕了 curl -s -X POST http://localhost:8000/v2/models/ocr/infer -H 'Content-Type: application/json' -d "{ |
|
提示是这个 {"error":"failed to parse the request JSON buffer: Missing a name for object member. at 4"} |
|
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.3-cpu 这个镜像进入到容器里 咋啥也没有呢？paddlex, paddle都没有安装么？只有一个python？ |
|
有没有把高稳定性服务化部署完成了的，分享一下咋搞呢？ 然后就报错了，Failed to initialize Python stub: ModuleNotFoundError: No module named 'numpy' |
|
请问你确定c4a02c2d3ac3是ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.3-cpu的image ID嘛？我刚刚试了下，这个镜像里是有Python、PaddleX和Paddle的 
|
|
我搞定了，应该是启动命令写的有问题。应该是-v的目录有问题 |
|
您好，我在910B上使用基础服务部署的，命令paddlex --serve --pipeline ocr 为什么发送请求每张新的图片显存就会增长，但是跑完显存不会释放。有没有解决办法？ |
|
我也遇到这个问题，而且快一年了还是这样，不知道有没有参数可以设置自动释放缓存 |
|
我想问下，可以在paddleOCR-VL产线的ymal配置文件中加入印章识别模块吗？我尝试了将ppstructure的中配置印章模块复制过去，模型路径也做了修改。但是返回结果一直是图片格式，没有返回印章文本。 |
|
在启动容器后，转TensorRT模型，出现这个错误,Error Code 3: API Usage Error,容器会一直循环下载转换模型这一过程。这是什么错误啊。显存还有20多G呢 Creating model: ('PP-LCNet_x1_0_doc_ori', None)
Using official model (PP-LCNet_x1_0_doc_ori), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Downloading [README.md]: 100%|██████████| 6.67k/6.67k [00:00<00:00, 15.0kB/s]
Downloading [inference.json]: 100%|██████████| 102k/102k [00:00<00:00, 226kB/s]
Downloading [config.json]: 100%|██████████| 2.50k/2.50k [00:00<00:00, 3.08kB/s]
Downloading [inference.yml]: 100%|██████████| 766/766 [00:00<00:00, 1.04kB/s]
Downloading [inference.pdiparams]: 100%|██████████| 6.44M/6.44M [00:02<00:00, 2.34MB/s]
Processing 5 items: 100%|██████████| 5.00/5.00 [00:03<00:00, 1.34it/s]<00:00, 1.94MB/s]
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvcaffe_parser.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvcaffe_parser.so.
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvinfer_plugin.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvinfer_plugin.so.
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvinfer.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvinfer.so.
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvonnxparser.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvonnxparser.so.
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvparsers.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvparsers.so.
Automatically converting PaddlePaddle model to ONNX format
Inference backend: tensorrt
Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 224, 224], [1, 3, 224, 224], [8, 3, 224, 224]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(567)::BuildTrtEngine [TrtBackend] Use FP16 to inference.
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine...
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(659)::BuildTrtEngine TensorRT Engine is built successfully.
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(661)::BuildTrtEngine Serialize TensorRTEngine to local file /root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori/.cache/tensorrt/trt_serialized.trt.
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(672)::BuildTrtEngine TensorRTEngine is serialized to local file /root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori/.cache/tensorrt/trt_serialized.trt, we can load this model from the serialized engine directly next time.
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU. |
|
如果不影响最终结果的话，这些ERROR信息可以忽略 |
|
有影响，日志显示已经有服务端口了，使用 服务端会去重复下载转换模型 E0128 00:48:28.966493 19 python.cc:1944] Stub process is unhealthy and it will be restarted.
[ INFO] [2026-01-28 00:48:35,433] [*] [*] - Triton model config: {'name': 'ocr', 'platform': '', 'backend': 'python', 'version_policy': {'latest': {'num_versions': 1}}, 'max_batch_size': 1, 'input': [{'name': 'input', 'data_type': 'TYPE_STRING', 'format': 'FORMAT_NONE', 'dims': [1], 'is_shape_tensor': False, 'allow_ragged_batch': False}], 'output': [{'name': 'output', 'data_type': 'TYPE_STRING', 'dims': [1], 'label_filename': '', 'is_shape_tensor': False}], 'batch_input': [], 'batch_output': [], 'optimization': {'priority': 'PRIORITY_DEFAULT', 'input_pinned_memory': {'enable': True}, 'output_pinned_memory': {'enable': True}, 'gather_kernel_buffer_threshold': 0, 'eager_batching': False}, 'instance_group': [{'name': 'ocr_0', 'kind': 'KIND_GPU', 'count': 3, 'gpus': [2], 'secondary_devices': [], 'profile': [], 'passive': False, 'host_policy': ''}], 'default_model_filename': '', 'cc_model_filenames': {}, 'metric_tags': {}, 'parameters': {}, 'model_warmup': []}
[ INFO] [2026-01-28 00:48:35,434] [*] [*] - Input names: ['input']
[ INFO] [2026-01-28 00:48:35,434] [*] [*] - Output names: ['output']
Creating model: ('PP-LCNet_x1_0_doc_ori', None)
Using official model (PP-LCNet_x1_0_doc_ori), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 8.46it/s]
TensorRT dynamic shapes will be loaded from the file.
Inference backend: tensorrt
Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 224, 224], [1, 3, 224, 224], [8, 3, 224, 224]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(719)::CreateTrtEngineFromOnnx Detect serialized TensorRT Engine file in /root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori/.cache/tensorrt/trt_serialized.trt, will load it directly.
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [8, 3, 224, 224], The shape range before: min_shape=[-1, 3, 224, 224], max_shape=[-1, 3, 224, 224].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[8, 3, 224, 224], max_shape=[8, 3, 224, 224].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [1, 3, 224, 224], The shape range before: min_shape=[8, 3, 224, 224], max_shape=[8, 3, 224, 224].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[1, 3, 224, 224], max_shape=[8, 3, 224, 224].
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(108)::LoadTrtCache Build TensorRT Engine from cache file: /root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori/.cache/tensorrt/trt_serialized.trt with shape range information as below,
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(111)::LoadTrtCache Input name: x, shape=[-1, 3, 224, 224], min=[1, 3, 224, 224], max=[8, 3, 224, 224]
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU.
Creating model: ('UVDoc', None)
Using official model (UVDoc), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 8.91it/s]
The Paddle Inference backend is selected with the default configuration. This may not provide optimal performance.
Using Paddle Inference backend
Paddle predictor option: device_type: gpu, device_id: 2, run_mode: paddle, trt_dynamic_shapes: {'img': [[1, 3, 128, 64], [1, 3, 256, 128], [8, 3, 512, 256]]}, cpu_threads: 10, delete_pass: [], enable_new_ir: True, enable_cinn: False, trt_cfg_setting: {}, trt_use_dynamic_shapes: True, trt_collect_shape_range_info: True, trt_discard_cached_shape_range_info: False, trt_dynamic_shape_input_data: None, trt_shape_range_info_path: None, trt_allow_rebuild_at_runtime: True, mkldnn_cache_capacity: 10
Creating model: ('PP-LCNet_x1_0_textline_ori', None)
Using official model (PP-LCNet_x1_0_textline_ori), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 6.42it/s]
TensorRT dynamic shapes will be loaded from the file.
Inference backend: tensorrt
Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 80, 160], [1, 3, 80, 160], [8, 3, 80, 160]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(719)::CreateTrtEngineFromOnnx Detect serialized TensorRT Engine file in /root/.paddlex/official_models/PP-LCNet_x1_0_textline_ori/.cache/tensorrt/trt_serialized.trt, will load it directly.
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [8, 3, 80, 160], The shape range before: min_shape=[-1, 3, 80, 160], max_shape=[-1, 3, 80, 160].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[8, 3, 80, 160], max_shape=[8, 3, 80, 160].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [1, 3, 80, 160], The shape range before: min_shape=[8, 3, 80, 160], max_shape=[8, 3, 80, 160].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[1, 3, 80, 160], max_shape=[8, 3, 80, 160].
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(108)::LoadTrtCache Build TensorRT Engine from cache file: /root/.paddlex/official_models/PP-LCNet_x1_0_textline_ori/.cache/tensorrt/trt_serialized.trt with shape range information as below,
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(111)::LoadTrtCache Input name: x, shape=[-1, 3, 80, 160], min=[1, 3, 80, 160], max=[8, 3, 80, 160]
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU.
Creating model: ('PP-OCRv5_server_det', None)
Using official model (PP-OCRv5_server_det), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 8.15it/s]
Inference backend: tensorrt
Inference backend config: precision='fp32' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 32, 32], [1, 3, 736, 736], [1, 3, 4000, 4000]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine...
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 1: No Myelin Error exists
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 1: [runnerBuilderBase.cpp::buildAndSerializeMyelinGraph::326] Error Code 1: Myelin (No Myelin Error exists)
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(636)::BuildTrtEngine Failed to call buildSerializedNetwork().
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(752)::CreateTrtEngineFromOnnx Failed to build tensorrt engine.
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU.
Creating model: ('PP-OCRv5_server_rec', None)
Using official model (PP-OCRv5_server_rec), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 6.34it/s]
TensorRT dynamic shapes will be loaded from the file.
Inference backend: tensorrt
Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 48, 160], [1, 3, 48, 320], [8, 3, 48, 3200]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(719)::CreateTrtEngineFromOnnx Detect serialized TensorRT Engine file in /root/.paddlex/official_models/PP-OCRv5_server_rec/.cache/tensorrt/trt_serialized.trt, will load it directly.
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [8, 3, 48, 3200], The shape range before: min_shape=[-1, 3, 48, -1], max_shape=[-1, 3, 48, -1].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[8, 3, 48, 3200], max_shape=[8, 3, 48, 3200].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [1, 3, 48, 160], The shape range before: min_shape=[8, 3, 48, 3200], max_shape=[8, 3, 48, 3200].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[1, 3, 48, 160], max_shape=[8, 3, 48, 3200].
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(108)::LoadTrtCache Build TensorRT Engine from cache file: /root/.paddlex/official_models/PP-OCRv5_server_rec/.cache/tensorrt/trt_serialized.trt with shape range information as below,
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(111)::LoadTrtCache Input name: x, shape=[-1, 3, 48, -1], min=[1, 3, 48, 160], max=[8, 3, 48, 3200]
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU.
[ INFO] [2026-01-28 00:52:34,475] [*] [*] - 79c490ea18004a21ba751b9a3ad1ff21 initialized successfully
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(445)::SetInputs TRTBackend SetInputs not find name:x
--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
No stack trace in paddle, may be caused by external reasons.
----------------------
Error Message Summary:
----------------------
FatalError: `Process abort signal` is detected by the operating system.
[TimeInfo: *** Aborted at 1769563483 (unix time) try "date -d @1769563483" if you are using GNU date ***]
[SignalInfo: *** SIGABRT (@0x165c) received by PID 5724 (TID 0x7fcf78777000) from PID 5724 ***] |
|
有可能是trt引擎构建失败导致的，可以试试给OCRv5_server_rec更换一个后端 |
|
麻烦问一下，文档场景信息抽取v3产线的端点名称是什么？ |
|
v2/models/layout-parsing/infer |
|
用默认构造请求的方式请求，不通。 |
|
你好，使用ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.4-gpu这个镜像。执行docker run |
|
想咨询下http请求解析本地图片，怎么构造请求体 读取本地图片并转换为 base64with open("11.png", "rb") as image_file: response = requests.post( 使用url是正常的，但是改成本地图片就不行了 |
|
已解决 |
|
您好，3.5的镜像拉取不了 docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.5-gpu |
|
请问昇腾如何实现服务化部署多实例+高性能推理，有无参考命令，目前官网文档只找到了高性能推理插件命令--use_hpip， |
|
2026年4月30日，为什么docker镜像还不支持50系列的显卡高性能部署，40系列显卡都淘汰了 |
|
什么时候支持50系列显卡高性能部署啊，有没有时间节点啊 |
|
高稳定性服务化部署 运行报错 docker run -it -e PADDLEX_HPS_DEVICE_TYPE=cpu -v "$(pwd)":/app -w /app --rm --init --network host --shm-size 8g ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.5-cpu /bin/bash server.sh
I0514 07:59:28.376264 7 model_repository_manager.cc:1022] loading: ocr:1
I0514 07:59:28.486763 7 python.cc:1880] TRITONBACKEND_ModelInstanceInitialize: ocr_0 (CPU device 0)
[ INFO] [2026-05-14 07:59:31,474] [*] [*] - Triton model config: {'name': 'ocr', 'platform': '', 'backend': 'python', 'version_policy': {'latest': {'num_versions': 1}}, 'max_batch_size': 8, 'input': [{'name': 'input', 'data_type': 'TYPE_STRING', 'format': 'FORMAT_NONE', 'dims': [-1], 'is_shape_tensor': False, 'allow_ragged_batch': False}], 'output': [{'name': 'output', 'data_type': 'TYPE_STRING', 'dims': [-1], 'label_filename': '', 'is_shape_tensor': False}], 'batch_input': [], 'batch_output': [], 'optimization': {'priority': 'PRIORITY_DEFAULT', 'input_pinned_memory': {'enable': True}, 'output_pinned_memory': {'enable': True}, 'gather_kernel_buffer_threshold': 0, 'eager_batching': False}, 'dynamic_batching': {'preferred_batch_size': [8], 'max_queue_delay_microseconds': 0, 'preserve_ordering': False, 'priority_levels': 0, 'default_priority_level': 0, 'priority_queue_policy': {}}, 'instance_group': [{'name': 'ocr_0', 'kind': 'KIND_CPU', 'count': 1, 'gpus': [], 'secondary_devices': [], 'profile': [], 'passive': False, 'host_policy': ''}], 'default_model_filename': '', 'cc_model_filenames': {}, 'metric_tags': {}, 'parameters': {}, 'model_warmup': []}
[ INFO] [2026-05-14 07:59:31,474] [*] [*] - Input names: ['input']
[ INFO] [2026-05-14 07:59:31,474] [*] [*] - Output names: ['output']
Creating model: ('PP-LCNet_x1_0_doc_ori', None, None)
Checking connectivity to the model hosters, this may take a while. To bypass this check, set `PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK` to `True`.
Using official model (PP-LCNet_x1_0_doc_ori), the model files will be automatically downloaded and saved in `/root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori`.
inference.yml: 100%|███████████████████████████████████████████████████████████████████| 766/766 [00:00<00:00, 1.47MB/s]
.gitattributes: 1.57kB [00:00, 378kB/s]
inference.json: 104kB [00:00, 12.1MB/s]
README.md: 6.85kB [00:00, 1.65MB/s] | 1/6 [00:00<00:02, 1.99it/s]
config.json: 2.56kB [00:00, 4.09MB/s]
inference.pdiparams: 100%|█████████████████████████████████████████████████████████| 6.75M/6.75M [00:01<00:00, 3.49MB/s]
Fetching 6 files: 100%|███████████████████████████████████████████████████████████████████| 6/6 [00:02<00:00, 2.36it/s]
Creating model: ('UVDoc', None, None)
Using official model (UVDoc), the model files will be automatically downloaded and saved in `/root/.paddlex/official_models/UVDoc`.
README.md: 4.46kB [00:00, 206kB/s] | 0/6 [00:00<?, ?it/s]
.gitattributes: 1.57kB [00:00, 220kB/s]
config.json: 1.49kB [00:00, 1.09MB/s]
inference.json: 191kB [00:00, 137MB/s] | 1/6 [00:00<00:01, 2.79it/s]
inference.yml: 100%|███████████████████████████████████████████████████████████████████| 330/330 [00:00<00:00, 1.38MB/s]
inference.pdiparams: 100%|█████████████████████████████████████████████████████████| 32.1M/32.1M [00:13<00:00, 2.35MB/s]
Fetching 6 files: 100%|███████████████████████████████████████████████████████████████████| 6/6 [00:14<00:00, 2.33s/it]
Creating model: ('PP-LCNet_x1_0_textline_ori', None, None)
Using official model (PP-LCNet_x1_0_textline_ori), the model files will be automatically downloaded and saved in `/root/.paddlex/official_models/PP-LCNet_x1_0_textline_ori`.
README.md: 3.55kB [00:00, 8.07MB/s] | 0/6 [00:00<?, ?it/s]
inference.yml: 100%|███████████████████████████████████████████████████████████████████| 735/735 [00:00<00:00, 2.56MB/s]
.gitattributes: 1.57kB [00:00, 1.97MB/s]
inference.json: 104kB [00:00, 83.6MB/s] | 1/6 [00:00<00:01, 2.90it/s]
config.json: 2.49kB [00:00, 11.8MB/s]
inference.pdiparams: 100%|█████████████████████████████████████████████████████████| 6.74M/6.74M [00:01<00:00, 3.48MB/s]
Fetching 6 files: 100%|███████████████████████████████████████████████████████████████████| 6/6 [00:02<00:00, 2.71it/s]
0514 07:59:56.121384 15 pb_stub.cc:352] Failed to initialize Python stub: FileNotFoundError: [Errno 2] No such file or directory: '/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/hpi_model_info_collection.json'
At:
/usr/lib/python3.10/pathlib.py(1117): open
/usr/lib/python3.10/importlib/abc.py(433): open_resource
/usr/lib/python3.10/importlib/resources.py(46): open_binary
/usr/lib/python3.10/importlib/resources.py(82): open_text
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/backend.py(37): _get_hpi_model_info_collection
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/backend.py(125): suggest_inference_backend_and_config
/paddlex/py310/lib/python3.10/site-packages/paddlex/utils/deps.py(145): _wrapper
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/runner.py(114): _determine_backend_and_config
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/runner.py(70): __init__
/paddlex/py310/lib/python3.10/site-packages/paddlex/utils/deps.py(158): _wrapper
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/engines/hpi.py(91): runner_builder
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/engines/_base.py(190): build_runner
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(311): _build_predictor_runner
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(338): _build_predictor_kwargs
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(469): create_predictor
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/base.py(141): create_model
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/ocr/pipeline.py(95): __init__
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/_parallel.py(190): _create_internal_pipeline
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/_parallel.py(135): __init__
/paddlex/py310/lib/python3.10/site-packages/paddlex/utils/deps.py(208): _wrapper
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/__init__.py(169): create_pipeline
/paddlex/py310/lib/python3.10/site-packages/paddlex_hps_server/base_model.py(205): _create_pipeline
/paddlex/py310/lib/python3.10/site-packages/paddlex_hps_server/base_model.py(80): initialize
/paddlex/var/paddlex_model_repo/ocr/1/model.py(47): initialize
E0514 07:59:56.122048 7 model_repository_manager.cc:1186] failed to load 'ocr' version 1: Internal: FileNotFoundError: [Errno 2] No such file or directory: '/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/hpi_model_info_collection.json'
At:
/usr/lib/python3.10/pathlib.py(1117): open
/usr/lib/python3.10/importlib/abc.py(433): open_resource
/usr/lib/python3.10/importlib/resources.py(46): open_binary
/usr/lib/python3.10/importlib/resources.py(82): open_text
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/backend.py(37): _get_hpi_model_info_collection
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/backend.py(125): suggest_inference_backend_and_config
/paddlex/py310/lib/python3.10/site-packages/paddlex/utils/deps.py(145): _wrapper
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/runner.py(114): _determine_backend_and_config
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/runner.py(70): __init__
/paddlex/py310/lib/python3.10/site-packages/paddlex/utils/deps.py(158): _wrapper
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/engines/hpi.py(91): runner_builder
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/engines/_base.py(190): build_runner
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(311): _build_predictor_runner
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(338): _build_predictor_kwargs
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(469): create_predictor
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/base.py(141): create_model
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/ocr/pipeline.py(95): __init__
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/_parallel.py(190): _create_internal_pipeline
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/_parallel.py(135): __init__
/paddlex/py310/lib/python3.10/site-packages/paddlex/utils/deps.py(208): _wrapper
/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/__init__.py(169): create_pipeline
/paddlex/py310/lib/python3.10/site-packages/paddlex_hps_server/base_model.py(205): _create_pipeline
/paddlex/py310/lib/python3.10/site-packages/paddlex_hps_server/base_model.py(80): initialize
/paddlex/var/paddlex_model_repo/ocr/1/model.py(47): initialize
I0514 07:59:56.122230 7 server.cc:522]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+
I0514 07:59:56.122271 7 server.cc:549]
+---------+------+--------+
| Backend | Path | Config |
+---------+------+--------+
+---------+------+--------+
I0514 07:59:56.122375 7 server.cc:592]
+-------+---------+--------------------------------------------------------------------------------------------------+
| Model | Version | Status |
+-------+---------+--------------------------------------------------------------------------------------------------+
| ocr | 1 | UNAVAILABLE: Internal: FileNotFoundError: [Errno 2] No such file or directory: '/paddlex/py310/l |
| | | ib/python3.10/site-packages/paddlex/inference/models/runners/hpi/hpi_model_info_collection.json' |
| | | |
| | | At: |
| | | /usr/lib/python3.10/pathlib.py(1117): open |
| | | /usr/lib/python3.10/importlib/abc.py(433): open_resource |
| | | /usr/lib/python3.10/importlib/resources.py(46): open_binary |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/backend.py(37 |
| | | ): _get_hpi_model_info_collection |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/backend.py(37): _get_hpi_model_info_collection |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/runners/hpi/runner.py(114 |
| | | ): _determine_backend_and_config |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/utils/deps.py(145): _wrapper |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/engines/hpi.py(91): runne |
| | | r_builder |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(311): _build_ |
| | | predictor_runner |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(469): create_ |
| | | predictor |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/ocr/pipeline.py(95): _ |
| | | _init__ |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/_parallel.py(135): __i |
| | | nit__ |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/__init__.py(169): crea |
| | | te_pipeline |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(338): _build_predictor_kwargs |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/models/__init__.py(469): create_predictor |
| | | /paddlex/py310/lib/python3.10/site-packages/paddlex/inference/pipelines/base.py(141): create_model |
+-------+---------+--------------------------------------------------------------------------------------------------+
I0514 07:59:56.122539 7 tritonserver.cc:1920]
+----------------------------------+----------------------------------------------------------------------------------+
| Option | Value |
+----------------------------------+----------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.15.0 |
| server_extensions | classification sequence model_repository model_repository(unload_dependents) sch |
| | edule_policy model_configuration system_shared_memory cuda_shared_memory binary_ |
| | tensor_data statistics |
| model_repository_path[0] | /paddlex/var/paddlex_model_repo |
| model_control_mode | MODE_NONE |
| strict_model_config | 1 |
| rate_limit | OFF |
| pinned_memory_pool_byte_size | 268435456 |
| response_cache_byte_size | 0 |
| min_supported_compute_capability | 0.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
+----------------------------------+----------------------------------------------------------------------------------+
I0514 07:59:56.122570 7 server.cc:252] Waiting for in-flight requests to complete.
I0514 07:59:56.122581 7 server.cc:267] Timeout 30: Found 0 live models and 0 in-flight non-inference requests
error: creating server: Internal - failed to load all models
|
pip install paddlex 全都跑了一遍 paddlex 3.5.2 pip show paddlex cuda 13.0 结果连值都没有，这bug太恶心了吧 |
and others
Uh oh!
There was an error while loading. Please reload this page.
PaddleX/latest/pipeline_deploy/serving
https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/serving.html
All reactions