[Bad Case]: Vllm 部署MiniCPM3-8B 报错

[zhanglt]

Description / 描述

INFO 08-06 08:34:26 [init.py:244] Automatically detected platform cuda.
INFO 08-06 08:34:31 [api_server.py:1287] vLLM API server version 0.9.1
INFO 08-06 08:34:32 [cli_args.py:309] non-default args: {'model': '/llm/models/MiniCPM4-8B', 'dtype': 'half', 'max_model_len': 16384, 'served_model_name': ['MiniCPM4-8B']}
Traceback (most recent call last):
File "/llm/anaconda3/envs/inference/bin/vllm", line 8, in
sys.exit(main())
^^^^^^
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/vllm/entrypoints/cli/main.py", line 59, in main
args.dispatch_function(args)
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/vllm/entrypoints/cli/serve.py", line 58, in cmd
uvloop.run(run_server(args))
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/uvloop/init.py", line 105, in run
return runner.run(wrapper())
^^^^^^^^^^^^^^^^^^^^^
File "/llm/anaconda3/envs/inference/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/uvloop/init.py", line 61, in wrapper
return await main
^^^^^^^^^^
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1323, in run_server
await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1343, in run_server_worker
async with build_async_engine_client(args, client_config) as engine_client:
File "/llm/anaconda3/envs/inference/lib/python3.11/contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 155, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/llm/anaconda3/envs/inference/lib/python3.11/contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 177, in build_async_engine_client_from_engine_args
vllm_config = engine_args.create_engine_config(usage_context=usage_context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/vllm/engine/arg_utils.py", line 1018, in create_engine_config
model_config = self.create_model_config()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/vllm/engine/arg_utils.py", line 910, in create_model_config
return ModelConfig(
^^^^^^^^^^^^
File "/llm/anaconda3/envs/inference/lib/python3.11/site-packages/pydantic/_internal/_dataclasses.py", line 123, in init
s.pydantic_validator.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
Value error, The repository /llm/models/MiniCPM4-8B contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co//llm/models/MiniCPM4-8B.
Please pass the argument trust_remote_code=True to allow custom code to be run. [type=value_error, input_value=ArgsKwargs((), {'model': ..., 'model_impl': 'auto'}), input_type=ArgsKwargs]
For further information visit https://errors.pydantic.dev/2.11/v/value_error

Case Explaination / 案例解释

No response

[suhmily10]

vllm 版本多少呢，试试升级到 0.9.2 以后的版本呢？

[aFcFzF]

需要加参数 --trust_remote_code