Skip to content

请问是否可以提供转成onnx的相关指导文档，谢谢 #259

Open
Open
请问是否可以提供转成onnx的相关指导文档，谢谢#259

Description

@IeohMingChan
IeohMingChan
opened on Nov 7, 2024
          您好，由于Reranker采用的是双向注意力，无kv cache机制，因此使用vllm部署并不会有较大的提升。您可以尝试转成onnx

Originally posted by @Kaguya-19 in #258 (comment)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions