Follow

vllm比sglang加載本地大模型速度要快不少, 有點意外, 本地部署的參數配置很有講究i

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.