Back to AI Tools
LM
Models · Large Language ModelsInternLM / Shanghai AI LabOpen source

LMDeploy

Open-source LLM serving toolkit by InternLM.

Company
InternLM / Shanghai AI Lab
Pricing
Free (OSS)
Website GitHub

About LMDeploy

What this tool does and where it fits best.

Toolkit for compressing & serving LLMs (KV quantization, blocked KV cache, batching, TP).

Prompts for LMDeploy

Challenges using LMDeploy

Tool details

Core technical and commercial details.

License
Open Source
Pricing
Free (OSS)
Supported languages

Python, C/C++

Similar Tools

Frequently Asked Questions about LMDeploy