Local LLM inference with one-line setup. CPU-optimized, edge deployment focus. 148K GitHub stars, 2.5M weekly downloads.
Metrics
GitHub Stars
148K
GitHub repository popularity
Radar Status
adopt
ThoughtWorks Technology Radar assessment
Framework Type
inference
Primary framework purpose
Weekly Downloads
2.5M
Package manager install frequency
Score Breakdown
adoption
30%96
Community size, growth velocity, and industry usage
ecosystem
25%86
Integrations, plugins, and extension availability
performance
30%82
Execution speed, latency, and resource efficiency