jundot/omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Category: General

★ 18.0k · forks 1.5k · watchers 94 · open issues 711

License: Apache-2.0 · Default: main · Topics: apple-silicon, inference-server, llm, macos, mlx, openai-api

github · apple-silicon · 0 reviews · updated 6 days ago

Critics score

—

Audience score

—

Scores hidden until this repo has at least 5 reviews.

Wiki, Report & Thread