Skip to main content

Tag · 4 repos

inference-engine

Repositories carrying the inference-engine tag.

Exact tag match4 sample repositories
925
nobodywho-ooo/nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

RustMobile & Desktop AppsLLM Inference & Serving
930
ARahim3/mlx-dspark

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

PythonLocal LLMsLLM Inference & Serving
2044
youssofal/MTPLX

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

PythonLocal LLMsLLM Inference & Serving
4850
FareedKhan-dev/kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

CLLM Inference & ServingMachine Learning & Data Science