inferssd
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
- apple-silicon
- cuda
- gguf
- inference
- llama-cpp
- llm
- metal
- mixture-of-experts
- ssd
- Stars
- 56
- Forks
- 10
- + today
- +2
- Created
- 9d
Ranking data as of October 31, 2026 (UTC).
Star History
Today, hour by hour
02
Overview
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp. It ranks #679 on GitTiger, gaining +2 stars on October 31, 2026 (UTC).
The project is written in C++ and has 10 forks. It was created 9d ago.
Installation
git clone https://github.com/wici-ai/inferssd.git
cd inferssd
# see README for setup