nemotron-30b-cpu-256k
Running a 30B MoE LLM on CPU at 30-50+ tok/s with a 256K Context Window via Hybrid Mamba-2 SSM, L3 NTA Slicing, AVX-512 VNNI Tile Repacking & 2MB HugePages
- avx512
- cpu-inference
- hpc
- llama-cpp
- llm
- mamba2
- mixture-of-experts
- nemotron
- speculative-decoding
- zen5
- Stars
- 41
- Forks
- 4
- + today
- +1
- Created
- 2d
Ranking data as of October 31, 2026 (UTC).
Star History
Today, hour by hour
01
Overview
Running a 30B MoE LLM on CPU at 30-50+ tok/s with a 256K Context Window via Hybrid Mamba-2 SSM, L3 NTA Slicing, AVX-512 VNNI Tile Repacking & 2MB HugePages It ranks #4144 on GitTiger, gaining +1 star on October 31, 2026 (UTC).
The project is written in C++ and has 4 forks. It was created 2d ago.
Installation
git clone https://github.com/synchopate/nemotron-30b-cpu-256k.git
cd nemotron-30b-cpu-256k
# see README for setup