ztxz16

fastllm is a high-performance, backend-independent inference library. Supports tensor-parallel dense models and hybrid MOE inference; GPUs with 10G+ can run full DeepSeek. A dual-socket 9004/9005 server with one GPU runs the original full-precision model at 20tps single-concurrency; INT4 reaches 30tps single and 60+ concurrent.