Neural Magic
Neural Magic (Acquired by Red Hat) empowers developers to optimize & deploy LLMs at scale. Our model compression & acceleration enable top performance with vLLM
Pinned Loading
Repositories
Showing 10 of 118 repositories
-
- vllm-beam-search-plugin Public
Beam search scheduler plugin for vLLM v1 with CoW block table forking
-
- nm-vllm-omni-ent Public Forked from vllm-project/vllm-omni
A framework for efficient model inference with omni-modality models
-
- vllm-openshift-recipes Public