Flyweight publishes its first public release, v0.2.0, an open-source native C++/CUDA GGUF inference runtime for MoE models
Flyweight published its first public release, v0.2.0, on 18 Sep 2026 as the flyweight llm wheel on PyPI and on GitHub (github.com/yairpatch/flyweight). It is a…
Published on MyPrivateClaw
Sep 18, 2026, 4:23 PM UTC
Coverage date
Sep 18, 2026
Last updated
Sep 18, 2026, 4:23 PM UTC
News summary
Flyweight published its first public release, v0.2.0, on PyPI as the flyweight llm wheel and on GitHub (github.com/yairpatch/flyweight) on 18 Sep 2026. The GitHub release v0.2.0 is tagged at 18 Sep 07:11 UTC. Flyweight is a native C++/CUDA GGUF inference runtime; Python supplies the CLI, tokenizer facing server adapter, and OpenAI/Anthropic compatible HTTP API. It runs Mixture of Experts models larger than a single GPU's VRAM by offloading experts to system RAM and CPU via CPU, hybrid, and strict resident expert placement, with the Python layer only orchestrating. Flyweight compiles its CUDA kernels at runtime with NVRTC through the driver API, so the build needs no CUDA toolkit and nvcc is never invoked. Wheels ship for Linux x86 64 (manylinux 2.28) and Windows x64, and it can serve on CPU without a GPU via backend cpu, a CPU fallback path. Flyweight is released under the Apache 2.0 li…