lyogavin/airllm
AirLLM 70B inference with single 4GB GPU
Key features
- Runs 70B models on a single 4GB GPU
- Runs a 2.8T model in about 3.7GB VRAM
- Trains 125B models under 6GB by streaming layers
- Block-wise compression speeds inference by up to 3x
- One AutoModel class loads many Hugging Face models
- Category
- Trending Repos
- Who it's for
- People who run or train large language models on small GPUs.
- Tags
- githublyogavingpu
- Website
- github.com
- Listed
- 2026-10-12
Frequently asked questions
Is lyogavin/airllm free?
Yes. lyogavin/airllm is open source and free to use.
What is lyogavin/airllm used for?
AirLLM 70B inference with single 4GB GPU. Typical uses: runs 70B models on a single 4GB GPU; runs a 2.8T model in about 3.7GB VRAM; trains 125B models under 6GB by streaming layers.
Related in Trending Repos
See all →- addyosmani/agent-skills Engineering skills for AI coding agents, from Addy Osmani.
- anthropics/knowledge-work-plugins Open source plugins for knowledge workers to use in Claude Cowork.
- aquasecurity/trivy Scanner for vulnerabilities, misconfigurations, and secrets.
- AtomicBot-ai/atomic-agent Local-first AI agent that runs open-weight models via llama.cpp.
- axios/axios Promise-based HTTP client for the browser and Node.js.
- bevyengine/bevy Data-driven game engine written in Rust.