unsloth: the 2x-faster, 70%-less-VRAM fine-tuning library, now a desktop runtime that wires local weights into Claude Code and Codex

From fine-tuning library to desktop runtime: the agent front door for local models

unsloth now describes itself as "the first desktop app to run and train models". That is a full step wider than its earlier identity as a Python library that made LoRA fine-tuning faster and lighter on VRAM. It ships native clients for Windows, macOS and Linux x64/ARM64 (both deb and AppImage), keeps the one-line installers (curl -fsSL https://unsloth.ai/install.sh | sh, or irm ... | iex on Windows) and a Docker image. On the hardware side it supports multi-GPU setups, NVIDIA / AMD / Intel GPUs, CPU-only, and a Vulkan backend.

The substance of the pivot is not that it grew a GUI, but that the exit path for "running a model locally" changed from a notebook to an agent. The Agents & Tools section is explicit: you can drive Claude Code, Codex and MCP with local models, including tool calling and code execution.

# Unsloth Start: one command wires Claude Code to local weights
unsloth start claude --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL

That single line is worth unpacking. It combines a subscription cloud coding agent with open weights on your own machine into a new configuration: the harness (context management, tool protocol, edit loop) comes from Claude Code or Codex, while inference happens locally - no token bill, and the code never leaves. In terms of this site's facets, unsloth therefore belongs to both llm (it changes cost per token and reachability) and code (it is the model supply side for coding agents), with the primary capability assigned to llm per the domain contract.

Training: 2x faster, 70% less VRAM, and a complete post-training menu

Fine-tuning is still what it is known for, claimed at "2x faster with 70% less VRAM and no accuracy loss". The first two are reproducible engineering results (hand-written backward passes, not materialising intermediate activations, fused operators); the third - "no accuracy loss" - is the project's own statement and depends on task and hyperparameters, so we record it as a project claim.

Post-training coverage is broader than most comparable libraries: LoRA, QLoRA, full fine-tuning, pretraining, reinforcement learning, GRPO, DPO and FP8. GRPO being on that list is the significant part - it is the mainstream algorithm for verifiable-reward RL in open source after DeepSeek-R1, and a consumer-grade fine-tuning library shipping GRPO out of the box moves the entry barrier for RL post-training from a cluster down to a single card. Export formats cover GGUF, NVFP4 and FP8; GGUF export plugs straight into the llama.cpp ecosystem while NVFP4 targets serving engines on Blackwell, so a freshly trained weight has both deployment routes open.

Data Recipes builds datasets from PDFs, CSVs, DOCX and similar files, which is the natural consequence of going desktop: the data a local user actually has is documents, not parquet.

Model coverage: a snapshot of the current open frontier

The model list in the README doubles as a dated snapshot of the open-weight frontier: Qwen3.8, GLM-5.3-Flash, Kimi K3, Qwen-Image-2.1, MiniMax-H3, DeepSeek-V4 and Gemma 4. By type it covers LLMs, MLX, GGUF, diffusion, embedding and audio models, and it can both run and train image and video diffusion as well as multimodal models.

Beyond models it bundles several capabilities that usually require external services: private and unlimited web search, deep research, auto-compaction (a rolling context window) and RAG. Remote access works from any device on the LAN, or from outside through secure Cloudflare HTTPS to a local model. Serving is via an OpenAI-compatible API, and existing ChatGPT/Codex subscriptions plus cloud providers can be connected - local and cloud are mixed inside one client.

Where it sits in the stack

All three roles have to be held in view at once to place it accurately: unsloth is a memory and speed optimiser on the training side, a desktop runtime for local inference, and the local model supply side for coding agents. Its relationship to llama.cpp is complementary rather than substitutive - llama.cpp provides the widest hardware backend coverage and the GGUF de facto standard, and unsloth adds training, a GUI and agent wiring on top, with GGUF as its main export format. It is largely not in the same job as vLLM or SGLang: those optimise aggregate cluster throughput, while unsloth optimises whether the "one person, one machine, one model" path works at all.

Provenance to flag: the 2x speed and 70% VRAM figures are project claims with no unified third-party benchmark, and "no accuracy loss" will resolve differently across tasks. What is verifiable is the method (hand-written backwards, non-materialised activations, fused operators), the list of supported algorithms, the export formats, and the existence of the desktop clients.