Running GPT-OSS locally with Ollama
OpenAI has released GPT‑OSS, its first open-weight model family in more than five years, and you can run it on your own machine today with Ollama. It’s more than a release. It changes how we think about owning AI, keeping data private and experimenting freely.
I’ve had Ollama installed for a while, but honestly, the models never really stuck for me — until now.
What’s new
Two models
- gpt‑oss‑20b — built for consumer hardware, runs with 16 GB of RAM
- gpt‑oss‑120b — the larger, higher-performance model, and fine-tunable
- Both are Apache 2.0 licensed and downloadable today
Run ollama run gpt-oss:20b and you have a model with chain-of-thought reasoning and tool use, running entirely on your machine.
Why this matters to me
I’ve got the 20B model downloading already, and the 120B is next. I’m excited to throw different tasks at them: prompt engineering, data analysis, agent orchestration and more.
Here’s why it matters:
- Homelab. I can experiment freely, without cloud limits.
- Client work. Secure, custom assistants that run where the data lives.
- Community. Anyone can build and share workflows now.
What I want to try
Some of the ideas on my list:
- RAG pipelines over custom data
- AI agents for automation
- Private assistants in sensitive environments
The big picture
This release isn’t just about OpenAI catching up with Meta or Mistral. It’s about taking back control of our tools. If GPT‑OSS lives up to its promise, the days of black-box, cloud-only AI could be numbered.
How are you planning to use GPT‑OSS? I’d love to hear from you — get in touch, and I’ll share benchmarks and workflows as I go.