How to Run an Open-Source Coding Model Locally with Ollama
A step-by-step guide to running Qwen2.5-Coder, Devstral or DeepSeek-Coder-V2 on your own machine with Ollama and connecting it to your editor.
CoderBaba Team1 min read
Running a coding model locally gives you privacy (your code never leaves your machine), zero per-token cost and offline access. With Ollama it takes about five minutes.
1. Install Ollama
On macOS or Windows, download the installer from ollama.com. On Linux:
curl -fsSL https://ollama.com/install.sh | sh2. Pick a model that fits your hardware
| Your hardware | Good starting model |
|---|---|
| 8 GB RAM laptop | qwen2.5-coder:1.5b or codegemma:2b |
| 16 GB RAM / 8 GB GPU | qwen2.5-coder:7b |
| 24 GB GPU or 32 GB Mac | qwen2.5-coder:14b or devstral |
| 48 GB+ | qwen2.5-coder:32b |
Browse the full model directory for licenses and specs.
3. Pull and run it
ollama run qwen2.5-coder:7bThe first run downloads the weights. After that you can chat with the model in your terminal.
4. Connect it to your editor
Ollama serves an OpenAI-compatible API at http://localhost:11434/v1. Most AI coding extensions, such as Continue, accept a custom endpoint. Point the extension there and choose your model.
Tip
Use a small model (1.5B to 3B) for autocomplete, where latency matters, and a larger one for chat and refactoring.
5. Try an agent
For multi-file tasks, use an agentic model like Devstral together with an agent framework such as OpenHands. Everything still runs on your machine.