Guides: Choosing and Understanding Open Coding Models

What our guides are for

Tutorials tell you how to do something. Guides help you decide what to do and why. Each guide in this topic tackles a decision developers face when adopting AI coding models, such as which model, which license, which hardware and whether to self-host. It gives you a framework you can reuse long after specific model names change.

Core concepts in one place

Before diving into individual guides, here is a primer on the ideas that come up again and again.

Open weights vs. open source

An open-weight model publishes its trained parameters so you can download and run it. A fully open model also publishes its training data, data-processing code and training recipe. Most popular coding models are open-weight. A few, such as StarCoder2, are fully open. For most practical uses, open weights are what you need. For research and compliance, full openness can matter a lot.

Licenses in three tiers

1. Permissive (Apache 2.0, MIT): use, modify and ship commercially with minimal obligations, such as keeping the license notice.
2. Custom with conditions: commercial use is allowed, but with strings attached, such as use-based restrictions (OpenRAIL-M, the DeepSeek Model License) or thresholds for very large companies (the Llama 2 Community License).
3. Non-production or research-only: fine for experiments and personal use, but commercial deployment needs a separate agreement, as with Codestral's open weights.

Quantization

Models are trained in 16-bit precision. Quantization stores weights with fewer bits, typically 8, 5 or 4, to cut memory and speed up inference. At 4-bit, memory drops to roughly a quarter of the 16-bit size with only a small quality loss for most coding tasks. Below 4-bit, quality degrades faster. Common formats include GGUF (used by llama.cpp and Ollama) and AWQ or GPTQ (popular with GPU servers like vLLM).

Dense vs. mixture of experts

Dense models use all of their parameters for every token. Mixture-of-experts (MoE) models activate only a subset per token. For example, DeepSeek-Coder-V2 has 236B parameters in total but only 21B active per token. MoE gives you big-model quality at lower compute cost, but memory requirements follow the total size, not the active size.

Context windows

The context window is the model's working memory for a request. Bigger windows let the model see more of your codebase at once, but memory use grows with context length, and quality can drop toward the far end of very long contexts. Agents and repository-wide tasks benefit most from 128K or more.

Benchmarks

HumanEval measures writing a single Python function from a description. SWE-bench Verified measures fixing real GitHub issues end to end, which is closer to agent work. Launch numbers are usually self-reported, so treat them as a rough guide and test on your own code.

Local vs. hosted: a decision framework

Choose local or self-hosted open models when:

• Your code can't leave your machine or network for legal, contractual or security reasons
• You run a lot of requests, such as autocomplete or agents, and per-token costs would add up
• You need predictable behavior from a pinned model version
• You want to work offline or avoid dependence on any vendor

Choose hosted frontier APIs when:

• You need the absolute best quality on the hardest tasks
• You don't have suitable hardware and need results today
• Request volume is low and convenience matters more than cost

Choose both in many real setups: route hard, rare tasks to a hosted API and frequent, routine work to a local model. Most open-source agents support this kind of model routing.

Questions to answer before you commit

1. What tasks will the model do: autocomplete, chat or autonomous agents?
2. What is the most demanding hardware you can rely on, for you or for every teammate?
3. Does the license allow your intended use, including in products you sell?
4. How much context do typical tasks need?
5. How will you evaluate quality? Pick five or ten real tasks from your own backlog and try them on two or three candidate models.
6. What is your exit plan if a model or tool stops being available? Keep state in git and prefer open formats.

Guides in this topic

Every post tagged #guide is listed above. Start with How to Choose an Open-Source Coding Model, then use the model directory to compare candidates side by side. For hands-on setup, continue to the tutorials.

A worked example: choosing for a small team

Imagine a five-person startup building a commercial web app. They want private code completion and an agent for routine tasks, and each developer has a laptop with 32 GB of unified memory.

1. License first. They're shipping a commercial product, so they filter for permissive licenses. That rules out non-production weights and leaves models such as Qwen2.5-Coder (most sizes), Devstral and Granite Code.
2. Hardware next. With 32 GB machines, models up to about 24B parameters at 4-bit are comfortable, leaving room for the IDE, a browser and context.
3. Task fit. For autocomplete they choose a small Qwen2.5-Coder model. For agentic tasks they shortlist Devstral.
4. Evaluation. They run ten real backlog tasks through each candidate and keep the model that passes the most tasks with the fewest corrections.
5. Review date. They put a calendar reminder for three months out to rerun the same tasks against newer releases.

The whole process takes about a day and gives them a defensible, documented choice.

Frequently asked questions

Do model recommendations go out of date?

Specific names do. A model recommended today may be overtaken within months, and we update guides when that happens. The frameworks in these guides, such as matching task to model size, checking license tiers and testing on your own code, stay valid much longer.

Is a bigger model always better?

For quality, usually yes within a model family. In practice it depends. A smaller model that fits entirely in your GPU and responds instantly often beats a bigger one that spills into system RAM and crawls. For autocomplete in particular, speed wins.

How much does a quantized model actually lose?

At 8-bit, differences are usually negligible. At 4-bit, most coding tasks see small losses that are hard to notice in daily use. Below 4-bit, errors in syntax and logic become more frequent. If you have the memory, prefer a higher-precision version of a mid-size model over an aggressively quantized large one.

Are self-reported benchmark scores trustworthy?

They are honest measurements under the authors' own conditions, which are rarely identical to yours. Use them to build a shortlist, then decide with a small evaluation on your own tasks.

Do I need a lawyer to use an open model commercially?

For permissive licenses like Apache 2.0 and MIT, most teams follow standard open-source compliance practices. Custom licenses and anything with use restrictions deserve a careful read, and legal review is wise if the model is central to a product you sell. Our license badges are a summary, not legal advice.