Understand supports several local large language models (LLMs) for code explanation, navigation help, and exploratory analysis. The “best” model depends on your goals, hardware, and any compliance requirements you may be working under.


This article is here to help you choose deliberately, not guess.


Note: visit our Hugging Face page to find direct links to the models we use!

Quick Recommendation (If You Just Want an Answer)

  • Most users: Stick with the default Qwen3-1.7B

  • Better reasoning, stronger answers: Qwen3-4B

  • Strict US-origin software requirements: Granite-4-1B or Granite-4-3B

  • High-end machines / deeper analysis: Qwen3-8B

If you want full control, you can also:

  • Load your own GGUF model

  • Connect Understand to Ollama or LM Studio

Why Qwen Is the Default

In our internal testing and real-world usage, Qwen models consistently perform better than Llama models for code understanding tasks, especially:

  • Explaining unfamiliar code

  • Following multi-step instructions

  • Staying grounded in the codebase instead of hallucinating

That’s why as of build 7.2.1242 Understand defaults to Qwen3-1.7B — it’s the best balance of quality, speed, and size for most users.

That said, defaults are just starting points.

Model Options Explained

Alibaba Cloud Qwen Models (Default & Recommended)

Best for: General use, code comprehension, accuracy, instruction following

  • Qwen3-1.7B (1.0 GB) ← Default

    • Strong baseline for most workflows

    • Good explanations without heavy hardware demands

  • Qwen3-4B (2.3 GB)

    • Noticeably better reasoning and context retention

    • A great upgrade if you have the RAM

  • Qwen3-8B (4.7 GB)

    • Much stronger at cross-file reasoning

    • Slower, but more “thoughtful”

IBM Granite Models (US-Origin Alternative)

Best for: Compliance-driven environments, US-only software requirements

Some organizations require models developed by US companies. In those cases, IBM Granite is a great choice and performed very well in our testing.

  • Granite-1B (1 GB)

    • Comparable in size to Qwen3-1.7B

    • Slightly less consistent in instruction following

  • Granite-3B (2 GB)

    • Best overall Granite balance

    • Recommended Granite starting point

Why choose Granite?

  • Regulatory or contractual requirements

  • Internal policy on model provenance

  • Consistency with other tooling already using Llama

Choosing Based on Your Needs

Choose a Smaller Model If:

  • You’re on limited RAM

  • You want fast, lightweight answers

  • You mostly ask simple questions or summaries

Choose a Larger Model If:

  • You work in large or complex codebases

  • You want better cross-file reasoning

  • You rely heavily on AI explanations during exploration

Choose Granite If:

  • You must use US-origin software or cannot use Chinese-origin software

Using Your Own Model or External Runtimes

Understand isn’t limited to the bundled models.

You can also:

  • Load your own GGUF file
    Useful if you’ve standardized on a specific model internally.

  • Connect to Ollama or LM Studio
    Ideal if you already run models centrally or want easy model switching.

This flexibility lets you align Understand with your existing AI stack instead of reshaping your workflow around it.

Final Thoughts

There’s no single “best” model — only the best fit for how you work and where you work.

If you want reliability and strong code understanding, start with Qwen.
If you need compliance and provenance guarantees, Granite is there for you.
And if you’re experimenting or optimizing, Understand gives you room to explore.

If you ever feel like the AI is almost right but not quite — that’s usually a signal to change model size, not abandon the feature.

And if you want help tuning that choice, you know where to find us.