Best Local Coding AI for Your GPU8 GB to 128 GB

Pick the file, not just the model

You have eight gigabytes of memory. You download an eighteen gigabyte coding model. It cannot live entirely in that pool. The download finished.

Your memory did not grow. Last video, we built the memory ladder. Today, we pick the files. Eight.

Sixteen. Twenty four. Thirty two. Sixty four.

And one hundred twenty eight gigabytes. One recommendation per rung.

Memory, context and benchmark rules

With the size checked. First, the rule that keeps this useful. A download size is not a complete memory requirement. You also need the running software, its working space, and the context.

That is the prompt and conversation the model keeps available. On a graphics card, count available video memory. On a machine with unified memory, the operating system and your apps share that pool. Close what you can.

Start with four thousand ninety six tokens. That is my conservative starting setting. Not a promise that a whole repository will fit. These are source checked starting picks.

Not a tournament I ran on six computers. Every model here has weights released in twenty twenty six. All four models use the Apache two point zero license. The download links and exact commands are on screen.

There is one rule for benchmarks. The scores come from the original model cards. They are not measurements of our downloaded quants. Keep those two labels separate.

Four bit and eight bit describe how weights are stored. They do not describe the number of parameters. And a bigger model name cannot tell you the size of its chosen download. The file listing does that.

That is why every rung names both. Eight gigabytes. My pick is Qwen three point five, four B.

8 GB: Qwen3.5-4B, Q4_K_M

The card calls the language model four billion parameters. The Ollama package at Q four K M is three point three gigabytes. Apache two point zero. The published coding result is fifty five point eight on Live Code Bench version six.

That gives us a coding number from the maker. It does not tell us how many bugs it will fix in your project. I would start with a small function. Ask for tests.

Then run those tests. Keep the request narrow enough that you can inspect the answer. Why four bit here? Room.

An eight gigabyte machine has other things to hold. I would spend that room on the prompt before chasing a larger file. Copy the command under this board. It selects this quant explicitly.

Sixteen gigabytes. Qwen three point five, nine B.

16 GB: Qwen3.5-9B, Q8_0

Nine billion language model parameters on the official card. My starting file is Q eight zero. Ollama lists the package at ten gigabytes. Same Apache license.

Its published Live Code Bench version six score is sixty five point six. Again. Original model result. Not a new score for the eight bit file.

This rung buys a bigger language model than the four B. It also keeps more weight precision than our four bit pick. That is the trade I am choosing with sixteen gigabytes. Ten gigabytes is still substantial.

On a unified memory laptop, keep other apps under control. If the context needs more room, a smaller quant is another trade to test. Do not decide from the parameter name alone. Read the file size beside it.

Twenty four gigabytes.

24 GB: Qwen3.8-27B, Q4_K_M

Move to Qwen three point eight, twenty seven B. The official language model count is twenty seven billion. Ollama lists the Q four K M package at eighteen gigabytes. Apache two point zero.

The model card reports ninety point three on Live Code Bench version six. It also reports sixty one point seven on SWE bench Pro. That second result uses a coding harness and corrected benchmark tasks. The testing setup matters.

This is my bigger model, lower precision rung. We give up weight precision to make space for a larger model. That can be a sensible experiment. It does not guarantee every answer beats a smaller model at higher precision.

At twenty four gigabytes, start with short context. The package fits below the headline capacity. You still have to check that it loads on your machine.

32 GB: Qwen3.8-27B, UD-Q6_K

Thirty two gigabytes. Keep Qwen three point eight, twenty seven B. Change the file. My pick is Unsloth's dynamic Q six K version.

Hugging Face lists it at twenty two gigabytes. The exact byte count is on the board. This is a community conversion of the same Apache licensed model. Not a new model from Qwen.

The original coding score stays attached to the original card. We have no verified score here for this particular six bit conversion. That is why this rung repeats the model. More memory can buy more weight precision.

It does not have to buy more parameters. This command selects the exact text model file. For this coding setup, we are not adding the separate vision projector. If you want screenshots later, budget for that too.

The twenty two gigabyte number is this file alone. Sixty four gigabytes.

64 GB: Qwen3-Coder-Next, Q4_K_M

My coding specialist pick is Qwen three Coder Next. Eighty billion parameters in total. Three billion active for each token. Apache two point zero.

Active is the compute story. Total is still the weights story. This is not a three billion parameter download. Ollama lists Q four K M at fifty two gigabytes.

The official card reports seventy point six on SWE bench Verified, and forty four point three on SWE bench Pro. Do not line those up as a clean race against the previous board. The evaluation setups differ. I am picking this rung for a coding specialist trained for agent workflows.

Not claiming that eighty billion automatically beats twenty seven billion. Fifty two gigabytes leaves limited room on sixty four. Set short context first. Check the actual load.

128 GB: Qwen3-Coder-Next, Q8_0

A large context preset can undo this fit. One hundred twenty eight gigabytes. Coder Next again. This time, Q eight zero.

Ollama lists eighty five gigabytes. Same eighty billion total parameters. Same three billion active. Same Apache license.

The published coding results are the same original card results. Seventy point six on Verified. Forty four point three on Pro. I am not assigning a new benchmark score to the eight bit download.

The extra memory buys a higher precision copy of the same model. Or, if you choose a smaller file instead, it can leave more room for context. You cannot spend the same memory twice. The native context on these cards is two hundred sixty two thousand, one hundred forty four tokens.

Run, inspect context and test the patch

That is a supported ceiling. Not a promise that your remaining memory can hold it. Start small. Increase deliberately.

Now run the line for your rung. In the chat, apply the short context setting shown here. Then ask for one function and its tests. In a second terminal, use the status command on screen.

Read processor. Read context. A split between CPU and GPU means the load is using both pools. One correction to our last video.

Ollama's dedicated context page now lists defaults that depend on video memory. Its FAQ still says four thousand ninety six. Set it yourself. Check what actually loaded.

No speed promises here. We have file sizes and published coding scores. We have not timed your GPU. For your first check, choose a function you already understand.

Give it an empty input. Give it an awkward boundary case. Read the proposed test before trusting the result. A model that explains its patch still needs that patch checked.

If the load spills into system memory, record that with your result. Otherwise, your next comparison starts with a different memory setup, and you may blame the wrong file. Pick the file. Leave memory for context.

Run the tests. Next video. Connect a local coding model to your editor, and check its first patch. If this helped, subscribe.

Build with AI. For real.