Give a Local Model Real Tools: Ollama Plus MCP, the Current Way

Here is the fact most guides bury. Ollama is a model server. Its API accepts a list of tools and returns tool calls, and the official docs show that loop in detail. Starting an MCP server, reading its tool list, running the call the model asked for, and carrying the result back is a second job, and Ollama does not do that job. You need an MCP client between the model and the servers. This tutorial installs that client, connects one safe server, and ends with a local model reading and writing real files on your own machine, with no cloud account in the loop.

What You Will Build or Learn

You will end with a terminal chat where a local model can use the official MCP filesystem server on one test folder you choose. The model will list that folder, create a file inside it when you approve the call, and read the file back. The pattern then repeats for any other MCP server you add later, because the client, not the model, owns the server list.

The route here uses ollmcp, the MCP Client for Ollama project, because it talks to Ollama in its native API and stays in the terminal. Coding agents launched through Ollama, such as the integrations behind ollama launch, are also MCP clients and follow the same shape. The client changes. The division of labour does not.

Before You Begin

Start with a test folder, never with your home folder or a drive root. The filesystem server can read and write every path you hand it, and a model that misreads a prompt can do real damage inside those paths. The approval prompt in ollmcp is on by default and shows each tool call before it runs. Leave it on for this whole tutorial.

Step 1: Install Ollama and Confirm the Server Answers

Action: install Ollama from its official site, or use the script on Linux and macOS. Then confirm the local server responds.

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
curl http://localhost:11434/

Expected result: the version command prints your Ollama version, and the final call answers with the plain text line Ollama is running. On Windows and macOS, open the Ollama app once if the server does not answer, since the app starts the background server. Nothing else in this tutorial works while that address stays silent.

Step 2: Pull a Model That Can Call Tools

Action: pull a model with tool support. The ollmcp project lists the qwen2.5 and qwen3 lines and llama3.1 and llama3.2 among the models that work well with tool use, and the Ollama tool calling docs build their examples on qwen3. Start with the 8B size.

ollama pull qwen3:8b
ollama list

Expected result: ollama list shows qwen3:8b on disk. A model without tool support will still chat inside the client and will never call a tool, which is the single most common cause of a build that looks finished and does nothing. If you swap models later, check the model page for tool support first.

Step 3: Raise the Context Before the Tool Lists Arrive

Action: restart Ollama with a larger context. Tool lists eat context fast. Ollama itself notes that a context window of 32k or higher improves the performance of tool calling, and a small default window can leave no room for the tool definitions plus your conversation. Stop the running server first, then start it with the setting in place.

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Expected result: Ollama starts and serves as usual, now with room for the filesystem tool list and a working conversation. Memory use rises with context, so this is a trade you are making on purpose. Run this server in its own terminal window and leave it open while you work through the rest of the steps.

Step 4: Install the MCP Client

Action: install ollmcp with pip, or run it in one step with the uv tool if you already use uv.

pip install ollmcp
ollmcp --help

Expected result: the help text prints the client options, including --mcp-server, --servers-json, --model, and --host. The default host is http://localhost:11434, which matches Step 1, so no address work is needed for a local install.

Step 5: Write the Server Configuration

Action: make a test folder and a configuration file that hands that one folder to the official filesystem server. Replace the example user path with your own absolute path, and keep it pointed at the test folder only.

mkdir -p ~/ollama-mcp/playground

Save this as servers.json inside the ~/ollama-mcp folder:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/home/yourname/ollama-mcp/playground"]
    }
  }
}

Expected result: the file names one server, started by npx, with one allowed root. The filesystem server refuses paths outside the roots in its argument list, which is the guardrail that makes the test safe. On Windows, use the full Windows path for the playground folder in the same argument slot.

Step 6: Start the Client and Check the Tools Loaded

Action: start ollmcp with your configuration file and your pulled model.

ollmcp --servers-json ~/ollama-mcp/servers.json --model qwen3:8b

Expected result: the client connects to Ollama, starts the filesystem server, and opens its chat screen. Type tools and press Enter. The tool list should show the filesystem tools, among them tools to list a directory, read a file, and write a file. If the list is empty, the server did not start, and Step 5 is where to look first.

Step 7: Give the Model a Real Task

Action: ask for work that only a tool can do, in plain words.

List the files in the playground folder. Then create a file called hello.txt in that folder with one sentence: the local model was here. Then read hello.txt back and tell me what it says.

Expected result: before the write runs, the approval prompt shows the tool name and its arguments. Read them, confirm the path sits inside the playground folder, and approve. The model should report an empty or near empty folder at first, create the file, and quote your sentence back from the read. That quote came from disk, not from memory of the chat, and that difference is the whole point of the build.

Verification

Open a second terminal and check the file yourself, without asking the model.

ls ~/ollama-mcp/playground
cat ~/ollama-mcp/playground/hello.txt

The listing shows hello.txt, and the printout shows your sentence exactly as written. Inside the client, type tools once more and confirm the filesystem tools still show as enabled. If the file exists on disk and the client still lists the tools, the loop works end to end: prompt, tool call, approval, server, result, answer.

Troubleshooting

Next Steps

Add a second server to the same servers.json file and the client will offer both tool sets in one chat. The official MCP servers collection is the safe shelf to shop from, and the filesystem pattern in Step 5 covers most stdio servers: a command, its arguments, and an env block where a key is needed, written as a placeholder in anything you share. Save a setup that works with the save-config command inside the client so plain ollmcp reopens the same build.

When the work turns from chat to code, the same servers follow you. Agents started through ollama launch, such as OpenCode and Claude Code, read their own MCP configuration and drive the local model the same way ollmcp does here. Learn the loop in the terminal first, where every call asks your permission. Then hand it bigger tools with your eyes open.

Commands and behaviour checked against the Ollama tool calling documentation, the Ollama note on streaming tool calls and context size, the ollmcp project documentation, and the MCP filesystem server configuration format, October 2026. This procedure was verified against those documents; run it on a test folder first, as written.