Ollama MCP Integration Docs

find documentation for MCP with Ollama

​To integrate Ollama with the Model Context Protocol (MCP), you can utilize several available resources and tools: 1. **Ollama MCP Server**: This server enables seamless integration between Ollama's local Large Language Models (LLMs) and MCP-compatible applications, allowing functionalities such as listing available models, pulling new models, and interacting with models using Ollama's chat API. citeturn0search0 2. **Ollama MCP Bridge**: This TypeScript implementation connects local LLMs via Ollama to MCP servers, allowing open-source models to utilize the same tools and capabilities as other MCP-integrated applications. citeturn0search2 3. **Dolphin MCP**: An open-source Python library and CLI tool that simplifies integration with various language models, including Ollama. It allows you to connect to multiple MCP servers simultaneously, making their tools available to language models through natural language queries. citeturn0search1 4. **Ollama Deep Researcher MCP Server**: This server adaptation of LangChain's Ollama Deep Researcher provides deep research capabilities as MCP tools, enabling AI assistants to perform in-depth research on topics using local LLMs via Ollama. citeturn0search5 For comprehensive documentation and further guidance on MCP, you can refer to the following resources: - **Understanding the Model Context Protocol (MCP)**: An article that provides an in-depth overview of MCP, explaining its purpose and how it standardizes interactions between applications and large language models. citeturn0search7 - **Model Context Protocol (MCP): The Future of LLM Function Calling**: This article discusses the future prospects of MCP in function calling for large language models, providing insights into its potential applications and benefits. citeturn0search8 These resources should assist you in effectively integrating Ollama with MCP and leveraging the protocol's capabilities.

I want to integrate Ollama and MCP, help me do so with a custom set up

To integrate **Ollama** with **MCP (Model Context Protocol)** in a custom setup, you’ll need to: 1. **Set up an MCP server** 2. **Connect it to Ollama** 3. **Create a custom MCP tool for interacting with Ollama** 4. **Test the integration** --- ### **Step 1: Install Ollama and MCP** First, make sure you have **Ollama** installed and running locally: ```bash curl -fsSL https://ollama.com/install.sh | sh ``` Then, install an **MCP server**. You can use an existing MCP implementation, or create your own. Let’s use a Python-based MCP server. ```bash pip install fastapi uvicorn pydantic ``` --- ### **Step 2: Create an MCP Server to Interact with Ollama** Here’s a **FastAPI MCP server** that connects to **Ollama**: ```python from fastapi import FastAPI, HTTPException import requests import json app = FastAPI() OLLAMA_API_URL = "http://localhost:11434/api/generate" # Ollama local API @app.post("/mcp/ollama") async def ollama_generate(prompt: str, model: str = "mistral"): """ MCP-compatible endpoint to interact with Ollama """ payload = {"model": model, "prompt": prompt, "stream": False} try: response = requests.post(OLLAMA_API_URL, json=payload) response_data = response.json() return {"response": response_data.get("response", "No response")} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) ``` Save this file as `mcp_server.py` and run it with: ```bash uvicorn mcp_server:app --host 0.0.0.0 --port 8000 ``` --- ### **Step 3: Create an MCP Tool for Ollama** Now, let’s create an **MCP-compatible tool** that connects to our MCP server. Save the following JSON as `ollama_mcp_tool.json`: ```json { "name": "ollama_mcp", "description": "An MCP tool that generates responses from Ollama models.", "parameters": [ { "name": "prompt", "type": "string", "description": "The text prompt to generate a response from Ollama." }, { "name": "model", "type": "string", "description": "The name of the Ollama model to use (e.g., mistral, llama2, codellama).", "default": "mistral" } ], "api_endpoint": "http://localhost:8000/mcp/ollama" } ``` --- ### **Step 4: Test the Integration** To verify that everything works, you can use **cURL** or a Python script: ```bash curl -X POST "http://localhost:8000/mcp/ollama" -H "Content-Type: application/json" \ -d '{"prompt": "Tell me a joke", "model": "mistral"}' ``` Or in Python: ```python import requests response = requests.post( "http://localhost:8000/mcp/ollama", json={"prompt": "Tell me a joke", "model": "mistral"} ) print(response.json()) ``` --- ### **Next Steps** - If you want to **register this MCP tool** into an existing system, make sure it follows the **MCP JSON schema**. - You can add **more tools** to MCP that interact with Ollama for **image generation, function calling, or data processing**. - Consider **Dockerizing** the solution to make deployment easier. Would you like help extending this to include **agents or multi-tool orchestration**? 🚀

(venv) venvdanielkliewer@Daniels-MacBook-Pro mcp01 % python3 main.py {'detail': [{'type': 'missing', 'loc': ['query', 'prompt'], 'msg': 'Field required', 'input': None}]}

Your FastAPI server is expecting the `prompt` parameter to be passed as a **query parameter**, but you're likely sending it in the request **body** instead. ### **Solution** Modify your FastAPI function to expect the parameters in the request **body** instead of query parameters. #### **Fix your `main.py` file:** Modify your function to use **`pydantic.BaseModel`** to parse the request body correctly. ```python from fastapi import FastAPI, HTTPException from pydantic import BaseModel import requests app = FastAPI() OLLAMA_API_URL = "http://localhost:11434/api/generate" # Ollama's local API # Define request body schema class OllamaRequest(BaseModel): prompt: str model: str = "mistral" @app.post("/mcp/ollama") async def ollama_generate(request: OllamaRequest): """ MCP-compatible endpoint to interact with Ollama """ payload = {"model": request.model, "prompt": request.prompt, "stream": False} try: response = requests.post(OLLAMA_API_URL, json=payload) response_data = response.json() return {"response": response_data.get("response", "No response")} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) ``` --- ### **Restart Your Server** After updating the code, restart your server: ```bash uvicorn main:app --host 0.0.0.0 --port 8000 ``` --- ### **Now, Send a Proper JSON Request** #### **Using cURL:** ```bash curl -X POST "http://localhost:8000/mcp/ollama" -H "Content-Type: application/json" \ -d '{"prompt": "Tell me a joke", "model": "mistral"}' ``` #### **Using Python:** ```python import requests response = requests.post( "http://localhost:8000/mcp/ollama", json={"prompt": "Tell me a joke", "model": "mistral"} ) print(response.json()) ``` --- ### **Why Did This Happen?** FastAPI **expects query parameters by default** when you define function arguments without using `BaseModel`. By switching to `BaseModel`, FastAPI now correctly **expects a JSON body** instead. Try this out and let me know if you run into any issues! 🚀

now how do we integrate tools for it to use

To integrate **MCP tools** into your Ollama-based FastAPI server, we need to follow a structured approach: 1. **Define MCP-compatible tool endpoints** in the FastAPI server. 2. **Register the tools in an MCP-compatible format** (JSON schema). 3. **Allow the LLM (Ollama) to use the tools** dynamically. 4. **Implement tool calling** within your API. --- ## **Step 1: Define MCP-Compatible Tool Endpoints** We'll add **multiple tools** (e.g., a simple calculator and a Wikipedia search tool) to our FastAPI server. ### **Update `main.py`** ```python from fastapi import FastAPI, HTTPException from pydantic import BaseModel import requests import wikipedia app = FastAPI() OLLAMA_API_URL = "http://localhost:11434/api/generate" # Request models class OllamaRequest(BaseModel): prompt: str model: str = "mistral" class CalculatorRequest(BaseModel): num1: float num2: float operation: str class WikiRequest(BaseModel): query: str # Tool: Generate response using Ollama @app.post("/mcp/ollama") async def ollama_generate(request: OllamaRequest): payload = {"model": request.model, "prompt": request.prompt, "stream": False} try: response = requests.post(OLLAMA_API_URL, json=payload) response_data = response.json() return {"response": response_data.get("response", "No response")} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) # Tool: Simple calculator @app.post("/mcp/tools/calculator") async def calculator(request: CalculatorRequest): try: if request.operation == "add": result = request.num1 + request.num2 elif request.operation == "subtract": result = request.num1 - request.num2 elif request.operation == "multiply": result = request.num1 * request.num2 elif request.operation == "divide": if request.num2 == 0: return {"error": "Cannot divide by zero"} result = request.num1 / request.num2 else: return {"error": "Invalid operation"} return {"result": result} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) # Tool: Wikipedia search @app.post("/mcp/tools/wiki") async def wiki_search(request: WikiRequest): try: summary = wikipedia.summary(request.query, sentences=2) return {"summary": summary} except wikipedia.exceptions.DisambiguationError as e: return {"error": f"Multiple results found: {e.options}"} except wikipedia.exceptions.PageError: return {"error": "No matching page found"} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) ``` --- ## **Step 2: Register Tools for MCP** Now we need to **define the tools in an MCP-compatible format** so that an LLM can understand how to use them. Save this JSON as **`mcp_tools.json`**: ```json [ { "name": "calculator", "description": "Performs basic arithmetic operations like add, subtract, multiply, and divide.", "parameters": [ {"name": "num1", "type": "float", "description": "First number"}, {"name": "num2", "type": "float", "description": "Second number"}, {"name": "operation", "type": "string", "description": "Operation type: add, subtract, multiply, divide"} ], "api_endpoint": "http://localhost:8000/mcp/tools/calculator" }, { "name": "wiki_search", "description": "Fetches a short summary of a Wikipedia topic.", "parameters": [ {"name": "query", "type": "string", "description": "The search query for Wikipedia."} ], "api_endpoint": "http://localhost:8000/mcp/tools/wiki" } ] ``` --- ## **Step 3: Allow the LLM to Use Tools** We can now **modify our Ollama prompt to let the LLM know that these tools exist**. 1. **Modify `ollama_generate` to inject available tools** ```python import json @app.post("/mcp/ollama_with_tools") async def ollama_generate_with_tools(request: OllamaRequest): """ Informs Ollama about available tools and processes tool calls. """ # Load MCP tools with open("mcp_tools.json", "r") as file: tools = json.load(file) # Inject tool info into prompt tool_instruction = "You can use the following tools to answer the user's request:\n" for tool in tools: tool_instruction += f"- {tool['name']}: {tool['description']}\n" full_prompt = f"{tool_instruction}\nUser Prompt: {request.prompt}" payload = {"model": request.model, "prompt": full_prompt, "stream": False} try: response = requests.post(OLLAMA_API_URL, json=payload) response_data = response.json() return {"response": response_data.get("response", "No response"), "tools": tools} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) ``` --- ## **Step 4: Test the Integration** ### **1️⃣ Start your server** ```bash uvicorn main:app --host 0.0.0.0 --port 8000 ``` ### **2️⃣ Test Ollama with tool access** #### **Using cURL** ```bash curl -X POST "http://localhost:8000/mcp/ollama_with_tools" -H "Content-Type: application/json" \ -d '{"prompt": "Calculate 5 + 3 using the calculator tool.", "model": "mistral"}' ``` #### **Using Python** ```python import requests response = requests.post( "http://localhost:8000/mcp/ollama_with_tools", json={"prompt": "Calculate 5 + 3 using the calculator tool.", "model": "mistral"} ) print(response.json()) ``` --- ## **What This Setup Does** 1. **MCP-compatible tool endpoints**: - `/mcp/tools/calculator` (arithmetic) - `/mcp/tools/wiki` (Wikipedia search) 2. **

write a guide on what we have done so far

Here's your guide on integrating Ollama with MCP! Let me know if you need any refinements or additional features. 🚀

now let's flesh out the ollama_mcp_tool.json to include a wealth of tools

I've expanded the `ollama_mcp_tool.json` to include multiple tools, such as **text generation, summarization, translation, and code generation**, making the MCP setup more versatile. Let me know if you'd like to refine or add any more functionalities! 🚀

no, I want tools which are actually useful, such as browser-use

Got it! Let's add tools that are more practical, such as **web browsing, file handling, data extraction, and interaction with APIs**. I'll update the `ollama_mcp_tool.json` with useful functionalities. Here are some tool ideas we can include: - **Web Browsing**: Fetch and summarize webpage content. - **File Handling**: Read and summarize local files (TXT, PDF, JSON). - **Web Search**: Query search engines for relevant information. - **API Interaction**: Send GET/POST requests to external APIs. - **OCR (Optical Character Recognition)**: Extract text from images. - **Speech-to-Text**: Convert audio files to text. I'll modify the JSON now to include these tools.

I've expanded the `ollama_mcp_tool.json` to include **web searching, webpage fetching, file reading, OCR for images, and speech-to-text conversion**—all practical tools to enhance your MCP setup. Let me know if you want any more refinements! 🚀

(venv) venvdanielkliewer@Daniels-MacBook-Pro mcp01 % python3 test.py {'detail': 'Not Found'}

It looks like the FastAPI server might not be running or the endpoint you're calling in `test.py` doesn't match what's defined in `main.py`. Let's debug this step by step: ### **1. Check if the FastAPI Server is Running** Run the following command to check if the FastAPI server is active: ```bash lsof -i :8000 ``` If nothing appears, it means the server isn't running. Start it with: ```bash uvicorn main:app --host 0.0.0.0 --port 8000 --reload ``` ### **2. Verify the Available Endpoints** Once the server is running, open your browser or use `curl` to check if the `/mcp/fetch_webpage` endpoint exists: ```bash curl http://localhost:8000/docs ``` This should return an OpenAPI page listing all available endpoints. If your expected endpoints (like `/mcp/fetch_webpage`) aren't listed, there might be an issue in `main.py`. ### **3. Confirm `test.py` is Hitting the Correct URL** If the test script is using the wrong URL, update it to match the correct endpoint from FastAPI: ```python import requests response = requests.post( "http://localhost:8000/mcp/fetch_webpage", json={"url": "https://example.com"} ) print(response.json()) ``` If it still fails, let me know what `test.py` contains so I can help debug further! 🚀