> ## Documentation Index
> Fetch the complete documentation index at: https://laddr.agnetlabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Ollama Integration

> Configure and use Ollama for running local LLM models with Laddr agents - Docker setup, configuration, and examples

Run Laddr agents with local LLM models using Ollama. This guide covers configuration, Docker setup, and usage examples.

***

## Configuration

### Environment Variables

Add these to your `.env` file:

```bash theme={null}
# ===================================
# Ollama Configuration
# ===================================

# Ollama server URL
# For local development (Ollama running on host):
OLLAMA_BASE_URL=http://localhost:11434

# For Docker containers (see Docker Setup section):
# OLLAMA_BASE_URL=http://host.docker.internal:11434

# Default Ollama model (used if not specified per-agent)
OLLAMA_MODEL=gemma2:2b

# ===================================
# Backend Selection
# ===================================

# Set Ollama as default LLM backend for all agents
LLM_BACKEND=ollama

# Or configure per-agent:
LLM_BACKEND_RESEARCHER=ollama
LLM_BACKEND_COORDINATOR=openai  # Mix with cloud models

# ===================================
# Per-Agent Models
# ===================================

# Use different Ollama models for different agents
LLM_MODEL_RESEARCHER=gemma2:2b        # Fast, cheap model
LLM_MODEL_COORDINATOR=llama3.2:latest # Better reasoning
LLM_MODEL_WRITER=mistral:7b           # Good at writing

```

### Configuration Priority

Laddr resolves Ollama configuration in this order (highest to lowest priority):

**For base URL:**

1. Per-agent base URL: `LLM_BASE_URL_RESEARCHER=http://custom:11434`
2. Global Ollama URL: `OLLAMA_BASE_URL=http://localhost:11434`
3. Default: `http://localhost:11434`

**For models:**

1. Per-agent model: `LLM_MODEL_RESEARCHER=gemma2:2b`
2. Global Ollama model: `OLLAMA_MODEL=llama3.2:latest`
3. Global LLM model: `LLM_MODEL=gemma2:2b`
4. Default: `gemma2:2b`

***

## Docker Setup

When running Laddr in Docker containers, special network configuration is needed so containers can reach Ollama running on the host machine.

### Problem: Container Network Isolation

By default, `http://localhost:11434` inside a Docker container refers to the **container itself**, not the host machine where Ollama is running.

**Error you might see:**

```text theme={null}
LLM generation failed: Ollama endpoints not reachable at http://localhost:11434
Cannot connect to host localhost:11434

```

### Solution: Use host.docker.internal

Docker provides a special hostname `host.docker.internal` that resolves to the host's IP address.

### Docker Compose Configuration

Update your `docker-compose.yml`:

```yaml theme={null}
services:
  api:
    build:
      context: ..
      dockerfile: tester/Dockerfile
    ports:
      - 8000:8000
    environment:
      # Point to host's Ollama via special hostname
      OLLAMA_BASE_URL: "http://host.docker.internal:11434"
    # Map host.docker.internal to host gateway
    extra_hosts:
      - "host.docker.internal:host-gateway"
    # ... other config

  researcher_worker:
    build:
      context: ..
      dockerfile: tester/Dockerfile
    command: python -m agents.researcher
    environment:
      AGENT_NAME: researcher
      # Worker must also use host.docker.internal
      OLLAMA_BASE_URL: "http://host.docker.internal:11434"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    # ... other config

  coordinator_worker:
    build:
      context: ..
      dockerfile: tester/Dockerfile
    command: python -m agents.coordinator
    environment:
      AGENT_NAME: coordinator
      OLLAMA_BASE_URL: "http://host.docker.internal:11434"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    # ... other config

```

### Key Configuration Points

**1. Environment Variable**

```yaml theme={null}
environment:
  OLLAMA_BASE_URL: "http://host.docker.internal:11434"

```

Tells Laddr to connect to Ollama via the special hostname.

**2. Extra Hosts Mapping**

```yaml theme={null}
extra_hosts:
  - "host.docker.internal:host-gateway"

```

Maps the hostname to the Docker host's gateway IP.

**3. All Services Need It**

Apply this configuration to **all services** that use Ollama:

* API server
* Worker containers
* Any agent services

### Verification in Docker

Test Ollama connectivity from inside a container:

```bash theme={null}
# Start your containers
docker compose up -d

# Exec into a worker container
docker compose exec researcher_worker bash

# Test connection
curl http://host.docker.internal:11434/api/generate -d '{
  "model": "gemma2:2b",
  "prompt": "test",
  "stream": false
}'

```

Expected: JSON response with generated text.

***

## Usage Examples

### Example 1: Simple Agent with Ollama

```python theme={null}
# my_agent.py
from laddr import Agent
from laddr.llms import ollama

researcher = Agent(
    name="researcher",
    role="Research Specialist",
    goal="Find and analyze information",
    backstory="You are an expert researcher.",
    
    # Use Ollama with gemma2:2b model
    llm=ollama(model="gemma2:2b"),
    
    tools=[],
    instructions="Research the given topic thoroughly."
)

```

### Example 2: Custom Ollama Server

```python theme={null}
from laddr import Agent
from laddr.llms import ollama

# Connect to Ollama on custom port/host
researcher = Agent(
    name="researcher",
    role="Researcher",
    goal="Research information",
    backstory="Expert researcher",
    
    llm=ollama(
        model="llama3.2:latest",
        base_url="http://192.168.1.100:11434"  # Remote Ollama server
    ),
    
    tools=[]
)

```

### Example 3: Mixed Backends

```python theme={null}
from laddr import Agent
from laddr.llms import ollama, gemini

# Coordinator uses cloud API (complex reasoning)
coordinator = Agent(
    name="coordinator",
    role="Task Coordinator",
    goal="Coordinate research tasks",
    backstory="Experienced coordinator",
    llm=gemini("gemini-2.0-flash"),  # Cloud API
    is_coordinator=True
)

# Researcher uses local Ollama (cost-effective)
researcher = Agent(
    name="researcher",
    role="Researcher",
    goal="Find information",
    backstory="Research specialist",
    llm=ollama("gemma2:2b"),  # Local model
    tools=[web_search]
)

```

### Example 4: Environment-Based Configuration

```python theme={null}
# agents/researcher.py
from laddr import Agent
from laddr.llms import ollama

# Model and URL from environment variables
researcher = Agent(
    name="researcher",
    role="Researcher",
    goal="Research topics",
    backstory="Expert researcher",
    
    # Automatically uses:
    # - LLM_MODEL_RESEARCHER or OLLAMA_MODEL or default
    # - OLLAMA_BASE_URL or default
    llm=ollama(),
    
    tools=[]
)

```

***

## Available Models

Popular Ollama models you can use:

* **gemma2:2b** - Fast, lightweight (good for simple tasks)
* **llama3.2:latest** - Balanced performance and quality
* **mistral:7b** - Good for writing and analysis
* **llama3.1:8b** - Strong reasoning capabilities
* **qwen2.5:7b** - Multilingual support

<Tip>
  Start with smaller models (2b-7b) for faster responses. Use larger models (13b+) for complex reasoning tasks.
</Tip>

***

## Troubleshooting

### Connection Errors

If you see connection errors:

1. **Verify Ollama is running:**
   ```bash theme={null}
   curl http://localhost:11434/api/tags
   ```

2. **Check Docker network:**
   ```bash theme={null}
   docker compose exec worker curl http://host.docker.internal:11434/api/tags
   ```

3. **Verify environment variables:**
   ```bash theme={null}
   docker compose exec worker env | grep OLLAMA
   ```

### Model Not Found

If a model is not found:

1. **Pull the model:**
   ```bash theme={null}
   ollama pull gemma2:2b
   ```

2. **List available models:**
   ```bash theme={null}
   ollama list
   ```

### Performance Issues

If responses are slow:

1. Use smaller models for simple tasks
2. Increase Ollama's context window if needed
3. Check system resources (CPU, RAM)
4. Consider using GPU acceleration

***

## Next Steps

* [Agent Configuration](/guides/agents/agent-config) - Configure agents
* [Local Runtime](/guides/local-runtime) - Local development
* [Scaling & Operations](/guides/scaling-and-ops) - Production deployment