# How to Deploy Multi-Agent AI: Setup CrewAI on Ubuntu 24.04 Bare Metal GPU

Multi-agent AI frameworks have shifted from experimental prototypes to production-grade automation engines. But running CrewAI or AutoGen agents via cloud APIs introduces crippling costs: a single agent task can trigger 20 to 50 LLM calls due to autonomous thinking loops.

By deploying your multi-agent AI framework on a Dedicated Bare Metal GPU server, your VRAM becomes a fixed cost. Whether your agent loops 10 times or 10,000 times, your infrastructure bill remains flat.

---

## Phase 1: VRAM & KV Cache Math

When multiple agents run concurrently, each agent consumes VRAM for model weights plus KV Cache (Key-Value Cache) for context memory:

* A **14B model** (like Qwen2.5 or Llama-3.3) takes ~15GB of VRAM in FP8 precision.
* Each concurrent agent context requires roughly **1.5GB of KV cache**.
* Running a 4-agent crew concurrently requires unthrottled GPU access and high PCIe throughput to prevent Out-Of-Memory (OOM) crashes and latency spikes.

---

## Phase 2: Server Hardening & UFW Setup

Never expose your local LLM engine to the public internet. Leaving inference ports open invites GPU hijacking. Lock down your Ubuntu 24.04 firewall:

```bash
# Update Ubuntu packages
sudo apt update && sudo apt upgrade -y

# Allow only SSH, HTTP, HTTPS
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
```

---

## Phase 3: Install Ollama & Pull the Model

Install Ollama to host your sovereign inference engine locally:

```bash
curl -fsSL [https://ollama.com/install.sh](https://ollama.com/install.sh) | sh

# Pull a 14B+ model for robust agent reasoning
ollama pull qwen2.5:14b
```

> **Engineering Tip:** 8B models struggle with complex JSON structured outputs required for CrewAI task delegation. Use at least 14B models for worker agents and 32B/70B models for manager agents.

---

## Phase 4: Deploying CrewAI with `uv`

Avoid raw `pip` to prevent Python environment conflicts. Use `uv` from Astral for ultra-fast environment isolation:

```bash
# Install UV package manager
curl -LsSf [https://astral.sh/uv/install.sh](https://astral.sh/uv/install.sh) | sh
source $HOME/.cargo/env

# Initialize project scaffold
uv tool install crewai
crewai create crew servermo_agents
cd servermo_agents
```

### The LiteLLM Trap
CrewAI utilizes LiteLLM under the hood, which crashes if it doesn't detect an OpenAI API key—even when routing strictly to `localhost`. Bypass this by creating a `.env` file with dummy values:

```env
OPENAI_API_KEY="NA"
OPENAI_API_BASE="http://localhost:11434/v1"
```

---

## Phase 5: Production Crew Code (`crew.py`)

```python
from crewai import Agent, Task, Crew, Process, LLM

# 1. Define Local GPU LLM
bare_metal_llm = LLM(
    model="ollama/qwen2.5:14b",
    base_url="http://localhost:11434",
    temperature=0.2
)

# 2. Architect Agents (Disable infinite delegation loops)
research_agent = Agent(
    role="Infrastructure Security Analyst",
    goal="Discover vulnerabilities in cloud VM networking",
    backstory="Elite SRE who trusts only bare metal servers.",
    llm=bare_metal_llm,
    verbose=True,
    allow_delegation=False
)

writer_agent = Agent(
    role="DevSecOps Technical Writer",
    goal="Draft an actionable security report based on findings",
    backstory="Synthesizes complex security data into Markdown.",
    llm=bare_metal_llm,
    verbose=True,
    allow_delegation=False
)

# 3. Define Tasks
research_task = Task(
    description="Analyze why multi-tenant Cloud VMs are less secure than Dedicated Bare Metal. List 3 key points.",
    expected_output="3 technical bullet points regarding hypervisor vulnerabilities.",
    agent=research_agent
)

write_task = Task(
    description="Write a 2-paragraph security advisory from the 3 bullet points.",
    expected_output="Formatted Markdown security advisory document.",
    agent=writer_agent,
    context=[research_task]
)

# 4. Initialize and Run Crew
production_crew = Crew(
    agents=[research_agent, writer_agent],
    tasks=[research_task, write_task],
    process=Process.sequential
)

if __name__ == "__main__":
    result = production_crew.kickoff()
    print("\n--- FINAL OUTPUT ---\n", result)
```

---

## Deployment Model Comparison

| Deployment Model | Compute Cost Model | Latency Stability | Data Sovereignty |
| :--- | :--- | :--- | :--- |
| **Public Cloud APIs** | Punishing per-token fees | High network jitter | Third-party privacy risk |
| **Shared Cloud VMs** | Low initial cost | Noisy neighbors cause OOMs | Shared hypervisor risk |
| **ServerMO Bare Metal GPU ★** | **100% Fixed monthly rate** | **Unthrottled PCIe & VRAM** | **100% Private network** |

***

👉 **Ready to deploy sovereign multi-agent AI on bare metal GPUs? Read the full deployment guide on ServerMO:**  
[Setup CrewAI on Ubuntu 24.04 Bare Metal GPU | ServerMO](https://www.servermo.com/howto/deploy-crewai-ubuntu-bare-metal/)