n8n Ollama Local LLM: Deployment Private AI Automation

I run local LLMs with Ollama and n8n because sending sensitive data to cloud APIs is a privacy risk. When I process customer records, financial data, or proprietary code, I need the AI to run entirely within my infrastructure. Ollama provides a lightweight local LLM server. n8n connects to it through its REST API, turning private AI into automated workflows.

Ollama runs models like Llama 3, Mistral, and Gemma on your own hardware. No data leaves your machine. The Ollama API mirrors OpenAI’s format, so I can swap between local and cloud models without changing my n8n workflows. This flexibility matters when I need to benchmark local model quality against cloud alternatives.

Deploying Ollama Locally

Setting up Ollama takes minutes. Pull a model and start the server:

`bash

Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Pull Llama 3 model

ollama pull llama3.2

Pull Mistral for lighter workloads

ollama pull mistral

Pull Gemma for code tasks

ollama pull gemma2

Verify the server is running

curl http://localhost:11434/api/tags
`

The Ollama server listens on port 11434 by default. It exposes endpoints for listing models, generating completions, and embedding generation. n8n reaches these endpoints through HTTP Request nodes.

For production deployment, I run Ollama in Docker for isolation and resource management:

`bash
docker run -d –name ollama -p 11434:11434 -v ollama-data:/root/.ollama ollama/ollama
`

The volume mount persists models between container restarts. Without it, pulling models again after a restart wastes bandwidth and time.

Basic LLM Calls from n8n

Ollama’s chat endpoint accepts the same message format as OpenAI’s API. I use n8n’s HTTP Request node to call it:

`javascript
// HTTP Request node: Ollama chat completion
Method: POST
URL: http://localhost:11434/api/chat
Body: {
“model”: “llama3.2”,
“messages”: [
{
“role”: “user”,
“content”: “{{user_query}}”
}
],
“stream”: false,
“options”: {
“temperature”: 0.7,
“num_predict”: 512
}
}
`

The response contains the model’s reply in message.content. I extract it for downstream processing:

`javascript
// Code node: extract Ollama response
const reply = item.json.message?.content || ‘No response received’;
return [{ json: { llm_reply: reply, model_used: item.json.model } }];
`

Streaming responses work too. Set "stream": true and process the incremental chunks in n8n. This gives faster perceived response times for long outputs:

`javascript
// HTTP Request node: streaming response
Method: POST
URL: http://localhost:11434/api/chat
Body: {
“model”: “llama3.2”,
“messages”: [{ “role”: “user”, “content”: “{{long_query}}” }],
“stream”: true
}
`

Each streamed chunk contains a partial message. The Code node accumulates them into a complete response.

Embedding Generation for Local Vector Search

Ollama generates embeddings through a separate endpoint. These vector representations power semantic search without sending data to external APIs:

`javascript
// HTTP Request node: generate embeddings
Method: POST
URL: http://localhost:11434/api/embed
Body: {
“model”: “nomic-embed-text”,
“input”: [“{{document_text}}”]
}
`

The nomic-embed-text model produces 768-dimensional vectors suitable for similarity search. I store these in a local vector store or pass them to n8n’s Code node for cosine similarity calculations:

`javascript
// Code node: cosine similarity between two embeddings
function cosineSimilarity(a, b) {
const dotProduct = a.reduce((sum, val, i) => sum + val * b[i], 0);
const magA = Math.sqrt(a.reduce((sum, val) => sum + val * val, 0));
const magB = Math.sqrt(b.reduce((sum, val) => sum + val * val, 0));
return dotProduct / (magA * magB);
}

const vecA = item.json.embedding || [];
const vecB = item.json.query_embedding || [];
const similarity = cosineSimilarity(vecA, vecB);

return [{ json: { similarity_score: similarity.toFixed(4), document: item.json.document } }];
`

This local embedding pipeline replaces cloud-dependent vector search. Documents get embedded on your hardware, similarity computed locally, and results returned without external API calls.

Prompt Engineering for Local Models

Local models vary in capability. Llama 3.2 handles general tasks well. Mistral excels at instruction following. Gemma performs strongly on code generation. I match models to tasks:

`javascript
// Code node: select model based on task type
const taskType = item.json.task_category || ‘general’;
const modelMap = {
coding: ‘gemma2’,
analysis: ‘llama3.2’,
writing: ‘mistral’,
general: ‘llama3.2’
};

return [{
json: {
…item.json,
selected_model: modelMap[taskType] || modelMap.general
}
}];
`

Temperature and token limits control output quality. Lower temperature (0.2-0.5) produces consistent, factual responses. Higher temperature (0.7-1.0) encourages creativity. I adjust these based on the workflow purpose:

`javascript
Body: {
“model”: “llama3.2”,
“messages”: [{ “role”: “user”, “content”: “{{instruction}}” }],
“stream”: false,
“options”: {
“temperature”: {{task_type === ‘creative’ ? 0.9 : 0.3}},
“num_predict”: {{task_type === ‘creative’ ? 1024 : 256}}
}
}
`

System prompts help local models stay focused. Ollama supports system messages through the API:

`javascript
Body: {
“model”: “llama3.2”,
“system”: “You are a data analyst. Provide numerical insights from the given data. Show your calculations.”,
“messages”: [{ “role”: “user”, “content”: “{{dataset_analysis_request}}” }],
“stream”: false
}
`

Connecting to Your Existing Setup

The n8n Docker Deployment guide covers running Ollama and n8n in the same Docker Compose network. Set the Ollama URL to http://ollama:11434/api when both containers share a network.

For resource constraints, the n8n Docker Environment Variables reference explains memory limits and timeout settings. Local LLMs consume significant RAM — allocate at least 8GB for Llama 3.2 8B models.

The n8n AI Agents guide extends local LLM usage into autonomous agent workflows where the model makes decisions and triggers n8n actions.

Action Card: Local LLM Test

Verify your Ollama setup with this minimal workflow:

`javascript
// 1. HTTP Request node:
// Method: POST
// URL: http://localhost:11434/api/chat
// Body: {“model”:”llama3.2″,”messages”:[{“role”:”user”,”content”:”What is n8n?”}],”stream”:false}

// 2. Response node: check message.content field

// 3. If node: if response contains “workflow”, the model is working correctly
`

Replace the model name if you pulled a different one. The response should appear in seconds on capable hardware.

References