🎯

ollama

🎯Skill
g1joshi/agent-skills3 Installs9Last Updated Feb 10, 2026

Installation

$npx skills add g1joshi/agent-skills --skill ollama

Run it in a terminal; it asks which AI coding tools to install it for (Claude Code, Codex, Cursor and more).

Uses skills.sh Β· Node.js 22.20+

After installing

Ask for related work and the agent uses it on its own, or call it yourself.

Run it (Claude Code)/ollama
Install by hand

Copy the whole skill folder from GitHub to the place below. Copying only SKILL.md can leave out files the skill uses.

Open the skill folder β†’
  • This project .claude/skills/ollama/
  • All projects ~/.claude/skills/ollama/

These paths are for Claude Code. For other tools, the command above puts it in the right place.

Expert Ollama assistance covering local LLM execution, Modelfile customization, REST API integration, and model quantizations. Use when running open models (Llama, Mistral, DeepSeek) locally on CPU/GPU.

Ollama

Ollama bundles model weights, configurations, and runtime dependencies into a streamlined CLI and local server for running open-source LLMs locally with native tool-calling.

When to Use

  • Local LLM Execution: Running Llama 3, DeepSeek-R1, Mistral, and Qwen locally with a single CLI command.
  • Zero-Cloud Air-Gapped Deployments: Privacy-critical enterprise environments requiring 100% on-premise execution.
  • Custom Modelfile Configuration: Tailoring system prompts, temperature, context length, and stop sequences.
  • REST & OpenAI-Compatible API: Seamless drop-in replacement for OpenAI SDKs in local test and dev pipelines.

Quick Start

# Run open-weight models locally with zero setup
ollama run llama3.2:3b "Write a Python script to compute Fibonacci numbers."
# Query local Ollama instance via Python SDK
import ollama

response = ollama.chat(
    model="llama3.2:3b",
    messages=[{"role": "user", "content": "Explain zero-trust security."}]
)
print(response['message']['content'])

Core Concepts

Local Inference with the Python Ollama SDK

Querying local models with streaming and structured JSON output:

import ollama

# Basic chat completion
response = ollama.chat(
    model='llama3.1:8b',
    messages=[
        {'role': 'system', 'content': 'You are an expert DevOps engineer.'},
        {'role': 'user', 'content': 'Write a minimal production Dockerfile for a Go API.'}
    ],
    options={
        'temperature': 0.2,
        'num_ctx': 8192
    }
)

print(response['message']['content'])

Structured Output with Pydantic & JSON Schema

Enforcing typed JSON responses from local models:

from pydantic import BaseModel
import ollama

class ThreatAnalysis(BaseModel):
    threat_level: str
    vulnerabilities: list[str]
    mitigation: str

response = ollama.chat(
    model='llama3.1:8b',
    messages=[
        {'role': 'user', 'content': 'Analyze the security of an unauthenticated Redis instance exposed to 0.0.0.0.'}
    ],
    format=ThreatAnalysis.model_json_schema(),
    options={'temperature': 0.1}
)

analysis = ThreatAnalysis.model_validate_json(response['message']['content'])
print("Threat Level:", analysis.threat_level)
print("Mitigation:", analysis.mitigation)

Custom Modelfile Authoring

Creating tailored local model artifacts:

# Modelfile
FROM llama3.1:8b

PARAMETER temperature 0.3
PARAMETER num_ctx 16384
PARAMETER stop "<|eot_id|>"

SYSTEM '''
You are Antigravity Code Guardian. You analyze git diffs for security vulnerabilities,
performance regressions, and design pattern anti-patterns. Respond in concise markdown tables.
'''
# Build and register the custom model
ollama create code-guardian -f ./Modelfile
ollama run code-guardian "Review this diff..."

Common Patterns

Custom System Instructions and Temperature via Modelfile

Problem: Re-sending identical system prompts and temperature settings with every API request.

Solution: Create a custom Modelfile:

FROM llama3.2:3b

# Set model temperature
PARAMETER temperature 0.2

# Set system instruction
SYSTEM """
You are a senior DevOps engineer. Always provide answers as copy-pasteable bash or Terraform snippets.
"""

Build model: ollama create devops-assistant -f ./Modelfile

Best Practices

Do:

  • Set OLLAMA_NUM_PARALLEL and OLLAMA_MAX_LOADED_MODELS to handle concurrent local requests efficiently.
  • Use format=Schema.model_json_schema() for reliable structured outputs without regex parsing.
  • Offload models to Apple Silicon Metal or NVIDIA CUDA automatically with proper driver setups.
  • Utilize ollama pull and model tags to test quantized variants (:q4_K_M, :q8_0).

Don't:

  • Expose Ollama port (11434) to the public internet without an authenticating reverse proxy (Nginx, Caddy).
  • Assume default context is unlimited; explicitly set num_ctx: 16384 or higher in options if needed.
  • Run heavy 70B models on systems with less than 64GB unified memory/RAM.

Troubleshooting

ErrorCauseSolution
Error: could not connect to ollama appOllama daemon not running in background.Start daemon with ollama serve.
CUDA out of memory on local GPUModel size exceeds discrete GPU VRAM.Pull a smaller quantization (e.g. q4_0) or specify CPU layers in Modelfile.
pull model failed: unexpected EOFNetwork timeout during multi-gigabyte weight download.Re-run ollama pull <model> to resume download from checkpoint.

References

More from this repository10