vector-index-tuning
🎯SkillA guide for making meaning-based search over large data faster, more accurate and lighter on memory. It helps you choose the right settings as your data grows from thousands to billions of items.
Installation
npx skills add wshobson/agents --skill vector-index-tuningRun it in a terminal; it asks which AI coding tools to install it for (Claude Code, Codex, Cursor and more).
Uses skills.sh · Node.js 22.20+
After installing
Ask for related work and the agent uses it on its own, or call it yourself.
/vector-index-tuningInstall by hand
Copy the whole skill folder from GitHub to the place below. Copying only SKILL.md can leave out files the skill uses.
Open the skill folder →- This project
.claude/skills/vector-index-tuning/ - All projects
~/.claude/skills/vector-index-tuning/
These paths are for Claude Code. For other tools, the command above puts it in the right place.
What this skill does
Overview
This skill helps you tune a vector index for real use. A vector index is a structure that speeds up searching through lists of numbers that stand for the meaning of data. The skill shows how to balance speed, memory and recall, which means how many of the true best matches a search finds.
Key Features
- Pick an index type based on how many items you have, from Flat up to IVF+PQ or DiskANN.
- Adjust the HNSW settings
M,efConstructionandefSearchwith clear notes on each trade-off. - Save memory with quantization, which stores numbers in a smaller, less precise form.
- Follow do's and don'ts, such as testing with real queries and planning for index rebuilds.
- Find full templates and worked examples in
references/details.md.
Who is this for?
It suits engineers running vector search in production who face slow searches or high memory use. It also helps teams who expect their data to grow quickly.
Summarized by AI from this skill's SKILL.md. Suggest a better one
SKILL.md
View on GitHub →Optimize vector index performance for latency, recall, and memory. Use when tuning HNSW parameters, selecting quantization strategies, or scaling vector search infrastructure.
Vector Index Tuning
Guide to optimizing vector indexes for production performance.
When to Use This Skill
- Tuning HNSW parameters
- Implementing quantization
- Optimizing memory usage
- Reducing search latency
- Balancing recall vs speed
- Scaling to billions of vectors
Core Concepts
1. Index Type Selection
Data Size Recommended Index
────────────────────────────────────────
< 10K vectors → Flat (exact search)
10K - 1M → HNSW
1M - 100M → HNSW + Quantization
> 100M → IVF + PQ or DiskANN
2. HNSW Parameters
| Parameter | Default | Effect |
|---|---|---|
| M | 16 | Connections per node, ↑ = better recall, more memory |
| efConstruction | 100 | Build quality, ↑ = better index, slower build |
| efSearch | 50 | Search quality, ↑ = better recall, slower search |
3. Quantization Types
Full Precision (FP32): 4 bytes × dimensions
Half Precision (FP16): 2 bytes × dimensions
INT8 Scalar: 1 byte × dimensions
Product Quantization: ~32-64 bytes total
Binary: dimensions/8 bytes
Templates and detailed worked examples
Full template library and detailed worked examples live in references/details.md. Read that file when you need the concrete templates.
Best Practices
Do's
- Benchmark with real queries - Synthetic may not represent production
- Monitor recall continuously - Can degrade with data drift
- Start with defaults - Tune only when needed
- Use quantization - Significant memory savings
- Consider tiered storage - Hot/cold data separation
Don'ts
- Don't over-optimize early - Profile first
- Don't ignore build time - Index updates have cost
- Don't forget reindexing - Plan for maintenance
- Don't skip warming - Cold indexes are slow
More from this repository10
Helps you build a consistent look for your app with Tailwind CSS v4. It sets up shared colors, sizes, dark mode and reusable components so every screen matches.
A guide to TypeScript's advanced type features, which describe the shape of your data so mistakes are caught before the code runs. It covers reusable types, types that change based on rules, and built-in helper types, with short examples.
Guidance for building reliable server-side apps with Node.js, using Express or Fastify. It covers login, error handling, databases, API design and a list of proven best practices.
Intelligent automation and multi-agent orchestration for Claude Code
Claude Code plugin for AI-powered framework migration workflows, coordinating specialized agents to plan and execute transitions between technology stacks as part of the wshobson/agents marketplace.
A Claude Code plugin for payment system integration covering Stripe, PayPal, and billing platforms with checkout flows, subscription management, and webhook handling in the wshobson/agents ecosystem.
A Claude Code plugin bundling core development tools and agents for everyday tasks like debugging and code generation, part of the wshobson/agents marketplace with 73 focused plugins.
Claude Code plugin for observability and monitoring including logging, metrics, distributed tracing, and alerting, part of the wshobson/agents marketplace with 73 focused plugins.
A Claude Code plugin providing specialized AI agents for deployment strategy planning and execution, covering blue-green deployments, canary releases, rolling updates, and zero-downtime patterns.
Claude Code plugin for JVM language development including Java, Kotlin, and Scala with specialized agents for idiomatic patterns, build tools, and runtime optimization in the wshobson/agents marketplace.