ml-paper-writing

• [Our Mission](#our-mission)
• [Path Towards AI Research Agent](#path-towards-ai-research-agent)
• [Available AI Research Engineering Skills](#available-ai-research-engineering-skills)
• [Demos](#demos)
• [Skill Structure](#skill-structure)
• [Roadmap](#roadmap)
• [Repository Structure](#repository-structure)
• [Use Cases](#use-cases)
• [Contributing](#contributing)
• [Community](#community)

We provide the layer of Engineering Ability that enable your coding agent to write and conduct AI research experiments, including preparing datasets, executing training pipelines, deploying models, and building your AI agents.

System diagram of an AI research agent

Modern AI research requires mastering dozens of specialized tools and frameworks.

AI Researchers spend more time debugging infrastructure than testing hypotheses—slowing the pace of scientific discovery.

We provide a comprehensive library of expert-level research engineering skills that enable AI agents to autonomously implement and execute different stages of AI research experiments—from data preparation and model training to evaluation and deployment.

- Specialized Expertise - Each skill provides deep, production-ready knowledge of a specific framework (Megatron-LM, vLLM, TRL, etc.)

- End-to-End Coverage - 82 skills spanning model architecture, tokenization, fine-tuning, mechanistic interpretability, data processing, post-training, distributed training, optimization, evaluation, inference, infrastructure, agents, RAG, multimodal, prompt engineering, MLOps, observability, emerging techniques, and ML paper writing

- Research-Grade Quality - Documentation sourced from official repos, real GitHub issues, and battle-tested production workflows

Quality over quantity: Each skill provides comprehensive, expert-level guidance with real code examples, troubleshooting guides, and production-ready workflows.

📦 Quick Install (Recommended)

Install skills to any coding agent (Claude Code, OpenCode, Cursor, Codex, Gemini CLI, Qwen Code) with one command:

```bash

npx @orchestra-research/ai-research-skills

```

This launches an interactive installer that:

• Auto-detects your installed coding agents
• Installs skills to ~/.orchestra/skills/ with symlinks to each agent
• Offers everything, quickstart bundle, by category, or individual skills
• Updates installed skills with latest versions
• Uninstalls all or selected skills

All 20 Categories (82 Skills)

| Category | Skills | Included |

|----------|--------|----------|

| Model Architecture | 5 | LitGPT, Mamba, NanoGPT, RWKV, TorchTitan |

| Tokenization | 2 | HuggingFace Tokenizers, SentencePiece |

| Fine-Tuning | 4 | Axolotl, LLaMA-Factory, PEFT, Unsloth |

| Mech Interp | 4 | TransformerLens, SAELens, pyvene, nnsight |

| Data Processing | 2 | NeMo Curator, Ray Data |

| Post-Training | 8 | TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge |

| Safety | 3 | Constitutional AI, LlamaGuard, NeMo Guardrails |

| Distributed | 6 | DeepSpeed, FSDP, Accelerate, Megatron-Core, Lightning, Ray Train |

| Infrastructure | 3 | Modal, Lambda Labs, SkyPilot |

| Optimization | 6 | Flash Attention, bitsandbytes, GPTQ, AWQ, HQQ, GGUF |

| Evaluation | 3 | lm-eval-harness, BigCode, NeMo Evaluator |

| Inference | 4 | vLLM, TensorRT-LLM, llama.cpp, SGLang |

| MLOps | 3 | W&B, MLflow, TensorBoard |

| Agents | 4 | LangChain, LlamaIndex, CrewAI, AutoGPT |

| RAG | 5 | Chroma, FAISS, Pinecone, Qdrant, Sentence Transformers |

| Prompt Eng | 4 | DSPy, Instructor, Guidance, Outlines |

| Observability | 2 | LangSmith, Phoenix |

| Multimodal | 7 | CLIP, Whisper, LLaVA, BLIP-2, SAM, Stable Diffusion, AudioCraft |

| Emerging | 6 | MoE, Model Merging, Long Context, Speculative Decoding, Distillation, Pruning |

| ML Paper Writing | 1 | ML Paper Writing (LaTeX templates, citation verification) |

🏗️ Model Architecture (5 skills)

• [LitGPT](01-model-architecture/litgpt/) - Lightning AI's 20+ clean LLM implementations with production training recipes (462 lines + 4 refs)
• [Mamba](01-model-architecture/mamba/) - State-space models with O(n) complexity, 5× faster than Transformers (253 lines + 3 refs)
• [RWKV](01-model-architecture/rwkv/) - RNN+Transformer hybrid, infinite context, Linux Foundation project (253 lines + 3 refs)
• [NanoGPT](01-model-architecture/nanogpt/) - Educational GPT in ~300 lines by Karpathy (283 lines + 3 refs)
• [TorchTitan](01-model-architecture/torchtitan/) - PyTorch-native distributed training for Llama 3.1 with 4D parallelism

🔤 Tokenization (2 skills)

• [HuggingFace Tokenizers](02-tokenization/huggingface-tokenizers/) - Rust-based, <20s/GB, BPE/WordPiece/Unigram algorithms (486 lines + 4 refs)
• **[