{storage_root}/backups/ 2026-01-06T12-30-45_doctor-repair/ manifest.json database.sqlite3 archive.bundle ``` ### Restore from backup If something goes wrong, restore from any backup: ```bash # Preview what would be restored uv run python -m mcp_agent_mail.cli doctor restore /path/to/backup --dry-run # Restore (prompts for confirmation) uv run python -m mcp_agent_mail.cli doctor restore /path/to/backup # Skip confirmation prompt uv run python -m mcp_agent_mail.cli doctor restore /path/to/backup --yes ``` **Restore process:** 1. Validates backup manifest exists and is readable 2. Shows backup metadata (creation time, reason, contents) 3. **Creates a pre-restore backup** of current state (safety net) 4. Restores SQLite database from backup 5. Restores Git archive from bundle 6. Reports any errors encountered **Safety features:** - Current database saved as `*.sqlite3.pre-restore` before overwrite - Current archive saved as `*.pre-restore` directory before overwrite - Errors during restore are captured and reported ### Best practices 1. **Run diagnostics regularly**: `am doctor check` is fast and non-destructive 2. **Review before repair**: Use `--dry-run` first to see what would change 3. **Keep backups**: Don't delete old backups until you've verified the system is healthy 4. **Automate checks**: Include `am doctor check --json` in your CI/monitoring for early warning ### Quick Start: Interactive Deployment Wizard The easiest way to export and deploy is the interactive wizard, which supports both GitHub Pages and Cloudflare Pages: ```bash # Via CLI (recommended) uv run python -m mcp_agent_mail.cli share wizard # Or run the script directly ./scripts/share_to_github_pages.py ``` #### What the wizard does The wizard provides a fully automated end-to-end deployment experience: 1. **Session resumption**: Detects interrupted sessions and offers to resume exactly where you left off, avoiding re-export 2. **Configuration management**: Remembers your last settings and offers to reuse them, saving time on repeated exports 3. **Deployment target selection**: Choose between GitHub Pages, Cloudflare Pages, or local export 4. **Automatic CLI installation**: Detects and installs missing tools (`gh` for GitHub, `wrangler` for Cloudflare) 5. **Guided authentication**: Step-by-step browser login flows for GitHub and Cloudflare 6. **Smart project selection**: - Shows all available projects in a formatted table - Supports multiple selection modes: `all`, single number (`1`), lists (`1,3,5`), or ranges (`1-3`, `2-5,8`) - Remembers your previous selection for quick re-export 7. **Redaction configuration**: Choose between `standard` (scrub secrets like API keys/tokens, keep agent names) or `strict` (redact all message bodies) 8. **Cryptographic signing**: Optional Ed25519 signing with automatic key generation or reuse of existing keys 9. **Pre-flight validation**: Checks that GitHub repo names are available before starting the export 10. **Deployment summary**: Shows what will be deployed (project count, bundle size, target, signing status) and asks for confirmation 11. **Export and preview**: Exports the bundle and launches an interactive preview server with automatic port detection (tries 9000-9100) 12. **Interactive preview controls**: - Press **'r'** to force browser refresh (manual cache bust) - Press **'d'** to skip preview and deploy immediately - Press **'q'** to quit preview server 13. **Automatic viewer asset refresh**: Always ensures latest HTML/JS/CSS from source tree are used, even when reusing bundles 14. **Real-time deployment**: Streams git and deployment output in real-time so you can follow the progress 15. **Automatic deployment**: Creates repos, enables Pages, pushes code, and gives you the live URL #### Session resumption (new in latest version) If you interrupt the wizard (close terminal, Ctrl+C during preview, etc.), it saves your progress to `~/.mcp-agent-mail/wizard-session/`. When you run the wizard again: ``` Incomplete session detected Projects: 3 selected Stage: preview Workspace: ~/.mcp-agent-mail/wizard-session/bundle Resume where you left off? (Y/n): ``` **What gets saved:** - Selected projects and scrub preset - Deployment configuration (target, repo name, etc.) - Signing key preferences and paths - Exported bundle (in session workspace) - Current stage (preview, deploy) **Resume scenarios:** - **Closed terminal during preview**: Resume → Skip re-export → Launch preview immediately - **Changed your mind after export**: Resume → "Reuse bundle?" → Preview or re-export - **Want to deploy later**: Resume → Press 'd' in preview → Deploy without re-exporting - **Made viewer code changes**: Resume → Assets auto-refresh from source tree After successful deployment, the session state is automatically cleared. Sessions also clear if they become invalid (workspace deleted, projects removed, etc.). #### Configuration persistence The wizard saves your configuration to `~/.mcp-agent-mail/wizard-config.json` after each successful deployment. On subsequent runs, it will show: ``` Previous Configuration Found Projects: 3 selected Redaction: standard Target: github-new Use these settings again? (Y/n): ``` This allows rapid re-deployment with the same settings. The saved configuration includes: - Selected project indices (validates against current project list) - Redaction preset - Deployment target and parameters (repo name, privacy, project name) - Signing preferences (whether to sign, whether to generate new key) - Last used signing key path (offered as default when not generating new key) Configuration is project-agnostic: if you add or remove projects, the wizard validates saved indices and prompts for re-selection if needed. **Difference between session and config:** - **Session state** (`wizard-session/`): Temporary, for resuming interrupted runs, includes exported bundle - **Config file** (`wizard-config.json`): Persistent, for "use last settings" across fresh runs, no bundle #### Multi-project selection The project selector supports flexible selection syntax: ``` Available Projects: # Slug Path 1 backend-abc123 /abs/path/backend 2 frontend-xyz789 /abs/path/frontend 3 infra-def456 /abs/path/infra 4 scripts-ghi789 /abs/path/scripts Select projects to export (e.g., 'all', '1,3,5', or '1-3'): ``` **Selection modes:** - `all`: Export all projects (default) - `1`: Export project #1 only - `1,3,5`: Export projects #1, #3, and #5 - `1-3`: Export projects #1, #2, and #3 (inclusive range) - `2-4,7`: Export projects #2, #3, #4, and #7 (combined range and list) Invalid selections (out of range, malformed) are rejected with helpful error messages and the wizard prompts again. #### Dynamic port allocation The preview server automatically detects an available port in the range 9000-9100 instead of failing if port 9000 is in use. The actual port is displayed: ``` Launching preview server... Using port 9001 (Ctrl+C to stop server) Waiting for server to start... ✓ Server ready, opening browser at http://127.0.0.1:9001 ``` This prevents port conflicts when multiple previews are running or when port 9000 is used by other services. #### Deployment summary panel Before starting the export, the wizard shows a comprehensive summary: ``` ═══ Deployment Summary ═══ Projects: 3 selected Bundle size: ~32 MB Redaction: standard Target: GitHub Pages Repository: mailbox-viewer-2024 Visibility: Private Signing: Enabled (Ed25519) Proceed with export and deployment? (Y/n): ``` This gives you a final chance to review all settings and cancel if needed. The bundle size is estimated based on ~10 MB per project plus ~2 MB for static assets. #### Real-time deployment streaming Git operations and Cloudflare deployments stream output in real-time so you can see exactly what's happening: ``` Initializing git repository and pushing... Initializing repository... Initialized empty Git repository in /tmp/mailbox-preview-abc123/.git/ ✓ Initializing repository complete Adding files... ✓ Adding files complete Creating commit... [main (root-commit) 1a2b3c4] Initial mailbox export 425 files changed, 123456 insertions(+) ✓ Creating commit complete Pushing to GitHub... Enumerating objects: 430, done. Counting objects: 100% (430/430), done. Delta compression using up to 8 threads Compressing objects: 100% (425/425), done. Writing objects: 100% (430/430), 12.34 MiB | 5.67 MiB/s, done. ✓ Pushing to GitHub complete ✓ Successfully pushed to owner/mailbox-viewer-2024 ``` This provides transparency and helps diagnose issues if deployment fails. #### Platform-specific details **For GitHub Pages:** - Wizard detects your package manager (brew/apt/dnf) and offers automated installation of `gh` CLI - For apt/dnf, shows complete manual installation instructions (including repo setup) since automation requires sudo - Runs `gh auth login` interactively to authenticate via browser - Creates new repository with your specified name and visibility (public/private) - Initializes git, commits, and pushes with streaming output - Enables GitHub Pages automatically via the GitHub API - Provides the GitHub Pages URL (may take 1-2 minutes to become live) **For Cloudflare Pages:** - Detects npm and offers automated installation of `wrangler` CLI - Runs `wrangler login` interactively to authenticate via browser - Deploys directly to Cloudflare's global CDN (no git repository needed) - Streams wrangler output in real-time - Provides the `.pages.dev` URL immediately (site is live instantly) - Benefits: instant deployment, 275+ global locations, automatic HTTPS, unlimited requests on free tier **For local export:** - Saves bundle to specified directory - No CLI installation or authentication required - Suitable for manual deployment to custom hosting or inspection #### Error handling and recovery The wizard includes comprehensive error handling: - **Pre-flight validation**: Checks GitHub repo availability before starting export to avoid conflicts - **Port conflict resolution**: Automatically finds an available port for preview server - **Invalid selection handling**: Validates project selections and prompts for correction - **CLI installation failures**: Shows manual installation instructions if automatic installation fails - **Git operation failures**: Each git step is validated; stops on first failure with clear error message - **Deployment failures**: Distinguishes between repo creation, push, and Pages enablement failures If deployment fails after export, the bundle remains in the temp directory and can be deployed manually using the git commands shown in the manual deployment section below. The wizard handles all operations automatically. For manual control or advanced options, see the detailed workflows below. ### Basic export workflow (manual) **1. Export a bundle** ```bash # Export all projects to a directory uv run python -m mcp_agent_mail.cli share export --output ./my-bundle # Export specific projects only uv run python -m mcp_agent_mail.cli share export \ --output ./my-bundle \ --project backend-abc123 \ --project frontend-xyz789 # Export with Ed25519 signing for tamper-evident distribution uv run python -m mcp_agent_mail.cli share export \ --output ./my-bundle \ --signing-key ./keys/signing.key \ --signing-public-out ./keys/signing.pub # Export and encrypt with age for secure distribution uv run python -m mcp_agent_mail.cli share export \ --output ./my-bundle \ --age-recipient age1abc...xyz \ --age-recipient age1def...uvw ``` The export process: 1. Creates a snapshot of the SQLite database (read-only, no WAL/SHM files) 2. Copies message bodies, attachments, and metadata into the bundle structure 3. Applies redaction rules based on the scrub preset (default: `standard`) 4. Generates `manifest.json` with SHA-256 hashes for all assets 5. Optionally signs the manifest with Ed25519 (produces `manifest.sig.json`) 6. Packages everything into a ZIP archive (optional, enabled by default) 7. If chunking is enabled, writes the segmented database plus a `chunks.sha256` manifest so streamed pages can be verified cheaply 8. Optionally encrypts the ZIP with age (produces `bundle.zip.age`) ### Refresh an existing bundle Once you have published a bundle you can refresh it in place without re-running the full wizard. Every export records the settings that were used (projects, scrub preset, attachment thresholds, chunking config) inside `manifest.json`. The new `share update` command reads those defaults, regenerates the SQLite snapshot and viewer assets in a temporary directory, and then replaces the bundle atomically—removing obsolete chunked files or attachments along the way. ```bash # Refresh bundle using the originally recorded settings uv run python -m mcp_agent_mail.cli share update ./my-bundle # Override one or more export options while updating uv run python -m mcp_agent_mail.cli share update ./my-bundle \ --project backend-abc123 \ --inline-threshold 16384 \ --chunk-threshold 104857600 # Re-sign and package the refreshed bundle uv run python -m mcp_agent_mail.cli share update ./my-bundle \ --zip \ --signing-key ./keys/signing.key ``` When chunking was enabled previously but the refreshed snapshot no longer needs it, `share update` cleans up the `chunks/` directory, `chunks.sha256`, and `mailbox.sqlite3.config.json` automatically, ensuring the bundle tree matches the new manifest. You can still tweak any setting at update time; overrides are written back into the `export_config` section of `manifest.json` for the next refresh. **2. Preview locally** ```bash # Serve the bundle on localhost:9000 uv run python -m mcp_agent_mail.cli share preview ./my-bundle # Custom port and auto-open browser uv run python -m mcp_agent_mail.cli share preview ./my-bundle \ --port 8080 \ --open-browser ``` This launches a lightweight HTTP server that serves the static files. Open `http://127.0.0.1:9000/viewer/` in your browser to explore the archive. **Interactive preview controls:** - **'r'**: Force browser reload (bumps manual cache-bust token, triggers viewer refresh) - **'d'**: Request deployment (exits with code 42; wizard detects and proceeds to deploy) - **'q'**: Quit preview server - **Ctrl+C**: Stop preview server The preview server automatically refreshes viewer assets from the source tree if available, ensuring you always see the latest HTML/JS/CSS during development. **3. Verify integrity** ```bash # Verify SRI hashes and signature uv run python -m mcp_agent_mail.cli share verify ./my-bundle # Verify with explicit public key (overrides manifest.sig.json) uv run python -m mcp_agent_mail.cli share verify ./my-bundle \ --public-key AAAA...base64... ``` Verification checks: - SHA-256 hashes for all vendor libraries (Marked.js, DOMPurify, SQL.js) - SHA-256 hashes for the SQLite database and attachments - Ed25519 signature over the canonical manifest (if present) **4. Decrypt (if encrypted)** ```bash # Decrypt with age identity file (private key) uv run python -m mcp_agent_mail.cli share decrypt bundle.zip.age \ --identity ~/.age/key.txt # Decrypt with passphrase (interactive prompt) uv run python -m mcp_agent_mail.cli share decrypt bundle.zip.age \ --passphrase # Specify custom output path uv run python -m mcp_agent_mail.cli share decrypt bundle.zip.age \ --output ./decrypted-bundle.zip \ --identity ~/.age/key.txt ``` After decryption, unzip the archive and use `share preview` to view it. ### Export options reference | Option | Type | Default | Description | | :-- | :-- | :-- | :-- | | `--output`, `-o` | Path | (required) | Directory where the static bundle will be written | | `--project`, `-p` | List | All projects | Limit export to specific project slugs or human keys (repeatable) | | `--inline-threshold` | Bytes | 65536 (64KB) | Inline attachments smaller than this as base64 data URIs | | `--detach-threshold` | Bytes | 26214400 (25MB) | Mark attachments larger than this as external (not bundled) | | `--scrub-preset` | String | `standard` | Redaction preset: `standard` or `strict` (see Redaction presets section) | | `--chunk-threshold` | Bytes | 20971520 (20MB) | Split SQLite database into chunks if it exceeds this size | | `--chunk-size` | Bytes | 4194304 (4MB) | Chunk size when splitting large databases | | `--dry-run` | Flag | false | Generate security summary and preview without writing files | | `--zip` / `--no-zip` | Flag | true | Package the bundle into a ZIP archive | | `--signing-key` | Path | None | Path to Ed25519 signing key (32-byte raw seed) | | `--signing-public-out` | Path | None | Write the Ed25519 public key to this file after signing | | `--age-recipient` | String | None | age public key for encryption (repeatable for multiple recipients) | | `--interactive`, `-i` | Flag | false | Launch interactive wizard (prints guidance; full wizard TBD) | ### Security features **XSS protection (DOMPurify + CSP)** Message bodies are rendered using a defense-in-depth pipeline: 1. **Marked.js** parses GitHub-Flavored Markdown into HTML 2. **DOMPurify** sanitizes the HTML, removing dangerous tags and attributes 3. **Content Security Policy** restricts script sources, blocks inline event handlers, and limits network access This prevents malicious content in message bodies from executing JavaScript or exfiltrating data. **CSP configuration notes:** - `script-src`: Allows self, CDNs (Tailwind, Alpine.js), and `'unsafe-eval'` (required for SQL.js WebAssembly) - `connect-src`: Allows `*` (all origins) to support preview mode polling and flexible deployment scenarios - `style-src`: Allows self, inline styles (for Tailwind), and font CDNs - Trusted Types removed for browser compatibility (Firefox, Safari don't support it yet) **Cryptographic signing (Ed25519)** When you provide a signing key, the export process: 1. Generates a canonical JSON representation of `manifest.json` 2. Signs it with Ed25519 (fast, 64-byte signatures, 128-bit security) 3. Writes the signature and public key to `manifest.sig.json` Recipients can verify the signature using `share verify` to ensure: - The bundle hasn't been modified since signing - The bundle was created by someone with the private key - All assets match their declared SHA-256 hashes **Requirements and fallback:** - Requires PyNaCl >= 1.6.0 (installed automatically with this package) - If PyNaCl is unavailable or signing fails, export gracefully falls back to unsigned mode - Wizard reuses existing signing keys by default (no re-generation unless requested) - Private keys are automatically excluded from git via `.gitignore` (signing-*.key pattern) **Encryption (age)** The `age` encryption tool (https://age-encryption.org/) provides modern, secure file encryption. When you provide recipient public keys, the export process encrypts the final ZIP archive. Only holders of the corresponding private keys can decrypt it. Generate keys with: ```bash # Install age (example for macOS) brew install age # Generate a new key pair age-keygen -o key.txt # Public key is printed to stdout; share it with exporters # Private key is saved to key.txt; keep it secret ``` **Redaction presets** The export pipeline supports configurable scrubbing to remove sensitive data: - `standard`: Clears acknowledgment/read state, removes file reservations and agent links, scrubs secrets (GitHub tokens, Slack tokens, OpenAI keys, bearer tokens, JWTs) from message bodies and attachment metadata. Retains agent names (which are already meaningless pseudonyms like "BlueMountain"), full message bodies, and attachments. - `strict`: All standard redactions plus replaces entire message bodies with "[Message body redacted]" placeholder and removes all attachments from the bundle. All presets apply redaction to message subjects, bodies, and attachment metadata before the bundle is written. ### Static viewer features The bundled HTML viewer provides: **Dashboard layout**: - **Gmail-style three-pane interface**: Projects sidebar, message list (center), and detail pane (right) - **Bundle metadata header**: Shows bundle creation time, export settings, and scrubbing preset - **Summary panels**: Side-by-side panels displaying projects included, attachment statistics, and redaction summary - **Message list**: Virtual-scrolled message list with sender, subject, snippet, and importance badges - **Raw manifest viewer**: Collapsible JSON display of the complete manifest for verification **Advanced boolean search** (new): Powered by SQLite FTS5 with LIKE fallback, supports complex queries: - **Boolean operators**: `(auth OR login) AND NOT admin` - **Quoted phrases**: `"build plan"` (exact match) - **Parentheses**: Control precedence like `(A OR B) AND (C OR D)` - **Operator precedence**: NOT > AND > OR (e.g., `A OR B AND C` = `A OR (B AND C)`) - **Automatic debouncing**: 140ms delay avoids hammering the database on every keystroke - **Performance**: FTS5 search is 10-100x faster than LIKE on large datasets **Lazy message loading** (performance optimization): - Initial load fetches only 280-character snippets for all messages (3-6x faster) - Full message body loaded on-demand when you click a message - Dramatically reduces memory usage and initial load time **Virtual scrolling** (new): Clusterize.js-powered virtual list rendering: - Smoothly handles 100,000+ messages without slowdown - Only ~30 DOM nodes exist at any time (visible rows + buffers) - Maintains native scrollbar feel with keyboard navigation **Markdown rendering**: Message bodies are rendered with GitHub-Flavored Markdown, supporting code blocks, tables, task lists, and inline images. **Opportunistic OPFS caching**: The SQLite database is cached in Origin Private File System (OPFS) in the background: - First load: Downloads from network, caches to OPFS during idle time - Subsequent loads: Instant from OPFS (even faster than IndexedDB) - Automatic cache key validation prevents stale data **Dark mode**: Toggle between light and dark themes with localStorage persistence. Dark mode state is managed by the main viewer controller for consistency. **Attachment preview**: Inline images render directly in message bodies. External attachments show file size and download links. **Message detail view**: Click any message in the list to load its full body (lazy), view metadata (sender, recipients, timestamp, importance), and browse attachments. **No server required**: After the initial HTTP serving (which can be a static file host like S3, GitHub Pages, or Netlify), all functionality runs client-side. No backend, no API calls, no authentication. **Browser compatibility**: Works in all modern browsers (Chrome, Firefox, Safari, Edge) with graceful fallbacks for missing features (OPFS, FTS5). ### Deployment options **Option 1: GitHub Pages (automated via wizard)** ```bash # Use the wizard for fully automated deployment uv run python -m mcp_agent_mail.cli share wizard # Select: GitHub Pages → provide repo name → wizard handles everything ``` Or manually: ```bash # Export and unzip uv run python -m mcp_agent_mail.cli share export --output ./bundle --no-zip cd bundle # Initialize git and push to GitHub git init git add . git commit -m "Initial export" git remote add origin git@github.com:yourorg/project-archive.git git push -u origin main # Enable GitHub Pages in repo settings (source: main branch, root directory) ``` **Option 2: Cloudflare Pages (automated via wizard)** ```bash # Use the wizard for instant global CDN deployment uv run python -m mcp_agent_mail.cli share wizard # Select: Cloudflare Pages → provide project name → wizard deploys directly ``` Or manually with wrangler CLI: ```bash # Export and deploy uv run python -m mcp_agent_mail.cli share export --output ./bundle --no-zip npx wrangler pages deploy ./bundle --project-name=project-archive # Your site is live at: https://project-archive.pages.dev ``` **Benefits of Cloudflare Pages:** - Instant deployment (no git repo required) - Global CDN with 275+ locations - Automatic HTTPS and DDoS protection - Zero-downtime updates - Generous free tier (500 builds/month, unlimited requests) **Option 3: S3 + CloudFront** ```bash # Export and unzip uv run python -m mcp_agent_mail.cli share export --output ./bundle --no-zip # Upload to S3 aws s3 sync ./bundle s3://your-bucket/archives/project-2024/ --acl public-read # Access via CloudFront # https://d123abc.cloudfront.net/archives/project-2024/ ``` **Option 4: Nginx static site** ```nginx server { listen 443 ssl; server_name archives.example.com; ssl_certificate /etc/letsencrypt/live/archives.example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/archives.example.com/privkey.pem; root /var/www/archives/project-2024; index index.html; # Enable gzip for efficient transfer gzip on; gzip_types text/html text/css application/javascript application/json application/wasm; # Cache static assets location ~* \.(js|css|wasm|png|jpg|webp)$ { expires 1y; add_header Cache-Control "public, immutable"; } # CSP headers are already in index.html meta tag # Add HTTPS-only and frame protection add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; add_header X-Frame-Options "DENY" always; add_header X-Content-Type-Options "nosniff" always; } ``` **Option 5: Encrypted distribution via file sharing** For confidential archives: ```bash # Export with age encryption uv run python -m mcp_agent_mail.cli share export \ --output ./bundle \ --signing-key ./signing.key \ --age-recipient age1auditor... \ --age-recipient age1manager... # This produces bundle.zip.age # Upload to Dropbox, Google Drive, or send via secure file transfer # Recipients decrypt locally uv run python -m mcp_agent_mail.cli share decrypt bundle.zip.age \ --identity ~/.age/key.txt # Verify integrity unzip bundle.zip uv run python -m mcp_agent_mail.cli share verify ./bundle # Preview locally uv run python -m mcp_agent_mail.cli share preview ./bundle ``` ### Example workflows **Quarterly audit package** ```bash # Export Q4 2024 communications for audit uv run python -m mcp_agent_mail.cli share export \ --output ./audit-q4-2024 \ --scrub-preset strict \ --signing-key ./audit-signing.key \ --signing-public-out ./audit-signing.pub \ --age-recipient age1auditor@firm.example # Produces: audit-q4-2024.zip.age # Send to auditor with audit-signing.pub # Auditor verifies: age -d -i auditor-key.txt audit-q4-2024.zip.age > audit-q4-2024.zip unzip audit-q4-2024.zip python -m mcp_agent_mail.cli share verify ./audit-q4-2024 \ --public-key $(cat audit-signing.pub) ``` **Executive summary for stakeholders** ```bash # Export high-importance threads only # (filter in UI after export, or use SQL to create filtered snapshot) uv run python -m mcp_agent_mail.cli share export \ --output ./exec-summary \ --project backend-prod \ --scrub-preset standard # Host on internal web server cp -r ./exec-summary /var/www/exec-archives/2024-12/ # Share link: https://internal.example.com/exec-archives/2024-12/ ``` **Disaster recovery backup** ```bash # Monthly encrypted backup uv run python -m mcp_agent_mail.cli share export \ --output ./backup-$(date +%Y-%m) \ --scrub-preset none \ --signing-key ./dr-signing.key \ --age-recipient age1dr@company.example # Store in off-site backup system aws s3 cp backup-2024-12.zip.age s3://dr-backups/mcp-mail/ \ --storage-class GLACIER_IR # Restore procedure documented in runbook ``` ### Troubleshooting exports **Export fails with "Database locked"** The export takes a snapshot using SQLite's Online Backup API. If the server is actively writing, wait a few seconds and retry. For large databases, consider temporarily stopping the server during export. **Bundle size is too large** Use `--detach-threshold` to mark large attachments as external references. These won't be included in the bundle but will show file metadata in the viewer. ```bash # Bundle files under 1MB, mark larger files as external uv run python -m mcp_agent_mail.cli share export \ --output ./bundle \ --detach-threshold 1048576 ``` Alternatively, filter to specific projects with `--project`. **Encrypted bundle won't decrypt** Verify you're using the correct identity file: ```bash # Get your public key from your identity file age-keygen -y identity.txt # Ensure this public key was included in the --age-recipient values during export # If you have multiple identity files, try each one age -d -i identity.txt bundle.zip.age > bundle.zip ``` **Signature verification fails** Signature verification requires: 1. The original `manifest.json` (unmodified) 2. The `manifest.sig.json` file (contains signature and public key) 3. All assets referenced in the manifest with matching SHA-256 hashes If verification fails, the bundle may have been tampered with or corrupted during transfer. Re-export and re-transfer. **Viewer shows blank page or errors** Check browser console for errors. Common issues: - **OPFS not supported**: Older browsers may not support Origin Private File System. The viewer will fall back to in-memory mode (slower). - **Database too large**: Browsers limit in-memory database size to ~1-2GB. Use chunking (`--chunk-threshold`) for very large archives. - **CSP violations**: If hosting the bundle, ensure the web server doesn't add conflicting CSP headers. The viewer's CSP is defined in `index.html` and should not be overridden. ### On-disk layout (per project) ``` /projects// agents//profile.json agents//inbox/YYYY/MM/.md agents//outbox/YYYY/MM/.md messages/YYYY/MM/.md messages/threads/.md # optional human digest maintained by the server file_reservations/.json attachments//.webp ``` ### Message file format Messages are GitHub-Flavored Markdown with JSON frontmatter (fenced by `---json`). Attachments are either WebP files referenced by relative path or inline base64 WebP data URIs. ```markdown ---json { "id": 1234, "thread_id": "TKT-123", "project": "/abs/path/backend", "project_slug": "backend-abc123", "from": "GreenCastle", "to": ["BlueLake"], "cc": [], "created": "2025-10-23T15:22:14Z", "importance": "high", "ack_required": true, "attachments": [ {"type": "file", "media_type": "image/webp", "path": "projects/backend-abc123/attachments/2a/2a6f.../diagram.webp"} ] } --- # Build plan for /api/users routes ... body markdown ... ``` ### Data model (SQLite) - `projects(id, human_key, slug, created_at)` - `agents(id, project_id, name, program, model, task_description, inception_ts, last_active_ts, attachments_policy, contact_policy)` - `messages(id, project_id, sender_id, thread_id, subject, body_md, created_ts, importance, ack_required, attachments)` - `message_recipients(message_id, agent_id, kind, read_ts, ack_ts)` - `file_reservations(id, project_id, agent_id, path_pattern, exclusive, reason, created_ts, expires_ts, released_ts)` - `agent_links(id, a_project_id, a_agent_id, b_project_id, b_agent_id, status, reason, created_ts, updated_ts, expires_ts)` - `project_sibling_suggestions(id, project_a_id, project_b_id, score, status, rationale, created_ts, evaluated_ts, confirmed_ts, dismissed_ts)` - `fts_messages(message_id UNINDEXED, subject, body)` + triggers for incremental updates ### Concurrency and lifecycle - One request/task = one isolated operation - Archive writes are guarded by a per-project `.archive.lock` under `projects//` - Git index/commit operations are serialized across the shared archive repo by a repo-level `.commit.lock` - DB operations are short-lived and scoped to each tool call; FTS triggers keep the search index current - Artifacts are written first, then committed as a cohesive unit with a descriptive message - Attachments are content-addressed (sha1) to avoid duplication ## How it works (key flows) 1) Create an identity - `register_agent(project_key, program, model, name?, task_description?)` → creates/updates a named identity, persists profile to Git, and commits. 2) Send a message - `send_message(project_key, sender_name, to[], subject, body_md, cc?, bcc?, attachment_paths?, convert_images?, importance?, ack_required?, thread_id?, auto_contact_if_blocked?)` - Writes a canonical message under `messages/YYYY/MM/`, an outbox copy for the sender, and inbox copies for each recipient; commits all artifacts. - Optionally converts images (local paths or data URIs) to WebP and embeds small ones inline. ```mermaid sequenceDiagram participant Agent participant Server participant DB participant Git Agent->>Server: call send_message Server->>DB: insert message and recipients DB-->>Server: ok Server->>Git: write canonical markdown Server->>Git: write outbox copy Server->>Git: write inbox copies Server->>Git: commit Server-->>Agent: result ``` 3) Check inbox - `fetch_inbox(project_key, agent_name, since_ts?, urgent_only?, include_bodies?, limit?)` returns recent messages, preserving thread_id where available. - `acknowledge_message(project_key, agent_name, message_id)` marks acknowledgements. 4) Avoid conflicts with file reservations (leases) - `file_reservation_paths(project_key, agent_name, paths[], ttl_seconds, exclusive, reason)` records an advisory lease in DB and writes JSON reservation artifacts in Git; conflicts are reported if overlapping active exclusives exist (reservations are still granted; conflicts are returned alongside grants). - `release_file_reservations(project_key, agent_name, paths? | file_reservation_ids?)` releases active leases (all if none specified). JSON artifacts remain for audit history. - Optional: install a pre-commit hook in your code repo that blocks commits conflicting with other agents' active exclusive file reservations. ```mermaid sequenceDiagram participant Agent participant Server participant DB participant Git Agent->>Server: call file_reservation_paths Server->>DB: expire old leases and check overlaps DB-->>Server: conflicts or grants Server->>DB: insert file reservation rows Server->>Git: write file reservation JSON files Server->>Git: commit Server-->>Agent: granted paths and any conflicts ``` 5) Search & summarize - `search_messages(project_key, query, limit?)` uses FTS5 over subject and body. - `summarize_thread(project_key, thread_id, include_examples?)` extracts key points, actions, and participants from the thread. - `reply_message(project_key, message_id, sender_name, body_md, ...)` creates a subject-prefixed reply, preserving or creating a thread. ### Semantics & invariants - Identity - Names are memorable adjective+noun and unique per project; `name_hint` is sanitized (alnum) and used if available - `whois` returns the stored profile; `list_agents` can filter by recent activity - `last_active_ts` is bumped on relevant interactions (messages, inbox checks, etc.) - Threads - Replies inherit `thread_id` from the original; if missing, the reply sets `thread_id` to the original message id - Subject lines are prefixed (e.g., `Re:`) for readability in mailboxes - Attachments - Image references (file path or data URI) are converted to WebP; small images embed inline when policy allows - Non-absolute paths resolve relative to the project repo root - Stored under `attachments//.webp` and referenced by relative path in frontmatter - File Reservations - TTL-based; exclusive means "please don't modify overlapping surfaces" for others until expiry or release - Conflict detection is per exact path pattern; shared reservations can coexist, exclusive conflicts are surfaced - JSON artifacts remain in Git for audit even after release (DB tracks release_ts) - Search - External-content FTS virtual table and triggers index subject/body on insert/update/delete - Queries are constrained to the project id and ordered by `created_ts DESC` ## Contact model and "consent-lite" messaging Goal: make coordination "just work" without spam across unrelated agents. The server enforces per-project isolation by default and adds an optional consent layer within a project so agents only contact relevant peers. ### Isolation by project - All tools require a `project_key`. Agents only see messages addressed to them within that project. - An agent working in Project A is invisible to agents in Project B unless explicit cross-project contact is established (see below). This avoids distraction between unrelated repositories. ### Policies (per agent) - `open`: accept any targeted messages in the project. - `auto` (default): allow messages when there is obvious shared context (e.g., same thread participants; recent overlapping active file reservations; recent prior direct contact within a TTL); otherwise requires a contact request. - `contacts_only`: require an approved contact link first. - `block_all`: reject all new contacts (errors with CONTACT_BLOCKED). Use `set_contact_policy(project_key, agent_name, policy)` to update. ### Request/approve contact - `request_contact(project_key, from_agent, to_agent, reason?, ttl_seconds?)` creates or refreshes a pending link and sends a small ack_required "intro" message to the recipient. - `respond_contact(project_key, to_agent, from_agent, accept, ttl_seconds?)` approves or denies; approval grants messaging until expiry. - `list_contacts(project_key, agent_name)` surfaces current links. ### Auto-allow heuristics (no explicit request required) - Same thread: replies or messages to thread participants are allowed. - Recent overlapping file reservations: if sender and recipient hold active file reservations in the project, messaging is allowed. - Recent prior contact: a sliding TTL allows follow-ups between the same pair. These heuristics minimize friction while preventing cold spam. ### Cross-project coordination (frontend vs backend repos) When two repos represent the same underlying project (e.g., `frontend` and `backend`), you have two options: 1) Use the same `project_key` across both workspaces. Agents in both repos operate under one project namespace and benefit from full inbox/outbox coordination automatically. 2) Keep separate `project_key`s and establish explicit contact: - In `backend`, agent `GreenCastle` calls: - `request_contact(project_key="/abs/path/backend", from_agent="GreenCastle", to_agent="BlueLake", reason="API contract changes")` - In `frontend`, `BlueLake` calls: - `respond_contact(project_key="/abs/path/backend", to_agent="BlueLake", from_agent="GreenCastle", accept=true)` - After approval, messages can be exchanged; in default `auto` policy the server allows follow-up threads/reservation-based coordination without re-requesting. Important: You can also create reciprocal links or set `open` policy for trusted pairs. The consent layer is on by default (CONTACT_ENFORCEMENT_ENABLED=true) but is designed to be non-blocking in obvious collaboration contexts. ## Resource layer (read-only URIs) Expose common reads as resources that clients can fetch. See API Quick Reference → Resources for the full list and parameters. Example (conceptual) resource read: ```json { "method": "resources/read", "params": {"uri": "resource://inbox/BlueLake?project=/abs/path/backend&limit=20"} } ``` ```mermaid sequenceDiagram participant Client participant Server participant DB Client->>Server: read inbox resource Server->>DB: select messages for agent DB-->>Server: rows Server-->>Client: inbox data ``` ## File Reservations and the optional pre-commit guard - Guard status and pre-push - Print guard status: - `mcp-agent-mail guard status /path/to/repo` - Install both guards (pre-commit + pre-push): - `mcp-agent-mail guard install --prepush` - Pre-commit honors `WORKTREES_ENABLED` and `AGENT_MAIL_GUARD_MODE` (`warn` advisory). - Pre-push enumerates to-be-pushed commits (`rev-list`) and uses `diff-tree` with `--no-ext-diff`. - Composition-safe install (chain-runner): - A Python chain-runner is written to `.git/hooks/pre-commit` and `.git/hooks/pre-push`. - It executes `hooks.d//*` in lexical order, then `.orig` if present (existing hooks are preserved, not overwritten). - Agent Mail installs its guard as `hooks.d/pre-commit/50-agent-mail.py` and `hooks.d/pre-push/50-agent-mail.py`. - Windows shims (`pre-commit.cmd/.ps1`, `pre-push.cmd/.ps1`) are written to invoke the Python chain-runner. - Matching and safety details: - Renames/moves are handled: both the old and new names are checked (`git diff --cached --name-status -M -z`). - NUL-safe end-to-end: paths are collected and forwarded as NUL-delimited to avoid ambiguity. - Git-native matching: reservations are checked using Git wildmatch pathspec semantics against repo-root relative paths; `core.ignorecase` is honored. - Emergency bypass (use sparingly): set `AGENT_MAIL_BYPASS=1`, or use native Git `--no-verify`. In `warn` mode the guard never blocks. ## Git-based project identity (opt-in) - Gate: `WORKTREES_ENABLED=1` or `GIT_IDENTITY_ENABLED=1` enables git-based identity features. Default off. - Identity modes (default `dir`): `dir`, `git-remote`, `git-toplevel`, `git-common-dir`. - Inspect identity for a path: - Resource (MCP): `resource://identity/{/abs/path}` (available when `WORKTREES_ENABLED=1`) - CLI (diagnostics): `mcp-agent-mail mail status /abs/path` - Precedence (when gate is on): 1) Committed marker `.agent-mail-project-id` (recommended) 2) Discovery YAML `.agent-mail.yaml` with `project_uid:` 3) Private marker under Git common dir `.git/agent-mail/project-id` 4) Remote fingerprint: normalized `origin` URL + default branch 5) `git-common-dir` hash; else dir hash - Migration helpers: - Write committed marker: `mcp-agent-mail projects mark-identity . --commit` - Scaffold discovery file: `mcp-agent-mail projects discovery-init . --product ` Example identity payload (resource): ```json { "project_uid": "c5b2c86b-7c36-4de6-9a0a-2c4e1c3a1c4a", "slug": "repo-a1b2c3d4e5", "identity_mode_used": "git-remote", "canonical_path": "github.com/owner/repo", "human_key": "/abs/worktree/path", "repo_root": "/abs/repo", "git_common_dir": "/abs/repo/.git", "branch": "feature/x", "worktree_name": "repo-wt-x", "core_ignorecase": true, "normalized_remote": "github.com/owner/repo" } ``` ## Adopt/Merge legacy projects (optional) Consolidate legacy per-worktree projects into a canonical one (safe, explicit, and auditable). - Plan the merge (no changes): - `mcp-agent-mail projects adopt --dry-run` - Apply the merge (moves artifacts and re-keys DB rows): - `mcp-agent-mail projects adopt --apply` - Safeguards and behavior: - Requires both projects be in the same repository (validated via `git-common-dir`). - Moves archived Git artifacts from `projects//…` to `projects//…` while preserving history. - Re-keys database rows (`agents`, `messages`, `file_reservations`) from source to target project. - Records `aliases.json` under the target with `"former_slugs": [...]` for discoverability. - Aborts if agent-name conflicts would break uniqueness in the target (fix names, then retry). - Idempotent where possible; dry-run always prints a clear plan before apply. ## Build slots and helpers (opt-in) - `amctl env` prints helpful environment keys: - `SLUG`, `PROJECT_UID`, `BRANCH`, `AGENT`, `CACHE_KEY`, `ARTIFACT_DIR` - Example: `mcp-agent-mail amctl env --path . --agent AliceDev` - `am-run` wraps a command with those keys set: - Example: `mcp-agent-mail am-run frontend-build -- npm run dev` - Build slots (advisory, per-project coarse locking): - Flags: - `--ttl-seconds`: lease duration (default 3600) - `--shared/--exclusive`: non-exclusive or exclusive lease (default exclusive) - `--block-on-conflicts`: exit non-zero if exclusive conflicts are detected before starting - Acquire: - Tool: `acquire_build_slot(project_key, agent_name, slot, ttl_seconds=3600, exclusive=true)` - Renew: - Tool: `renew_build_slot(project_key, agent_name, slot, extend_seconds=1800)` - Release (non-destructive; marks released): - Tool: `release_build_slot(project_key, agent_name, slot)` - Notes: - Slots are recorded under the project archive `build_slots//__.json` - `exclusive=true` reports conflicts if another active exclusive holder exists - Intended for long-running tasks (dev servers, watchers); pair with `am-run` and `amctl env` ## Product Bus Group multiple repositories (e.g., frontend, backend, infra) under a single product for product‑wide inbox/search and shared threads. - Ensure a product: - `mcp-agent-mail products ensure MyProduct --name "My Product"` - Link a project (slug or path) into the product: - `mcp-agent-mail products link MyProduct .` - Inspect product and linked projects: - `mcp-agent-mail products status MyProduct` - Product‑wide message search (FTS): - `mcp-agent-mail products search MyProduct "urgent AND deploy" --limit 50` - Product‑wide inbox: - `mcp-agent-mail products inbox MyProduct Alice --limit 50 --urgent-only --include-bodies --since-ts "2025-11-01T00:00:00Z"` - Product‑wide thread summarization: - `mcp-agent-mail products summarize-thread MyProduct "bd-123" --per-thread-limit 100 --no-llm` ## Containers - Build and run locally: ```bash docker build -t mcp-agent-mail . docker run --rm -p 8765:8765 \ -e HTTP_HOST=0.0.0.0 \ -e STORAGE_ROOT=/data/mailbox \ -v agent_mail_data:/data \ mcp-agent-mail ``` - Or with Compose: ```bash docker compose up --build ``` - Notes: - Runs as an unprivileged user (`appuser`, uid 10001). - Includes a HEALTHCHECK against `/health/liveness`. - The server reads config from `.env` via python-decouple. You can mount it read-only into the container at `/app/.env`. - Default bind host is `0.0.0.0` in the container; port `8765` is exposed. - Persistent archive lives under `/data/mailbox` (mapped to the `agent_mail_data` volume by default). Notes - A unique `product_uid` is stored for each product; you can reference a product by uid or name. - Server tools also exist for orchestration: `ensure_product`, `products_link`, `search_messages_product`, and `resource://product/{key}`. Exclusive file reservations are advisory but visible and auditable: - A reservation JSON is written to `file_reservations/.json` capturing holder, pattern, exclusivity, created/expires - The pre-commit guard scans active exclusive reservations and blocks commits that touch conflicting paths held by another agent - Agents must set `AGENT_NAME` so the guard knows who "owns" the commit - The server continuously evaluates reservations for staleness (agent inactivity + mail/filesystem/git silence) and releases abandoned locks automatically; the `force_release_file_reservation` tool uses the same heuristics and notifies the previous holder when another agent clears a stale lease Install the guard into a code repo (conceptual tool call): ```json { "method": "tools/call", "params": { "name": "install_precommit_guard", "arguments": { "project_key": "/abs/path/backend", "code_repo_path": "/abs/path/backend" } } } ``` ## Configuration Configuration is loaded from an existing `.env` via `python-decouple`. Do not use `os.getenv` or auto-dotenv loaders. ### Changing the HTTP Port If port 8765 is already in use (e.g., by Cursor's Python extension), you can change it: **Option 1: During installation** One-liner with custom port: ```bash curl -fsSL "https://raw.githubusercontent.com/Dicklesworthstone/mcp_agent_mail/main/scripts/install.sh?$(date +%s)" | bash -s -- --port 9000 --yes ``` Or with local script: ./scripts/install.sh --port 9000 --yes ``` **Option 2: After installation (CLI)** ```bash # Change port using CLI command uv run python -m mcp_agent_mail.cli config set-port 9000 # View current port configuration uv run python -m mcp_agent_mail.cli config show-port # Restart server for changes to take effect scripts/run_server_with_token.sh ``` **Option 3: Manual .env edit** ```bash # Edit .env file manually with your text editor (recommended) nano .env # or vim, code, etc. # Or append (⚠️ warning: will create duplicate if HTTP_PORT already exists) echo "HTTP_PORT=9000" >> .env ``` **Option 4: CLI server override** ```bash # Override port at server startup (doesn't modify .env) uv run python -m mcp_agent_mail.cli serve-http --port 9000 ``` ```python from decouple import Config as DecoupleConfig, RepositoryEnv decouple_config = DecoupleConfig(RepositoryEnv(".env")) STORAGE_ROOT = decouple_config("STORAGE_ROOT", default="~/.mcp_agent_mail_git_mailbox_repo") HTTP_HOST = decouple_config("HTTP_HOST", default="127.0.0.1") HTTP_PORT = int(decouple_config("HTTP_PORT", default=8765)) HTTP_PATH = decouple_config("HTTP_PATH", default="/mcp/") ``` Common variables you may set: ### Configuration reference | Name | Default | Description | | :-- | :-- | :-- | | `STORAGE_ROOT` | `~/.mcp_agent_mail_git_mailbox_repo` | Root for per-project repos and SQLite DB | | `HTTP_HOST` | `127.0.0.1` | Bind host for HTTP transport | | `HTTP_PORT` | `8765` | Bind port for HTTP transport | | `HTTP_PATH` | `/mcp/` | HTTP path where MCP endpoint is mounted | | `HTTP_JWT_ENABLED` | `false` | Enable JWT validation middleware | | `HTTP_JWT_SECRET` | | HMAC secret for HS* algorithms (dev) | | `HTTP_JWT_JWKS_URL` | | JWKS URL for public key verification | | `HTTP_JWT_ALGORITHMS` | `HS256` | CSV of allowed algs | | `HTTP_JWT_AUDIENCE` | | Expected `aud` (optional) | | `HTTP_JWT_ISSUER` | | Expected `iss` (optional) | | `HTTP_JWT_ROLE_CLAIM` | `role` | JWT claim name containing role(s) | | `HTTP_RBAC_ENABLED` | `true` | Enforce read-only vs tools RBAC | | `HTTP_RBAC_READER_ROLES` | `reader,read,ro` | CSV of reader roles | | `HTTP_RBAC_WRITER_ROLES` | `writer,write,tools,rw` | CSV of writer roles | | `HTTP_RBAC_DEFAULT_ROLE` | `reader` | Role used when none present | | `HTTP_RBAC_READONLY_TOOLS` | `health_check,fetch_inbox,whois,search_messages,summarize_thread` | CSV of read-only tool names | | `HTTP_RATE_LIMIT_ENABLED` | `false` | Enable token-bucket limiter | | `HTTP_RATE_LIMIT_BACKEND` | `memory` | `memory` or `redis` | | `HTTP_RATE_LIMIT_PER_MINUTE` | `60` | Legacy per-IP limit (fallback) | | `HTTP_RATE_LIMIT_TOOLS_PER_MINUTE` | `60` | Per-minute for tools/call | | `HTTP_RATE_LIMIT_TOOLS_BURST` | `0` | Optional burst for tools (0=auto=rpm) | | `HTTP_RATE_LIMIT_RESOURCES_PER_MINUTE` | `120` | Per-minute for resources/read | | `HTTP_RATE_LIMIT_RESOURCES_BURST` | `0` | Optional burst for resources (0=auto=rpm) | | `HTTP_RATE_LIMIT_REDIS_URL` | | Redis URL for multi-worker limits | | `HTTP_REQUEST_LOG_ENABLED` | `false` | Print request logs (Rich + JSON) | | `LOG_JSON_ENABLED` | `false` | Output structlog JSON logs | | `INLINE_IMAGE_MAX_BYTES` | `65536` | Threshold (bytes) for inlining WebP images during send_message | | `CONVERT_IMAGES` | `true` | Convert images to WebP (and optionally inline small ones) | | `KEEP_ORIGINAL_IMAGES` | `false` | Also store original image bytes alongside WebP (attachments/originals/) | | `LOG_LEVEL` | `INFO` | Server log level | | `HTTP_CORS_ENABLED` | `false` | Enable CORS middleware when true | | `HTTP_CORS_ORIGINS` | | CSV of allowed origins (e.g., `https://app.example.com,https://ops.example.com`) | | `HTTP_CORS_ALLOW_CREDENTIALS` | `false` | Allow credentials on CORS | | `HTTP_CORS_ALLOW_METHODS` | `*` | CSV of allowed methods or `*` | | `HTTP_CORS_ALLOW_HEADERS` | `*` | CSV of allowed headers or `*` | | `HTTP_BEARER_TOKEN` | | Static bearer token (only when JWT disabled) | | `HTTP_ALLOW_LOCALHOST_UNAUTHENTICATED` | `true` | Allow localhost requests without auth (dev convenience) | | `HTTP_OTEL_ENABLED` | `false` | Enable OpenTelemetry instrumentation | | `OTEL_SERVICE_NAME` | `mcp-agent-mail` | Service name for telemetry | | `OTEL_EXPORTER_OTLP_ENDPOINT` | | OTLP exporter endpoint URL | | `APP_ENVIRONMENT` | `development` | Environment name (development/production) | | `DATABASE_URL` | `sqlite+aiosqlite:///./storage.sqlite3` | SQLAlchemy async database URL | | `DATABASE_ECHO` | `false` | Echo SQL statements for debugging | | `GIT_AUTHOR_NAME` | `mcp-agent` | Git commit author name | | `GIT_AUTHOR_EMAIL` | `mcp-agent@example.com` | Git commit author email | | `LLM_ENABLED` | `true` | Enable LiteLLM for thread summaries and discovery | | `LLM_DEFAULT_MODEL` | `gpt-5-mini` | Default LiteLLM model identifier | | `LLM_TEMPERATURE` | `0.2` | LLM temperature for text generation | | `LLM_MAX_TOKENS` | `512` | Max tokens for LLM completions | | `LLM_CACHE_ENABLED` | `true` | Enable LLM response caching | | `LLM_CACHE_BACKEND` | `memory` | LLM cache backend (`memory` or `redis`) | | `LLM_CACHE_REDIS_URL` | | Redis URL for LLM cache (if backend=redis) | | `LLM_COST_LOGGING_ENABLED` | `true` | Log LLM API costs and token usage | | `FILE_RESERVATIONS_CLEANUP_ENABLED` | `false` | Enable background cleanup of expired file reservations | | `FILE_RESERVATIONS_CLEANUP_INTERVAL_SECONDS` | `60` | Interval for file reservations cleanup task | | `FILE_RESERVATION_INACTIVITY_SECONDS` | `1800` | Inactivity threshold (seconds) before a reservation is considered stale | | `FILE_RESERVATION_ACTIVITY_GRACE_SECONDS` | `900` | Grace window for recent mail/filesystem/git activity to keep a reservation active | | `FILE_RESERVATIONS_ENFORCEMENT_ENABLED` | `true` | Block message writes on conflicting file reservations | | `ACK_TTL_ENABLED` | `false` | Enable overdue ACK scanning (logs/panels; see views/resources) | | `ACK_TTL_SECONDS` | `1800` | Age threshold (seconds) for overdue ACKs | | `ACK_TTL_SCAN_INTERVAL_SECONDS` | `60` | Scan interval for overdue ACKs | | `ACK_ESCALATION_ENABLED` | `false` | Enable escalation for overdue ACKs | | `ACK_ESCALATION_MODE` | `log` | `log` or `file_reservation` escalation mode | | `ACK_ESCALATION_CLAIM_TTL_SECONDS` | `3600` | TTL for escalation file reservations | | `ACK_ESCALATION_CLAIM_EXCLUSIVE` | `false` | Make escalation file reservation exclusive | | `ACK_ESCALATION_CLAIM_HOLDER_NAME` | | Ops agent name to own escalation file reservations | | `CONTACT_ENFORCEMENT_ENABLED` | `true` | Enforce contact policy before messaging | | `CONTACT_AUTO_TTL_SECONDS` | `86400` | TTL for auto-approved contacts (1 day) | | `CONTACT_AUTO_RETRY_ENABLED` | `true` | Auto-retry contact requests on policy violations | | `MESSAGING_AUTO_REGISTER_RECIPIENTS` | `true` | Automatically create missing local recipients during `send_message` and retry routing | | `MESSAGING_AUTO_HANDSHAKE_ON_BLOCK` | `true` | When contact policy blocks delivery, attempt a contact handshake (auto-accept) and retry | | `TOOLS_LOG_ENABLED` | `true` | Log tool invocations for debugging | | `LOG_RICH_ENABLED` | `true` | Enable Rich console logging | | `LOG_INCLUDE_TRACE` | `false` | Include trace-level logs | | `TOOL_METRICS_EMIT_ENABLED` | `false` | Emit periodic tool usage metrics | | `TOOL_METRICS_EMIT_INTERVAL_SECONDS` | `60` | Interval for metrics emission | | `RETENTION_REPORT_ENABLED` | `false` | Enable retention/quota reporting | | `RETENTION_REPORT_INTERVAL_SECONDS` | `3600` | Interval for retention reports (1 hour) | | `RETENTION_MAX_AGE_DAYS` | `180` | Max age for retention policy reporting | | `QUOTA_ENABLED` | `false` | Enable quota enforcement | | `QUOTA_ATTACHMENTS_LIMIT_BYTES` | `0` | Max attachment storage per project (0=unlimited) | | `QUOTA_INBOX_LIMIT_COUNT` | `0` | Max inbox messages per agent (0=unlimited) | | `RETENTION_IGNORE_PROJECT_PATTERNS` | `demo,test*,testproj*,testproject,backendproj*,frontendproj*` | CSV of project patterns to ignore in retention/quota reports | | `AGENT_NAME_ENFORCEMENT_MODE` | `coerce` | Agent naming policy: `strict` (reject invalid adjective+noun names), `coerce` (auto-generate if invalid), `always_auto` (always auto-generate) | ## Development quick start Prerequisite: complete the setup above (Python 3.14 + uv venv + uv sync). Dev helpers: ```bash # Quick endpoint smoke test (server must be running locally) bash scripts/test_endpoints.sh # Pre-commit guard smoke test (no pytest) bash scripts/test_guard.sh ``` Database schema (automatic): ```bash # Tables are created from SQLModel definitions on first run. # If models change, delete the SQLite DB (and WAL/SHM) and run migrate again. uv run python -m mcp_agent_mail.cli migrate ``` Run the server (HTTP-only). Use the Typer CLI or module entry: ```bash uv run python -m mcp_agent_mail.cli serve-http uv run python -m mcp_agent_mail.http --host 127.0.0.1 --port 8765 ``` Connect with your MCP client using the HTTP (Streamable HTTP) transport on the configured host/port. The endpoint tolerates both `/mcp` and `/mcp/`. ## Search syntax tips (SQLite FTS5) - Basic terms: `plan users` - Phrase search: `"build plan"` - Prefix search: `mig*` - Boolean operators: `plan AND users NOT legacy` - Field boosting is not enabled by default; subject and body are indexed. Keep queries concise. When FTS is unavailable, the UI/API automatically falls back to SQL LIKE on subject/body. ## Design choices and rationale - **HTTP-only FastMCP**: Streamable HTTP is the modern remote transport; STDIO is not exposed here by design - **Git + Markdown**: Human-auditable, diffable artifacts that fit developer workflows (inbox/outbox mental model) - **SQLite + FTS5**: Efficient indexing/search with minimal ops footprint - **Advisory file reservations**: Make intent explicit and reviewable; optional guard enforces reservations at commit time - **WebP attachments**: Compact images by default; inline embedding keeps small diagrams in context - Optional: keep original binaries and dedup manifest under `attachments/` for audit and reuse ## Examples (conceptual tool calls) This section has been removed to keep the README focused. See API Quick Reference below for canonical method signatures. ## Operational notes - One async session per request/task; don't share across concurrent coroutines - Use explicit loads in async code; avoid implicit lazy loads - Use async-friendly file operations when needed; Git operations are serialized with a file lock - Clean shutdown should dispose any async engines/resources (if introduced later) ## Security and ops - Transport - HTTP-only (Streamable HTTP). Place behind a reverse proxy (e.g., NGINX) with TLS termination for production - Auth - Optional JWT (HS*/JWKS) via HTTP middleware; enable with `HTTP_JWT_ENABLED=true` - Static bearer token (`HTTP_BEARER_TOKEN`) is independent of JWT; when set, BearerAuth protects all routes (including UI). You may use it alone or together with JWT. - When JWKS is configured (`HTTP_JWT_JWKS_URL`), incoming JWTs must include a matching `kid` header; tokens without `kid` or with unknown `kid` are rejected - Starter RBAC (reader vs writer) using role configuration; see `HTTP_RBAC_*` settings - Reverse proxy + TLS (minimal example) - NGINX location block: ```nginx upstream mcp_mail { server 127.0.0.1:8765; } server { listen 443 ssl; server_name mcp.example.com; ssl_certificate /etc/letsencrypt/live/mcp.example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/mcp.example.com/privkey.pem; location /mcp/ { proxy_pass http://mcp_mail; proxy_set_header Host $host; proxy_set_header X-Forwarded-Proto https; } } ``` - Backups and retention - The Git repos and SQLite DB live under `STORAGE_ROOT`; back them up together for consistency - Observability - Add logging and metrics at the ASGI layer returned by `mcp.http_app()` (Prometheus, OpenTelemetry) - Concurrency - Archive writes: per-project `.archive.lock` prevents cross-project head-of-line blocking - Commits: repo-level `.commit.lock` serializes Git index/commit to avoid races across projects ## Python client example (HTTP JSON-RPC) This section has been removed to keep the README focused. Client code samples belong in `examples/`. ## Troubleshooting - "sender_name not registered" - Create the agent first with `register_agent` or `create_agent_identity`, or check the `project_key` you're using matches the sender's project - Pre-commit hook blocks commits - Set `AGENT_NAME` to your agent identity; release or wait for conflicting exclusive file reservations; inspect `.git/hooks/pre-commit` - Inline images didn't embed - Ensure `convert_images=true`; images are automatically inlined if the compressed WebP size is below the server's `INLINE_IMAGE_MAX_BYTES` threshold (default 64KB). Larger images are stored as attachments instead. - Message not found - Confirm the `project` disambiguation when using `resource://message/{id}`; ids are unique per project - Inbox empty but messages exist - Check `since_ts`, `urgent_only`, and `limit`; verify recipient names match exactly (case-sensitive) ## FAQ - Why Git and SQLite together? - Git provides human-auditable artifacts and history; SQLite provides fast queries and FTS search. Each is great at what the other isn't. - Are file reservations enforced? - Yes, optionally. The server can block message writes when a conflicting active exclusive reservation exists (`FILE_RESERVATIONS_ENFORCEMENT_ENABLED=true`, default). Reservations themselves are advisory and always return both `granted` and `conflicts`. The optional pre-commit hook adds local enforcement at commit time in your code repo. - Why HTTP-only? - Streamable HTTP is the modern remote transport for MCP; avoiding extra transports reduces complexity and encourages a uniform integration path. - Why JSON-RPC instead of REST or gRPC? - MCP defines a tool/resource method call model that maps naturally to JSON-RPC over a single endpoint. It keeps clients simple (one URL, method name + params), plays well with proxies, and avoids SDK lock-in while remaining language-agnostic. - Why separate "resources" (reads) from "tools" (mutations)? - Clear semantics enable aggressive caching and safe prefetch for resources, while tools remain explicit, auditable mutations. This split also powers RBAC (read-only vs writer) without guesswork. - Why canonical message storage in Git, not only in the database? - Git gives durable, diffable, human-reviewable artifacts you can clone, branch, and audit. SQLite provides fast indexing and FTS. The combo preserves governance and operability without a heavyweight message bus. - Why advisory file reservations instead of global locks? - Agents coordinate asynchronously; hard locks create head-of-line blocking and brittle failures. Advisory reservations surface intent and conflicts while the optional pre-commit guard enforces locally where it matters. - Why are agent names adjective+noun? - Memorable identities reduce confusion in inboxes, commit logs, and UI. The scheme yields low collision risk while staying human-friendly (vs GUIDs) and predictable for directory listings. - Why is `project_key` an absolute path? - Using the workspace's absolute path creates a stable, collision-resistant project identity across shells and agents. Slugs are derived deterministically from it, avoiding accidental forks of the same project. - Why WebP attachments and optional inlining? - WebP provides compact, high-quality images. Small images can be inlined for readability; larger ones are stored as attachments. You can keep originals when needed (`KEEP_ORIGINAL_IMAGES=true`). - Why both static bearer and JWT/JWKS support? - Local development should be zero-friction (single bearer). Production benefits from verifiable JWTs with role claims, rotating keys via JWKS, and layered RBAC. - Why SQLite FTS5 instead of an external search service? - FTS5 delivers fast, relevant search with minimal ops. It’s embedded, portable, and easy to back up with the Git archive. If FTS isn’t available, we degrade to SQL LIKE automatically. - Why is LLM usage optional? - Summaries and discovery should enhance, not gate, core functionality. Keeping LLM usage optional controls cost and latency while allowing richer UX when enabled. ## API Quick Reference ### Tools > Tip: to see tools grouped by workflow with recommended playbooks, fetch `resource://tooling/directory`. | Name | Signature | Returns | Notes | | :-- | :-- | :-- | :-- | | `health_check` | `health_check()` | `{status, environment, http_host, http_port, database_url}` | Lightweight readiness probe | | `ensure_project` | `ensure_project(human_key: str)` | `{id, slug, human_key, created_at}` | Idempotently creates/ensures project | | `register_agent` | `register_agent(project_key: str, program: str, model: str, name?: str, task_description?: str, attachments_policy?: str)` | Agent profile dict | Creates/updates agent; writes profile to Git | | `whois` | `whois(project_key: str, agent_name: str, include_recent_commits?: bool, commit_limit?: int)` | Agent profile dict | Enriched profile for one agent (optionally includes recent commits) | | `create_agent_identity` | `create_agent_identity(project_key: str, program: str, model: str, name_hint?: str, task_description?: str, attachments_policy?: str)` | Agent profile dict | Always creates a new unique agent | | `send_message` | `send_message(project_key: str, sender_name: str, to: list[str], subject: str, body_md: str, cc?: list[str], bcc?: list[str], attachment_paths?: list[str], convert_images?: bool, importance?: str, ack_required?: bool, thread_id?: str, auto_contact_if_blocked?: bool)` | `{deliveries: list, count: int, attachments?}` | Writes canonical + inbox/outbox, converts images | | `reply_message` | `reply_message(project_key: str, message_id: int, sender_name: str, body_md: str, to?: list[str], cc?: list[str], bcc?: list[str], subject_prefix?: str)` | `{thread_id, reply_to, deliveries: list, count: int, attachments?}` | Preserves/creates thread, inherits flags | | `request_contact` | `request_contact(project_key: str, from_agent: str, to_agent: str, to_project?: str, reason?: str, ttl_seconds?: int)` | Contact link dict | Request permission to message another agent | | `respond_contact` | `respond_contact(project_key: str, to_agent: str, from_agent: str, accept: bool, from_project?: str, ttl_seconds?: int)` | Contact link dict | Approve or deny a contact request | | `list_contacts` | `list_contacts(project_key: str, agent_name: str)` | `list[dict]` | List contact links for an agent | | `set_contact_policy` | `set_contact_policy(project_key: str, agent_name: str, policy: str)` | Agent dict | Set policy: `open`, `auto`, `contacts_only`, `block_all` | | `fetch_inbox` | `fetch_inbox(project_key: str, agent_name: str, limit?: int, urgent_only?: bool, include_bodies?: bool, since_ts?: str)` | `list[dict]` | Non-mutating inbox read | | `mark_message_read` | `mark_message_read(project_key: str, agent_name: str, message_id: int)` | `{message_id, read, read_at}` | Per-recipient read receipt | | `acknowledge_message` | `acknowledge_message(project_key: str, agent_name: str, message_id: int)` | `{message_id, acknowledged, acknowledged_at, read_at}` | Sets ack and read | | `macro_start_session` | `macro_start_session(human_key: str, program: str, model: str, task_description?: str, agent_name?: str, file_reservation_paths?: list[str], file_reservation_reason?: str, file_reservation_ttl_seconds?: int, inbox_limit?: int)` | `{project, agent, file_reservations, inbox}` | Orchestrates ensure→register→optional file reservation→inbox fetch | | `macro_prepare_thread` | `macro_prepare_thread(project_key: str, thread_id: str, program: str, model: str, agent_name?: str, task_description?: str, register_if_missing?: bool, include_examples?: bool, inbox_limit?: int, include_inbox_bodies?: bool, llm_mode?: bool, llm_model?: str)` | `{project, agent, thread, inbox}` | Bundles registration, thread summary, and inbox context | | `macro_file_reservation_cycle` | `macro_file_reservation_cycle(project_key: str, agent_name: str, paths: list[str], ttl_seconds?: int, exclusive?: bool, reason?: str, auto_release?: bool)` | `{file_reservations, released}` | File Reservation + optionally release surfaces around a focused edit block | | `macro_contact_handshake` | `macro_contact_handshake(project_key: str, requester|agent_name: str, target|to_agent: str, to_project?: str, reason?: str, ttl_seconds?: int, auto_accept?: bool, welcome_subject?: str, welcome_body?: str)` | `{request, response, welcome_message}` | Automates contact request/approval and optional welcome ping | | `search_messages` | `search_messages(project_key: str, query: str, limit?: int)` | `list[dict]` | FTS5 search (bm25) | | `summarize_thread` | `summarize_thread(project_key: str, thread_id: str, include_examples?: bool, llm_mode?: bool, llm_model?: str, per_thread_limit?: int)` | Single: `{thread_id, summary, examples}` Multi (comma-sep): `{threads[], aggregate}` | Extracts participants, key points, actions. Use comma-separated thread_id for multi-thread digest. | | `install_precommit_guard` | `install_precommit_guard(project_key: str, code_repo_path: str)` | `{hook}` | Install a Git pre-commit guard in a target repo | | `uninstall_precommit_guard` | `uninstall_precommit_guard(code_repo_path: str)` | `{removed}` | Remove the guard from a repo | | `file_reservation_paths` | `file_reservation_paths(project_key: str, agent_name: str, paths: list[str], ttl_seconds?: int, exclusive?: bool, reason?: str)` | `{granted: list, conflicts: list}` | Advisory leases; Git artifact per path | | `release_file_reservations` | `release_file_reservations(project_key: str, agent_name: str, paths?: list[str], file_reservation_ids?: list[int])` | `{released, released_at}` | Releases agent's active file reservations | | `force_release_file_reservation` | `force_release_file_reservation(project_key: str, agent_name: str, file_reservation_id: int, notify_previous?: bool, note?: str)` | `{released, released_at, reservation}` | Clears stale reservations using inactivity/mail/fs/git heuristics and notifies the previous holder | | `renew_file_reservations` | `renew_file_reservations(project_key: str, agent_name: str, extend_seconds?: int, paths?: list[str], file_reservation_ids?: list[int])` | `{renewed, file reservations[]}` | Extend TTL of existing file reservations | ### Resources | URI | Params | Returns | Notes | | :-- | :-- | :-- | :-- | | `resource://config/environment` | — | `{environment, database_url, http}` | Inspect server settings | | `resource://tooling/directory` | — | `{generated_at, metrics_uri, clusters[], playbooks[]}` | Grouped tool directory + workflow playbooks | | `resource://tooling/schemas` | — | `{tools: {: {required[], optional[], aliases{}}}}` | Argument hints for tools | | `resource://tooling/metrics` | — | `{generated_at, tools[]}` | Aggregated call/error counts per tool | | `resource://tooling/locks` | — | `{locks[], summary}` | Active locks and owners (debug only). Categories: `archive` (per-project `.archive.lock`) and `custom` (e.g., repo `.commit.lock`). | | `resource://tooling/capabilities/{agent}{?project}` | listed| `{generated_at, agent, project, capabilities[]}` | Capabilities assigned to the agent (see `deploy/capabilities/agent_capabilities.json`) | | `resource://tooling/recent/{window_seconds}{?agent,project}` | listed | `{generated_at, window_seconds, count, entries[]}` | Recent tool usage filtered by agent/project | | `resource://projects` | — | `list[project]` | All projects | | `resource://project/{slug}` | `slug` | `{project..., agents[]}` | Project detail + agents | | `resource://file_reservations/{slug}{?active_only}` | `slug`, `active_only?` | `list[file reservation]` | File reservations plus staleness metadata (heuristics, last activity timestamps) | | `resource://message/{id}{?project}` | `id`, `project` | `message` | Single message with body | | `resource://thread/{thread_id}{?project,include_bodies}` | `thread_id`, `project`, `include_bodies?` | `{project, thread_id, messages[]}` | Thread listing | | `resource://inbox/{agent}{?project,since_ts,urgent_only,include_bodies,limit}` | listed | `{project, agent, count, messages[]}` | Inbox listing | | `resource://mailbox/{agent}{?project,limit}` | `project`, `limit` | `{project, agent, count, messages[]}` | Mailbox listing (recent messages with basic commit ref) | | `resource://mailbox-with-commits/{agent}{?project,limit}` | `project`, `limit` | `{project, agent, count, messages[]}` | Mailbox listing enriched with commit metadata | | `resource://outbox/{agent}{?project,limit,include_bodies,since_ts}` | listed | `{project, agent, count, messages[]}` | Messages sent by the agent | | `resource://views/acks-stale/{agent}{?project,ttl_seconds,limit}` | listed | `{project, agent, ttl_seconds, count, messages[]}` | Ack-required older than TTL without ack | | `resource://views/urgent-unread/{agent}{?project,limit}` | listed | `{project, agent, count, messages[]}` | High/urgent importance messages not yet read | | `resource://views/ack-required/{agent}{?project,limit}` | listed | `{project, agent, count, messages[]}` | Pending acknowledgements for an agent | | `resource://views/ack-overdue/{agent}{?project,ttl_minutes,limit}` | listed | `{project, agent, ttl_minutes, count, messages[]}` | Ack-required older than TTL without ack | ### Client Integration Guide 1. **Fetch onboarding metadata first.** Issue `resources/read` for `resource://tooling/directory` (and optionally `resource://tooling/metrics`) before exposing tools to an agent. Use the returned clusters and playbooks to render a narrow tool palette for the current workflow rather than dumping every verb into the UI. 2. **Scope tools per workflow.** When the agent enters a new phase (e.g., "Messaging Lifecycle"), remount only the cluster's tools in your MCP client. This mirrors the workflow macros already provided and prevents "tool overload." 3. **Monitor real usage.** Periodically pull or subscribe to log streams containing the `tool_metrics_snapshot` events emitted by the server (or query `resource://tooling/metrics`) so you can detect high-error-rate tools and decide whether to expose macros or extra guidance. 4. **Fallback to macros for smaller models.** If you're routing work to a lightweight model, prefer the macro helpers (`macro_start_session`, `macro_prepare_thread`, `macro_file_reservation_cycle`, `macro_contact_handshake`) and hide the granular verbs until the agent explicitly asks for them. 5. **Show recent actions.** Read `resource://tooling/recent/60?agent=&project=` (adjust window as needed) to display the last few successful tool invocations relevant to the agent/project. See `examples/client_bootstrap.py` for a runnable reference implementation that applies the guidance above. ```json { "steps": [ "resources/read -> resource://tooling/directory", "select active cluster (e.g. messaging)", "mount tools listed in cluster.tools plus macros if model size <= S", "optional: resources/read -> resource://tooling/metrics for dashboard display", "optional: resources/read -> resource://tooling/recent/60?agent=&project= for UI hints" ] } ``` ### Monitoring & Alerts 1. **Enable metric emission.** Set `TOOL_METRICS_EMIT_ENABLED=true` and choose an interval (`TOOL_METRICS_EMIT_INTERVAL_SECONDS=120` is a good starting point). The server will periodically emit a structured log entry such as: ```json { "event": "tool_metrics_snapshot", "tools": [ {"name": "send_message", "cluster": "messaging", "calls": 42, "errors": 1}, {"name": "file_reservation_paths", "cluster": "file reservations", "calls": 11, "errors": 0} ] } ``` 2. **Ship the logs.** Forward the structured stream (stderr/stdout or JSON log files) into your observability stack (e.g., Loki, Datadog, Elastic) and parse the `tools[]` array. 3. **Alert on anomalies.** Create a rule that raises when `errors / calls` exceeds a threshold for any tool (for example 5% over a 5-minute window) so you can decide whether to expose a macro or improve documentation. 4. **Dashboard the clusters.** Group by `cluster` to see where agents are spending time and which workflows might warrant additional macros or guard-rails. See `docs/observability.md` for a step-by-step cookbook (Loki/Prometheus example pipelines included), and `docs/GUIDE_TO_OPTIMAL_MCP_SERVER_DESIGN.md` for a comprehensive design guide covering tool curation, capability gating, security, and observability best practices. Operations teams can follow `docs/operations_alignment_checklist.md`, which links to the capability templates in `deploy/capabilities/` and the sample Prometheus alert rules in `deploy/observability/`. --- ## Deployment quick notes - **Direct uvicorn**: `uvicorn mcp_agent_mail.http:build_http_app --factory --host 0.0.0.0 --port 8765` - **Python module**: `python -m mcp_agent_mail.http --host 0.0.0.0 --port 8765` - **Gunicorn**: `gunicorn -c deploy/gunicorn.conf.py mcp_agent_mail.http:build_http_app --factory` - **Docker**: `docker compose up --build` ### CI/CD - Lint and Typecheck CI: GitHub Actions workflow runs Ruff and Ty on pushes/PRs to main/develop. - Release: Pushing a tag like `v0.1.0` builds and pushes a multi-arch Docker image to GHCR under `ghcr.io//` with `latest` and version tags. - Nightly: A scheduled workflow runs migrations and lists projects daily for lightweight maintenance visibility. ### Log rotation (optional) If not using journald, a sample logrotate config is provided at `deploy/logrotate/mcp-agent-mail` to rotate `/var/log/mcp-agent-mail/*.log` weekly, keeping 7 rotations. ### Logging (journald vs file) - Default systemd unit (`deploy/systemd/mcp-agent-mail.service`) is configured to send logs to journald (StandardOutput/StandardError=journal). - For file logging, configure your process manager to write to files under `/var/log/mcp-agent-mail/*.log` and install the provided logrotate config. - Environment file path for systemd is `/etc/mcp-agent-mail.env` (see `deploy/systemd/mcp-agent-mail.service`). ### Container build and multi-arch push Use Docker Buildx for multi-arch images. Example flow: ```bash # Create and select a builder (once) docker buildx create --use --name mcp-builder || docker buildx use mcp-builder # Build and test locally (linux/amd64) docker buildx build --load -t your-registry/mcp-agent-mail:dev . # Multi-arch build and push (amd64, arm64) docker buildx build \ --platform linux/amd64,linux/arm64 \ -t your-registry/mcp-agent-mail:latest \ -t your-registry/mcp-agent-mail:v0.1.0 \ --push . ``` Recommended tags: a moving `latest` and immutable version tags per release. Ensure your registry login is configured (`docker login`). ### Systemd manual deployment steps 1. Copy project files to `/opt/mcp-agent-mail` and ensure permissions (owner `appuser`). 2. Place environment file at `/etc/mcp-agent-mail.env` based on `deploy/env/production.env`. 3. Install service file `deploy/systemd/mcp-agent-mail.service` to `/etc/systemd/system/`. 4. Reload systemd and start: ```bash sudo systemctl daemon-reload sudo systemctl enable mcp-agent-mail sudo systemctl start mcp-agent-mail sudo systemctl status mcp-agent-mail ``` Optional (non-journald log rotation): install `deploy/logrotate/mcp-agent-mail` into `/etc/logrotate.d/` and write logs to `/var/log/mcp-agent-mail/*.log` via your process manager or app config. See `deploy/gunicorn.conf.py` for a starter configuration. For project direction and planned areas, read `project_idea_and_guide.md`. ## CLI Commands The project exposes a developer CLI for common operations: - `serve-http`: run the HTTP transport (Streamable HTTP only) - `migrate`: ensure schema and FTS structures exist - `lint` / `typecheck`: developer helpers - `list-projects [--include-agents]`: enumerate projects - `guard install `: install the pre-commit guard into a repo - `guard uninstall `: remove the guard from a repo - `share wizard`: launch interactive deployment wizard (auto-installs CLIs, authenticates, exports, deploys to GitHub Pages or Cloudflare Pages) - `share export --output [options]`: export mailbox to a static HTML bundle (see Static Mailbox Export section for full options) - `share update [options]`: refresh an existing bundle using recorded (or overridden) export settings - `share preview [--port N] [--open-browser]`: serve a static bundle locally for inspection - `share verify [--public-key ]`: verify bundle integrity (SRI hashes and Ed25519 signature) - `share decrypt [--identity | --passphrase]`: decrypt an age-encrypted bundle - `config set-port `: change the HTTP server port (updates .env) - `config show-port`: display the current configured HTTP port - `clear-and-reset-everything [--force] [--archive/--no-archive]`: DELETE the SQLite database (incl. WAL/SHM) and WIPE all contents under `STORAGE_ROOT` after optionally saving a restore point. Without flags it prompts to create an archive first; `--force --no-archive` skips all prompts for automation. - `list-acks --project --agent [--limit N]`: list messages requiring acknowledgement for an agent where ack is missing - `acks pending [--limit N]`: show pending acknowledgements for an agent - `acks remind [--min-age-minutes N] [--limit N]`: highlight pending ACKs older than a threshold - `acks overdue [--ttl-minutes N] [--limit N]`: list overdue ACKs beyond TTL - `file_reservations list [--active-only/--no-active-only]`: list file reservations - `file_reservations active [--limit N]`: list active file reservations - `file_reservations soon [--minutes N]`: show file reservations expiring soon - `doctor check [PROJECT] [--verbose] [--json]`: run comprehensive diagnostics on mailbox health - `doctor repair [PROJECT] [--dry-run] [--yes] [--backup-dir PATH]`: semi-automatic repair with backup before changes - `doctor backups [--json]`: list available diagnostic backups - `doctor restore [--dry-run] [--yes]`: restore from a diagnostic backup Examples: ```bash # Interactive wizard: export + deploy to GitHub Pages (easiest) ./scripts/share_to_github_pages.py # Export a static bundle with signing and encryption uv run python -m mcp_agent_mail.cli share export \ --output ./bundle \ --signing-key ./keys/signing.key \ --age-recipient age1abc...xyz # Preview a bundle locally uv run python -m mcp_agent_mail.cli share preview ./bundle --port 9000 --open-browser # Verify bundle integrity uv run python -m mcp_agent_mail.cli share verify ./bundle # Refresh an existing bundle in place with recorded settings uv run python -m mcp_agent_mail.cli share update ./bundle # Change server port uv run python -m mcp_agent_mail.cli config set-port 9000 # Install guard into a repo uv run python -m mcp_agent_mail.cli guard install /abs/path/backend /abs/path/backend # List pending acknowledgements for an agent uv run python -m mcp_agent_mail.cli acks pending /abs/path/backend BlueLake --limit 10 # Run mailbox health diagnostics uv run python -m mcp_agent_mail.cli doctor check # Preview repairs without making changes uv run python -m mcp_agent_mail.cli doctor repair --dry-run # Run repairs (creates backup first, prompts for data changes) uv run python -m mcp_agent_mail.cli doctor repair # List available backups uv run python -m mcp_agent_mail.cli doctor backups # Restore from a backup uv run python -m mcp_agent_mail.cli doctor restore /path/to/backup --dry-run # WARNING: Destructive reset (clean slate) uv run python -m mcp_agent_mail.cli clear-and-reset-everything --force ``` ## Client integrations Use the automated installer to wire up supported tools automatically (e.g., Claude Code, Cline, Windsurf, OpenCode). Run `scripts/automatically_detect_all_installed_coding_agents_and_install_mcp_agent_mail_in_all.sh` or the one-liner in the Quickstart above. ### Tool-specific integration scripts For manual integration or customization, dedicated scripts are available: | Tool | Script | What it configures | |------|--------|-------------------| | Claude Code | `scripts/integrate_claude_code.sh` | `.claude/settings.json`, hooks, MCP server | | Codex CLI | `scripts/integrate_codex_cli.sh` | `~/.codex/config.toml`, MCP server, notify handler | | Gemini CLI | `scripts/integrate_gemini_cli.sh` | `~/.gemini/settings.json`, MCP server, hooks | Each script: - Detects the MCP server endpoint from your settings - Generates or reuses a bearer token for authentication - Configures the MCP server connection - Installs hooks/notify handlers for inbox reminders - Bootstraps your project and agent identity on the server ### Automatic inbox reminders Agents often get absorbed in their work and forget to check their mail. The integration scripts install lightweight hooks that periodically remind agents when they have unread messages. **How it works:** - A rate-limited hook script (`scripts/hooks/check_inbox.sh`) runs after certain tool invocations - It checks the inbox via a fast curl call (avoids Python import overhead) - If there are unread messages, it outputs a brief reminder - Rate limited to at most once per 2 minutes to avoid noise **Claude Code / Gemini CLI:** The hook is configured as a `PostToolUse` hook that fires after `Bash` or `shell` tool invocations: ```json { "hooks": { "PostToolUse": [ { "matcher": "Bash", "hooks": [{ "type": "command", "command": "...check_inbox.sh" }] } ] } } ```

MCP Agent Mail

Installation