compliance-automation

🏗️ Data Engineering Plugin

• 15 Commands: Complete pipeline lifecycle from planning to deployment
• 18 Specialized Agents: Expert code review and optimization
• 8 Skills: Reusable architecture patterns and templates
• 3 MCP Servers: Deep Databricks integration

🤖 MLOps Plugin (Optional)

• Model training and deployment automation
• Feature store management
• MLflow experiment tracking
• Model monitoring and drift detection

🔒 Governance Plugin (Optional)

• Unity Catalog access control
• Compliance checking and reporting
• Data lineage tracking
• Audit log analysis

Installation

```bash

# Recommended: Install via npx

npx claude-plugins install @vivekgana/databricks-platform-marketplace/databricks-engineering

# Or add marketplace in Claude

/plugin marketplace add https://github.com/yourcompany/databricks-platform-marketplace

/plugin install databricks-engineering

```

Prerequisites

```bash

# Set up Databricks credentials

export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"

export DATABRICKS_TOKEN="your-token-here"

# Optional: Configure specific resources

export DATABRICKS_WAREHOUSE_ID="your-warehouse-id"

export DATABRICKS_CLUSTER_ID="your-cluster-id"

```

Your First Pipeline

```bash

# 1. Plan a new data pipeline

claude /databricks:plan-pipeline "Build customer 360 with real-time updates"

# 2. Implement the pipeline

claude /databricks:work-pipeline plans/customer-360.md

# 3. Review before merging

claude /databricks:review-pipeline https://github.com/your-org/repo/pull/42

# 4. Deploy to production

claude /databricks:deploy-bundle --environment prod

```

Commands

| Command | Description | Category |

|---------|-------------|----------|

| plan-pipeline | Plan data pipeline with architecture and costs | Planning |

| work-pipeline | Execute implementation systematically | Development |

| review-pipeline | Multi-agent code review | Quality |

| create-data-product | Design data products with SLAs | Data Products |

| configure-delta-share | Set up external data sharing | Sharing |

| deploy-bundle | Deploy with Asset Bundles | Deployment |

| optimize-costs | Analyze and reduce costs | Optimization |

| test-data-quality | Generate quality tests | Testing |

| monitor-data-product | Set up monitoring | Observability |

[See all 15 commands →](docs/commands-reference.md)

Specialized Agents

• PySpark Optimizer: Performance tuning and best practices
• Delta Lake Expert: Storage optimization and time travel
• Data Quality Sentinel: Validation and monitoring
• Unity Catalog Expert: Governance and permissions
• Cost Analyzer: Compute and storage optimization
• Delta Sharing Expert: External data distribution
• Data Product Architect: Product design and SLAs
• Pipeline Architect: Medallion architecture patterns

[See all 18 agents →](docs/agents-reference.md)

Skills & Templates

• Medallion Architecture: Bronze/Silver/Gold patterns
• Delta Live Tables: Streaming pipeline tem