quality-metrics

When to Use

• Building quality dashboards
• Defining quality gates
• Evaluating testing effectiveness
• Justifying quality investments

Meaningful vs Vanity Metrics

| ✅ Meaningful | ❌ Vanity |

|--------------|-----------|

| Bug escape rate | Test case count |

| MTTD (detection) | Lines of test code |

| MTTR (recovery) | Test executions |

| Change failure rate | Coverage % (alone) |

| Lead time for changes | Requirements traced |

DORA Metrics

| Metric | Elite | High | Medium | Low |

|--------|-------|------|--------|-----|

| Deploy Frequency | On-demand | Weekly | Monthly | Yearly |

| Lead Time | < 1 hour | < 1 week | < 1 month | > 6 months |

| Change Failure Rate | < 5% | < 15% | < 30% | > 45% |

| MTTR | < 1 hour | < 1 day | < 1 week | > 1 month |

Quality Gate Thresholds

| Metric | Blocking Threshold | Warning |

|--------|-------------------|---------|

| Test pass rate | 100% | - |

| Critical coverage | > 80% | > 70% |

| Security critical | 0 | - |

| Performance p95 | < 200ms | < 500ms |

| Flaky tests | < 2% | < 5% |

---

Bug Escape Rate

```

Bug Escape Rate = (Production Bugs / Total Bugs Found) × 100

Target: < 10% (90% caught before production)

```

Test Effectiveness

```

Test Effectiveness = (Bugs Found by Tests / Total Bugs) × 100

Target: > 70%

```

Defect Density

```

Defect Density = Defects / KLOC

Good: < 1 defect per KLOC

```

Mean Time to Detect (MTTD)

```

MTTD = Time(Bug Reported) - Time(Bug Introduced)

Target: < 1 day for critical, < 1 week for others

```

---

```typescript

// Agent generates quality dashboard

await Task("Generate Dashboard", {

metrics: {

delivery: ['deployment-frequency', 'lead-time', 'change-failure-rate'],

quality: ['bug-escape-rate', 'test-effectiveness', 'defect-density'],

stability: ['mttd', 'mttr', 'availability'],

process: ['code-review-time', 'flaky-test-rate', 'coverage-trend']

},

visualization: 'grafana',

alerts: {

critical: { bug_escape_rate: '>20%', mttr: '>24h' },

warning: { coverage: '<70%', flaky_rate: '>5%' }

}

}, "qe-quality-analyzer");

```

---

```json

{

"qualityGates": {

"commit": {

"coverage": { "min": 80, "blocking": true },

"lint": { "errors": 0, "blocking": true }

},

"pr": {

"tests": { "pass": "100%", "blocking": true },

"security": { "critical": 0, "blocking": true },

"coverage_delta": { "min": 0, "blocking": false }

},

"release": {

"e2e": { "pass": "100%", "blocking": true },

"performance_p95": { "max_ms": 200, "blocking": true },

"bug_escape_rate": { "max": "10%", "blocking": false }

}

}

}

```

---

```typescript

// Calculate quality trends

await Task("Quality Trend Analysis", {

timeframe: '90d',

metrics: ['bug-escape-rate', 'mttd', 'test-effectiveness'],

compare: 'previous-90d',

predictNext: '30d'

}, "qe-quality-analyzer");

// Evaluate quality gate

await Task("Quality Gate Evaluation", {

buildId: 'build-123',

environment: 'staging',

metrics: currentMetrics,

policy: qualityPolicy

}, "qe-quality-gate");

```

---

Memory Namespace

```

aqe/quality-metrics/

├── dashboards/* - Dashboard configurations

├── trends/* - Historical metric data

├── gates/* - Gate evaluation results

└── alerts/* - Triggered alerts

```

Fleet Coordination

```typescript

const metricsFleet = await FleetManager.coordinate({

strategy: 'quality-metrics',

agents: [

'qe-quality-analyzer', // Trend analysis

'qe-test-executor', // Test metrics

'qe-coverage-analyzer', // Coverage data

'qe-production-intelligence', // Production metrics

'qe-quality-gate' // Gate decisions

],

topology: 'mesh'

});

```

---

| Trap | Problem | Solution |

|------|---------|----------|

| Coverage worship | 100% coverage, bugs still escape | Measure bug escape rate instead |

| Test count focus | Many tests, slow feedback | Measure execution time |

| Activity metrics | Busy work, no outcomes | Measure outcomes (MTTD, MTTR) |

| Point-in-time | Snapshot without context | Track trends over time |

---

• [agentic-quality-engineering](../agentic-quality-engineering/) - Agent coordination
• [cicd-pipeline-qe-orchestrator](../cicd-pipeline-qe-orchestrator/) - Quality gates
• [risk-based-testing](../risk-based-testing/) - Risk-informed metrics
• [shift-right-testing](../shift-right-testing/) - Production metrics

---

Measure outcomes, not activities. Bug escape rate > test count. MTTD/MTTR > coverage %. Trends > snapshots. Set gates that block bad code. What you measure is what you optimize.

With Agents: Agents track metrics automatically, analyze trends, trigger alerts, and make gate decisions. Use agents to maintain continuous quality visibility.