Anti-Pattern 1: Using Generic Sentiment Analysis
Novice thinking: "Negative sentiment = crisis"
Problem: Mental health language is nuanced, context-dependent.
Wrong approach:
```typescript
// β Generic sentiment misses mental health signals
const sentiment = analyzeSentiment(text);
if (sentiment.score < -0.5) {
alertCrisis(); // Too broad!
}
```
Why wrong: "I'm tired" vs "I'm tired of living" - different meanings, same sentiment.
Correct approach:
```typescript
// β
Mental health-specific model
import { pipeline } from '@huggingface/transformers';
const detector = await pipeline('text-classification', 'mental/bert-base-uncased');
const result = await detector(text, {
labels: ['suicidal_ideation', 'self_harm', 'substance_relapse', 'safe']
});
if (result[0].label === 'suicidal_ideation' && result[0].score > 0.8) {
await escalateToCrisisCounselor({
text,
confidence: result[0].score,
timestamp: Date.now()
});
// IMMEDIATELY show crisis resources
showCrisisResources({
phone: '988',
text: 'Text "HELLO" to 741741',
chat: 'https://988lifeline.org/chat'
});
}
```
Timeline context:
- 2015: Rule-based keyword matching
- 2020: BERT fine-tuning for mental health
- 2024: Multi-label models with context understanding
---
Anti-Pattern 2: Automated Responses Without Human Review
Problem: AI cannot replace empathy, may escalate distress.
Wrong approach:
```typescript
// β AI auto-responds to crisis
if (isCrisis(text)) {
await sendMessage(userId, "I'm concerned about you. Are you okay?");
}
```
Why wrong:
- Feels robotic, invalidating
- May increase distress
- No human judgment
Correct approach:
```typescript
// β
Flag for human review, show resources
if (isCrisis(text)) {
// 1. Flag for counselor review
await flagForReview({
userId,
text,
severity: 'high',
detectedAt: Date.now(),
requiresImmediate: true
});
// 2. Notify on-call counselor
await notifyOnCallCounselor({
userId,
summary: 'Suicidal ideation detected',
urgency: 'immediate'
});
// 3. Show resources (no AI message)
await showInAppResources({
type: 'crisis_support',
resources: [
{ name: '988 Suicide & Crisis Lifeline', link: 'tel:988' },
{ name: 'Crisis Text Line', link: 'sms:741741' },
{ name: 'Chat Now', link: 'https://988lifeline.org/chat' }
]
});
// 4. DO NOT send automated "are you okay" message
}
```
Human review flow:
```
AI Detection β Flag β On-call counselor notified β Human reaches out
```
---
Anti-Pattern 3: Not Providing Immediate Resources
Problem: User in crisis needs help NOW, not later.
Wrong approach:
```typescript
// β Just flags, no immediate help
if (isCrisis(text)) {
await logCrisisEvent(userId, text);
// User left with no resources
}
```
Correct approach:
```typescript
// β
Immediate resources + escalation
if (isCrisis(text)) {
// Show resources IMMEDIATELY (blocking modal)
await showCrisisModal({
title: 'Resources Available',
resources: [
{
name: '988 Suicide & Crisis Lifeline',
description: 'Free, confidential support 24/7',
action: 'tel:988',
type: 'phone'
},
{
name: 'Crisis Text Line',
description: 'Text support with trained counselor',
action: 'sms:741741',
message: 'HELLO',
type: 'text'
},
{
name: 'Chat with counselor',
description: 'Online chat support',
action: 'https://988lifeline.org/chat',
type: 'web'
}
],
dismissible: true, // User can close, but resources shown first
analytics: { event: 'crisis_resources_shown', source: 'ai_detection' }
});
// Then flag for follow-up
await flagForReview({ userId, text, severity: 'high' });
}
```
---
Anti-Pattern 4: Storing Crisis Data Insecurely
Problem: Crisis content is extremely sensitive PHI.
Wrong approach:
```typescript
// β Plain text storage
await db.logs.insert({
userId: user.id,
type: 'crisis',
content: text, // Stored in plain text!
timestamp: Date.now()
});
```
Why wrong: Data breach exposes most vulnerable moments.
Correct approach:
```typescript
// β
Encrypted, access-logged, auto-deleted
import { encrypt, decrypt } from './encryption';
await db.crisisEvents.insert({
id: generateId(),
userId: hashUserId(user.id), // Hash, not plain ID
contentHash: hashContent(text), // For deduplication only
encryptedContent: encrypt(text, process.env.CRISIS_DATA_KEY),
detectedAt: Date.now(),
reviewedAt: null,
reviewedBy: null,
autoDeleteAt: Date.now() + (30 24 60 60 1000), // 30 days
accessLog: []
});
// Log all access
await logAccess({
eventId: crisisEvent.id,
accessedBy: counselorId,
accessedAt: Date.now(),
reason: 'Review for follow-up',
ipAddress: hashedIp
});
// Auto-delete after retention period
schedule.daily(() => {
db.crisisEvents.deleteMany({
autoDeleteAt: { $lt: Date.now() }
});
});
```
HIPAA Requirements:
- Encryption at rest and in transit
- Access logging
- Auto-deletion after retention period
- Minimum necessary access
---
Anti-Pattern 5: No Escalation Protocol
Problem: No clear path from detection to human intervention.
Wrong approach:
```typescript
// β Flags crisis but no escalation process
if (isCrisis(text)) {
await db.flags.insert({ userId, text, flaggedAt: Date.now() });
// Now what? Who responds?
}
```
Correct approach:
```typescript
// β
Clear escalation protocol
enum CrisisSeverity {
LOW = 'low', // Distress, no immediate danger
MEDIUM = 'medium', // Self-harm thoughts, no plan
HIGH = 'high', // Suicidal ideation with plan
IMMEDIATE = 'immediate' // Imminent danger
}
async function escalateCrisis(detection: CrisisDetection): Promise {
const severity = assessSeverity(detection);
switch (severity) {
case CrisisSeverity.IMMEDIATE:
// Notify on-call counselor (push notification)
await notifyOnCall({
userId: detection.userId,
severity,
requiresResponse: 'immediate',
text: detection.text
});
// Send SMS to backup on-call if no response in 5 min
setTimeout(async () => {
if (!await hasResponded(detection.id)) {
await notifyBackupOnCall(detection);
}
}, 5 60 1000);
// Show 988 modal (blocking)
await show988Modal(detection.userId);
break;
case CrisisSeverity.HIGH:
// Notify on-call counselor (email + push)
await notifyOnCall({ severity, requiresResponse: '1 hour' });
// Show crisis resources
await showCrisisResources(detection.userId);
break;
case CrisisSeverity.MEDIUM:
// Add to review queue for next business day
await addToReviewQueue({ priority: 'high' });
// Suggest self-help resources
await suggestResources(detection.userId, 'coping_strategies');
break;
case CrisisSeverity.LOW:
// Add to review queue
await addToReviewQueue({ priority: 'normal' });
break;
}
// Always log for audit
await logEscalation({
detectionId: detection.id,
severity,
actions: ['notified_on_call', 'showed_resources'],
timestamp: Date.now()
});
}
```
---
|