domain-ml

| Domain Rule | Design Constraint | Rust Implication |

|-------------|-------------------|------------------|

| Large data | Efficient memory | Zero-copy, streaming |

| GPU acceleration | CUDA/Metal support | candle, tch-rs |

| Model portability | Standard formats | ONNX |

| Batch processing | Throughput over latency | Batched inference |

| Numerical precision | Float handling | ndarray, careful f32/f64 |

| Reproducibility | Deterministic | Seeded random, versioning |

---

Memory Efficiency

```

RULE: Avoid copying large tensors

WHY: Memory bandwidth is bottleneck

RUST: References, views, in-place ops

```

GPU Utilization

```

RULE: Batch operations for GPU efficiency

WHY: GPU overhead per kernel launch

RUST: Batch sizes, async data loading

```

Model Portability

```

RULE: Use standard model formats

WHY: Train in Python, deploy in Rust

RUST: ONNX via tract or candle

```

---

From constraints to design (Layer 2):

```

"Need efficient data pipelines"

↓ m10-performance: Streaming, batching

↓ polars: Lazy evaluation

"Need GPU inference"

↓ m07-concurrency: Async data loading

↓ candle/tch-rs: CUDA backend

"Need model loading"

↓ m12-lifecycle: Lazy init, caching

↓ tract: ONNX runtime

```

---

| Use Case | Recommended | Why |

|----------|-------------|-----|

| Inference only | tract (ONNX) | Lightweight, portable |

| Training + inference | candle, burn | Pure Rust, GPU |

| PyTorch models | tch-rs | Direct bindings |

| Data pipelines | polars | Fast, lazy eval |

| Purpose | Crate |

|---------|-------|

| Tensors | ndarray |

| ONNX inference | tract |

| ML framework | candle, burn |

| PyTorch bindings | tch-rs |

| Data processing | polars |

| Embeddings | fastembed |

| Pattern | Purpose | Implementation |

|---------|---------|----------------|

| Model loading | Once, reuse | OnceLock |

| Batching | Throughput | Collect then process |

| Streaming | Large data | Iterator-based |

| GPU async | Parallelism | Data loading parallel to compute |

```rust

use std::sync::OnceLock;

use tract_onnx::prelude::*;

static MODEL: OnceLock, Graph>>> = OnceLock::new();

fn get_model() -> &'static SimplePlan<...> {

MODEL.get_or_init(|| {

tract_onnx::onnx()

.model_for_path("model.onnx")

.unwrap()

.into_optimized()

.unwrap()

.into_runnable()

.unwrap()

})

}

async fn predict(input: Vec) -> anyhow::Result> {

let model = get_model();

let input = tract_ndarray::arr1(&input).into_shape((1, input.len()))?;

let result = model.run(tvec!(input.into()))?;

Ok(result[0].to_array_view::()?.iter().copied().collect())

}

```

```rust

async fn batch_predict(inputs: Vec>, batch_size: usize) -> Vec> {

let mut results = Vec::with_capacity(inputs.len());

for batch in inputs.chunks(batch_size) {

// Stack inputs into batch tensor

let batch_tensor = stack_inputs(batch);

// Run inference on batch

let batch_output = model.run(batch_tensor).await;

// Unstack results

results.extend(unstack_outputs(batch_output));

}

results

}

```

---

| Mistake | Domain Violation | Fix |

|---------|-----------------|-----|

| Clone tensors | Memory waste | Use views |

| Single inference | GPU underutilized | Batch processing |

| Load model per request | Slow | Singleton pattern |

| Sync data loading | GPU idle | Async pipeline |

---

| Constraint | Layer 2 Pattern | Layer 1 Implementation |

|------------|-----------------|------------------------|

| Memory efficiency | Zero-copy | ndarray views |

| Model singleton | Lazy init | OnceLock |

| Batch processing | Chunked iteration | chunks() + parallel |

| GPU async | Concurrent loading | tokio::spawn + GPU |

---

| When | See |

|------|-----|

| Performance | m10-performance |

| Lazy initialization | m12-lifecycle |

| Async patterns | m07-concurrency |

| Memory efficiency | m01-ownership |