scikit-learn

This skill provides comprehensive guidance for machine learning tasks using scikit-learn, the industry-standard Python library for classical machine learning. Use this skill for classification, regression, clustering, dimensionality reduction, preprocessing, model evaluation, and building production-ready ML pipelines.

```bash

# Install scikit-learn using uv

uv uv pip install scikit-learn

# Optional: Install visualization dependencies

uv uv pip install matplotlib seaborn

# Commonly used with

uv uv pip install pandas numpy

```

Use the scikit-learn skill when:

• Building classification or regression models
• Performing clustering or dimensionality reduction
• Preprocessing and transforming data for machine learning
• Evaluating model performance with cross-validation
• Tuning hyperparameters with grid or random search
• Creating ML pipelines for production workflows
• Comparing different algorithms for a task
• Working with both structured (tabular) and text data
• Need interpretable, classical machine learning approaches

Classification Example

```python

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import StandardScaler

from sklearn.ensemble import RandomForestClassifier

from sklearn.metrics import classification_report

# Split data

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.2, stratify=y, random_state=42

)

# Preprocess

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

X_test_scaled = scaler.transform(X_test)

# Train model

model = RandomForestClassifier(n_estimators=100, random_state=42)

model.fit(X_train_scaled, y_train)

# Evaluate

y_pred = model.predict(X_test_scaled)

print(classification_report(y_test, y_pred))

```

Complete Pipeline with Mixed Data

```python

from sklearn.pipeline import Pipeline

from sklearn.compose import ColumnTransformer

from sklearn.preprocessing import StandardScaler, OneHotEncoder

from sklearn.impute import SimpleImputer

from sklearn.ensemble import GradientBoostingClassifier

# Define feature types

numeric_features = ['age', 'income']

categorical_features = ['gender', 'occupation']

# Create preprocessing pipelines

numeric_transformer = Pipeline([

('imputer', SimpleImputer(strategy='median')),

('scaler', StandardScaler())

])

categorical_transformer = Pipeline([

('imputer', SimpleImputer(strategy='most_frequent')),

('onehot', OneHotEncoder(handle_unknown='ignore'))

])

# Combine transformers

preprocessor = ColumnTransformer([

('num', numeric_transformer, numeric_features),

('cat', categorical_transformer, categorical_features)

])

# Full pipeline

model = Pipeline([

('preprocessor', preprocessor),

('classifier', GradientBoostingClassifier(random_state=42))

])

# Fit and predict

model.fit(X_train, y_train)

y_pred = model.predict(X_test)

```

1. Supervised Learning

Comprehensive algorithms for classification and regression tasks.

Key algorithms:

• Linear models: Logistic Regression, Linear Regression, Ridge, Lasso, ElasticNet
• Tree-based: Decision Trees, Random Forest, Gradient Boosting
• Support Vector Machines: SVC, SVR with various kernels
• Ensemble methods: AdaBoost, Voting, Stacking
• Neural Networks: MLPClassifier, MLPRegressor
• Others: Naive Bayes, K-Nearest Neighbors

When to use:

• Classification: Predicting discrete categories (spam detection, image classification, fraud detection)
• Regression: Predicting continuous values (price prediction, demand forecasting)

See: references/supervised_learning.md for detailed algorithm documentation, parameters, and usage examples.

2. Unsupervised Learning

Discover patterns in unlabeled data through clustering and dimensionality reduction.

Clustering algorithms:

• Partition-based: K-Means, MiniBatchKMeans
• Density-based: DBSCAN, HDBSCAN, OPTICS
• Hierarchical: AgglomerativeClustering
• Probabilistic: Gaussian Mixture Models
• Others: MeanShift, SpectralClustering, BIRCH

Dimensionality reduction:

• Linear: PCA, TruncatedSVD, NMF
• Manifold learning: t-SNE, UMAP, Isomap, LLE
• Feature extraction: FastICA, LatentDirichletAllocation

When to use:

• Customer segmentation, anomaly detection, data visualization
• Reducing feature dimensions, exploratory data analysis
• Topic modeling, image compression

See: references/unsupervised_learning.md for detailed documentation.

3. Model Evaluation and Selection

Tools for robust model evaluation, cross-validation, and hyperparameter tuning.

Cross-validation strategies:

• KFold, StratifiedKFold (classification)
• TimeSeriesSplit (temporal data)
• GroupKFold (grouped samples)

Hyperparameter tuning:

• GridSearchCV (exhaustive search)
• RandomizedSearchCV (random sampling)
• HalvingGridSearchCV (successive halving)

Metrics:

• Classification: accuracy, precision, recall, F1-score, ROC AUC, confusion matrix
• Regression: MSE, RMSE, MAE, R², MAPE
• Clustering: silhouette score, Calinski-Harabasz, Davies-Bouldin

When to use:

• Comparing model performance objectively
• Finding optimal hyperparameters
• Preventing overfitting through cross-validation
• Understanding model behavior with learning curves

See: references/model_evaluation.md for comprehensive metrics and tuning strategies.

4. Data Preprocessing

Transform raw data into formats suitable for machine learning.

Scaling and normalization:

• StandardScaler (zero mean, unit variance)
• MinMaxScaler (bounded range)
• RobustScaler (robust to outliers)
• Normalizer (sample-wise normalization)

Encoding categorical variables:

• OneHotEncoder (nominal categories)
• OrdinalEncoder (ordered categories)
• LabelEncoder (target encoding)

Handling missing values:

• SimpleImputer (mean, median, most frequent)
• KNNImputer (k-nearest neighbors)
• IterativeImputer (multivariate imputation)

Feature engineering:

• PolynomialFeatures (interaction terms)
• KBinsDiscretizer (binning)
• Feature selection (RFE, SelectKBest, SelectFromModel)

When to use:

• Before training any algorithm that requires scaled features (SVM, KNN, Neural Networks)
• Converting categorical variables to numeric format
• Handling missing data systematically
• Creating non-linear features for linear models

See: references/preprocessing.md for detailed preprocessing techniques.

5. Pipelines and Composition

Build reproducible, production-ready ML workflows.

Key components:

• Pipeline: Chain transformers and estimators sequentially
• ColumnTransformer: Apply different preprocessing to different columns
• FeatureUnion: Combine multiple transformers in parallel
• TransformedTargetRegressor: Transform target variable

Benefits:

• Prevents data leakage in cross-validation
• Simplifies code and improves maintainability
• Enables joint hyperparameter tuning
• Ensures consistency between training and prediction

When to use:

• Always use Pipelines for production workflows
• When mixing numerical and categorical features (use ColumnTransformer)
• When performing cross-validation with preprocessing steps
• When hyperparameter tuning includes preprocessing parameters

See: references/pipelines_and_composition.md for comprehensive pipeline patterns.

Classification Pipeline

Run a complete classification workflow with preprocessing, model comparison, hyperparameter tuning, and evaluation:

```bash

python scripts/classification_pipeline.py

```

This script demonstrates:

• Handling mixed data types (numeric and categorical)
• Model comparison using cross-validation
• Hyperparameter tuning with GridSearchCV
• Comprehensive evaluation with multiple metrics
• Feature importance analysis

Clustering Analysis

Perform clustering analysis with algorithm comparison and visualization:

```bash

python scripts/clustering_analysis.py

```

This script demonstrates:

• Finding optimal number of clusters (elbow method, silhouette analysis)
• Comparing multiple clustering algorithms (K-Means, DBSCAN, Agglomerative, Gaussian Mixture)
• Evaluating clustering quality without ground truth
• Visualizing results with PCA projection

This skill includes comprehensive reference files for deep dives into specific topics:

Quick Reference

File: references/quick_reference.md

• Common import patterns and installation instructions
• Quick workflow templates for common tasks
• Algorithm selection cheat sheets
• Common patterns and gotchas
• Performance optimization tips

Supervised Learning

File: references/supervised_learning.md

• Linear models (regression and classification)
• Support Vector Machines
• Decision Trees and ensemble methods
• K-Nearest Neighbors, Naive Bayes, Neural Networks
• Algorithm selection guide

Unsupervised Learning

File: references/unsupervised_learning.md

• All clustering algorithms with parameters and use cases
• Dimensionality reduction techniques
• Outlier and novelty detection
• Gaussian Mixture Models
• Method selection guide

Model Evaluation

File: references/model_evaluation.md

• Cross-validation strategies
• Hyperparameter tuning methods
• Classification, regression, and clustering metrics
• Learning and validation curves
• Best practices for model selection

Preprocessing

File: references/preprocessing.md

• Feature scaling and normalization
• Encoding categorical variables
• Missing value imputation
• Feature engineering techniques
• Custom transformers

Pipelines and Composition

File: references/pipelines_and_composition.md

• Pipeline construction and usage
• ColumnTransformer for mixed data types
• FeatureUnion for parallel transformations
• Complete end-to-end examples
• Best practices

Building a Classification Model

1. Load and explore data

```python

import pandas as pd

df = pd.read_csv('data.csv')

X = df.drop('target', axis=1)

y = df['target']

```

1. Split data with stratification

```python

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.2, stratify=y, random_state=42

)

```

1. Create preprocessing pipeline

```python

from sklearn.pipeline import Pipeline

from sklearn.preprocessing import StandardScaler

from sklearn.compose import ColumnTransformer

# Handle numeric and categorical features separately

preprocessor = ColumnTransformer([

('num', StandardScaler(), numeric_features),

('cat', OneHotEncoder(), categorical_features)

])

```

1. Build complete pipeline

```python

model = Pipeline([

('preprocessor', preprocessor),

('classifier', RandomForestClassifier(random_state=42))

])

```

1. Tune hyperparameters

```python

from sklearn.model_selection import GridSearchCV

param_grid = {

'classifier__n_estimators': [100, 200],

'classifier__max_depth': [10, 20, None]

}

grid_search = GridSearchCV(model, param_grid, cv=5)

grid_search.fit(X_train, y_train)

```

1. Evaluate on test set

```python

from sklearn.metrics import classification_report

best_model = grid_search.best_estimator_

y_pred = best_model.predict(X_test)

print(classification_report(y_test, y_pred))

```

Performing Clustering Analysis

1. Preprocess data

```python

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)

```

1. Find optimal number of clusters

```python

from sklearn.cluster import KMeans

from sklearn.metrics import silhouette_score

scores = []

for k in range(2, 11):

kmeans = KMeans(n_clusters=k, random_state=42)

labels = kmeans.fit_predict(X_scaled)

scores.append(silhouette_score(X_scaled, labels))

optimal_k = range(2, 11)[np.argmax(scores)]

```

1. Apply clustering

```python

model = KMeans(n_clusters=optimal_k, random_state=42)

labels = model.fit_predict(X_scaled)

```

1. Visualize with dimensionality reduction

```python

from sklearn.decomposition import PCA

pca = PCA(n_components=2)

X_2d = pca.fit_transform(X_scaled)

plt.scatter(X_2d[:, 0], X_2d[:, 1], c=labels, cmap='viridis')

```

Always Use Pipelines

Pipelines prevent data leakage and ensure consistency:

```python

# Good: Preprocessing in pipeline

pipeline = Pipeline([

('scaler', StandardScaler()),

('model', LogisticRegression())

])

# Bad: Preprocessing outside (can leak information)

X_scaled = StandardScaler().fit_transform(X)

```

Fit on Training Data Only

Never fit on test data:

```python

# Good

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

X_test_scaled = scaler.transform(X_test) # Only transform

# Bad

scaler = StandardScaler()

X_all_scaled = scaler.fit_transform(np.vstack([X_train, X_test]))

```

Use Stratified Splitting for Classification

Preserve class distribution:

```python

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.2, stratify=y, random_state=42

)

```

Set Random State for Reproducibility

```python

model = RandomForestClassifier(n_estimators=100, random_state=42)

```

Choose Appropriate Metrics

• Balanced data: Accuracy, F1-score
• Imbalanced data: Precision, Recall, ROC AUC, Balanced Accuracy
• Cost-sensitive: Define custom scorer

Scale Features When Required

Algorithms requiring feature scaling:

• SVM, KNN, Neural Networks
• PCA, Linear/Logistic Regression with regularization
• K-Means clustering

Algorithms not requiring scaling:

• Tree-based models (Decision Trees, Random Forest, Gradient Boosting)
• Naive Bayes

ConvergenceWarning

Issue: Model didn't converge

Solution: Increase max_iter or scale features

```python

model = LogisticRegression(max_iter=1000)

```

Poor Performance on Test Set

Issue: Overfitting

Solution: Use regularization, cross-validation, or simpler model

```python

# Add regularization

model = Ridge(alpha=1.0)

# Use cross-validation

scores = cross_val_score(model, X, y, cv=5)

```

Memory Error with Large Datasets

Solution: Use algorithms designed for large data

```python

# Use SGD for large datasets

from sklearn.linear_model import SGDClassifier

model = SGDClassifier()

# Or MiniBatchKMeans for clustering

from sklearn.cluster import MiniBatchKMeans

model = MiniBatchKMeans(n_clusters=8, batch_size=100)

```

• Official Documentation: https://scikit-learn.org/stable/
• User Guide: https://scikit-learn.org/stable/user_guide.html
• API Reference: https://scikit-learn.org/stable/api/index.html
• Examples Gallery: https://scikit-learn.org/stable/auto_examples/index.html