xgboost
π―SkillInstallation
npx skills add g1joshi/agent-skills --skill xgboostRun it in a terminal; it asks which AI coding tools to install it for (Claude Code, Codex, Cursor and more).
Uses skills.sh Β· Node.js 22.20+
After installing
Ask for related work and the agent uses it on its own, or call it yourself.
/xgboostInstall by hand
Copy the whole skill folder from GitHub to the place below. Copying only SKILL.md can leave out files the skill uses.
Open the skill folder β- This project
.claude/skills/xgboost/ - All projects
~/.claude/skills/xgboost/
These paths are for Claude Code. For other tools, the command above puts it in the right place.
SKILL.md
View on GitHub βExpert XGBoost assistance covering extreme gradient boosting, tree pruners, DMatrix, GPU hist tree method, and early stopping. Use when training high-accuracy tabular models for competitions and production.
XGBoost
XGBoost is an optimized distributed gradient boosting library designed for high efficiency, speed, and predictive accuracy on tabular datasets.
When to Use
- Tabular Data Competitions & Industry Production: High-performance gradient boosted decision trees for classification and regression.
- GPU-Accelerated Tree Building: Training massive datasets on NVIDIA GPUs with
tree_method="hist"anddevice="cuda". - Handling Missing Values & Sparsity: Automatic direction assignment for missing values without manual imputation.
- Monotonic Feature Constraints: Enforcing domain-specific rules (e.g., higher credit score must never increase interest rate).
Quick Start
import xgboost as xgb
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=1000, n_features=20, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
# Scikit-learn compatible API with GPU histogram support
model = xgb.XGBClassifier(
n_estimators=100,
learning_rate=0.05,
max_depth=6,
tree_method="hist", # Ultra-fast histogram method
device="cuda" if xgb.rabit.get_rank() >= 0 else "cpu"
)
model.fit(X_train, y_train)
accuracy = model.score(X_test, y_test)
print(f"XGBoost Test Accuracy: {accuracy:.4f}")
Core Concepts
High-Performance XGBClassifier with Early Stopping
Training XGBoost using Scikit-Learn API with GPU acceleration:
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score
import pandas as pd
import numpy as np
# Sample dataset
X = pd.DataFrame(np.random.randn(20000, 10), columns=[f"feat_{i}" for i in range(10)])
y = (X["feat_0"] + X["feat_1"] * 2 > 0.5).astype(int)
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)
# Configure modern XGBoost 2.x classifier
clf = xgb.XGBClassifier(
n_estimators=1000,
learning_rate=0.03,
max_depth=6,
subsample=0.8,
colsample_bytree=0.8,
tree_method="hist",
device="cuda" if xgb.rabit.is_distributed() else "cpu",
early_stopping_rounds=30,
eval_metric="auc",
random_state=42
)
clf.fit(
X_train, y_train,
eval_set=[(X_val, y_val)],
verbose=50
)
preds = clf.predict_proba(X_val)[:, 1]
print(f"Validation AUC: {roc_auc_score(y_val, preds):.4f}")
Enforcing Monotonic Constraints
Ensuring predictions strictly increase or decrease with specific features:
# Feature 0 must have positive monotonic relationship (+1), Feature 1 negative (-1)
monotonic_rules = (1, -1, 0, 0, 0, 0, 0, 0, 0, 0)
constrained_clf = xgb.XGBClassifier(
monotone_constraints=monotonic_rules,
tree_method="hist",
n_estimators=100
)
constrained_clf.fit(X_train, y_train)
Saving Model in Universal JSON Format
Exporting trained model for portable cross-platform serving:
# Save model in standard JSON format (recommended for XGBoost 2.x)
clf.save_model("xgboost_model.json")
# Load model in production service
loaded_clf = xgb.XGBClassifier()
loaded_clf.load_model("xgboost_model.json")
Common Patterns
DMatrix Native Training with Early Stopping Callback
Problem: Preventing overfitting without training hundreds of redundant trees after validation loss plateaus.
Solution:
Use native xgb.train with early_stopping:
dtrain = xgb.DMatrix(X_train, label=y_train)
dval = xgb.DMatrix(X_test, label=y_test)
params = {
"objective": "binary:logistic",
"eval_metric": "logloss",
"tree_method": "hist"
}
evals = [(dtrain, "train"), (dval, "val")]
bst = xgb.train(
params,
dtrain,
num_boost_round=1000,
evals=evals,
callbacks=[xgb.callback.EarlyStopping(rounds=50, save_best=True)],
verbose_eval=50
)
Best Practices
Do:
- Target XGBoost 2.x with
tree_method="hist"anddevice="cuda"for extreme training speedups. - Save models in the native
.jsonformat (model.save_model("model.json")) rather than binary pickle files. - Set
early_stopping_roundsin constructor parameters to prevent overfitting on validation data. - Utilize
monotone_constraintswhen business or regulatory logic demands monotonic behavior.
Don't:
- Use deprecated
gpu_histtree method; in modern XGBoost usetree_method="hist"withdevice="cuda". - Perform manual one-hot encoding on high-cardinality categoricals; use
enable_categorical=True. - Tune hyperparameters without early stopping active; it wastes compute on overfitted trees.
Troubleshooting
| Error | Cause | Solution |
|---|---|---|
XGBoostError: Invalid device ordinal | Requesting GPU when no CUDA device is present or wrong GPU ID. | Set device="cpu" or verify GPU with nvidia-smi. |
DataFrame columns must be the same as when the model was trained | Inference DataFrame features do not match training order/names. | Align inference DataFrame columns: df = df[model.feature_names_in_]. |
Overfitting on training set | Trees too deep or learning rate too high. | Decrease max_depth (e.g. 4-6) and tune regularization reg_alpha and reg_lambda. |
References
More from this repository10
A template skill providing a starter framework for creating new agent skills with structured sections for triggers, quick start examples, and capability descriptions.
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill