MiniBoosts

A Rust library for implementing and comparing boosting algorithms.

crates.io version docs.rs status license

Overview

MiniBoosts collects boosting algorithms behind shared Booster, WeakLearner, and Classifier traits, so the same data and weak learner can be used across experiments for fair comparisons. Linear, quadratic, and conic subproblems are solved with Clarabel; no commercial solver is required. Optional Python bindings expose the same algorithms with a scikit-learn-style API.

Install

cargo add miniboosts

Example

use miniboosts::prelude::*;

let sample = SampleReader::default()
    .file("train.csv")
    .has_header(true)
    .target_feature("class")
    .read()
    .unwrap();
let tree = DecisionTreeBuilder::new(&sample)
    .max_depth(2)
    .build();
let f = LpBoost::init(&sample)
    .nu(0.1 * sample.shape().0 as f64)
    .run(&tree);
let predictions = f.predict_all(&sample);

Algorithms

AdaBoostFreund and Schapire
BrownBoostFreund
AdaBoostVRätsch and Warmuth
MadaBoostDomingo and Watanabe
SmoothBoostServedio
LPBoostDemiriz, Bennett, and Shawe-Taylor
ERLPBoostWarmuth, Glocer, and Vishwanathan
Corrective ERLPBoostShalev-Shwartz and Singer
MLPBoostMitsuboshi, Hatano, and Takimoto
SoftBoostWarmuth, Glocer, and Rätsch
TotalBoostWarmuth, Liao, and Rätsch
Graph Separation BoostingAlon, Gonen, Hazan, and Moran
SparsiBoostGrønlund, Larsen, and Mathiasen

Built-in weak learners: decision trees, logistic regression, bounded linear functions, Gaussian naive Bayes, and Gaussian RBF bumps. Any type implementing WeakLearner can be used instead.

Playground

Place points, choose an algorithm, and see the decision boundary. Training runs entirely in your browser: MiniBoosts is compiled to WebAssembly. Shading shows the ensemble's confidence (relative to its largest value on the plane); the black line is where the confidence is zero.

Click: add a point of the selected class. Shift-click or right-click: add the other class. Alt-click: remove the nearest point.

Data
New points:

Algorithm
Weak learner

Exact edge maximization over bounded linear functions. A combination of linear functions is linear, so every booster returns a linear classifier.

Fits a weighted logistic loss; no edge guarantee.

One bump ±kc per training point; returns the bump with the largest edge.

Add points of both classes, then press Run.