Skip to content

feat: add tuple support and exam_cb for callbacks#665

Open
ugbotueferhire wants to merge 3 commits intointerpretml:mainfrom
ugbotueferhire:feature/callback-tuples
Open

feat: add tuple support and exam_cb for callbacks#665
ugbotueferhire wants to merge 3 commits intointerpretml:mainfrom
ugbotueferhire:feature/callback-tuples

Conversation

@ugbotueferhire
Copy link
Copy Markdown
Contributor

Fixes #635 Callback tuple support and exam callback

Description

This PR implements Phase 2 of the callback redesign for EBM training.

Phase 1 changed the callback API to keyword-only arguments and ensured the
progress callback only fires on progressing boosting steps. That work was
merged in #662.

This PR builds on top of that by allowing callback to accept either:

  • a single progress callback
  • a single examination callback
  • a tuple containing one progress callback and one examination callback

The supported callback signatures are:

def progress_cb(*, bag, stage, step, term, metric):
    ...

def exam_cb(*, bag, stage, step, term, gain):
    ...

Why

The original callback parameter was carrying more than one kind of signal.
As discussed in the review for #662, there are really two different callback
use cases:

  • a progress callback that reports accepted boosting progress
  • an examination callback that reports when a term has been examined and what
    gain was computed for it

Splitting those concepts makes the API clearer while still keeping a single
public callback parameter.

What Changed

  • Added callback normalization and validation in _ebm.py
  • Classified callbacks by signature using metric vs gain
  • Allowed callback to be either a callable or a tuple of callables
  • Enforced at most one progress callback and one examination callback
  • Added an exam_callback hook in _boost.py at the point where avg_gain
    is computed
  • Kept the existing progress callback behavior after accepted term updates
  • Allowed either callback type to stop training early by returning True
  • Updated callback parameter docstrings for EBM model, classifier, and regressor

Validation Rules

This PR raises a ValueError for invalid callback configurations such as:

  • empty callback tuples
  • more than one progress callback
  • more than one examination callback
  • callbacks that do not match either supported signature

Tests

Added or covered tests for:

  • tuple callback support calling both callback types
  • valid finite gain values in the examination callback
  • early termination from the examination callback
  • invalid tuple configurations
  • invalid callback signatures

Verified with:

python -m pytest tests/glassbox/ebm/test_callback.py -q
python -m pytest tests/glassbox/ebm/test_ebm.py::test_callbacks_short tests/glassbox/ebm/test_ebm.py::test_callbacks_long tests/glassbox/ebm/test_merge_ebms.py::test_merge_ebms_callback_is_none -q

Notes

@ugbotueferhire ugbotueferhire force-pushed the feature/callback-tuples branch from 3c62aa9 to 5b7a76f Compare May 2, 2026 16:05
@codecov
Copy link
Copy Markdown

codecov Bot commented May 2, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.86%. Comparing base (7ad1c39) to head (e4c93fc).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #665      +/-   ##
==========================================
+ Coverage   66.60%   66.86%   +0.26%     
==========================================
  Files          76       76              
  Lines       11611    11681      +70     
==========================================
+ Hits         7733     7811      +78     
+ Misses       3878     3870       -8     
Flag Coverage Δ
bdist_linux_311_python ?
bdist_linux_312_python ?
bdist_linux_313_python ?
bdist_linux_314_python ?
bdist_linuxarm_311_python ?
bdist_linuxarm_312_python 45.40% <97.33%> (-20.95%) ⬇️
bdist_linuxarm_313_python 64.66% <100.00%> (-1.71%) ⬇️
bdist_linuxarm_314_python ?
bdist_mac_311_python ?
bdist_mac_312_python ?
bdist_mac_313_python ?
bdist_mac_314_python ?
bdist_win_311_python ?
bdist_win_312_python ?
bdist_win_313_python ?
bdist_win_314_python ?
sdist_linux_311_python 66.58% <100.00%> (+0.28%) ⬆️
sdist_linux_312_python 66.56% <100.00%> (+0.27%) ⬆️
sdist_linux_313_python 66.57% <100.00%> (+0.27%) ⬆️
sdist_linux_314_python 66.50% <100.00%> (+0.32%) ⬆️
sdist_linuxarm_311_python 66.60% <100.00%> (+0.29%) ⬆️
sdist_linuxarm_312_python 66.61% <100.00%> (+0.30%) ⬆️
sdist_linuxarm_313_python 66.59% <100.00%> (+0.28%) ⬆️
sdist_linuxarm_314_python 66.50% <100.00%> (+0.28%) ⬆️
sdist_mac_311_python 66.71% <100.00%> (+0.29%) ⬆️
sdist_mac_312_python 66.73% <100.00%> (+0.31%) ⬆️
sdist_mac_313_python 66.72% <100.00%> (+0.30%) ⬆️
sdist_mac_314_python 66.63% <100.00%> (+0.31%) ⬆️
sdist_win_311_python 66.83% <100.00%> (+0.30%) ⬆️
sdist_win_312_python 66.83% <100.00%> (+0.32%) ⬆️
sdist_win_313_python 66.83% <100.00%> (+0.32%) ⬆️
sdist_win_314_python 66.72% <100.00%> (+0.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ugbotueferhire ugbotueferhire force-pushed the feature/callback-tuples branch from 5b7a76f to 07f9ff4 Compare May 3, 2026 06:29
Signed-off-by: ugbotueferhire <ugbotueferhire@gmail.com>
@ugbotueferhire ugbotueferhire force-pushed the feature/callback-tuples branch from 07f9ff4 to ca55033 Compare May 3, 2026 19:05
ugbotueferhire and others added 2 commits May 3, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Repeated iterations in callback

2 participants