feature: introduce optional bootstrap level tuning to Catboost by thomasATbayer · Pull Request #65 · Bayer-Group/MotherML

thomasATbayer · 2026-07-28T13:53:53Z

Currently the opitmization chooses the Boostsrap type but not the level of that Bootstrap, as this is important especially for uncertainty estimation should enable that at least optionally.

Add the tune_bootstrap_level flag so Optuna can optionally tune the bootstrap level parameters (subsample for Bernoulli, bagging_temperature for Bayesian) alongside the existing bootstrap_type search.

Wrap the _CatboostHyperParams constructor signature across multiple lines and strip trailing whitespace from the surrounding docstrings. No behaviour change.

Copilot

🟡 Changes recommended

The new tune_bootstrap_level flag is not consistently propagated/exposed across CatBoost wrappers (notably the Gaussian Process regressor and classifier parameter round-trips), and the new search-space behavior lacks targeted unit test coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR extends the MotherML CatBoost model integration to optionally tune CatBoost bootstrap level parameters (e.g., subsample / bagging_temperature) in Optuna search spaces, which is particularly relevant for uncertainty estimation workflows.

Changes:

Added tune_bootstrap_level flag to CatBoost hyperparameter-space generation to optionally tune bootstrap level parameters based on the selected bootstrap_type.
Threaded tune_bootstrap_level through CatboostRegressorMother and CatboostRankerMother and included it in parameter/state handling where applicable.
Added serialization / parameter plumbing in multiple CatBoost wrappers (but currently inconsistently across wrappers).

File summaries

File	Description
src/mother/ml/models/m_catboost.py	Introduces optional bootstrap-level tuning in the Optuna hyperparameter space and adds plumbing to expose/persist the new flag across CatBoost model wrappers.

Review details

Suppressed comments (2)

src/mother/ml/models/m_catboost.py:1267

tune_bootstrap_level is accepted by set_params (and is also included in pickling state), but CatboostClassifierMother does not expose it via __init__ and get_params doesn’t return it. This makes the parameter hard to configure consistently and can cause it to be dropped during sklearn cloning.

        our_params = [
            "target_type",
            "tune_boosting_type",
            "model_type",
            "tune_tree_structure_type",
            "tune_bootstrap_level",
        ]

src/mother/ml/models/m_catboost.py:1497

Docstring is inaccurate: tune_bootstrap_level controls tuning of bootstrap level parameters (e.g., subsample / bagging_temperature), not whether bootstrap_type itself is included.

            tune_bootstrap_level : bool, optional
                Whether to include the "bootstrap_type" parameter in hyperparameter tuning.

Files reviewed: 1/1 changed files
Comments generated: 3
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Add tune_bootstrap_level to CatboostClassifierMother.__init__ and return it from get_params so the parameter follows the same sklearn-compatible round trip as the regressor and ranker implementations. Previously it was accepted by set_params and persisted in the pickling state, but was absent from the constructor and get_params, which allowed sklearn.clone to drop the value. Also clarify the ranker documentation: tune_bootstrap_level controls bootstrap level parameters such as subsample and bagging_temperature, not whether bootstrap_type itself is included in the search space.

Copilot

🟡 Changes recommended

tune_bootstrap_level is not consistently wired/exposed in all touched estimators (notably the GP regressor init and Ranker params/cloning), which can silently disable the new functionality and break cloning-based workflows.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/mother/ml/models/m_catboost.py:174

Line exceeds the repo's 120-char formatting limit and is inconsistent with surrounding formatting; this is likely to fail ruff format/style checks. Please wrap the suggest_float call like the bagging_temperature block below.

src/mother/ml/models/m_catboost.py:747

CatboostGaussianProcessRegressorMother exposes tune_bootstrap_level in its constructor (and accepts it in set_params/pickle state), but the constructor never forwards it to _CatboostHyperParams.__init__ or otherwise initializes self.tune_bootstrap_level. As a result, passing tune_bootstrap_level=True at construction time is silently ignored and bootstrap-level tuning will remain disabled (and likely won’t survive cloning since get_params doesn’t include it).

        eps: float = 1e-4,
        tune_boosting_type: bool = False,
        tune_tree_structure_type: bool = True,
        tune_bootstrap_level: bool = False,
        verbose: bool = False,

Files reviewed: 2/2 changed files
Comments generated: 1
Review effort level: Lite

github-actions · 2026-09-02T15:31:39Z

Release Preflight

Automated check that complements semantic-release changelog generation.

# Release Preflight Report

Base ref: origin/main
Commit count: 7

## Conventional commit summary

- feat: 1
- fix: 4
- style: 1
- test: 1

## Commits missing issue or PR reference

- 359a3e42 test(catboost): verify bootstrap tuning search space distributions
- 949c01e7 fix(catboost): scope bootstrap-level tuning to estimators that use it
- 63ec0fd0 fix(tabpfn): disable autocast for pre-fitted embedding extraction

## Suggested manual follow-up

- Run the update-docs skill before release for issue and milestone reconciliation.
- Confirm docs updates for release-specific changes.

TabPFN's "auto" inference precision can select autocast, which makes get_embeddings() return bfloat16 tensors. Converting those to NumPy fails with `TypeError: Got unsupported ScalarType BFloat16`, breaking TabPFNEmbeddingTransformer whenever a user-supplied pre-fitted model is used (test_prefitted_model). Models the transformer creates itself already pass inference_precision=torch.float32, so only the pre-fitted path was unguarded. Add _prepare_prefitted_model_for_embeddings(), which forces use_autocast_ = False on the pre-fitted model, and call it in both the pre-fitted fit() branch and transform(). The failure is not specific to this branch — m_tabpfn.py is identical to main; it only surfaces when the autocast heuristic picks bfloat16 on a given runner. This restores the fix from #71, which never landed on main.

Copilot

🟡 Changes recommended

The new tune_bootstrap_level flag is inconsistently wired in CatboostGaussianProcessRegressorMother (constructor/get_params behavior), and TabPFN autocast disabling should be made robust against missing attributes.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

src/mother/ml/models/m_catboost.py:174

The new tune_bootstrap_level behavior (adding subsample for Bernoulli and bagging_temperature for Bayesian) isn’t covered by unit tests, so regressions in the Optuna search space could go unnoticed. Consider adding deterministic tests using optuna.trial.FixedTrial to cover Bernoulli/Bayesian/MVS cases.

        if self.tune_bootstrap_level:
            if suggested_params[prefix + "bootstrap_type"] == "Bernoulli":
                suggested_params[prefix + "subsample"] = trial.suggest_float(prefix + "subsample", 0.25, 1.0, log=False)

src/mother/ml/models/m_catboost.py:749

CatboostGaussianProcessRegressorMother.__init__ now accepts tune_bootstrap_level, but it isn’t wired into the _CatboostHyperParams initialization, so setting it in the constructor won’t affect get_hyperparameter_space() for this estimator.

        eps: float = 1e-4,
        tune_boosting_type: bool = False,
        tune_tree_structure_type: bool = True,
        tune_bootstrap_level: bool = False,
        verbose: bool = False,
        model_type: str = "regression",
        target_type: props.TargetType = "single_target",

src/mother/ml/models/m_catboost.py:896

set_params() now supports tune_bootstrap_level, but get_params() doesn’t include it, so sklearn clone/round-tripping may drop this flag.

            "target_type",
            "tune_boosting_type",
            "tune_tree_structure_type",
            "tune_bootstrap_level",
            "samples",
            "prior_iterations",
            "sigma",

Files reviewed: 3/3 changed files
Comments generated: 2
Review effort level: Lite

`tune_bootstrap_level` was wired inconsistently into `CatboostGaussianProcessRegressorMother`: the constructor accepted it but never forwarded it to `_CatboostHyperParams`, and `set_params()` accepted it while `get_params()` omitted it. As a result the flag had no effect on `get_hyperparameter_space()` and sklearn clone round-trips silently dropped it. The GP estimator fits via `catboost.core.sample_gaussian_process`, which ignores bootstrap parameters entirely, so exposing the knob is misleading. Rather than complete the plumbing, drop the constructor parameter and pin `tune_bootstrap_level=False` when initialising `_CatboostHyperParams` — the same treatment the other unsupported CatBoost tuning knobs already receive for this estimator. Also harden the TabPFN embedding path: `use_autocast_` is not present on every pre-fitted TabPFN model, so guard the assignment with `hasattr()` instead of assuming the attribute exists. Adds deterministic `optuna.trial.FixedTrial` tests covering Bernoulli (`subsample`), Bayesian (`bagging_temperature`) and MVS (neither), the default-off behaviour, the clone round-trip of the flag, and that the GP estimator neither exposes nor applies it. Refs #65

Copilot

🟢 Approval recommended

The new flag is consistently wired through CatBoost estimators and is backed by targeted unit tests, with no verified regressions found in the reviewed changes.

Review details

Files reviewed: 4/4 changed files
Comments generated: 0 new
Review effort level: Lite

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved bootstrap-type, ranker API, and posterior-sampling compatibility issues remain.

Review effort: Lite
Findings: 3 · 2

Open (5)

Expose bootstrap tuning flag on CatBoost ranker · New
Include MVS when tuning bootstrap sampling level · New
Correct test to expect MVS subsample tuning · New
The bootstrap-level tuning block uses two independent if checks and includes an overlong…
New bootstrap-level tuning logic is added here, but there are no unit tests asserting that enabling…

Resolved since last review (4)

_prepare_prefitted_model_for_embeddings() unconditionally sets self.model.use_autocast_, but…
CatboostRankerMother now accepts tune_bootstrap_level, but get_params/set_params are still…
CatboostGaussianProcessRegressorMother.__init__ now accepts tune_bootstrap_level, but the value…
Docstring is inaccurate: tune_bootstrap_level controls tuning of bootstrap level parameters…

+            if suggested_params[prefix + "bootstrap_type"] == "Bernoulli":
+                suggested_params[prefix + "subsample"] = trial.suggest_float(prefix + "subsample", 0.25, 1.0, log=False)


+    params = model.get_hyperparameter_space(X, y, _fixed_trial("MVS"))
+
+    assert _SUBSAMPLE not in params
+    assert _BAGGING_TEMPERATURE not in params


Copilot

Copilot review overview

🔵 Needs a closer look

Moderate unresolved issues can cause incompatible parameters or prevent MVS bootstrap-level tuning.

Review effort: Lite
Findings: 2 · 1

Open (3)

Correct test to expect MVS subsample tuning
Include MVS when tuning bootstrap sampling level
New bootstrap-level tuning logic is added here, but there are no unit tests asserting that enabling…

Resolved since last review (2)

Expose bootstrap tuning flag on CatBoost ranker
The bootstrap-level tuning block uses two independent if checks and includes an overlong…

Copilot

Copilot review overview

🔵 Needs a closer look

MVS bootstrap-level tuning remains unresolved.

Review effort: Lite
Findings: 2

Open (2)

Correct test to expect MVS subsample tuning
Include MVS when tuning bootstrap sampling level

Resolved since last review (1)

New bootstrap-level tuning logic is added here, but there are no unit tests asserting that enabling…

feat(catboost): introduce optional bootstrap level tuning (#65)

65210d5

Add the tune_bootstrap_level flag so Optuna can optionally tune the bootstrap level parameters (subsample for Bernoulli, bagging_temperature for Bayesian) alongside the existing bootstrap_type search.

thomasATbayer linked an issue Jul 28, 2026 that may be closed by this pull request

Make bootstrap level in Catboost tunable #63

Open

style(catboost): reformat bootstrap tuning signatures (#65)

2c8b64c

Wrap the _CatboostHyperParams constructor signature across multiple lines and strip trailing whitespace from the surrounding docstrings. No behaviour change.

thomasATbayer added this to the RDKIT UGM Release milestone Jul 28, 2026

thomasATbayer added the enhancement New feature or request label Jul 29, 2026

thomasATbayer requested a review from agzieba August 5, 2026 18:45

Merge branch 'main' into enableBootstrapLevelTuning

3092135

Copilot AI lite review requested due to automatic review settings September 2, 2026 14:26

Copilot started reviewing on behalf of thomasATbayer September 2, 2026 14:26 View session

Copilot AI reviewed Sep 2, 2026

View reviewed changes

Comment thread src/mother/ml/models/m_catboost.py Outdated

Comment thread src/mother/ml/models/m_catboost.py Outdated

Comment thread src/mother/ml/models/m_catboost.py Outdated

Copilot AI review requested due to automatic review settings September 2, 2026 14:33

Copilot started reviewing on behalf of thomasATbayer September 2, 2026 14:34 View session

Copilot AI reviewed Sep 2, 2026

View reviewed changes

Comment thread src/mother/ml/models/m_catboost.py Outdated

thomasATbayer force-pushed the enableBootstrapLevelTuning branch from 5a03f73 to 28b9ce7 Compare September 2, 2026 15:39

Copilot AI review requested due to automatic review settings September 3, 2026 11:01

Copilot started reviewing on behalf of thomasATbayer September 3, 2026 11:01 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

Comment thread src/mother/ml/models/m_tabpfn.py Outdated

Comment thread src/mother/ml/models/m_catboost.py

Copilot AI review requested due to automatic review settings September 3, 2026 12:03

Copilot started reviewing on behalf of thomasATbayer September 3, 2026 12:03 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

thomasATbayer marked this pull request as draft September 29, 2026 16:57

fix(catboost): resolve merge conflicts and align bootstrap tuning (#65)

44ff463

thomasATbayer marked this pull request as ready for review October 1, 2026 09:57

thomasATbayer requested review from SommerKai and a lite review from Copilot October 1, 2026 09:57

Copilot started reviewing on behalf of thomasATbayer October 1, 2026 09:57 View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

fix(catboost): expose bootstrap tuning on ranker (#65)

9b81fd9

Copilot AI lite review requested due to automatic review settings October 1, 2026 12:17

Copilot started reviewing on behalf of thomasATbayer October 1, 2026 12:17 View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

test(catboost): verify bootstrap tuning search space distributions

359a3e4

Copilot AI lite review requested due to automatic review settings October 1, 2026 13:35

Copilot started reviewing on behalf of thomasATbayer October 1, 2026 13:36 View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

thomasATbayer self-assigned this Oct 2, 2026

Navigation Menu

Repository navigation

feature: introduce optional bootstrap level tuning to Catboost - #65

feature: introduce optional bootstrap level tuning to Catboost#65
thomasATbayer wants to merge 9 commits into
mainfrom
enableBootstrapLevelTuning

thomasATbayer commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot commented Sep 2, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		if suggested_params[prefix + "bootstrap_type"] == "Bernoulli":
		suggested_params[prefix + "subsample"] = trial.suggest_float(prefix + "subsample", 0.25, 1.0, log=False)

Conversation

thomasATbayer commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (2)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (2)

Uh oh!

Uh oh!

github-actions Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release Preflight

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (3)

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Sep 2, 2026 •

edited

Loading