SNOW-2434518: XGBRegressor.fit() call fails with imports/packages/session error

[EliasAthey]

On Oct 14, python code that was in a notebook began failing with this error:

TypeError: resolve_imports_and_packages() got multiple values for argument 'session'

Here is the (simplified) code that is failing.
EATHEY package debugging.ipynb

Here is the environment file for the notebook.
environment (1).yml

I have tried changing versions of python and versions of packages, but I keep getting this error.

I would love to understand where I should head next. Is the versioning/packages the right path? Could this be some issue on the snowflake warehouse runtime? Anything would be helpful.

Here is the full stack trace:

File "Cell [cell3]", line 19, in <module>
    final_surgeon_model.fit(surgeon_df)
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/ml/_internal/telemetry.py", line 565, in wrap
    return ctx.run(execute_func_with_statement_params)
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/ml/_internal/telemetry.py", line 533, in execute_func_with_statement_params
    result = func(*args, **kwargs)
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/ml/modeling/framework/base.py", line 440, in fit
    return self._fit(dataset)
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/ml/modeling/xgboost/xgb_regressor.py", line 554, in _fit
    self._sklearn_object = model_trainer.train()
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/ml/modeling/_internal/snowpark_implementations/snowpark_trainer.py", line 566, in train
    fit_wrapper_sproc = self._get_fit_wrapper_sproc(statement_params=statement_params, anonymous=False)
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/ml/modeling/_internal/snowpark_implementations/snowpark_trainer.py", line 236, in _get_fit_wrapper_sproc
    fit_wrapper_sproc = self.session.sproc.register(
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/snowpark/_internal/utils.py", line 1148, in call_wrapper
    return func(*args, **kwargs)
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/snowpark/stored_procedure.py", line 622, in register
    return self._do_register_sp(
File "/usr/lib/python_udf/e2688d3416449b8a63dd4780e2c439ed34af3be81650e17a4a30c0e5a15cf953/lib/python3.10/site-packages/snowflake/snowpark/stored_procedure.py", line 942, in _do_register_sp
    ) = resolve_imports_and_packages(

[sfc-gh-kunli]

Did you see this error in container runtime notebook?

[Y-TU-Corey]

I'm currently see this error on Snowpark Notebooks running on the warehouse; I started observing it yesterday. The error is occurring when I call .fit() on objects of the following classes:

• GradientBoostingClassifier
• XGBClassifier
• RandomForestClassifier

I no longer see the error after I shifted the notebook execution environment to a Container (CPU Runtime).

@sfc-gh-kunli Thanks for the unblocking me. Can you explain why Container Exec environment might not be running into multiple session definitions?

[sfc-gh-kunli]

Container Runtime is currently what we actively maintain and develop, not the warehouse notebook. For any ML workload, just use Container Runtime to avoid potential issues.