NameError: name 'HTML' is not defined

[fbarfi]

Description of your problem

Running the example: SMC-ABC_Lotka-Volterra_example. exactly as is. Getting the error down below. Thanks!

with pm.Model() as example:
    a = pm.Normal("a", mu=0, sigma=5)
    b = pm.HalfNormal("b", sigma=1)
    s = pm.Simulator("s", normal_sim, params=(a, b), sum_stat="sort", epsilon=1, observed=data)

    idata = pm.sample_smc()
    idata.extend(pm.sample_posterior_predictive(idata))

Please provide the full traceback.

Complete error traceback

RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/multiprocessing/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/pymc/smc/sample_smc.py", line 439, in _apply_args_and_kwargs
    return fn(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/pymc/smc/sample_smc.py", line 361, in _sample_smc_int
    progressbar.update_bar(getattr(progressbar, "offset", 0) + 0)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/fastprogress/fastprogress.py", line 76, in update_bar
    else: self.on_update(val, f'{100 * val/self.total:.2f}% [{val}/{self.total} {elapsed_t}<{remaining_t}{end}]')
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/fastprogress/fastprogress.py", line 127, in on_update
    if self.display: self.out.update(HTML(self.progress))
NameError: name 'HTML' is not defined
"""

The above exception was the direct cause of the following exception:

NameError                                 Traceback (most recent call last)
Input In [5], in <cell line: 1>()
      3 b = pm.HalfNormal("b", sigma=1)
      4 s = pm.Simulator("s", normal_sim, params=(a, b), sum_stat="sort", epsilon=1, observed=data)
----> 6 idata = pm.sample_smc()
      7 idata.extend(pm.sample_posterior_predictive(idata))

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/pymc/smc/sample_smc.py:227, in sample_smc(draws, kernel, start, model, random_seed, chains, cores, compute_convergence_checks, return_inferencedata, idata_kwargs, progressbar, **kernel_kwargs)
    224 t1 = time.time()
    226 if cores > 1:
--> 227     results = run_chains_parallel(
    228         chains, progressbar, _sample_smc_int, params, random_seed, kernel_kwargs, cores
    229     )
    230 else:
    231     results = run_chains_sequential(
    232         chains, progressbar, _sample_smc_int, params, random_seed, kernel_kwargs
    233     )

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/pymc/smc/sample_smc.py:408, in run_chains_parallel(chains, progressbar, to_run, params, random_seed, kernel_kwargs, cores)
    406 params = tuple(cloudpickle.dumps(p) for p in params)
    407 kernel_kwargs = {key: cloudpickle.dumps(value) for key, value in kernel_kwargs.items()}
--> 408 results = _starmap_with_kwargs(
    409     pool,
    410     to_run,
    411     [(*params, random_seed[chain], chain, pbars[chain]) for chain in range(chains)],
    412     repeat(kernel_kwargs),
    413 )
    414 results = tuple(cloudpickle.loads(r) for r in results)
    415 pool.close()

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/pymc/smc/sample_smc.py:435, in _starmap_with_kwargs(pool, fn, args_iter, kwargs_iter)
    431 def _starmap_with_kwargs(pool, fn, args_iter, kwargs_iter):
    432     # Helper function to allow kwargs with Pool.starmap
    433     # Copied from https://stackoverflow.com/a/53173433/13311693
    434     args_for_starmap = zip(repeat(fn), args_iter, kwargs_iter)
--> 435     return pool.starmap(_apply_args_and_kwargs, args_for_starmap)

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/multiprocessing/pool.py:372, in Pool.starmap(self, func, iterable, chunksize)
    366 def starmap(self, func, iterable, chunksize=None):
    367     '''
    368     Like `map()` method but the elements of the `iterable` are expected to
    369     be iterables as well and will be unpacked as arguments. Hence
    370     `func` and (a, b) becomes func(a, b).
    371     '''
--> 372     return self._map_async(func, iterable, starmapstar, chunksize).get()

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/multiprocessing/pool.py:771, in ApplyResult.get(self, timeout)
    769     return self._value
    770 else:
--> 771     raise self._value

NameError: name 'HTML' is not defined

Please provide any additional information below.

Versions and main components

Python implementation: CPython
Python version : 3.10.4
IPython version : 8.4.0

Compiler : Clang 12.0.1
OS : Darwin
Release : 21.5.0
Machine : arm64
Processor : arm
CPU cores : 10
Architecture: 64bit

arviz : 0.12.1
matplotlib: 3.5.2
numpy : 1.22.4
pymc : 4.0.0b6

[aloctavodia]

I am not able to reproduce this error locally. Maybe this is related to AnswerDotAI/fastprogress#32

[fbarfi]

Thanks aloctavodia for the suggestion. I followed the link and checked the suggestion in it about reinstalling ipywidgets which I did. It does not solve the problem. I am running the latest ipywidgets 7.7.0. I tried to run the notebook on collar but then was faced with other issues (after I installed pymc there).

[aloctavodia]

Thanks for checking. I will check this tomorrow, in the meantime could you try installing the latest version (pymc 4.0). https://github.com/pymc-devs/pymc/wiki/Installation-Guide-(MacOS)

[fbarfi]

Thanks. I have already installed pymc : 4.0.0b6. Isn't that the latest?

[aloctavodia]

almost, we have recently released the first stable version of the 4.x series. If you install from main or following the intructions in the link i posterd you will get the version 4.0.

[fbarfi]

will do. thanks.

[fbarfi]

Just reinstalled everything afresh as directed. Yet getting the same error. thanks.

[fbarfi]

Error about HTML disappears when I do:

pm.sample_smc(progressbar=False)

Now I get the following error:

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/multiprocessing/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py", line 439, in _apply_args_and_kwargs
    return fn(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py", line 363, in _sample_smc_int
    smc._initialize_kernel()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/smc.py", line 225, in _initialize_kernel
    initial_point, [self.model.datalogpt], self.variables, shared
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/model.py", line 847, in datalogpt
    return self.observedlogpt + self.potentiallogpt
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/model.py", line 864, in observedlogpt
    return self.logpt(vars=self.observed_RVs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/model.py", line 745, in logpt
    rv_logps = joint_logpt(list(rv_values.keys()), rv_values, sum=False, jacobian=jacobian)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/distributions/logprob.py", line 226, in joint_logpt
    temp_logp_var_dict = factorized_joint_logprob(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/aeppl/joint_logprob.py", line 147, in factorized_joint_logprob
    q_logprob_vars = _logprob(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/functools.py", line 889, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/aeppl/logprob.py", line 85, in _logprob
    raise NotImplementedError(f"Logprob method not implemented for {op}")
NotImplementedError: Logprob method not implemented for Simulator_rv{0, (0, 0), floatX, False}
"""

The above exception was the direct cause of the following exception:

NotImplementedError                       Traceback (most recent call last)
Input In [6], in <cell line: 1>()
      3 b = pm.HalfNormal("b", sigma=1)
      4 s = pm.Simulator("s", normal_sim, params=(a, b), 
      5                  sum_stat="sort", epsilon=1, observed=data)
----> 7 idata = pm.sample_smc(progressbar=False)
      8 idata.extend(pm.sample_posterior_predictive(idata))

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py:227, in sample_smc(draws, kernel, start, model, random_seed, chains, cores, compute_convergence_checks, return_inferencedata, idata_kwargs, progressbar, **kernel_kwargs)
    224 t1 = time.time()
    226 if cores > 1:
--> 227     results = run_chains_parallel(
    228         chains, progressbar, _sample_smc_int, params, random_seed, kernel_kwargs, cores
    229     )
    230 else:
    231     results = run_chains_sequential(
    232         chains, progressbar, _sample_smc_int, params, random_seed, kernel_kwargs
    233     )

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py:408, in run_chains_parallel(chains, progressbar, to_run, params, random_seed, kernel_kwargs, cores)
    406 params = tuple(cloudpickle.dumps(p) for p in params)
    407 kernel_kwargs = {key: cloudpickle.dumps(value) for key, value in kernel_kwargs.items()}
--> 408 results = _starmap_with_kwargs(
    409     pool,
    410     to_run,
    411     [(*params, random_seed[chain], chain, pbars[chain]) for chain in range(chains)],
    412     repeat(kernel_kwargs),
    413 )
    414 results = tuple(cloudpickle.loads(r) for r in results)
    415 pool.close()

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py:435, in _starmap_with_kwargs(pool, fn, args_iter, kwargs_iter)
    431 def _starmap_with_kwargs(pool, fn, args_iter, kwargs_iter):
    432     # Helper function to allow kwargs with Pool.starmap
    433     # Copied from https://stackoverflow.com/a/53173433/13311693
    434     args_for_starmap = zip(repeat(fn), args_iter, kwargs_iter)
--> 435     return pool.starmap(_apply_args_and_kwargs, args_for_starmap)

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/multiprocessing/pool.py:372, in Pool.starmap(self, func, iterable, chunksize)
    366 def starmap(self, func, iterable, chunksize=None):
    367     '''
    368     Like `map()` method but the elements of the `iterable` are expected to
    369     be iterables as well and will be unpacked as arguments. Hence
    370     `func` and (a, b) becomes func(a, b).
    371     '''
--> 372     return self._map_async(func, iterable, starmapstar, chunksize).get()

File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/multiprocessing/pool.py:771, in ApplyResult.get(self, timeout)
    769     return self._value
    770 else:
--> 771     raise self._value

NotImplementedError: Logprob method not implemented for Simulator_rv{0, (0, 0), floatX, False}

[ricardoV94]

Can you try running with a single core? pm.sample_smc(cores=1) or something.

Also, please provide the whole snippet you are using so that we can try to reproduce it locally.

[fbarfi]

this is the example I was running from the examples posted on pymc

[SMC-ABC_Lotka-Volterra_example.ipynb](https://github.com/pymc-devs/pymc-examples/blob/main/examples/samplers/SMC-ABC_Lotka-Volterra_example.ipynb)

I did what you suggested with cores=1 and everything works perfectly. Does this mean that one cannot use more than one core? Thanks.

[ricardoV94]

It should work, but there seems to be an issue with multiprocessing on your end :/

[fbarfi]

I am not sure what to say. The moment used cores =2 I get the message at the end as

NotImplementedError: Logprob method not implemented for Simulator_rv{0, (0, 0), floatX, False}

it does not mention anywhere the multiprocessing. Could it be a macOS issue, or perhaps an apple M1 issue? thanks.

[fbarfi]

I am looking into the other examples for samplers that use the pm.sample_smc and they all work with cores=1 and multiple chains (4). thanks.

[michaelosthege]

The traceback above indicates Python 3.10.
Are we sure that Aesara/PyMC work with 3.10?

Because #5209

[fbarfi]

Yes they do work. I ve tried many codes, including even with sampling_jax.