[prophet] prophet installation fails with ModuleNotFoundError: No module named 'lunarcalendar'

[mdeshmu]

How to reproduce the bug

Following methods to install prophet on docker image of Superset 1.5.1 fails with ModuleNotFoundError: No module named 'lunarcalendar'

docker pull apache/superset:1.5.1
docker run -it <imageid> bash
pip install apache-superset[prophet]
pip install --no-cache-dir apache-superset[prophet]
pip install prophet==1.0.1
pip install --no-cache-dir prophet==1.0.1

Note that the missing dependency can be anything, depending of what it is already installed or not, I personally saw errors about missing lunarcalendar, pystan, tqdm. Same has been explained here

For me, this worked finally: pip install lunarcalendar tqdm "pystan<3.0" && pip install "prophet>=1.0.1, <1.1" but i think even this might break in future if version of any of the other common dependencies with superset are changed or dependency is removed altogether.

Note: prophet dependency issues were solved recently in 1.1 release.
I can install latest 1.1 prophet package with pip install prophet without errors but holidays package gets upgraded and then superset db upgrade breaks and gives below error.
pkg_resources.ContextualVersionConflict: (holidays 0.14.2 (/usr/local/lib/python3.8/site-packages), Requirement.parse('holidays==0.10.3'), {'apache-superset'})
pkg_resources.DistributionNotFound: The 'holidays==0.10.3' distribution was not found and is required by apache-superset

Can holidays package dependency be bumped to 0.14.2 in Superset ?

Expected results

It should be easy to install prophet

Actual results

its really messy

Screenshots

Here are the logs:

prophet-latest-directly.txt
prophet-via-setup-py.txt
prophet-1.0.1-directly.txt

Environment

• browser type and version: NA
• superset version: 1.5.1
• python version: 3.8.13
• node.js version: NA
• any feature flags active: NA

Checklist

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

[wiktor2200]

I have the same issue, but I was unable to fix it with pip install lunarcalendar tqdm "pystan<3.0" && pip install "prophet>=1.0.1, <1.1". Have you changed something else?

[mdeshmu]

This is what i have exactly in my Dockerfile.

RUN pip install --upgrade pip
RUN pip install lunarcalendar tqdm "pystan<3.0" && pip install "prophet>=1.0.1, <1.1"

[haydarmiezanie]

I have some problem, I can't use prophet. I'm using 2.0 superset

[mdeshmu]

@villebro @zhaoyongjie @etr2460 As more people are reporting this issue, request you to provide your opinion for a permanent solution.

[EliottTardieu]

Hello, I used the command @mdeshmu suggested to use, but it's still not working. Whenever I toggle forecast, it says "No results were returned for this query". Any idea ? (Not using Superset 2.0 btw)

[wiktor2200]

With this Dockerfile:

FROM apache/superset:2.0.0 AS superset
USER root

RUN rm -rf /var/lib/apt/lists/* && \
    pip install \
        Authlib==1.0.1 \
        flask-oidc==1.3.0 \
        psycopg2-binary==2.9.1 \
        redis==3.5.3

## Install prophet and dependencies - there is open issue on Github: https://github.com/apache/superset/issues/20901
RUN pip install \
        lunarcalendar==0.0.9 \
        tqdm==4.64.0 \
        pystan==2.19.1.1   
RUN pip install prophet==1.0.1

USER superset

I was able to build Superset with prophet, but it still doesn't work. I'm having the same problem as above, no more errors, but forecast always returns 0 rows.
I've tried with both 2.0.0 and 1.5.1.

[mdeshmu]

@wiktor2200 @EliottTardieu Can you confirm you don't have multiple App containers running and prophet is installed on the container you are browsing for. You may check using pip freeze.

[wiktor2200]

I'm using Dockerfile from above and when I exec into container/pod, prophet is there, but returning 0 rows in every query when Enable Forecast is turned on.

pip freeze result (prophet is installed in version 1.0.1)

[...]
progress==1.6
prompt-toolkit==3.0.28
prophet==1.0.1
psycopg2-binary==2.9.1
ptyprocess==0.7.0
pure-eval==0.2.2
[...]

[EliottTardieu]

pip freeze gives

[...]
tqdm==4.64.0
LunarCalendar==0.0.9
pystan==2.19.1.1
prophet==1.0.1
[...]

I'm having the exact same issue as @wiktor2200 (0 rows when Enable Forecast is turned on). Running on Superset v1.5.1.

[wiktor2200]

Hello! Today I've tried again with my Dockerfile and now prophet works in both 1.5.1 and 2.0.0 versions, so I suppose there were some problems with dependencies which were solved during the weekend.

[EliottTardieu]

Same here, but even Friday. For some reason when I tried again (using the exact same dockerfile), it worked.

[ecederstrand]

Preinstalling lunarcalendar, tqdm and pystan is necessary because prophet <= 1.0 has a bug that requires these packages to be available when running setup.py to build the package. This was fixed in prophet 1.1 but that version can't be installed along with superset 2.0 due to a version conflict on the holidays package. I just contributed a fix for that conflict in #21091.

[596050]

Just tried @john-bodley's branch, after getting stuck on this issue when trying to use prophet with superset in docker. Initially it did not work for me on the master branch or @john-bodley's. However, after pulling @john-bodley's branch: #24129 and then running

docker exec -it superset_app /bin/bash
# in container
pip uninstall fbprophet pystan
pip --no-cache-dir install pystan==2.19.1.1
pip install --upgrade pip
pip install lunarcalendar tqdm "pystan<3.0" && pip install "prophet>=1.0.1, <1.1"

Forecasting seems to be working consistently.