Time-series line chart errors with data containing "Nan" value

[michellethomas]

A user reported an error on a chart with data that contains "Nan" values: TypeError: Could not convert "NaN" to numeric. It looks like this is happening in post_processing.

How to reproduce the bug

1. Create a time-series line chart using data that contains "Nan" values. For us this occurred using presto where the user's query was occasionally dividing by 0 and this occurred grouping by a column with sparse data.
2. Run the query

Expected results

Chart loaded
You can see the expected result when switching to the old line chart (the chart shows up correctly using the old line chart).

Actual results

500 error

Environment

(please complete the following information):

• browser type and version:
• superset version: superset version
• python version: python --version
• node.js version: node -v
• any feature flags active:

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset. checked on a version of superset a few days old.
• I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

Stacktrace:

File "/srv/superset-internal/superset-fork/superset/common/query_actions.py", line 186, in get_query_results
return result_func(query_context, query_obj, force_cached)
File "/srv/superset-internal/superset-fork/superset/common/query_actions.py", line 98, in _get_full
payload = query_context.get_df_payload(query_obj, force_cached=force_cached)
File "/srv/superset-internal/superset-fork/superset/common/query_context.py", line 468, in get_df_payload
query_result = self.get_query_result(query_obj)
File "/srv/superset-internal/superset-fork/superset/common/query_context.py", line 274, in get_query_result
df = query_object.exec_post_processing(df)
File "/srv/superset-internal/superset-fork/superset/common/query_object.py", line 408, in exec_post_processing
df = getattr(pandas_postprocessing, operation)(df, **options)
File "/srv/superset-internal/superset-fork/superset/utils/pandas_postprocessing.py", line 145, in wrapped
return func(df, **options)
File "/srv/superset-internal/superset-fork/superset/utils/pandas_postprocessing.py", line 282, in pivot
df = df.pivot_table(
TypeError: Could not convert "NaN" to numeric

[etr2460]

@junlincc fyi for the echarts migration. Looks like this needs to be fixed

[etr2460]

Hey @villebro, do you know the right engineer/poc to tag on echarts migration issues?

[villebro]

@etr2460 can you loop in @kgabryje and @zhaoyongjie and me - now that monorepo finally hit the repo, we've started gradually working on these types of issues (see e.g. #17587). I would almost suggest breaking these up into as small tickets as possible, assigning them priorities, so we can start tackling them one by one. Also, perhaps we can place them on a dedicated project board where we can track progress (there might already be one, I can't remember).

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[rusackas]

I was tempted to close this as stale, but now I'm wondering if anyone can confirm/deny that this is still an issue in modern-era Superset. Thanks in advance!

[sfirke]

@rusackas It's been another 6 months, no activity from reporters since the initial report - I would say let's close as stale rather than invest time trying to reproduce?

[rusackas]

Agreed, closing.