Skip to content

BUG: pd.json_normalize outputs meta argument as list #55559

Open
@WillAyd

Description

@WillAyd

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

In [1]: import pandas as pd
rec
In [2]: records = [{"dimensions": ["foo"], "metrics": {"count": [1, 2, 3]}}]

In [3]: pd.json_normalize(records, meta=["dimensions"], record_path=["metrics",
   ...: "count"])
Out[3]: 
   0 dimensions
0  1      [foo]
1  2      [foo]
2  3      [foo]


### Issue Description

The meta argument to `pd.json_normalize` starting returning an object-dtype containing lists sometime between 2.0.5 and 2.1.1 (still exists on main)

### Expected Behavior

Noticed this in 2.1.1; haven't done a full bisect, but in prior versions of pandas you would get a scalar for the `dimensions` column:

In [1]: import pandas as pd
rec
In [2]: records = [{"dimensions": ["foo"], "metrics": {"count": [1, 2, 3]}}]

In [3]: pd.json_normalize(records, meta=["dimensions"], record_path=["metrics",
...: "count"])
Out[3]:
0 dimensions
0 1 foo
1 2 foo
2 3 foo


### Installed Versions

<details>

INSTALLED VERSIONS
------------------
commit              : 76457275a89df6d7609dac99c258b3db1d982437
python              : 3.10.12.final.0
python-bits         : 64
OS                  : Linux
OS-release          : 6.2.0-34-generic
Version             : #34-Ubuntu SMP PREEMPT_DYNAMIC Mon Sep  4 13:06:55 UTC 2023
machine             : x86_64
processor           : x86_64
byteorder           : little
LC_ALL              : None
LANG                : en_US.UTF-8
LOCALE              : en_US.UTF-8

pandas              : 2.2.0dev0+343.g76457275a8
numpy               : 1.24.4
pytz                : 2023.3.post1
dateutil            : 2.8.2
setuptools          : 68.0.0
pip                 : 23.2.1
Cython              : 0.29.36
pytest              : 7.4.2
hypothesis          : 6.87.1
sphinx              : 7.2.6
blosc               : None
feather             : None
xlsxwriter          : 3.1.6
lxml.etree          : 4.9.3
html5lib            : 1.1
pymysql             : 1.4.6
psycopg2            : 2.9.7
jinja2              : 3.1.2
IPython             : 8.16.1
pandas_datareader   : None
bs4                 : 4.12.2
bottleneck          : 1.3.7
dataframe-api-compat: None
fastparquet         : 2023.8.0
fsspec              : 2023.9.2
gcsfs               : 2023.9.2
matplotlib          : 3.7.3
numba               : 0.57.1
numexpr             : 2.8.7
odfpy               : None
openpyxl            : 3.1.2
pandas_gbq          : None
pyarrow             : 13.0.0
pyreadstat          : 1.2.3
python-calamine     : None
pyxlsb              : 1.0.10
s3fs                : 2023.9.2
scipy               : 1.11.3
sqlalchemy          : 2.0.21
tables              : 3.8.0
tabulate            : 0.9.0
xarray              : 2023.9.0
xlrd                : 2.0.1
zstandard           : 0.21.0
tzdata              : 2023.3
qtpy                : None
pyqt5               : None


</details>

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugIO JSONread_json, to_json, json_normalizeNeeds TriageIssue that has not been reviewed by a pandas team member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions