Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
pd.read_html('<table><tr><td>a</td></tr><tr><td colspan="1000000000">b</td></tr></table>')Issue Description
I am parsing dataframes from varied untrusted HTML sources. Occasionally I'll encounter an HTML table that has a large colspan such as in the example, which causes pandas to allocate memory until the process OOMs.
Related: #17054
Expected Behavior
At the very least, it would be nice to have a column limit to prevent pandas from crashing the program on some non-sensical HTML such as this.
It's possible of course to parse the HTML first and look for colspan attributes that are too big, but this is expensive since the HTML is parsed twice, since I can't pass a parsed tree into read_html(), and it also requires deep knowledge of how pandas is parsing the HTML.
Installed Versions
INSTALLED VERSIONS
commit : 2e218d1
python : 3.10.12.final.0
python-bits : 64
OS : Linux
OS-release : 6.4.11-200.fc38.x86_64
Version : #1 SMP PREEMPT_DYNAMIC Wed Aug 16 17:42:12 UTC 2023
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 1.5.3
numpy : 1.25.2
pytz : 2023.3
dateutil : 2.8.2
setuptools : 68.0.0
pip : 23.2.1
Cython : None
pytest : 7.4.0
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.3
html5lib : 1.1
pymysql : None
psycopg2 : 2.9.6
jinja2 : 3.1.2
IPython : 8.14.0
pandas_datareader: None
bs4 : 4.12.2
bottleneck : None
brotli : None
fastparquet : None
fsspec : 2023.9.0
gcsfs : 2023.9.0
matplotlib : 3.7.2
numba : None
numexpr : None
odfpy : None
openpyxl : 3.1.2
pandas_gbq : None
pyarrow : 12.0.1
pyreadstat : None
pyxlsb : 1.0.10
s3fs : 2023.9.0
scipy : 1.11.1
snappy : None
sqlalchemy : None
tables : None
tabulate : 0.9.0
xarray : None
xlrd : 2.0.1
xlwt : None
zstandard : None
tzdata : None