Open
Description
Description
I'm attempting to run the T5 tutorial on an A6000.
I am getting the following error:
~$ python test_kernl.py
~/.local/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory
warn(f"Failed to load image Python extension: {e}")
~/.local/lib/python3.9/site-packages/transformers/models/t5/tokenization_t5_fast.py:156: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.
For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with `truncation is True`.
- Be aware that you SHOULD NOT rely on t5-small automatically truncating your input to 512 when padding/encoding.
- If you want to encode/pad to sequences longer than 512 you can either instantiate this tokenizer with `model_max_length` or pass `max_length` when encoding/padding.
- To avoid this warning, please instantiate this tokenizer with `model_max_length` set to your preferred value.
warnings.warn(
0.10650781099997175
free(): invalid pointer
Aborted (core dumped)
The package versions are listed here:
Package Version
--------------------------------- -----------------------
absl-py 1.4.0
agate 1.6.0
agate-dbf 0.2.0
agate-excel 0.2.3
agate-sql 0.5.2
aiohttp 3.8.3
aiosignal 1.3.1
anyio 3.6.2
apturl 0.5.2
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
asttokens 2.2.1
astunparse 1.6.3
async-timeout 4.0.2
attrs 22.2.0
Automat 0.8.0
awscli 1.18.69
Babel 2.11.0
backcall 0.2.0
beautifulsoup4 4.11.1
black 23.1.0
bleach 5.0.1
blinker 1.4
blis 0.7.9
boto3 1.24.66
botocore 1.27.96
Brlapi 0.7.0
cachetools 5.2.1
catalogue 2.0.8
certifi 2019.11.28
cffi 1.15.1
chardet 3.0.4
charset-normalizer 2.1.1
click 8.1.3
click-completion 0.5.2
click-didyoumean 0.3.0
click-help-colors 0.9.1
cloud-init 22.4.2
cloudpickle 2.1.0
cmake 3.25.2
colorama 0.4.3
comm 0.1.2
command-not-found 0.3
confection 0.0.4
configobj 5.0.6
constantly 15.1.0
cryptography 2.8
csvkit 1.0.2
cupshelpers 1.0
cycler 0.11.0
cymem 2.0.7
Cython 0.29.32
datasets 2.4.0
dbfread 2.0.7
dbus-python 1.2.16
debugpy 1.6.5
decorator 5.1.1
defer 1.0.6
defusedxml 0.7.1
dill 0.3.5.1
distro 1.4.0
distro-info 0.23ubuntu1
docker-pycreds 0.4.0
docutils 0.16
entrypoints 0.3
et-xmlfile 1.0.1
exceptiongroup 1.1.0
executing 1.2.0
fastjsonschema 2.16.2
filelock 3.9.0
flake8 6.0.0
flatbuffers 1.12
fonttools 4.38.0
frozenlist 1.3.3
fsspec 2022.11.0
future 0.18.2
gast 0.4.0
gdown 4.5.1
gitdb 4.0.10
GitPython 3.1.30
google-auth 2.16.0
google-auth-oauthlib 0.4.6
google-pasta 0.2.0
gql 3.0.0a6
gradient 2.0.6
gradient-utils 0.5.0
graphql-core 3.1.7
greenlet 2.0.1
grpcio 1.51.1
h5py 3.7.0
halo 0.0.31
httplib2 0.14.0
huggingface-hub 0.11.1
hyperlink 19.0.0
idna 2.8
imageio 2.24.0
importlib-metadata 6.0.0
incremental 16.10.1
iniconfig 2.0.0
ipykernel 6.15.2
ipython 8.5.0
ipython-genutils 0.2.0
ipywidgets 8.0.2
isodate 0.6.0
isort 5.12.0
jax 0.4.1
jaxlib 0.4.1+cuda11.cudnn82
jdcal 1.0
jedi 0.18.2
Jinja2 3.1.2
jmespath 0.9.4
joblib 1.2.0
json5 0.9.11
jsonify 0.5
jsonpatch 1.22
jsonpointer 2.0
jsonschema 4.17.3
jupyter_client 7.4.8
jupyter-contrib-core 0.4.2
jupyter-contrib-nbextensions 0.7.0
jupyter_core 5.1.3
jupyter-highlight-selected-word 0.2.0
jupyter-nbextensions-configurator 0.6.1
jupyter-server 1.23.5
jupyter-server-mathjax 0.2.6
jupyterlab 3.4.6
jupyterlab-git 0.41.0
jupyterlab-pygments 0.2.2
jupyterlab_server 2.18.0
jupyterlab-widgets 3.0.5
keras 2.9.0
Keras-Preprocessing 1.1.2
kernl 0.2.2
keyring 18.0.1
kiwisolver 1.4.4
langcodes 3.3.0
language-selector 0.1
launchpadlib 1.10.13
lazr.restfulclient 0.14.2
lazr.uri 1.0.3
leather 0.3.3
libclang 15.0.6.1
log-symbols 0.0.14
louis 3.12.0
lxml 4.5.0
macaroonbakery 1.3.1
Markdown 3.4.1
MarkupSafe 2.1.1
marshmallow 2.21.0
matplotlib 3.5.3
matplotlib-inline 0.1.6
mccabe 0.7.0
mistune 2.0.4
more-itertools 4.2.0
mpmath 1.3.0
multidict 6.0.4
multiprocess 0.70.13
murmurhash 1.0.9
mypy-extensions 1.0.0
nbclassic 0.4.8
nbclient 0.7.2
nbconvert 7.2.7
nbdime 3.1.1
nbformat 5.7.3
nest-asyncio 1.5.6
netifaces 0.10.4
networkx 3.0
nltk 3.7
notebook 6.5.2
notebook_shim 0.2.2
numpy 1.23.2
oauthlib 3.1.0
olefile 0.46
opencv-python 4.6.0.66
openpyxl 3.0.3
opt-einsum 3.3.0
packaging 23.0
pandas 1.4.4
pandocfilters 1.5.0
parsedatetime 2.4
parso 0.8.3
pathspec 0.11.0
pathtools 0.1.2
pathy 0.10.1
pexpect 4.6.0
pickleshare 0.7.5
Pillow 9.2.0
pip 22.3.1
platformdirs 2.6.2
pluggy 1.0.0
preshed 3.0.8
progressbar2 4.2.0
prometheus-client 0.9.0
promise 2.3
prompt-toolkit 3.0.36
protobuf 3.19.6
psutil 5.9.4
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 10.0.1
pyasn1 0.4.2
pyasn1-modules 0.2.1
pycairo 1.16.2
pycodestyle 2.10.0
pycparser 2.21
pycups 1.9.73
pydantic 1.9.2
pyflakes 3.0.1
Pygments 2.14.0
PyGObject 3.36.0
PyHamcrest 1.9.0
PyJWT 1.7.1
pymacaroons 0.13.0
pymongo 3.13.0
PyNaCl 1.3.0
pyOpenSSL 19.0.0
pyparsing 3.0.9
pyRFC3339 1.1
pyrsistent 0.15.5
pyserial 3.4
PySocks 1.7.1
pytest 7.2.2
python-apt 2.0.0+ubuntu0.20.4.8
python-dateutil 2.8.2
python-debian 0.1.36ubuntu1
python-distutils-extra 2.39
python-slugify 4.0.0
python-utils 3.4.5
pytimeparse 1.1.5
pytorch-triton 2.0.0+0d7e753227
pytz 2022.7
PyWavelets 1.4.1
pyxdg 0.26
PyYAML 5.4.1
pyzmq 25.0.0
regex 2022.10.31
reportlab 3.5.34
requests 2.28.2
requests-oauthlib 1.3.1
requests-toolbelt 0.10.1
requests-unixsocket 0.2.0
responses 0.18.0
roman 2.0.0
rsa 4.0
s3transfer 0.6.0
scikit-image 0.19.3
scikit-learn 1.1.2
scipy 1.9.1
screen-resolution-extra 0.0.0
seaborn 0.12.0
SecretStorage 2.3.1
Send2Trash 1.8.0
sentence-transformers 2.2.2
sentencepiece 0.1.97
sentry-sdk 1.13.0
service-identity 18.1.0
setproctitle 1.3.2
setuptools 45.2.0
shellingham 1.5.0.post1
shortuuid 1.0.11
simplejson 3.16.0
six 1.14.0
smart-open 6.3.0
smmap 5.0.0
sniffio 1.3.0
sos 4.4
soupsieve 2.3.2.post1
spacy 3.4.1
spacy-legacy 3.0.11
spacy-loggers 1.0.4
spinners 0.0.24
SQLAlchemy 1.4.40
srsly 2.4.5
ssh-import-id 5.10
stack-data 0.6.2
sympy 1.11.1
systemd-python 234
tabulate 0.8.10
tensorboard 2.9.1
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorflow 2.9.2
tensorflow-estimator 2.9.0
tensorflow-io-gcs-filesystem 0.29.0
termcolor 2.2.0
terminado 0.17.1
terminaltables 3.1.10
thinc 8.1.6
threadpoolctl 3.1.0
tifffile 2022.10.10
tinycss2 1.2.1
tokenize-rt 5.0.0
tokenizers 0.12.1
tomli 2.0.1
torch 2.0.0.dev20230128+cu117
torchaudio 0.12.1+cu116
torchvision 0.13.1+cu116
tornado 6.2
tqdm 4.64.1
traitlets 5.8.1
transformers 4.21.3
triton 2.0.0.dev20221202
Twisted 18.9.0
typer 0.4.2
typing_extensions 4.4.0
ubuntu-advantage-tools 27.12
ubuntu-drivers-common 0.0.0
ufw 0.36
unattended-upgrades 0.1
Unidecode 1.1.1
urllib3 1.26.14
wadllib 1.3.3
wandb 0.13.4
wasabi 0.10.1
wcwidth 0.2.5
webencodings 0.5.1
websocket-client 0.57.0
Werkzeug 2.2.2
wheel 0.35.1
widgetsnbextension 4.0.5
wrapt 1.14.1
xgboost 1.6.2
xkit 0.0.0
xlrd 1.1.0
xxhash 3.2.0
yarl 1.8.2
zipp 1.0.0
zope.interface 4.7.1
Steps to reproduce
- Install
kernlusing:
pip install 'git+https://github.com/ELS-RD/kernl' --extra-index-url https://download.pytorch.org/whl/nightly/cu117
# or for local dev, after git clone ...
pip install -e . --extra-index-url https://download.pytorch.org/whl/nightly/cu117
- Try to execute T5 Optimization.
Expected Behavior
Code should execute without error. Speed-up is a plus.
Actual Behavior
Errors.
Your environment
CPU Info:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 106
Model name: Intel(R) Xeon(R) Gold 5315Y CPU @ 3.20GHz
Stepping: 6
CPU MHz: 3200.270
BogoMIPS: 6400.13
Hypervisor vendor: Xen
Virtualization type: full
L1d cache: 384 KiB
L1i cache: 256 KiB
L2 cache: 10 MiB
L3 cache: 96 MiB
NUMA node0 CPU(s): 0-7
OS: Linux psfbhdmgj 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Self-service
- I would be willing to help fix this bug myself.
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
No labels