Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

gluon.data.DataLoader when set num_workers>0, Exceptions always occurred #21084

Open
CookingANut opened this issue Jun 30, 2022 · 2 comments
Open

Comments

@CookingANut
Copy link

Description

train_iter = gluon.data.DataLoader(
mnist_train.transform_first(transformer),
batch_size,
shuffle=True,
num_workers=4
)

when i set num_workers = 4
seems code can run through, but will appear below exception:

Exception ignored in: <function DataLoader.del at 0x7f1d7fe165e0>
Traceback (most recent call last):
File "/home/daemonh/miniconda3/envs/d2l/lib/python3.8/site-packages/mxnet/gluon/data/dataloader.py", line 720, in del
AttributeError: 'NoneType' object has no attribute 'Pool'
Exception ignored in: <function Pool.del at 0x7f1d7fe84670>
Traceback (most recent call last):
File "/home/daemonh/miniconda3/envs/d2l/lib/python3.8/multiprocessing/pool.py", line 268, in del
File "/home/daemonh/miniconda3/envs/d2l/lib/python3.8/multiprocessing/queues.py", line 362, in put
AttributeError: 'NoneType' object has no attribute 'dumps'

Does it need to focus or just common issue need to ignore, thanks you very much!

Environment

----------Python Info----------
Version : 3.8.13
Compiler : GCC 7.5.0
Build : ('default', 'Mar 28 2022 11:38:47')
Arch : ('64bit', 'ELF')
----------MXNet Info-----------
Version : 1.9.1
Directory : /home/daemonh/miniconda3/envs/d2l/lib/python3.8/site-packages/mxnet
Commit hash file "/home/daemonh/miniconda3/envs/d2l/lib/python3.8/site-packages/mxnet/COMMIT_HASH" not found. Not installed from pre-built package or built from source.
Library : ['/home/daemonh/miniconda3/envs/d2l/lib/python3.8/site-packages/mxnet/libmxnet.so']
Build features:
✔ CUDA
✔ CUDNN
✔ NCCL
✔ CUDA_RTC
✖ TENSORRT
✔ CPU_SSE
✔ CPU_SSE2
✔ CPU_SSE3
✖ CPU_SSE4_1
✖ CPU_SSE4_2
✖ CPU_SSE4A
✖ CPU_AVX
✖ CPU_AVX2
✔ OPENMP
✖ SSE
✖ F16C
✖ JEMALLOC
✔ BLAS_OPEN
✖ BLAS_ATLAS
✖ BLAS_MKL
✖ BLAS_APPLE
✔ LAPACK
✔ MKLDNN
✔ OPENCV
✖ CAFFE
✖ PROFILER
✔ DIST_KVSTORE
✖ CXX14
✖ INT64_TENSOR_SIZE
✔ SIGNAL_HANDLER
✖ DEBUG
✖ TVM_OP
----------System Info----------
Platform : Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.17
system : Linux
release : 5.10.102.1-microsoft-standard-WSL2
version : #1 SMP Wed Mar 2 00:30:59 UTC 2022

@github-actions
Copy link

Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue.
Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly.
If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on contributing to MXNet and our development guides wiki.

@CookingANut
Copy link
Author

@apache/mxnet-committers

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant