Dataloader预处理将数据转成Tensor，num_workers大于0报错 #31371
Description
- PaddlePaddle 2.0.0
- Ubuntu 18.04
- Python 3.7
Dataloader预处理将数据转成Tensor，num_workers大于0报错，num_workers==0不报错。
2021-03-01 14:07:02,535 - ERROR - DataLoader reader thread raised an exception! 12952
Traceback (most recent call last):
File "test_datasets.py", line 93, in <module>
for i in loader():
File "/home/ps/anaconda3/envs/paddle_dev/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 791, in __next__
data = self._reader.read_next_var_list()
SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception.
[Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:158)
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/ps/anaconda3/envs/paddle_dev/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/home/ps/anaconda3/envs/paddle_dev/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/home/ps/anaconda3/envs/paddle_dev/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 633, in _thread_loop
six.reraise(type(batch), batch)
File "/home/ps/anaconda3/envs/paddle_dev/lib/python3.7/site-packages/six.py", line 703, in reraise
raise value
OSError: (External) Cuda error(3), initialization error.
[Advise: The API call failed because the CUDA driver and runtime could not be initialized. ] (at /paddle/paddle/fluid/platform/gpu_info.cc:274)
import numpy as np
from paddle.io import DataLoader, Dataset
import paddle
class DefaultDataset(Dataset):
def __init__(self):
self.samples = np.arange(1).astype('float32')
def __getitem__(self, index):
fname = self.samples[index]
return paddle.to_tensor(fname)
def __len__(self):
return len(self.samples)
if __name__ == '__main__':
dataset = DefaultDataset()
loader = DataLoader(dataset, batch_size=1, collate_fn=None, num_workers=1, shuffle=False)
for i in loader():
print(i)
Metadata
Metadata
Assignees
Labels
No labels