ImportError: cannot import name 'array_record_module' from 'array_record.python'

[Zcchill]

/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET

Short description
Description of the bug.

Environment information

• Operating System: Win64

• Python version: 3.8

• tensorflow-datasets/tfds-nightly version: 4.8.3

• tensorflow/tf-nightly version:

• Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ?yes

Link to logs

Traceback (most recent call last):
  File "flan/v2/run_example.py", line 10, in <module>
    import seqio
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\seqio\__init__.py", line 18, in <module>
    from seqio.dataset_providers import *
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\seqio\dataset_providers.py", line 34, in <module>
    from seqio import utils
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\seqio\utils.py", line 25, in <module>
    import tensorflow_datasets as tfds
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\tensorflow_datasets\__init__.py", line 43, in <module>
    import tensorflow_datasets.core.logging as _tfds_logging
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\tensorflow_datasets\core\__init__.py", line 22, in <module>
    from tensorflow_datasets.core import community
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\tensorflow_datasets\core\community\__init__.py", line 18, in <module>
    from tensorflow_datasets.core.community.huggingface_wrapper import mock_builtin_to_use_gfile
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\tensorflow_datasets\core\community\huggingface_wrapper.py", line 31, in <module>
    from tensorflow_datasets.core import dataset_builder
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 34, in <module>
    from tensorflow_datasets.core import dataset_info
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\tensorflow_datasets\core\dataset_info.py", line 46, in <module>
    from tensorflow_datasets.core import file_adapters
  File "D:\anaconda\envs\zzc_flan\lib\site-packages\tensorflow_datasets\core\file_adapters.py", line 29, in <module>
    from array_record.python import array_record_module
ImportError: cannot import name 'array_record_module' from 'array_record.python' (D:\anaconda\envs\zzc_flan\lib\site-packages\array_record\python\__init__.py)```


**Expected behavior*
How to solve this bug?

[pppipppi]

need help same problem

[luni-moon]

Same here, here is my stuff:

Short description
Running tfds new <dataset_name> sends the following stacktrace, instead of creating the desired dataset, nor does running anything using tfds work. I provided two stacktraces, with two tfds commands, with similar and/or same output below.

Environment information

• Operating System: Windows 10 Pro, x86_64
• Python version: 3.11.2
• tensorflow-datasets/tfds-nightly version: tensorflow-datasets, 4.9.0
• tensorflow/tf-nightly version: tensorflow, 2.12.0 (Oddly enough, it installed the Intel package on my AMD system...)

Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ?
Yes

Reproduction instructions
Run the command tfds or any subcommand and/or argument to get the results.

Link to logs
Command 1
$ tfds

BirthStoneAI master % $ tfds
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Lukas\scoop\apps\python\current\Scripts\tfds.exe\__main__.py", line 4, in <module>
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\__init__.py", line 43, in <module>
    import tensorflow_datasets.core.logging as _tfds_logging
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\__init__.py", line 22, in <module>
    from tensorflow_datasets.core import community
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\community\__init__.py", line 18, in <module>
    from tensorflow_datasets.core.community.huggingface_wrapper import mock_builtin_to_use_gfile
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\community\huggingface_wrapper.py", line 31, in <module>
    from tensorflow_datasets.core import dataset_builder
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 34, in <module>
    from tensorflow_datasets.core import dataset_info
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\dataset_info.py", line 47, in <module>
    from tensorflow_datasets.core import file_adapters
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\file_adapters.py", line 29, in <module>
    from array_record.python import array_record_module
ImportError: cannot import name 'array_record_module' from 'array_record.python' (C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\array_record\python\__init__.py)
NativeCommandExitException: Program "tfds.exe" ended with non-zero exit code: 1.

Command 2
$ tfds new my_dataset

BirthStoneAI master % $ tfds new my_dataset
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Lukas\scoop\apps\python\current\Scripts\tfds.exe\__main__.py", line 4, in <module>
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\__init__.py", line 43, in <module>
    import tensorflow_datasets.core.logging as _tfds_logging
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\__init__.py", line 22, in <module>
    from tensorflow_datasets.core import community
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\community\__init__.py", line 18, in <module>
    from tensorflow_datasets.core.community.huggingface_wrapper import mock_builtin_to_use_gfile
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\community\huggingface_wrapper.py", line 31, in <module>
    from tensorflow_datasets.core import dataset_builder
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 34, in <module>
    from tensorflow_datasets.core import dataset_info
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\dataset_info.py", line 47, in <module>
    from tensorflow_datasets.core import file_adapters
  File "C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\tensorflow_datasets\core\file_adapters.py", line 29, in <module>
    from array_record.python import array_record_module
ImportError: cannot import name 'array_record_module' from 'array_record.python' (C:\Users\Lukas\scoop\apps\python\current\Lib\site-packages\array_record\python\__init__.py)
NativeCommandExitException: Program "tfds.exe" ended with non-zero exit code: 1.

Expected behavior
Create a new dataset with the desired name, and tfds to work at all.

Additional context
I think the problem is, if the user is running an AMD CPU, pip still installs the Intel package on the AMD system (at least in my case). Here is my output below:

~ $ pip install tensorflow
Collecting tensorflow
  Downloading tensorflow-2.12.0-cp311-cp311-win_amd64.whl (1.9 kB)
Requirement already satisfied: tensorflow-intel==2.12.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow) (2.12.0)
Requirement already satisfied: absl-py>=1.0.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.4.0)
Requirement already satisfied: astunparse>=1.6.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.6.3)
Requirement already satisfied: flatbuffers>=2.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (23.3.3)
Requirement already satisfied: gast<=0.4.0,>=0.2.1 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (0.4.0)
Requirement already satisfied: google-pasta>=0.1.1 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (0.2.0)
Requirement already satisfied: h5py>=2.9.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (3.8.0)
Requirement already satisfied: jax>=0.3.15 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (0.4.8)
Requirement already satisfied: libclang>=13.0.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (16.0.0)
Requirement already satisfied: numpy<1.24,>=1.22 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.23.5)
Requirement already satisfied: opt-einsum>=2.3.2 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (3.3.0)
Requirement already satisfied: packaging in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (23.0)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (3.20.3)
Requirement already satisfied: setuptools in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (65.5.0)
Requirement already satisfied: six>=1.12.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.16.0)
Requirement already satisfied: termcolor>=1.1.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (2.2.0)
Requirement already satisfied: typing-extensions>=3.6.6 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (4.5.0)
Requirement already satisfied: wrapt<1.15,>=1.11.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.14.1)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.53.0)
Requirement already satisfied: tensorboard<2.13,>=2.12 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (2.12.1)
Requirement already satisfied: tensorflow-estimator<2.13,>=2.12.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (2.12.0)
Requirement already satisfied: keras<2.13,>=2.12.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (2.12.0)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow-intel==2.12.0->tensorflow) (0.31.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from astunparse>=1.6.0->tensorflow-intel==2.12.0->tensorflow) (0.40.0)
Requirement already satisfied: ml-dtypes>=0.0.3 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from jax>=0.3.15->tensorflow-intel==2.12.0->tensorflow) (0.0.4)
Requirement already satisfied: scipy>=1.7 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from jax>=0.3.15->tensorflow-intel==2.12.0->tensorflow) (1.10.1)
Requirement already satisfied: google-auth<3,>=1.6.3 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (2.17.1)
Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (1.0.0)
Requirement already satisfied: markdown>=2.6.8 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (3.4.3)
Requirement already satisfied: requests<3,>=2.21.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (2.28.2)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (0.7.0)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (1.8.1)
Requirement already satisfied: werkzeug>=1.0.1 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (2.2.3)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (5.3.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (1.3.1)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (2022.12.7)
Requirement already satisfied: MarkupSafe>=2.1.1 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from werkzeug>=1.0.1->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (2.1.2)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (3.2.2)
Installing collected packages: tensorflow
Successfully installed tensorflow-2.12.0

It starts off by saying it collects tensorflow-2.12.0-cp311-cp311-win_amd64.whl, but then on the next line, it says Requirement already satisfied: tensorflow-intel==2.12.0 in c:\users\lukas\scoop\apps\python\current\lib\site-packages (from tensorflow) (2.12.0)

[NeilGirdhar]

Related google/array_record#44

[luni-moon]

Related google/array_record#44

That looks like it was for an Apple machine, not a Windows. I don't think we use .so files on Windows, but correct me if I am wrong.

[NeilGirdhar]

@Lukas-Batema True, but I don't think this problem is platform specific.

[luni-moon]

@Lukas-Batema True, but I don't think this problem is platform specific.

Yeah, I agree, but there is another file on Windows, that is causing the issue, similar to the UNIX/LINUX .so file(s)

[ferand12]

well, I have a Lenovo with an nvidia quadro T1000 but I have the same issue.

[luni-moon]

I also have a NVIDIA GPU, could that be the issue? Not sure if the OP has a NVIDIA GPU though.

On Wed, Apr 5, 2023 at 15:52, ferand12 ***@***.***> wrote: well, I have a Lenovo with an nvidia quadro T1000 but I have the same issue. — Reply to this email directly, view it on GitHub <#4805 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANI5RBIP3BWCQ6R2YCHJE5DW7XEPDANCNFSM6AAAAAAV7J2TMI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Best Wishes, Luke Batema (JavaScript, C#, DiscordJS, Python, and Front End Developer, Creator of cubelang)

[przem93]

I have Mac os m1. Does somebody know how to resolve that issue?

[luni-moon]

What is the GPU (I assume embedded)

On Wed, Apr 5, 2023 at 17:21, Przemyslaw Ratajczak ***@***.***> wrote: I have Mac os m1. Does somebody know how to resolve that issue? — Reply to this email directly, view it on GitHub <#4805 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANI5RBJOZC7BWKCZBZGEGE3W7XO5TANCNFSM6AAAAAAV7J2TMI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Best Wishes, Luke Batema (JavaScript, C#, DiscordJS, Python, and Front End Developer, Creator of cubelang)

[przem93]

@Lukas-Batema my specification: https://support.apple.com/kb/SP854?locale=pl_PL&viewlocale=en_US

[luni-moon]

@Lukas-Batema my specification: support.apple.com/kb/SP854?locale=pl_PL&viewlocale=en_US

Doesn't look like a GPU error then, it could just be the VRAM/RAM

Memory

Apple M1 Pro Chip

16GB unified memory
Configurable to 32GB (M1 Pro or M1 Max) or 64GB (M1 Max)

Taken from the specs page provided

Can you confirm how much RAM you have?
(To do so, click the Apple icon in the status bar, click About this Mac, then click Memory, IIRC)

[JakeV-1]

Basically get the same error here

Environment information

• Operating System: Windows 10, x86_64
• Ryzen 5 5600x CPU. gtx 760 gpu (not used)
• Python version: 3.9
• tensorflow-datasets version: 4.9.0
• tensorflow version: 2.12.0

Clean install + downgrading tfds to tensorflow-datasets==4.8.3 fixes the error for me.
Clean install + tfds==4.9 still gives the error.