Skip to content

_struct.cpython-310-darwin.so is not a fat binary #439

Closed
Closed
_struct.cpython-310-darwin.so is not a fat binary#439
Assignees
IvanZosimovpanticmilos
Labels
bugSomething isn't working

Description

@charles-cooper
charles-cooper
opened on Jun 18, 2022

Description:
since actions/python-versions#114, the python binary is universal2. but apparently, some of the .so files are not.
https://github.com/vyperlang/vyper/runs/6945662103?check_suite_focus=true raises the following error

  File "/Users/runner/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/PyInstaller/utils/osx.py", line 312, in binary_to_target_arch
    raise IncompatibleBinaryArchError(f"{display_name} is not a fat binary!")
PyInstaller.utils.osx.IncompatibleBinaryArchError: /Users/runner/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/lib-dynload/_struct.cpython-310-darwin.so is not a fat binary!

i downloaded the build that was used in the action from https://github.com/actions/python-versions/releases/download/3.10.5-2452607758/python-3.10.5-darwin-x64.tar.gz, and inspected the files.
the main binary looks good!

$ file bin/python3.10
bin/python3.10: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>] [arm64:Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>]

however, the library in question does not:

$ file lib/python3.10/lib-dynload/_struct.cpython-310-darwin.so 
lib/python3.10/lib-dynload/_struct.cpython-310-darwin.so: Mach-O 64-bit x86_64 bundle, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL>

Action version:
v4

Platform:

  • Ubuntu
  • macOS
  • Windows

Runner type:

  • Hosted
  • Self-hosted

Tools version:
3.10.5

Activity

  1. charles-cooper
    added
    bugSomething isn't working
    needs triage
    on Jun 18, 2022

  2. bwoodsend commented on Jun 18, 2022

    @bwoodsend
    bwoodsend
    on Jun 18, 2022

    Looks like the entirety of lib/python3.10/ḷib-dynload is single arch.

    ~/D/python-3.10.5-darwin-x64> for file in bin/* **.so **.dylib
                                  echo $file (llvm-lipo -archs $file 2>/dev/null)
                              end
bin/2to3
bin/2to3-3.10
bin/idle3
bin/idle3.10
bin/pip3
bin/pip3.10
bin/pydoc3
bin/pydoc3.10
bin/python3 x86_64 arm64 
bin/python3-config
bin/python3-intel64 x86_64 
bin/python3.10 x86_64 arm64 
bin/python3.10-config
bin/python3.10-intel64 x86_64 
lib/python3.10/lib-dynload/_asyncio.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_bisect.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_blake2.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_bz2.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_cn.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_hk.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_iso2022.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_jp.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_kr.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_tw.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_contextvars.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_crypt.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_csv.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_ctypes.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_ctypes_test.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_curses.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_curses_panel.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_datetime.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_dbm.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_decimal.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_elementtree.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_gdbm.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_hashlib.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_heapq.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_json.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_lsprof.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_lzma.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_md5.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_multibytecodec.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_multiprocessing.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_opcode.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_pickle.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_posixshmem.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_posixsubprocess.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_queue.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_random.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_scproxy.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sha1.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sha3.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sha256.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sha512.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_socket.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sqlite3.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_ssl.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_statistics.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_struct.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testbuffer.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testcapi.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testimportmultiple.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testinternalcapi.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testmultiphase.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_tkinter.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_uuid.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_xxsubinterpreters.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_xxtestfuzz.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_zoneinfo.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/array.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/audioop.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/binascii.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/cmath.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/fcntl.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/grp.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/math.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/mmap.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/nis.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/pyexpat.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/readline.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/resource.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/select.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/syslog.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/termios.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/unicodedata.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/xxlimited.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/xxlimited_35.cpython-310-darwin.so x86_64 
lib/python3.10/ḷib-dynload/zlib.cpython-310-darwin.so x86_64 
lib/libpython3.10.dylib x86_64 arm64
  3. panticmilos
    removed
    needs triage
    on Jun 19, 2022

  4. panticmilos commented on Jun 19, 2022

    @panticmilos
    panticmilos
    on Jun 19, 2022
    Contributor

    Hi @charles-cooper,

    Thank you for submitting the issue, we will take look into it.

  5. panticmilos
    self-assigned this
    on Jun 21, 2022
  6. panticmilos
    assigned
    IvanZosimov
    on Jun 29, 2022

  7. charles-cooper commented on Aug 23, 2022

    @charles-cooper
    charles-cooper
    on Aug 23, 2022
    Author

    @panticmilos @bwoodsend bump -- any news on this?

  8. panticmilos commented on Sep 14, 2022

    @panticmilos
    panticmilos
    on Sep 14, 2022
    Contributor

    Hi @charles-cooper, we have reverted logic here actions/python-versions#175, so I am going to close the issue.

    And regarding this 3.10.5 version, it's not going to be using universal2 builds anyway since this PR actions/python-versions#184 will affect versions from 3.11.0-beta.1

    Feel free to continue the conversation, reopen the issue, or open a new issue.

  9. panticmilos
    closed this as completedon Sep 14, 2022

  10. charles-cooper commented on Sep 14, 2022

    @charles-cooper
    charles-cooper
    on Sep 14, 2022
    Author

    @panticmilos i'm not sure i am following what the resolution is here. so universal2 builds of python3 are no longer allowed?

  11. IvanZosimov commented on Sep 15, 2022

    @IvanZosimov
    IvanZosimov
    on Sep 15, 2022
    Contributor

    Hi, @charles-cooper 👋 Let me clarify the situation here - now python builds in universal2 are unavailable, because making them from source code leads to many unexpected problems as you, for example, mentioned in this issue. So, it was decided not to build python in universal2 and this PR revert the logic of building python. Now, we are going not to build python for macOS in universal2 on our own, but use prebuilt packages right from the python dist. This PR should add this functionality, and starting from the 3.11.0-beta.1 you will be able to use unversal2 python .

  12. charles-cooper
    added 2 commits that reference this issue on May 7, 2023

    commit message

    d8e72b1

    chore: update published build to python 3.11 (#3385)

    895fbf2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions