Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install FMT and download ccache on MacOS #10933

Closed
wants to merge 3 commits into from

Conversation

majetideepak
Copy link
Collaborator

@majetideepak majetideepak commented Sep 5, 2024

Brew fmt11 is not supported.
ccache brings in fmt11 from brew.
The brew fmt11 headers are taking precedence over bundled headers.
Bundle fmt and download prebuilt ccache that is statically linked.
We will fix the header include issue in #10920.
Resolves #10936

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 5, 2024
Copy link

netlify bot commented Sep 5, 2024

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit e576f24
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/66da03463178690008432ce4

@majetideepak majetideepak changed the title Bundle FMT on MacOS Install FMT on MacOS Sep 5, 2024
@@ -35,7 +35,7 @@ PYTHON_VENV=${PYHTON_VENV:-"${SCRIPTDIR}/../.venv"}
NPROC=$(getconf _NPROCESSORS_ONLN)

DEPENDENCY_DIR=${DEPENDENCY_DIR:-$(pwd)}
MACOS_VELOX_DEPS="bison boost double-conversion flex fmt gflags glog googletest icu4c libevent libsodium lz4 lzo openssl protobuf@21 simdjson snappy thrift xz xsimd zstd"
MACOS_VELOX_DEPS="bison boost double-conversion flex gflags glog googletest icu4c libevent libsodium lz4 lzo openssl protobuf@21 simdjson snappy thrift xz xsimd zstd"
Copy link
Collaborator

@czentgr czentgr Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be enough. Problem is that line 39 installs ccache. This brings in fmt11 as well. And since we have the ordering issues will still use fmt11 regardless of whether or not it is installed in the CI script.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point! Let's remove ccache for now and add it back in #10920

@@ -93,12 +93,6 @@ jobs:
- name: Build
run: |
cmake --build _build/$BUILD_TYPE -j $NJOBS
ccache -s

- uses: assignUser/stash/save@v1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just do if: false and also add it to the restore step

@majetideepak majetideepak force-pushed the fix-fmt-ci branch 2 times, most recently from 5c9f8dc to 4fe1473 Compare September 5, 2024 18:51
@majetideepak
Copy link
Collaborator Author

@assignUser thanks for the feedback!

Copy link
Collaborator

@czentgr czentgr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: pleas squash the commits.

@assignUser
Copy link
Collaborator

assignUser commented Sep 5, 2024

pleas squash the commits.

Why the bot will squash them anyway before pushing?

@majetideepak majetideepak changed the title Install FMT on MacOS Install FMT and download ccache on MacOS Sep 5, 2024
@majetideepak
Copy link
Collaborator Author

@czentgr, @assignUser suggested we could download ccache instead. I implemented this.

Copy link
Contributor

@kgpai kgpai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fix @majetideepak

@facebook-github-bot
Copy link
Contributor

@kgpai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@kgpai merged this pull request in c722aa8.

@@ -77,6 +77,9 @@ function install_build_prerequisites {
python3 -m venv ${PYTHON_VENV}
fi
source ${PYTHON_VENV}/bin/activate; pip3 install cmake-format regex pyyaml
wget -O ccache.tar.gz https://github.com/ccache/ccache/releases/download/v4.10.2/ccache-4.10.2-darwin.tar.gz
tar -xf ccache.tar.gz
mv ccache-4.10.2-darwin/ccache /usr/local/bin/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmhh, this might be restricted. For the same reason we want INSTALL_PREFIX on macOS.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be okay. We install minio here as well.

Copy link

Conbench analyzed the 1 benchmark run on commit c722aa87.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

yingsu00 added a commit to yingsu00/velox that referenced this pull request Sep 13, 2024
In "Install FMT and download ccache on MacOS (facebookincubator#10933)", wget was used to
download ccache in setup-macos.sh. However, wget is not pre-installed
on MacOs by default, and the following error was thrown:

+ wget -O ccache.tar.gz https://github.com/ccache/ccache/releases/download/v4.10.2/ccache-4.10.2-darwin.tar.gz
../scripts/setup-macos.sh: line 80: wget: command not found
+ tar -xf ccache.tar.gz
tar: Error opening archive: Failed to open 'ccache.tar.gz'

This commit changes wget to curl, which is guaranteed to be pre-installed
on MacOS.
yingsu00 added a commit to yingsu00/velox that referenced this pull request Sep 13, 2024
In "Install FMT and download ccache on MacOS (facebookincubator#10933)", wget was used to
download ccache in setup-macos.sh. However, wget is not pre-installed
on MacOs by default, and the following error was thrown:

+ wget -O ccache.tar.gz https://github.com/ccache/ccache/releases/download/v4.10.2/ccache-4.10.2-darwin.tar.gz
../scripts/setup-macos.sh: line 80: wget: command not found
+ tar -xf ccache.tar.gz
tar: Error opening archive: Failed to open 'ccache.tar.gz'

This commit changes wget to curl, which is guaranteed to be pre-installed
on MacOS.
facebook-github-bot pushed a commit that referenced this pull request Sep 13, 2024
Summary:
In "Install FMT and download ccache on MacOS (#10933)", wget was used to download ccache in setup-macos.sh. However, wget is not pre-installed on MacOs by default, and the following error was thrown:

```
+ wget -O ccache.tar.gz https://github.com/ccache/ccache/releases/download/v4.10.2/ccache-4.10.2-darwin.tar.gz ../scripts/setup-macos.sh: line 80: wget: command not found
+ tar -xf ccache.tar.gz tar: Error opening archive: Failed to open 'ccache.tar.gz'
```

This commit changes wget to curl, which is guaranteed to be pre-installed on MacOS.

Pull Request resolved: #10995

Reviewed By: pedroerp

Differential Revision: D62646279

Pulled By: kevinwilfong

fbshipit-source-id: aaa2365ae013bb012f25556d4741e3a08ea7ef82
@majetideepak majetideepak deleted the fix-fmt-ci branch October 5, 2024 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Broken macos 13 and 14 build
5 participants