Update build-release.yml

[aliking]

Adds back in the mac arm build. @warren-animoto I'm hoping this will be as simple as this, I just re-instated this value that you removed a couple of commits ago.

[warren-animoto]

I think you'll probably need to add this platform back into the list here:

sharp-libvips/build.sh

Line 104 in 883eb62

for flavour in linux-x64 linuxmusl-x64 linux-armv6 linux-armv7 linux-arm64v8 linuxmusl-arm64v8 linux-s390x; do

[warren-animoto]

Oh wait, that's just the linux ones, so never mind...

[warren-animoto]

Makes sense. We'll have to make a new release in this repo so that mediamoto can pull it down by release/tag name (currently 8.15.2).

We should probably do some mediamoto testing (with HEIC and other files) before we push up the updated install_custom_libvips.sh file in that repo because we'll be grabbing this new sharp version in production as well as locally.

[megantaite]

My hope with cherry picking commits (aside from getting us more up-to-date) was that the build errors were:

patching file libvips/foreign/heifsave.c
Hunk #1 FAILED at 501.
Hunk #2 succeeded at 730 (offset 79 lines).
Hunk #3 FAILED at 689.
Hunk #4 succeeded at 821 (offset 79 lines).
2 out of 4 hunks FAILED -- saving rejects to file libvips/foreign/heifsave.c.rej
Error: Process completed with exit code 1.

and the heif-related patch was updated in this commit: 3f9da07
Also noteworthy - this other commit was the only commit that struck me as a change from our HEIC support: 49b1d66 but I believe I kept HEIC changes by making sure we still have -DWITH_LIBDE265=1 ... 🤔

which also gives us a new version. waiting to see if that passes... 👀 and inclined to pull in more changes

So far only wasm has failed... fingers crossed on arm. Happy to delete wasm builds and leave it here if the rest passes... 😅

On macos 12 ones taking forever...
macOS runners — especially older ones like macos-12 — are in super high demand and limited in availability on GitHub-hosted runners. So it's just sitting in the queue until one frees up.

& we're not far from the macos-13 commit... 🤔 but also we won't run this often... 👀

[megantaite]

GitHub Actions is starting the deprecation process for macOS 12. While the image is being deprecated, You may experience longer queue times during peak usage hours. Deprecation will begin on 10/7/24 and the image will be fully unsupported by 12/3/24 for GitHub and by 01/13/25 for ADO.

Oh 😁 to the latest macos-13 commit we go!

[megantaite]

Hmm.. arm failure seems related to the heic fix and & we're already on the latest for de265 (i.e. the fix) 🤔

libtool: link: clang++ -fno-stack-check -Werror=unguarded-availability-new -target arm64-apple-macos11 -Os -fPIC -Werror=return-type -Werror=unused-result -Werror=reorder -std=gnu++11 -DDE265_LOG_ERROR -Wl,-dead_strip -o .libs/block-rate-estim block_rate_estim-block-rate-estim.o  -L/Users/runner/work/sharp-libvips/sharp-libvips/target/lib ../libde265/.libs/libde265.dylib -lstdc++
ld: warning: ignoring duplicate libraries: '-lc++'
ld: warning: ignoring file '/Users/runner/work/sharp-libvips/sharp-libvips/deps/de265/libde265/.libs/libde265.0.dylib': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring duplicate libraries: '-lc++'
ld: warning: ignoring file '/Users/runner/work/sharp-libvips/sharp-libvips/deps/de265/libde265/.libs/libde265.0.dylib': found architecture 'x86_64', required architecture 'arm64'
ld: Undefined symbols:
  MSE(unsigned char const*, int, unsigned char const*, int, int, int), referenced from:
      _main in yuv_distortion-yuv-distortion.o
  PSNR(double), referenced from:
      _main in yuv_distortion-yuv-distortion.o
      _main in yuv_distortion-yuv-distortion.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [yuv-distortion] Error 1
make[2]: *** Waiting for unfinished jobs....
libtool: link: clang++ -fno-stack-check -Werror=unguarded-availability-new -target arm64-apple-macos11 -Os -fPIC -Werror=return-type -Werror=unused-result -Werror=reorder -std=gnu++11 -DDE265_LOG_ERROR -Wl,-dead_strip -o .libs/tests tests-tests.o  -L/Users/runner/work/sharp-libvips/sharp-libvips/target/lib ../libde265/.libs/libde265.dylib -lstdc++
ld: warning: ignoring duplicate libraries: '-lc++'
ld: warning: ignoring file '/Users/runner/work/sharp-libvips/sharp-libvips/deps/de265/libde265/.libs/libde265.0.dylib': found architecture 'x86_64', required architecture 'arm64'
make[1]: *** [install-recursive] Error 1
make: *** [install-strip] Error 2

Unfortunately running locally with ./build.sh 8.16.0 darwin-arm64v8 (had to run brew install meson first) gives me a different error

-- Installing: /Users/megantaite/animoto/stack/applications/sharp-libvips/target/lib/cmake/libheif/libheif-config-version.cmake
CMake Warning:
  No source or binary directory provided.  Both will be assumed to be the
  same as the current working directory, but note that this warning will
  become a fatal error in future CMake releases.


CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 has been removed from CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.

  Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.

🤔

[aliking]

I'm assuming that the way to read this error is that when we build libde265, that we're accidentally building it for x86_64. You might be able to fix that by setting target_cpu to something that begins with arm if the DARWIN_ARM is true. But why that, why now?

Alternatively... dunno if this would massively complicate things, but I saw that ARM mac runners are available https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
I'm wondering if instead of cross-compiling for arm in macos-13, we could just compile on arm in macos-14. Although, I'm unclear if there's cross-compilation settings that would need unpicking. It's probably not as simple as just changing the github runner.

Also/related of interest:

sharp-libvips/build.sh

Lines 60 to 69 in 883eb62

	if [ $PLATFORM = "darwin-arm64v8" ]; then
	# ARM64 builds work via cross compilation from an x86_64 machine
	export CHOST="aarch64-apple-darwin"
	export RUST_TARGET="aarch64-apple-darwin"
	export FLAGS+=" -target arm64-apple-macos11"
	# macOS 11 Big Sur is the first version to support ARM-based macs
	export MACOSX_DEPLOYMENT_TARGET="11.0"
	# Set SDKROOT to the latest SDK available
	export SDKROOT=$(xcrun -sdk macosx --show-sdk-path)
	fi

-- targeting arm64-apple-macos11

[aliking]

Sorry - meant to post why I thought about target_cpu - it's this bit in libde265

[megantaite]

I'm wondering if instead of cross-compiling for arm in macos-13, we could just compile on arm in macos-14. Although, I'm unclear if there's cross-compilation settings that would need unpicking. It's probably not as simple as just changing the github runner.

Worth a try anyways 😅

Lots of changes since approved c:

[aliking]

I'm assuming we've ended up with a bunch of redundant stuff in here from what you've pulled in from upstream, and I think that's probably fine - this fork always had some unused stuff in it. But I think you can probably delete these lines that were added. @warren-animoto does that match your original forking - just build for linux-x64?

[warren-animoto]

Yes, see 4fe02d8

Apparently I worked directly in the main branch, so the last few commits there are the tweaks I had to do to get this building again last time.

[megantaite]

If I'm understanding next steps correctly - it's:

1. Get this reviewed/approved/merged (I plan to squash ONLY our commits, NOT squashing the ones from upstream 🤔 oh jk, this button doesn't say "Squash and merge".... so I think the best option is to not squash commits at all)
2. After this is merged:

git checkout master
git pull origin master
git tag v8.16.0-heic
git push origin v8.16.0-heic

3. Go to the release tab in this repo and use the info above to make a new release & upload all the assets I get from... somewhere I'm hoping is obvious once this is merged 😁 I didn't need to do anything for this step - the builds ran again and things auto-magically added to the new release as they passed https://github.com/animoto/sharp-libvips/releases
4. Go to mediamoto and update all instances of libvips version being 8.15.2 to 8.16.0
5. Do some local mediamoto testing with 8.16.0, including HEIC uploads, attempt to unskip media test, deploy to QA, etc.
6. Go to stackimoto and attempt to dev update w/ installing the arm libvips
7. Merge mediamoto PR
8. Merge stackimoto PR

🤔

[warren-animoto]

If I'm understanding next steps correctly - it's:

Here's the stackimoto change I did last time - I assume we'll need to pass either darwin-64 or darwin-arm64v8 to the install_custom_libvips script depending on the architecture (in addition to revving the version number):
https://github.com/animoto/stackimoto/pull/338/files

[aliking]

That sounds correct to me, I've never really seen the important difference between tags and releases..
You also may have to update the install_custom_libvips.sh call in Dockerfile to handle if there ends up being something different about the download url for linux.

great job pushing this through despite the horrible dev loop!

I'd approve, but github thinks I shouldn't be allowed to.

[megantaite]

Here's the stackimoto change I did last time - I assume we'll need to pass either darwin-64 or darwin-arm64v8 to the install_custom_libvips script depending on the architecture (in addition to revving the version number):
https://github.com/animoto/stackimoto/pull/338/files

Thanks! @aliking added this some time ago...

- name: Install custom libvips (x64)
  shell: "{{ ansible_env.SHELL }} -lc 'NODENV_VERSION={{ node_version.stdout }} ANIMOTO_NPM_TOKEN={{ npm.token }} ./script/util/install_custom_libvips.sh darwin-x64' chdir={{ project.path }}"
  when: is_target_branch and arch == 'x64'
  tags: mediamoto

- name: Install custom libvips (arm64)
  shell: "{{ ansible_env.SHELL }} -lc 'NODENV_VERSION={{ node_version.stdout }} ANIMOTO_NPM_TOKEN={{ npm.token }} ./script/util/install_custom_libvips.sh darwin-arm64v8' chdir={{ project.path }}"
  when: is_target_branch and arch == 'arm64'
  tags: mediamoto

(i think i only modified the when statement) so I am hoping this will work as it is now 🤔