Skip to content

Generate custom source tarball #2248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 5, 2021
Merged

Generate custom source tarball #2248

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8fdc885
Generate source tarball
jneira Sep 29, 2021
c49b756
Update release doc
jneira Sep 29, 2021
baa7bf3
Checkout code and create subdir
jneira Sep 30, 2021
2737062
Separate src-tar in a new job
jneira Sep 30, 2021
3063833
Merge branch 'master' into gen-src-tarball
mergify[bot] Oct 4, 2021
22f2cfc
Merge branch 'master' into gen-src-tarball
mergify[bot] Oct 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 35 additions & 4 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ on:
types: [created]

jobs:

build:
runs-on: ${{ matrix.os }}

Expand Down Expand Up @@ -156,9 +155,41 @@ jobs:
with:
name: haskell-language-server-wrapper-${{ runner.OS }}${{env.EXE_EXT}}.${{ steps.compress_wrapper_binary.outputs.extension }}
path: ${{ steps.compress_wrapper_binary.outputs.path }}

# generates a custom tarball with sources, used by `ghcup compile hls`
src-tar:
needs: build
runs-on: ubuntu-18.04

steps:
- uses: actions/checkout@v2
with:
submodules: true

- name: "Create source tarball"
run: |
mkdir src-dist
git archive --prefix=haskell-language-server-${{ github.event.release.tag_name }}/ \
--format=tar.gz \
-o src-dist/haskell-language-server.tar.gz \
HEAD

- uses: actions/upload-release-asset@v1.0.2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ github.event.release.upload_url }}
asset_path: src-dist/haskell-language-server.tar.gz
asset_name: haskell-language-server-${{ github.event.release.tag_name }}-src.tar.gz
asset_content_type: application/gzip

# this generates .gz tarfiles containing all the GHC versions for
# macOS and Linux, used by ghcup
- uses: actions/upload-artifact@v2
with:
name: haskell-language-server-${{ github.event.release.tag_name }}-src.tar.gz
path: src-dist/haskell-language-server.tar.gz

# this generates .gz tarfiles containing binaries for all GHC versions and OS's
# used by `ghcup install hls`
tar:
needs: build
runs-on: ubuntu-18.04
Expand Down Expand Up @@ -198,7 +229,7 @@ jobs:
path: bin/haskell-language-server.tar.gz

sha256sum:
needs: tar
needs: [tar, src-tar]
runs-on: ubuntu-18.04
steps:
- uses: actions/download-artifact@v2
Expand Down
47 changes: 24 additions & 23 deletions docs/contributing/releases.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,18 +57,24 @@ only one wrapper binary is created per platform, and it should be built with the
most recent GHC version.

### ghcup
It also creates a `haskell-language-server-${os}-${hlsVersion}.tar.gz` tarball with
the binaries for *all* supported ghc versions, to help downstream publishers in
the distribution of the release. The most prominent publisher using them is `ghcup`.
The `tar` job in the workflow file automates the creation of this.

Once all these binaries are present
It creates a `haskell-language-server-${os}-${hlsVersion}.tar.gz` tarball with
the binaries for *all* supported ghc versions and a custom source tarball to help
downstream publishers in the distribution of the release.

The most prominent publisher using them is `ghcup`.

### checksums

The sha256 checksum of all artifacts are listed in the `SHA256SUMS` release file.

## Distributable binaries

In order to compile a hls binary on one machine and have it run on another, you
need to make sure there are **no hardcoded paths or data-files**.

### ghc libdir

One noteable thing which cannot be hardcoded is the **GHC libdir** – this is
a path to `/usr/local/lib/ghc` or something like that, which was previously
baked in at compile-time with ghc-paths. Note that with static binaries we
Expand All @@ -78,22 +84,24 @@ Therefore, hie-bios provides `getGhcRuntimeLibDir` to obtain this path on the fl
by consulting the cradle.

### Static binaries

We use the word "distributable" here because technically only the Linux builds
are static. They are built by passing `--enable-executable-static` to cabal.
Static binaries don't really exist on macOS, and there are issues with
proprietary code being linked in on Windows. However, the `.dylib`s linked on
macOS are all already provided by the system:

```
```bash
$ objdump -macho --dylibs-used haskell-language-server
haskell-language-server:
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
/usr/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
/usr/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0)
```

## The GitHub Actions workflow

It just kicks off a matrix of jobs varying across GHC versions and OSs, building
the binaries with Cabal and extracting them from the dist-newstyle directory.
The binaries are built with -O2.
Expand All @@ -120,19 +128,12 @@ its secure distribution using their hashes. We should only add new ones.*

To manually upload a new binary we should:

- Add the new tar/zip following the name conventions of existing ones
- `haskell-language-server-${os}-${ghcVersion}.gz` for `Linux` and `macOS` and `haskell-language-server-Windows-${ghcVersion}.exe.zip` for `Windows`
- the binary inside the gz file is named `haskell-language-server-${ghcVersion}` (with the `.exe` extension for `Windows`). Note that the binary name does not contain the `${os}` part.
- Add the executable to the existing tar `haskell-language-server-${os}-${ghcVersion}.tar.gz` *locally* and upload it under a new name `haskell-language-server-${os}-${ghcVersion}-rev${n}.tar.gz` following the same schema for the binary as the previous one.
- `-rev${n}` is the next revision number of the tarball, starting at 1.
- we should contact users of the tarball (particularly ghcup) to notify the change

### Windows
Currently building HLS with GHC 8.8.x on Windows is very flakey and so
is not included by default in the GitHub Actions build matrix. Instead
they need to be built and uploaded manually. See [this
PR](https://github.com/haskell/haskell-language-server/issues/276) for
more details
* Add the new tar/zip following the name conventions of existing ones
* `haskell-language-server-${os}-${ghcVersion}.gz` for `Linux` and `macOS` and `haskell-language-server-Windows-${ghcVersion}.exe.zip` for `Windows`
* the binary inside the gz file is named `haskell-language-server-${ghcVersion}` (with the `.exe` extension for `Windows`). Note that the binary name does not contain the `${os}` part.
* Add the executable to the existing tar `haskell-language-server-${os}-${ghcVersion}.tar.gz` *locally* and upload it under a new name `haskell-language-server-${os}-${ghcVersion}-rev${n}.tar.gz` following the same schema for the binary as the previous one.
* `-rev${n}` is the next revision number of the tarball, starting at 1.
* we should contact users of the tarball (particularly ghcup) to notify the change

## Hackage release workflow

Expand Down