Skip to content

Making local mirror

Jump to bottom
Eugene Lazutkin edited this page Dec 5, 2023 · 5 revisions

Sometimes for technical or security reasons, it is preferable to access precompiled files locally either using a web-hosted mirror or a set of local files. While all of it is described in install from cache, this guide provides simple recipes to set up a local mirror.

Command-line parameters describe how the installer retrieves an artifact.

In the examples below we will assume that we use the default environment variable names to retrieve an artifact for https://github.com/uhop/node-re2 version 1.15.2 running on the Linux x64 platform.

Setting up a web server

Imagine that for security reasons we decided to implement a local mirror with company-approved prebuilt artifacts.

Starting with version 1.3.0 it is possible to host a mirror as an HTTP or HTTPS server.

No mirror

Our artifact will be retrieved from:

https://github.com/uhop/node-re2/releases/download/1.15.2/linux-x64-83.br

Serve locally

We can just override DOWNLOAD_HOST=https://local.mirror/github. In this case, it will be retrieved from:

https://local.mirror/github/uhop/node-re2/releases/download/1.15.2/linux-x64-83.br

As you can see our server at local.mirror will serve as a mirror of github files from /github directory. The rest should be organized as subdirectories:

/github/
   uhop/
      node-re2/
         releases/
            download/
               1.15.2/
                  linux-x64-83.br
                  linux-x64-83.gz
   {user}/
      {repo}/
         releases/
            download/
               {tag}/
                  {name}.br
                  {name}.gz

Different way to structure subdirectories

Sometimes our mirror should be organized differently for a variety of reasons. For example, we want to serve files like that: /third-party/re2/1.15.2/linux-x64-83.br. It can be done with the following settings:

  • DOWNLOAD_HOST=https://local.mirror/third-party/re2
  • DOWNLOAD_SKIP_PATH=1

In this case, our artifact will be retrieved from:

https://local.mirror/third-party/re2/1.15.2/linux-x64-83.br

In order to achieve that, we need to create the following folder structure:

/third-party/re2/
   1.15.2/
      linux-x64-83.br
      linux-x64-83.gz
   {tag}/
      {name}.br
      {name}.gz

No version

While it is not generally recommended, in some cases a developer can decide to fix a version. In this case, versioning will be excessive. Borrowing the previous example we can modify the setting like that:

  • DOWNLOAD_HOST=https://local.mirror/third-party/re2
  • DOWNLOAD_SKIP_PATH=1
  • DOWNLOAD_SKIP_VER=1

In this case, our artifact will be retrieved from:

https://local.mirror/third-party/re2/linux-x64-83.br

Serving from the local filesystem

If ${host} does not start from http:// or https://, it is assumed to be a local path. In this case, it will be retrieved locally.

For example, we already copied pre-approved binary artifacts to /opt/third-party/re2, which is available in our image. By doing so we don't need to open our build system to the internet. It can be done with settings like that:

  • DOWNLOAD_HOST=/opt/third-party/re2
  • DOWNLOAD_SKIP_PATH=1

In this case, our artifact will be retrieved from:

/opt/third-party/re2/1.15.2/linux-x64-83.br

Clone this wiki locally