xmake v2.5.5 released, Support to download and install precompiled packages
xmake is a lightweight cross-platform build tool based on Lua. It uses xmake.lua to maintain project builds. Compared with makefile/CMakeLists.txt, the configuration syntax is more Concise and intuitive, it is very friendly to novices, and you can get started quickly in a short time, allowing users to focus more on actual project development.
In version 2.5.5, we continue to improve the experience of remote package integration, realize the pre-compiled package in the cloud, and then directly download the integrated pre-compiled binary package. This can greatly reduce the installation time of some packages that are very slow to compile.
In addition, in the new version, we have also re-implemented the new version of the local package generation scheme, which fully supports add_requires and add_packages seamlessly. From then on, remote packages and local packages can be maintained in a unified way.
Each time you install a package by the built-in package manager of xmake, you must download the corresponding package source code, and then perform local compilation and installation integration. This is for some large packages that compile very slowly, and some packages that rely on a lot of compilation tools. It will be very slow.
Especially on windows, not only the dependence of the third party package on the compilation environment is more complicated, but also many packages and compilation are very slow, such as boost, openssl and so on.
To this end, we implement cloud pre-compilation of packages based on github action, and pre-compile all commonly used packages, and then store them in build-artifacts under Releases of the repository.
Then, when we install the package, we will automatically download it from the binary image package source to achieve rapid integration (currently only pre-compiled windows packages are supported, and will be gradually released to other platforms in the future).
We will pre-compile various configuration combinations such as plat/arch/MT/MD/static/shared of each package, and accurately pull the packages that users actually need according to the unique buildhash. All compiled products will be compressed and packaged with 7zip, as follows Picture:
Since our pre-compiled products are all placed on github, for Chinese users, considering that access to github is not very stable, we can also use the xmake mirror proxy function to automatically switch the actual download to fastgit and other mirror sites to speed up the download.
We can configure mirror proxy rules through a pac.lua file, for example, access to all github.com domain names is switched to the hub.fastgit.org domain name to speed up downloading packages.
pac.lua configuration:
function mirror(url)
return url:gsub("github.com", "hub.fastgit.org")
endThen we set this pac.lua file, the default path is ~/.xmake/pac.lua, or you can manually configure pac.lua using the specified location.
$ xmake g --proxy_pac=/tmp/pac.luaThen, when we install the package, if we encounter the package source under the github.com domain name, it will automatically switch to the fastgit mirror to accelerate the download when downloading.
$ xrepo install libpng
> curl https://hub.fastgit.org/glennrp/libpng/archive/v1.6.37.zip -o v1.6.37.zipTherefore, all downloads of pre-compiled products will also be speeded up. Of course, more than Fastgit provides github mirror acceleration in China, and users can also switch to other mirror sources, such as cnpmjs.org and so on.
By default, xmake will not actively perform cloud pre-compilation and caching of all packages, which is too time-consuming and labor-intensive. Currently, only the pr is submitted to xmake-repo The official package repository, when a new package is included or the package version is updated, the cloud pre-compilation behavior of the corresponding package will be automatically triggered.
Therefore, if users want to contribute packages to our warehouse, they can basically be precompiled and cached (except for headeronly libraries), and if users do not want to contribute packages, but also want to get the precompilation acceleration of the corresponding package, it is also possible.
Just submit pr to the build branch of the build-artifacts repository, edit build.txt file, just modify the package name and version list that needs to trigger pre-compilation, for example:
build.txt
{
name = "asmjit",
versions = {
"2021.06.27"
}
}As long as pr is merged, it will automatically trigger the pre-compilation behavior, and then generate the final compilation product to releases.
Although we provide a pre-compilation, download and installation method, if users still want to compile and install from source code, we can also manually pass in the --build parameter to the xrepo command to force switch to source code compilation and installation mode.
$ xrepo install --build opensslIn xmake.lua, we can also support source code compilation and installation.
add_requires("openssl", {build = true})If it is not specified, then xmake will automatically try to download and install the precompiled package first.
Our official pre-compiled package repository is at: build-artifacts.
Similarly, we can also configure and add our own pre-compiled warehouse, the way to add is similar:
$ xmake repo --add local-repo git@github.com:xmake-mirror/myrepo-artifacts.gitYou can also add in xmake.lua:
add_repositories("local-repo git@github.com:xmake-mirror/myrepo-artifacts.git")In the new version, we provide a new local package packaging solution that will seamlessly connect add_requires and add_packages.
We can execute the xmake package command to generate the default new version of the packaging format.
$ xmake package
package(foo): build/packages/f/foo generatedIt will generate the file build/packages/f/foo/xmake.lua with the following content:
package("foo")
set_description("The foo package")
set_license("Apache-2.0")
add_deps("add", "sub")
on_load(function (package)
package:set("installdir", path.join(os.scriptdir(), package:plat(), package:arch(), package:mode()))
end)
on_fetch(function (package)
local result = {}
result.links = "foo"
result.linkdirs = package:installdir("lib")
result.includedirs = package:installdir("include")
return result
end)In fact, it uses package() to define and describe local packages, just like remote packages.
The generated directory structure is as follows:
$ tree build/packages/f/foo/
build/packages/f/foo/
├── macosx
│ └── x86_64
│ └── release
│ ├── include
│ │ └── foo.h
│ └── lib
│ └── libfoo.a
└── xmake.luaWe can also use the add_requires/add_repositories interface to seamlessly integrate this package.
add_rules("mode.debug", "mode.release")
add_repositories("local-repo build")
add_requires("foo")
target("bar")
set_kind("binary")
add_files("src/*.cpp")
add_packages("foo")Among them, the add_repositories configuration specifies the warehouse root directory of the local package, and then this package can be referenced through add_requires.
In addition, the generated local package has another feature, which is to support target/add_deps, which automatically associates the dependencies of multiple packages, and automatically connects all dependency links during integration.
Here is the complete test example.
"/usr/bin/xcrun -sdk macosx clang++" -o build/macosx/x86_64/release/bar build/.objs/bar/macosx/x86_64/release/src/main.cpp.o -arch x86_64 -mmacosx-version -min=10.15 -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -stdlib=libc++
-L/Users/ruki/projects/personal/xmake/tests/actions/package/localpkg/bar/build/packages/f/foo/macosx/x86_64/release/lib
-L/Users/ruki/projects/personal/xmake/tests/actions/package/localpkg/bar/build/packages/s/sub/macosx/x86_64/release/lib
-L/Users/ruki/projects/personal/xmake/tests/actions/package/localpkg/bar/build/packages/a/add/macosx/x86_64/release/lib
-Wl,-x -lfoo -lsub -ladd -lzNote: The previous old version of the local packaging format is an early product and will still be retained, but it is not recommended to continue to use. If you want to continue to use it, you can execute the following command to package:
$ xmake package -f oldpkgIn addition to the local package format, xmake package now also supports generating remote packages, so that users can quickly submit them to remote warehouses.
We only need to modify the package format when packaging.
$ xmake package -f remoteHe will also generate packages/f/foo/xmake.lua file.
package("foo")
set_description("The foo package")
set_license("Apache-2.0")
add_deps("add", "sub")
add_urls("https://github.com/myrepo/foo.git")
add_versions("1.0", "<shasum256 or gitcommit>")
on_install(function (package)
local configs = {}
if package:config("shared") then
configs.kind = "shared"
end
import("package.tools.xmake").install(package, configs)
end)
on_test(function (package)
- TODO check includes and interfaces
- assert(package:has_cfuncs("foo", {includes = "foo.h"})
end)Compared with the local package, the package definition configuration has more actual installation logic, as well as the settings of urls and versions,
We can also modify urls, versions and other configuration values through additional parameters, for example:
$ xmake package -f remote --url=https://xxxx/xxx.tar.gz --shasum=xxxxx --homepage=xxxxx`xmake will also read the relevant configuration information from the target's set_license and set_version configurations.
The built-in xrepo package manager command of xmake previously supported searching the built-in packages in the xmake-repo repository.
$ xrepo search zlib "pcr*"
zlib:
-> zlib: A Massively Spiffy Yet Delicately Unobtrusive Compression Library (in xmake-repo)
pcr*:
-> pcre2: A Perl Compatible Regular Expressions Library (in xmake-repo)
-> pcre: A Perl Compatible Regular Expressions Library (in xmake-repo)And now, we can also search for their packages from third-party package managers such as vcpkg, conan, conda and apt, just add the corresponding package namespace, for example:
$ xrepo search vcpkg::pcre
The package names:
vcpkg::pcre:
-> vcpkg::pcre-8.44#8: Perl Compatible Regular Expressions
-> vcpkg::pcre2-10.35#2: PCRE2 is a re-working of the original Perl Compatible Regular Expressions library$ xrepo search conan::openssl
The package names:
conan::openssl:
-> conan::openssl/1.1.1g:
-> conan::openssl/1.1.1h:We know that for the modification of the target file name, we can use set_basename or use the set_filename interface to configure the implementation. The former modifies the name of the xxx part of libxxx.so and the latter can modify the complete file name.
But in some cases, we just want to modify: extension .so, prefix name lib, or adding suffix name such as: libxxx-d.so will be very troublesome, or use set_filename for complete modification.
Now, we newly provide three independent interfaces set_prefixname, set_suffixname and set_extension to configure them more flexibly.
For example, change the default: libtest.so to test.so
target("test")
set_prefixname("")For example, change the default: libtest.so to libtest-d.so
target("test")
set_suffixname("-d")For example, change the default: libtest.so to test.dll
target("test")
set_prefixname("")
set_extension(".dll")In the new version, if the user does not specify the target type in the target setting set_kind, then the default is the binary program.
Therefore, we can achieve smaller configurations, such as:
target("test")
add_files("src/*.c")Compilation of some small projects can be completed in just two lines, or even shorter:
target("test", {files = "src/*.c"})We have also added a new appletvos compilation platform to support the compilation of programs on Apple's TVOS system. All you need is:
$ xmake f -p appletvos
$ xmakeWe can also import and export the configured configuration set to facilitate the rapid migration of the configuration.
$ xmake f --export=/tmp/config.txt
$ xmake f -m debug --xxx=y --export=/tmp/config.txt$ xmake f --import=/tmp/config.txt
$ xmake f -m debug --xxx=y --import=/tmp/config.txt$ xmake f --menu --export=/tmp/config.txt
$ xmake f --menu -m debug --xxx=y --export=/tmp/config.txt$ xmake f --menu --import=/tmp/config.txt
$ xmake f --menu -m debug --xxx=y --import=/tmp/config.txtIn addition, in the new version, we have also added support for the preview version of vs2020.
In the last version, we supported customizing some package configurations by adding the xmake.lua file in the current directory, and then entering a specific package shell environment.
add_requires("zlib 1.2.11")
add_requires("python 3.x", "luajit")$ xrepo env shell
> python --version
> luajit --versionAnd now, we can also configure and load the corresponding toolchain environment in xmake.lua, for example, load the VS compilation environment.
set_toolchains("msvc")- #1421: Add prefix, suffix and extension options for target names
- #1422: Support search packages from vcpkg, conan
- #1424: Set binary as default target kind
- #1140: Add a way to ask xmake to try to download dependencies from a certain package manager
-
#1339: Improve
xmake packageto generate new local/remote packages - Add
appletvosplatform support for AppleTV,xmake f -p appletvos -
#1437: Add headeronly library type for package to ignore
vs_runtime - #1351: Support export/import current configs
- #1454: Support to download and install precompiled image packages from xmake-mirror