Skip to content

Vendor the terminfo database for use with base/terminfo.jl #55411

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 8, 2024
Merged

Vendor the terminfo database for use with base/terminfo.jl #55411

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
aa7ecba
Vendor the terminfo database for use with base/terminfo.jl
ararslan Aug 7, 2024
7d0ff66
Test REPL precompilation with `TERM` unset
ararslan Aug 8, 2024
e04ab16
# -*- makefile -*-
ararslan Aug 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -403,6 +403,10 @@ endif
# Install appdata file
mkdir -p $(DESTDIR)$(datarootdir)/metainfo/
$(INSTALL_F) $(JULIAHOME)/contrib/julia.appdata.xml $(DESTDIR)$(datarootdir)/metainfo/
# Install terminal info database
ifneq ($(WITH_TERMINFO),0)
cp -R -L $(build_datarootdir)/terminfo $(DESTDIR)$(datarootdir)
endif

# Update RPATH entries and JL_SYSTEM_IMAGE_PATH if $(private_libdir_rel) != $(build_private_libdir_rel)
ifneq ($(private_libdir_rel),$(build_private_libdir_rel))
Expand Down
4 changes: 4 additions & 0 deletions NEWS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,10 @@ Deprecated or removed
External dependencies
---------------------

- The terminal info database, `terminfo`, is now vendored by default, providing a better
REPL user experience when `terminfo` is not available on the system. Julia can be built
without vendoring the database using the Makefile option `WITH_TERMINFO=0`. ([#55411])

Tooling Improvements
--------------------

Expand Down
11 changes: 10 additions & 1 deletion base/terminfo.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,8 @@ end
Locate the terminfo file for `term`, return `nothing` if none could be found.

The lookup policy is described in `terminfo(5)` "Fetching Compiled
Descriptions".
Descriptions". A terminfo database is included by default with Julia and is
taken to be the first entry of `@TERMINFO_DIRS@`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While @TERMINFO_DIRS@ came up while discussing the manpage (and was helpful to distinguish from the environment variable of the same name), I don't think this should actually be included in the docstring like this since it's a template pattern that's replaced when documentation is built: people who run man terminfo(5) on their system won't see @TERMINFO_DIRS@ mentioned 😅.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually do see it mentioned on macOS:

  • Finally, ncurses searches these compiled-in locations:
    • a list of directories (@TERMINFO_DIRS@), and
    • the system terminfo directory, /usr/share/terminfo (the compiled-in default).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's just a broken docs build 😛 the source text is:

   •   Finally, ncurses searches these compiled-in locations:
       •   a list of directories (@TERMINFO_DIRS@), and
       •   the system terminfo directory, @TERMINFO@ (the compiled-
           in default).

In your example, only @TERMINFO@ is substituted.

On my system, for comparison:

     •   Finally, ncurses searches these compiled‐in locations:
         •   a list of directories (/etc/terminfo:/usr/share/terminfo), and
         •   the system terminfo directory, /usr/share/terminfo

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha, incredible. Well, the function isn't really user-facing anyway; I'm sure a sufficiently motivated individual could decipher what this docstring means. 😅

"""
function find_terminfo_file(term::String)
isempty(term) && return
Expand All @@ -261,15 +262,23 @@ function find_terminfo_file(term::String)
append!(terminfo_dirs,
replace(split(ENV["TERMINFO_DIRS"], ':'),
"" => "/usr/share/terminfo"))
push!(terminfo_dirs, normpath(Sys.BINDIR, DATAROOTDIR, "terminfo"))
Sys.isunix() &&
push!(terminfo_dirs, "/etc/terminfo", "/lib/terminfo", "/usr/share/terminfo")
for dir in terminfo_dirs
if isfile(joinpath(dir, chr, term))
return joinpath(dir, chr, term)
elseif isfile(joinpath(dir, chrcode, term))
return joinpath(dir, chrcode, term)
elseif isfile(joinpath(dir, lowercase(chr), lowercase(term)))
# The vendored terminfo database is fully lowercase to avoid issues on
# case-sensitive filesystems. On Unix-like systems, terminfo files with
# different cases are hard links to one another, so this is still
# correct for non-vendored terminfo, just redundant.
return joinpath(dir, lowercase(chr), lowercase(term))
end
end
return nothing
end

"""
Expand Down
8 changes: 7 additions & 1 deletion deps/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,10 @@ ifeq ($(WITH_NVTX),1)
DEP_LIBS += nvtx
endif

ifneq ($(WITH_TERMINFO),0)
DEP_LIBS += terminfo
endif

# Only compile standalone LAPACK if we are not using OpenBLAS.
# OpenBLAS otherwise compiles LAPACK as part of its build.
# This is useful where one wants to use the vendor BLAS, but
Expand All @@ -197,7 +201,8 @@ DEP_LIBS_STAGED := $(DEP_LIBS)
DEP_LIBS_STAGED_ALL := llvm llvm-tools clang llvmunwind unwind libuv pcre \
openlibm dsfmt blastrampoline openblas lapack gmp mpfr patchelf utf8proc \
objconv mbedtls libssh2 nghttp2 curl libgit2 libwhich zlib p7zip csl \
sanitizers libsuitesparse lld libtracyclient ittapi nvtx JuliaSyntax
sanitizers libsuitesparse lld libtracyclient ittapi nvtx JuliaSyntax \
terminfo
DEP_LIBS_ALL := $(DEP_LIBS_STAGED_ALL)

ifneq ($(USE_BINARYBUILDER_OPENBLAS),0)
Expand Down Expand Up @@ -259,6 +264,7 @@ include $(SRCDIR)/libgit2.mk
include $(SRCDIR)/libwhich.mk
include $(SRCDIR)/p7zip.mk
include $(SRCDIR)/libtracyclient.mk
include $(SRCDIR)/terminfo.mk

# vendored Julia libs
include $(SRCDIR)/JuliaSyntax.mk
Expand Down
2 changes: 2 additions & 0 deletions deps/checksums/terminfo
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
TermInfoDB-v2023.12.9.any.tar.gz/md5/573d9b5adaf6af500e3dfae6e3d15ebf
TermInfoDB-v2023.12.9.any.tar.gz/sha512/e0a5bfe54346f9d5690a840628b329f6fac7375b0d29337bc70813ae3553a72bb397f8034d221c544289e40c4cfc685d5805777b7528f05bbe0123b5905c24a4
43 changes: 43 additions & 0 deletions deps/terminfo.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
## TERMINFO-DB ##
include $(SRCDIR)/terminfo.version

$(SRCCACHE)/TermInfoDB-v$(TERMINFO_VER).any.tar.gz: | $(SRCCACHE)
$(JLDOWNLOAD) $@ https://github.com/JuliaBinaryWrappers/TermInfoDB_jll.jl/releases/download/$(TERMINFO_TAG)/TermInfoDB.v$(TERMINFO_VER).any.tar.gz
touch -c $@

$(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/source-extracted: $(SRCCACHE)/TermInfoDB-v$(TERMINFO_VER).any.tar.gz
$(JLCHECKSUM) $<
rm -rf $(dir $@)
mkdir -p $(dir $@)
$(TAR) -C $(dir $@) --strip-components 1 -xf $<
echo 1 > $@

checksum-terminfo: $(SRCCACHE)/TermInfoDB-v$(TERMINFO_VER).any.tar.gz
$(JLCHECKSUM) $<

$(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/build-compiled: $(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/source-extracted
echo 1 > $@

$(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/build-checked: $(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/build-compiled
echo 1 > $@

define TERMINFO_INSTALL
mkdir -p $2/$$(build_datarootdir)
cp -R $1/terminfo $2/$$(build_datarootdir)
endef
$(eval $(call staged-install, \
terminfo,TermInfoDB-v$(TERMINFO_VER), \
TERMINFO_INSTALL,,,,))

clean-terminfo:
-rm -f $(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/build-compiled

distclean-terminfo:
rm -rf $(SRCCACHE)/TermInfoDB*.tar.gz $(SRCCACHE)/TermInfoDB-v$(TERMINFO_VER) $(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)

get-terminfo: $(SRCCACHE)/TermInfoDB-v$(TERMINFO_VER).any.tar.gz
extract-terminfo: $(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/source-extracted
configure-terminfo: extract-terminfo
compile-terminfo: $(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/build-compiled
fastcheck-terminfo: check-terminfo
check-terminfo: $(BUILDDIR)/TermInfoDB-v$(TERMINFO_VER)/build-checked
3 changes: 3 additions & 0 deletions deps/terminfo.version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# -*- makefile -*-
TERMINFO_VER := 2023.12.9
TERMINFO_TAG := TermInfoDB-v$(TERMINFO_VER)+0
7 changes: 5 additions & 2 deletions stdlib/REPL/test/precompilation.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,11 @@ if !Sys.iswindows()
@testset "No interactive startup compilation" begin
f, _ = mktemp()

# start an interactive session
cmd = `$(Base.julia_cmd()[1]) --trace-compile=$f -q --startup-file=no -i`
# start an interactive session, ensuring `TERM` is unset since it can trigger
# different amounts of precompilation stemming from `base/terminfo.jl` depending
# on the value, making the test here unreliable
cmd = addenv(`$(Base.julia_cmd()[1]) --trace-compile=$f -q --startup-file=no -i`,
Dict("TERM" => ""))
pts, ptm = open_fake_pty()
p = run(cmd, pts, pts, pts; wait=false)
Base.close_stdio(pts)
Expand Down