Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download Pkg from GitHub releases. #29615

Merged
merged 2 commits into from
Oct 27, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ all: debug release
# sort is used to remove potential duplicates
DIRS := $(sort $(build_bindir) $(build_depsbindir) $(build_libdir) $(build_private_libdir) $(build_libexecdir) $(build_includedir) $(build_includedir)/julia $(build_sysconfdir)/julia $(build_datarootdir)/julia $(build_datarootdir)/julia/stdlib $(build_man1dir))
ifneq ($(BUILDROOT),$(JULIAHOME))
BUILDDIRS := $(BUILDROOT) $(addprefix $(BUILDROOT)/,base src ui doc deps test test/embedding test/llvmpasses)
BUILDDIRS := $(BUILDROOT) $(addprefix $(BUILDROOT)/,base src ui doc deps stdlib test test/embedding test/llvmpasses)
BUILDDIRMAKE := $(addsuffix /Makefile,$(BUILDDIRS))
DIRS := $(DIRS) $(BUILDDIRS)
$(BUILDDIRMAKE): | $(BUILDDIRS)
Expand Down Expand Up @@ -39,9 +39,6 @@ endif
$(foreach dir,$(DIRS),$(eval $(call dir_target,$(dir))))
$(foreach link,base $(JULIAHOME)/test,$(eval $(call symlink_target,$(link),$(build_datarootdir)/julia,$(notdir $(link)))))

build_defaultpkgdir = $(build_datarootdir)/julia/stdlib/$(shell echo $(VERSDIR))
$(eval $(call symlink_target,$(JULIAHOME)/stdlib,$(build_datarootdir)/julia/stdlib,$(shell echo $(VERSDIR))))

julia_flisp.boot.inc.phony: julia-deps
@$(MAKE) $(QUIET_MAKE) -C $(BUILDROOT)/src julia_flisp.boot.inc.phony

Expand All @@ -56,9 +53,12 @@ ifndef JULIA_VAGRANT_BUILD
endif
endif

julia-deps: | $(DIRS) $(build_datarootdir)/julia/base $(build_datarootdir)/julia/test $(build_defaultpkgdir)
julia-deps: | $(DIRS) $(build_datarootdir)/julia/base $(build_datarootdir)/julia/test
@$(MAKE) $(QUIET_MAKE) -C $(BUILDROOT)/deps

julia-stdlib: | $(DIRS)
@$(MAKE) $(QUIET_MAKE) -C $(BUILDROOT)/stdlib

julia-base: julia-deps $(build_sysconfdir)/julia/startup.jl $(build_man1dir)/julia.1 $(build_datarootdir)/julia/julia-config.jl
@$(MAKE) $(QUIET_MAKE) -C $(BUILDROOT)/base

Expand All @@ -74,10 +74,10 @@ julia-ui-release julia-ui-debug : julia-ui-% : julia-src-%
julia-sysimg : julia-base julia-ui-$(JULIA_BUILD_MODE)
@$(MAKE) $(QUIET_MAKE) -C $(BUILDROOT) $(build_private_libdir)/sys.ji JULIA_EXECUTABLE='$(JULIA_EXECUTABLE)'

julia-sysimg-release : julia-sysimg julia-ui-release
julia-sysimg-release : julia-stdlib julia-sysimg julia-ui-release
@$(MAKE) $(QUIET_MAKE) -C $(BUILDROOT) $(build_private_libdir)/sys.$(SHLIB_EXT)

julia-sysimg-debug : julia-sysimg julia-ui-debug
julia-sysimg-debug : julia-stdlib julia-sysimg julia-ui-debug
@$(MAKE) $(QUIET_MAKE) -C $(BUILDROOT) $(build_private_libdir)/sys-debug.$(SHLIB_EXT)

julia-debug julia-release : julia-% : julia-ui-% julia-sysimg-% julia-symlink julia-libccalltest julia-base-cache
Expand Down Expand Up @@ -527,10 +527,11 @@ source-dist:
full-source-dist: light-source-dist.tmp
# Get all the dependencies downloaded
@$(MAKE) -C deps getall NO_GIT=1
@$(MAKE) -C stdlib getall
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this was supposed to include Pkg in the source tarballs? Apparently it didn't work: the 1.0.2 tarball doesn't contain it. That's annoying for building distribution packages without network access.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm 90% sure I tried this, but maybe I didn't do it from a clean repo. We should be able to re-upload the source tarballs with this included, @ararslan? I'll have a look at it.

Copy link
Member

@ararslan ararslan Nov 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we can re-create and re-upload the source tarballs, it's just a bit annoying since we can't go through the full-source-dist Make target; we'll have to do everything manually. Apologies for not catching this during the release preparation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was talking about light-source-dist, but maybe full-source-dist has the same problem.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be included there? I interpreted https://github.com/JuliaLang/julia/blob/master/Makefile#L516 as only including code from this repo, and thus Pkg is just an external dependency that should only be in the full variant, which it seems to be btw?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, all other external dependencies are libraries that could be installed separately from Julia, and which generally exist as distribution packages. OTC Pkg cannot be used without Julia. Also, in practical terms, if the plan is to move progressively all stdlib modules to separate repos, it would be quite painful for packagers to have to handle all of them manually (and update versions one by one for each new release).


# Create file full-source-dist.tmp to hold all the filenames that go into the tarball
cp light-source-dist.tmp full-source-dist.tmp
-ls deps/srccache/*.tar.gz deps/srccache/*.tar.bz2 deps/srccache/*.tar.xz deps/srccache/*.tgz deps/srccache/*.zip deps/srccache/*.pem >> full-source-dist.tmp
-ls deps/srccache/*.tar.gz deps/srccache/*.tar.bz2 deps/srccache/*.tar.xz deps/srccache/*.tgz deps/srccache/*.zip deps/srccache/*.pem stdlib/srccache/*.tar.gz >> full-source-dist.tmp

# Prefix everything with the current directory name (usually "julia"), then create tarball
DIRNAME=$$(basename $$(pwd)); \
Expand All @@ -543,6 +544,7 @@ clean: | $(CLEAN_TARGETS)
@-$(MAKE) -C $(BUILDROOT)/src clean
@-$(MAKE) -C $(BUILDROOT)/ui clean
@-$(MAKE) -C $(BUILDROOT)/test clean
@-$(MAKE) -C $(BUILDROOT)/stdlib clean-pkg
-rm -f $(BUILDROOT)/julia
-rm -f $(BUILDROOT)/*.tar.gz
-rm -f $(build_depsbindir)/stringreplace \
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
a633a375be6e5649865d8feee23bc177
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
9a16e4932017d0d159bcf65750eeeda1a637a51b6a1a4ebff957a81bc31fadc6db4be69426d6e99118129db9607e8b4d77df894c5df4fa55f88c45ebbb12d7b1
2 changes: 2 additions & 0 deletions stdlib/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/srccache
/Pkg
46 changes: 46 additions & 0 deletions stdlib/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
default: install

.PHONY: default extract-pkg get-pkg clean-pkg getall install

SRCDIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST))))
JULIAHOME := $(abspath $(SRCDIR)/..)
SRCCACHE := $(abspath $(SRCDIR)/srccache)
BUILDDIR := $(SRCCACHE)

include $(JULIAHOME)/Make.inc
include $(JULIAHOME)/deps/tools/common.mk
include $(JULIAHOME)/deps/tools/git-external.mk

VERSDIR := v`cut -d. -f1-2 < $(JULIAHOME)/VERSION`

# Download and extract Pkg
PKG := https://github.com/JuliaLang/Pkg.jl
PKG_TAR_URL = https://api.github.com/repos/JuliaLang/Pkg.jl/tarball/$1
$(eval $(call git-external,Pkg,PKG,,,$(BUILDDIR)))

extract-pkg: $(BUILDDIR)/$(PKG_SRC_DIR)/source-extracted
Pkg: $(BUILDDIR)/$(PKG_SRC_DIR)/source-extracted
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this check for Pkg/source-extracted rather than the file in the build location?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$(call git-external,Pkg,PKG,,,$(BUILDDIR))

This tells git-external to download and extract it into $(BUILDDIR). See this line.

rm -rf $@
cp -r $(BUILDDIR)/$(PKG_SRC_DIR) $@
get-pkg: Pkg
clean-pkg:
-rm -rf Pkg
-rm -rf $(BUILDDIR)/$(PKG_SRC_DIR)

# Generate symlinks to all stdlibs from usr/share/julia/stdlib/vX.Y/
STDLIBS = Base64 CRC32c Dates DelimitedFiles Distributed FileWatching \
Future InteractiveUtils Libdl LibGit2 LinearAlgebra Logging \
Markdown Mmap Printf Profile Random REPL Serialization SHA \
SharedArrays Sockets SparseArrays Statistics SuiteSparse Test Unicode UUIDs

$(foreach pkg, $(STDLIBS), $(eval $(call symlink_target,$(JULIAHOME)/stdlib/$(pkg),$(build_datarootdir)/julia/stdlib/$(shell echo $(VERSDIR)),$(pkg))))
$(eval $(call symlink_target,$(BUILDROOT)/stdlib/Pkg,$(build_datarootdir)/julia/stdlib/$(shell echo $(VERSDIR)),Pkg))

$(build_datarootdir)/julia/stdlib/$(shell echo $(VERSDIR)):
mkdir -p $@

STDLIBS_LINK_TARGETS = $(addprefix $(build_datarootdir)/julia/stdlib/$(shell echo $(VERSDIR))/, $(STDLIBS) Pkg)

install: Pkg $(STDLIBS_LINK_TARGETS)
getall: get-pkg

2 changes: 2 additions & 0 deletions stdlib/Pkg.version
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
PKG_BRANCH = master
PKG_SHA1 = d305e82fd353cb67e8a064800b9972ee1cb7b5e0
1 change: 0 additions & 1 deletion stdlib/Pkg/.codecov.yml

This file was deleted.

5 changes: 0 additions & 5 deletions stdlib/Pkg/.gitignore

This file was deleted.

28 changes: 0 additions & 28 deletions stdlib/Pkg/.travis.yml

This file was deleted.

36 changes: 0 additions & 36 deletions stdlib/Pkg/Compat.toml

This file was deleted.

47 changes: 0 additions & 47 deletions stdlib/Pkg/LICENSE.md

This file was deleted.

21 changes: 0 additions & 21 deletions stdlib/Pkg/Project.toml

This file was deleted.

17 changes: 0 additions & 17 deletions stdlib/Pkg/README.md

This file was deleted.

45 changes: 0 additions & 45 deletions stdlib/Pkg/appveyor.yml

This file was deleted.

1 change: 0 additions & 1 deletion stdlib/Pkg/bin/.gitignore

This file was deleted.

Loading