Skip to content

FEAT: Disable APT-Cacher by default #249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

FEAT: Disable APT-Cacher by default #249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 18 additions & 7 deletions bin/make-base-vm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ LXC=0
VBOX=0
DOCKER=0
DOCKER_IMAGE_HASH=""
APT_CACHER_DEFAULT=0

usage() {
echo "Usage: ${0##*/} [OPTION]..."
Expand All @@ -24,6 +25,7 @@ usage() {
--vbox use VirtualBox instead of kvm
--docker use docker instead of kvm
--docker-image-hash D digest of the docker image to build from
--enable-apt-cacher enable APT Cacher

The MIRROR_HOST environment variable can be used to change the
apt-cacher host. It should be something that both the host and the
Expand Down Expand Up @@ -88,6 +90,10 @@ if [ $# != 0 ] ; then
DOCKER=1
shift 1
;;
--enable-apt-cacher)
APT_CACHER=1
shift 1
;;
--docker-image-digest)
DOCKER_IMAGE_HASH="$2"
shift 2
Expand All @@ -103,13 +109,11 @@ if [ $# != 0 ] ; then
done
fi

if [ $DOCKER = "1" ]; then
MIRROR_DEFAULT=172.17.0.1
else
MIRROR_DEFAULT=127.0.0.1
fi
MIRROR_DEFAULT=127.0.0.1
MIRROR_BASE=http://${MIRROR_HOST:-$MIRROR_DEFAULT}:3142

APT_CACHER=${APT_CACHER:-$APT_CACHER_DEFAULT}

if [ $DISTRO = "ubuntu" ]; then
MIRROR=$MIRROR_BASE/archive.ubuntu.com/ubuntu
SECURITY_MIRROR=$MIRROR_BASE/security.ubuntu.com/ubuntu
Expand Down Expand Up @@ -193,12 +197,19 @@ if [ $DOCKER = "1" ]; then
base_image="$DISTRO:$SUITE"
fi

apt_cacher=""
if [ "$APT_CACHER" = 1 ]; then
apt_cacher="RUN echo 'Acquire::http { Proxy "$MIRROR_BASE"; };' > /etc/apt/apt.conf.d/50cacher"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line introduces a syntax error when using apt-cacher.

Within the 50cacher file of the Dockerfile, will render :

Acquire::http { Proxy http://172.17.0.1:3142; };

instead of

Acquire::http { Proxy "http://172.17.0.1:3142"; };

Need to escape quotes : { Proxy \"$MIRROR_BASE\"; }.

Fixed in 525e33a

fi

# Generate the dockerfile
cat << EOF > $OUT.Dockerfile
FROM $base_image

ENV DEBIAN_FRONTEND=noninteractive
RUN echo 'Acquire::http { Proxy "$MIRROR_BASE"; };' > /etc/apt/apt.conf.d/50cacher
# DELETE ESM Files: W: Failed to fetch https://esm.ubuntu.com/ubuntu/dists/trusty-infra-security/main/binary-amd64/Packages Received HTTP code 403 from proxy after CONNECT
RUN [ -f /etc/apt/sources.list.d/*esm*.list ] && rm /etc/apt/sources.list.d/*esm*.list
$apt_cacher
RUN apt-get update && apt-get --no-install-recommends -y install $addpkg

RUN useradd -ms /bin/bash -U $DISTRO
Expand All @@ -208,7 +219,7 @@ WORKDIR /home/$DISTRO
CMD ["sleep", "infinity"]
EOF

docker build --pull -f $OUT.Dockerfile -t $OUT .
docker build --network host --pull -f $OUT.Dockerfile -t $OUT .
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately, this only works on Linux. if this is desired, it has to be behind a flag or a conditional based on the host OS.

(this also relates to the change above, which points to 127.0.0.1 unconditionally)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to investigate the purpose of this, why he had to specify the host network and the relation with apt-cacher. Try to understand why he had to play with MIRROR_DEFAULT.

Copy link
Contributor

@AbcSxyZ AbcSxyZ Sep 16, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, if I'm right, it's related to the configuration of apt-cacher when you're using docker.

He's just using adding conditionally the following line in make-base-vm :

gitian-builder/bin/make-base-vm

Line 201 in 32b1145

RUN echo 'Acquire::http { Proxy "$MIRROR_BASE"; };' > /etc/apt/apt.conf.d/50cacher

So, I guess if mirrors configurations were working with apt-cacher before, not touching to those configurations when disabling apt-cacher shouldn't be a problem ?

Would keeping MIRROR_DEFAULT and docker network like it was be a solution ?


exit 0
fi
Expand Down
2 changes: 1 addition & 1 deletion libexec/start-target
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,6 @@ case $VMSW in
echo "Gitian-${2}" > var/target.vmname
;;
DOCKER)
docker run -d --name gitian-target base-$SUFFIX:latest > /dev/null
docker run --network host -d --name gitian-target base-$SUFFIX:latest > /dev/null
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comment above regarding this being Linux only

;;
esac