[ros] swap key and list file to trigger rebuild #10272

mikaelarguedas · 2021-05-31T21:26:27Z

The ROS apt repo key expired, a new key has been pushed, for apt to work in the docker images, they need to be rebuilt without cache.
To do so we modified our images to swap the creation of the source.list file and the retrieval of the key from the keyserver.

Another more futureprool approach would be to ADD the key using a fixed URL. If the key changes the cache would be burst and the images rebuilt, otherwise the cache would be kept. Is this an approach that would be acceptable in official images ?

Relates to osrf/docker_images#535

Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>

github-actions · 2021-05-31T21:28:33Z

Diff for 66be924:

diff --git a/_bashbrew-cat b/_bashbrew-cat
index c6de720..229b655 100644
--- a/_bashbrew-cat
+++ b/_bashbrew-cat
@@ -3,12 +3,12 @@ GitRepo: https://github.com/osrf/docker_images.git
 
 Tags: dashing-ros-core, dashing-ros-core-bionic
 Architectures: amd64, arm32v7, arm64v8
-GitCommit: df19ab7d5993d3b78a908362cdcd1479a8e78b35
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/dashing/ubuntu/bionic/ros-core
 
 Tags: eloquent-ros-core, eloquent-ros-core-bionic
 Architectures: amd64, arm32v7, arm64v8
-GitCommit: b3e79c3aef3687b56b3c1052ae38aa7010234834
+GitCommit: 45dbb7bd0bb08303e50ecde4a60e827f7cec0ab0
 Directory: ros/eloquent/ubuntu/bionic/ros-core
 
 Tags: foxy-ros-core, foxy-ros-core-focal
@@ -18,22 +18,22 @@ Directory: ros/foxy/ubuntu/focal/ros-core
 
 Tags: galactic-ros-core, galactic-ros-core-focal
 Architectures: amd64, arm64v8
-GitCommit: 6511d8fc0754616550b7f5ea31a40084c2462938
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/galactic/ubuntu/focal/ros-core
 
 Tags: kinetic-ros-core, kinetic-ros-core-xenial
 Architectures: amd64, arm32v7, arm64v8
-GitCommit: df19ab7d5993d3b78a908362cdcd1479a8e78b35
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/kinetic/ubuntu/xenial/ros-core
 
 Tags: melodic-ros-core, melodic-ros-core-bionic
 Architectures: amd64, arm32v7, arm64v8
-GitCommit: df19ab7d5993d3b78a908362cdcd1479a8e78b35
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/melodic/ubuntu/bionic/ros-core
 
 Tags: melodic-ros-core-stretch
 Architectures: amd64, arm64v8
-GitCommit: df19ab7d5993d3b78a908362cdcd1479a8e78b35
+GitCommit: d017429ffef82c2ae91e5f81a4a60640b2ad6c1b
 Directory: ros/melodic/debian/stretch/ros-core
 
 Tags: noetic-ros-core, noetic-ros-core-focal
@@ -43,12 +43,12 @@ Directory: ros/noetic/ubuntu/focal/ros-core
 
 Tags: noetic-ros-core-buster
 Architectures: amd64, arm64v8
-GitCommit: df19ab7d5993d3b78a908362cdcd1479a8e78b35
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/noetic/debian/buster/ros-core
 
 Tags: rolling-ros-core, rolling-ros-core-focal
 Architectures: amd64, arm64v8
-GitCommit: a5644adacdca4a49faf10221620048175cdd7262
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/rolling/ubuntu/focal/ros-core
 
 Tags: dashing-ros-base, dashing-ros-base-bionic, dashing
@@ -103,7 +103,7 @@ Directory: ros/rolling/ubuntu/focal/ros-base
 
 Tags: dashing-ros1-bridge, dashing-ros1-bridge-bionic
 Architectures: amd64, arm32v7, arm64v8
-GitCommit: 0d38100e2fec914106507817e3122d467617e2ee
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/dashing/ubuntu/bionic/ros1-bridge
 
 Tags: eloquent-ros1-bridge, eloquent-ros1-bridge-bionic
@@ -113,12 +113,12 @@ Directory: ros/eloquent/ubuntu/bionic/ros1-bridge
 
 Tags: foxy-ros1-bridge, foxy-ros1-bridge-focal
 Architectures: amd64, arm64v8
-GitCommit: bf35442257b504eac671418bcb70481807a7fa69
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/foxy/ubuntu/focal/ros1-bridge
 
 Tags: galactic-ros1-bridge, galactic-ros1-bridge-focal
 Architectures: amd64, arm64v8
-GitCommit: 6511d8fc0754616550b7f5ea31a40084c2462938
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/galactic/ubuntu/focal/ros1-bridge
 
 Tags: kinetic-perception, kinetic-perception-xenial
@@ -173,5 +173,5 @@ Directory: ros/noetic/debian/buster/robot
 
 Tags: rolling-ros1-bridge, rolling-ros1-bridge-focal
 Architectures: amd64, arm64v8
-GitCommit: bf35442257b504eac671418bcb70481807a7fa69
+GitCommit: 11c613986e35a1f36fd0fa18b49173e0c564cf1d
 Directory: ros/rolling/ubuntu/focal/ros1-bridge
diff --git a/ros_dashing-ros-core-bionic/Dockerfile b/ros_dashing-ros-core-bionic/Dockerfile
index 95d7bcc..b05284f 100644
--- a/ros_dashing-ros-core-bionic/Dockerfile
+++ b/ros_dashing-ros-core-bionic/Dockerfile
@@ -15,12 +15,12 @@ RUN apt-get update && apt-get install -q -y --no-install-recommends \
     gnupg2 \
     && rm -rf /var/lib/apt/lists/*
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros2/ubuntu bionic main" > /etc/apt/sources.list.d/ros2-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 # setup environment
 ENV LANG C.UTF-8
 ENV LC_ALL C.UTF-8
diff --git a/ros_dashing-ros1-bridge-bionic/Dockerfile b/ros_dashing-ros1-bridge-bionic/Dockerfile
index 14e5d46..b0f620c 100644
--- a/ros_dashing-ros1-bridge-bionic/Dockerfile
+++ b/ros_dashing-ros1-bridge-bionic/Dockerfile
@@ -2,18 +2,18 @@
 # generated from docker_images_ros2/ros1_bridge/create_ros_ros1_bridge_image.Dockerfile.em
 FROM ros:dashing-ros-base-bionic
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros/ubuntu bionic main" > /etc/apt/sources.list.d/ros1-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 ENV ROS1_DISTRO melodic
 ENV ROS2_DISTRO dashing
 
 # install ros packages
 RUN apt-get update && apt-get install -y --no-install-recommends \
-    ros-melodic-ros-comm=1.14.10-1* \
+    ros-melodic-ros-comm=1.14.11-1* \
     ros-melodic-roscpp-tutorials=0.9.3-1* \
     ros-melodic-rospy-tutorials=0.9.3-1* \
     && rm -rf /var/lib/apt/lists/*
diff --git a/ros_eloquent-ros-core-bionic/Dockerfile b/ros_eloquent-ros-core-bionic/Dockerfile
index 8f7f6d8..97e1085 100644
--- a/ros_eloquent-ros-core-bionic/Dockerfile
+++ b/ros_eloquent-ros-core-bionic/Dockerfile
@@ -15,11 +15,11 @@ RUN apt-get update && apt-get install -q -y --no-install-recommends \
     gnupg2 \
     && rm -rf /var/lib/apt/lists/*
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
-RUN echo "deb http://packages.ros.org/ros2/ubuntu bionic main" > /etc/apt/sources.list.d/ros2-latest.list
+RUN echo "deb http://snapshots.ros.org/eloquent/final/ubuntu bionic main" > /etc/apt/sources.list.d/ros2-snapshots.list
+
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 4B63CF8FDE49746E98FA01DDAD19BAB3CBF125EA
 
 # setup environment
 ENV LANG C.UTF-8
diff --git a/ros_foxy-ros1-bridge-focal/Dockerfile b/ros_foxy-ros1-bridge-focal/Dockerfile
index 4425b41..b0f6d56 100644
--- a/ros_foxy-ros1-bridge-focal/Dockerfile
+++ b/ros_foxy-ros1-bridge-focal/Dockerfile
@@ -2,12 +2,12 @@
 # generated from docker_images_ros2/ros1_bridge/create_ros_ros1_bridge_image.Dockerfile.em
 FROM ros:foxy-ros-base-focal
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros/ubuntu focal main" > /etc/apt/sources.list.d/ros1-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 ENV ROS1_DISTRO noetic
 ENV ROS2_DISTRO foxy
 
diff --git a/ros_galactic-ros-core-focal/Dockerfile b/ros_galactic-ros-core-focal/Dockerfile
index d91c1ed..dece158 100644
--- a/ros_galactic-ros-core-focal/Dockerfile
+++ b/ros_galactic-ros-core-focal/Dockerfile
@@ -15,12 +15,12 @@ RUN apt-get update && apt-get install -q -y --no-install-recommends \
     gnupg2 \
     && rm -rf /var/lib/apt/lists/*
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros2/ubuntu focal main" > /etc/apt/sources.list.d/ros2-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 # setup environment
 ENV LANG C.UTF-8
 ENV LC_ALL C.UTF-8
diff --git a/ros_galactic-ros1-bridge-focal/Dockerfile b/ros_galactic-ros1-bridge-focal/Dockerfile
index f367e4a..80cd353 100644
--- a/ros_galactic-ros1-bridge-focal/Dockerfile
+++ b/ros_galactic-ros1-bridge-focal/Dockerfile
@@ -2,12 +2,12 @@
 # generated from docker_images_ros2/ros1_bridge/create_ros_ros1_bridge_image.Dockerfile.em
 FROM ros:galactic-ros-base-focal
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros/ubuntu focal main" > /etc/apt/sources.list.d/ros1-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 ENV ROS1_DISTRO noetic
 ENV ROS2_DISTRO galactic
 
diff --git a/ros_kinetic-ros-core-xenial/Dockerfile b/ros_kinetic-ros-core-xenial/Dockerfile
index d030f14..112c984 100644
--- a/ros_kinetic-ros-core-xenial/Dockerfile
+++ b/ros_kinetic-ros-core-xenial/Dockerfile
@@ -8,12 +8,12 @@ RUN apt-get update && apt-get install -q -y --no-install-recommends \
     gnupg2 \
     && rm -rf /var/lib/apt/lists/*
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros/ubuntu xenial main" > /etc/apt/sources.list.d/ros1-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 # setup environment
 ENV LANG C.UTF-8
 ENV LC_ALL C.UTF-8
diff --git a/ros_melodic-ros-core-bionic/Dockerfile b/ros_melodic-ros-core-bionic/Dockerfile
index a7e316a..1a55916 100644
--- a/ros_melodic-ros-core-bionic/Dockerfile
+++ b/ros_melodic-ros-core-bionic/Dockerfile
@@ -15,12 +15,12 @@ RUN apt-get update && apt-get install -q -y --no-install-recommends \
     gnupg2 \
     && rm -rf /var/lib/apt/lists/*
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros/ubuntu bionic main" > /etc/apt/sources.list.d/ros1-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 # setup environment
 ENV LANG C.UTF-8
 ENV LC_ALL C.UTF-8
diff --git a/ros_melodic-ros-core-stretch/Dockerfile b/ros_melodic-ros-core-stretch/Dockerfile
index 0133531..dd3f8cf 100644
--- a/ros_melodic-ros-core-stretch/Dockerfile
+++ b/ros_melodic-ros-core-stretch/Dockerfile
@@ -8,11 +8,11 @@ RUN apt-get update && apt-get install -q -y --no-install-recommends \
     gnupg2 \
     && rm -rf /var/lib/apt/lists/*
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
-RUN echo "deb http://packages.ros.org/ros/ubuntu stretch main" > /etc/apt/sources.list.d/ros1-latest.list
+RUN echo "deb http://snapshots.ros.org/melodic/final/debian stretch main" > /etc/apt/sources.list.d/ros1-snapshots.list
+
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 4B63CF8FDE49746E98FA01DDAD19BAB3CBF125EA
 
 # setup environment
 ENV LANG C.UTF-8
diff --git a/ros_noetic-ros-core-buster/Dockerfile b/ros_noetic-ros-core-buster/Dockerfile
index 830522f..8f8345c 100644
--- a/ros_noetic-ros-core-buster/Dockerfile
+++ b/ros_noetic-ros-core-buster/Dockerfile
@@ -8,12 +8,12 @@ RUN apt-get update && apt-get install -q -y --no-install-recommends \
     gnupg2 \
     && rm -rf /var/lib/apt/lists/*
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros/ubuntu buster main" > /etc/apt/sources.list.d/ros1-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 # setup environment
 ENV LANG C.UTF-8
 ENV LC_ALL C.UTF-8
diff --git a/ros_rolling-ros-core-focal/Dockerfile b/ros_rolling-ros-core-focal/Dockerfile
index 81a509c..78459ea 100644
--- a/ros_rolling-ros-core-focal/Dockerfile
+++ b/ros_rolling-ros-core-focal/Dockerfile
@@ -15,12 +15,12 @@ RUN apt-get update && apt-get install -q -y --no-install-recommends \
     gnupg2 \
     && rm -rf /var/lib/apt/lists/*
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros2/ubuntu focal main" > /etc/apt/sources.list.d/ros2-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 # setup environment
 ENV LANG C.UTF-8
 ENV LC_ALL C.UTF-8
diff --git a/ros_rolling-ros1-bridge-focal/Dockerfile b/ros_rolling-ros1-bridge-focal/Dockerfile
index b11c4fe..efdf2e0 100644
--- a/ros_rolling-ros1-bridge-focal/Dockerfile
+++ b/ros_rolling-ros1-bridge-focal/Dockerfile
@@ -2,12 +2,12 @@
 # generated from docker_images_ros2/ros1_bridge/create_ros_ros1_bridge_image.Dockerfile.em
 FROM ros:rolling-ros-base-focal
 
-# setup keys
-RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
-
 # setup sources.list
 RUN echo "deb http://packages.ros.org/ros/ubuntu focal main" > /etc/apt/sources.list.d/ros1-latest.list
 
+# setup keys
+RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
+
 ENV ROS1_DISTRO noetic
 ENV ROS2_DISTRO rolling

mikaelarguedas · 2021-06-02T06:44:56Z

The build failure on eloquent is expected and addressed by #10270

yosifkit · 2021-06-02T17:37:56Z

As for ADD to download the key, we don't recommend ADD for any remote URL. We only recommend using ADD for a base image to extract the tar contents. An explicit PR when keys change or are updated is the expected flow so that everything is transparent to users.

Related: https://github.com/docker-library/faq#openpgp--gnupg-keys-and-verification

mikaelarguedas · 2021-06-02T18:41:24Z

Thanks for the clarification and the link.
Our issue was that the key was updated but had the same fingerprint so we were not sure how to ensure a rebuild (as the dockerfile could stay identical but just needed a rebuild without cache). Is there a way to request such a rebuild ? or do we need to submit a PR with a change to the Dockerfiles such as this one ?

yosifkit · 2021-06-03T21:26:52Z

Is there a way to request such a rebuild ?

Not really, we make heavy use of Docker build cache.

or do we need to submit a PR with a change to the Dockerfiles such as this one ?

This type of PR is the one option. These scenarios are quite rare; most software goes out of support before a key is updated. What I'd suggest is ensuring that the key is updated more than 30 days before expiration and then the natural rebuilds that we do every 30 days when Debian and Ubuntu images are updated will just pull in the updated key automatically.

[ros] swap key and list file to trigger rebuild

66be924

Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>

github-actions bot added the library/ros label May 31, 2021

This was referenced Jun 1, 2021

[ros] Swap key active ros images osrf/docker_images#538

Merged

[ros] eloquent and melodic/debian stretch now use snapshots repository #10270

Merged

yosifkit merged commit 12f58b8 into docker-library:master Jun 2, 2021

mikaelarguedas deleted the swap_key_install branch June 2, 2021 18:42

mikaelarguedas mentioned this pull request Jun 2, 2021

ROS GPG Key Expiration Incident osrf/docker_images#535

Closed

bjsowa mentioned this pull request Jun 4, 2021

Dockerfile build fails EuropeanRoverChallenge/ERC-Remote-Navigation-Sim#8

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ros] swap key and list file to trigger rebuild #10272

[ros] swap key and list file to trigger rebuild #10272

mikaelarguedas commented May 31, 2021

github-actions bot commented May 31, 2021

mikaelarguedas commented Jun 2, 2021

yosifkit commented Jun 2, 2021

mikaelarguedas commented Jun 2, 2021

yosifkit commented Jun 3, 2021

[ros] swap key and list file to trigger rebuild #10272

[ros] swap key and list file to trigger rebuild #10272

Conversation

mikaelarguedas commented May 31, 2021

github-actions bot commented May 31, 2021

mikaelarguedas commented Jun 2, 2021

yosifkit commented Jun 2, 2021

mikaelarguedas commented Jun 2, 2021

yosifkit commented Jun 3, 2021