Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add podman driver #6515

Merged
merged 63 commits into from
Feb 13, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
24b1b99
Add initial podman
medyagh Feb 6, 2020
e8a7137
Add cgroup option for podman and remove resource limit for podman
medyagh Feb 6, 2020
f896450
typo
medyagh Feb 6, 2020
ac55bf7
add podman to linux supported too
medyagh Feb 6, 2020
ef19064
Add podman to IsKic
medyagh Feb 6, 2020
ca6f91a
remove cp -a for podman
medyagh Feb 6, 2020
843fec9
add network inspect for podman
medyagh Feb 6, 2020
2093545
string
medyagh Feb 6, 2020
780a395
podman cp fix
medyagh Feb 6, 2020
f88e96d
chmod podman fix
medyagh Feb 6, 2020
c946018
more comments
medyagh Feb 6, 2020
f57b924
struggling podman cp
medyagh Feb 6, 2020
a77e173
fix inspect ip for podman
medyagh Feb 6, 2020
6c3d07c
fix hard coded docker
medyagh Feb 6, 2020
293585b
refactor all hard-coded var lib path
medyagh Feb 7, 2020
2d49e23
remove white space from ip result
medyagh Feb 7, 2020
f5158a8
Make podman run with sudo
medyagh Feb 7, 2020
5578403
typo
medyagh Feb 7, 2020
1280ed8
warn that podman doesnt allow cpu memory
medyagh Feb 7, 2020
4dd5d16
fix link to correct upcoming docs
medyagh Feb 7, 2020
9187219
add helpers for container ips
medyagh Feb 7, 2020
745ef7e
More acurate error for too slow podman and docker
medyagh Feb 7, 2020
376bbe4
Organize constants file
medyagh Feb 7, 2020
34af2f5
lint
medyagh Feb 7, 2020
97c3584
add github action job for podman
medyagh Feb 7, 2020
a7de263
add podman jenkins script
medyagh Feb 7, 2020
347dfe9
update unit test data with new structure
medyagh Feb 7, 2020
e68aa7b
fix unit test
medyagh Feb 7, 2020
37fd7f4
try a different way to install podman
medyagh Feb 7, 2020
13c3384
try installing github actions podman with sudo
medyagh Feb 7, 2020
444e1c7
try one more time
medyagh Feb 7, 2020
de4a385
set pending for podman jenkins job
medyagh Feb 7, 2020
d5fbf6a
diferent way of installing gopogh
medyagh Feb 7, 2020
f145991
install golang using script for jobs
medyagh Feb 7, 2020
83e6831
dont fail on golang as not needed
medyagh Feb 7, 2020
7dfc144
check install golang +x
medyagh Feb 7, 2020
aae4790
try to fix goopogh installer
medyagh Feb 7, 2020
3710ac2
print podman version in integeration tests
medyagh Feb 7, 2020
80f77ba
resolve docker-env
medyagh Feb 7, 2020
b815ae3
spelling
medyagh Feb 7, 2020
48e0b25
revert back to var lib minikube
medyagh Feb 8, 2020
db30634
run with context timeout
medyagh Feb 8, 2020
ffece53
fix needs root to inlude poxman
medyagh Feb 8, 2020
7339605
fix integration test script for podman
medyagh Feb 8, 2020
aef1374
get rid of var volume
medyagh Feb 8, 2020
4b82ced
enable exec in var for podman
medyagh Feb 8, 2020
d20e66f
try not setting root kubeconif for podman script
medyagh Feb 8, 2020
ed7a636
adpot new configure for podman
medyagh Feb 8, 2020
a44e133
run podman tests non parallel
medyagh Feb 8, 2020
ecdf62d
change test run time for podman to containerd
medyagh Feb 8, 2020
acecdc6
remove sprintf from test files for clarity
medyagh Feb 10, 2020
5255fc7
installer in path based on git
medyagh Feb 10, 2020
d471ccf
delete left over file
medyagh Feb 10, 2020
1b44a8a
podman command runner
medyagh Feb 11, 2020
de19331
add noisy log only for podman
medyagh Feb 11, 2020
14cc32b
podman cp
medyagh Feb 11, 2020
e2aa6ff
lint
medyagh Feb 11, 2020
b30fbd3
podman cp
medyagh Feb 11, 2020
987858e
still fixing podman cp
medyagh Feb 11, 2020
5cfd32c
move kubeconfig up in test script to make it effective
medyagh Feb 11, 2020
3425d9d
correct host path vol for podman
medyagh Feb 12, 2020
fb29982
create dirs for volume car
medyagh Feb 12, 2020
4d227c7
resolve code review comments
medyagh Feb 13, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,37 @@ jobs:
with:
name: none_on_ubuntu_latest_report
path: report
podman_ubuntu_18_04:
runs-on: ubuntu-18.04
medyagh marked this conversation as resolved.
Show resolved Hide resolved
steps:
- uses: actions/checkout@v2
- name: install podman
run: |
. /etc/os-release
sudo sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list"
wget -q https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/xUbuntu_${VERSION_ID}/Release.key -O- | sudo apt-key add -
sudo apt-key add - < Release.key || true
sudo apt-get update -qq
sudo apt-get -qq -y install podman
- name: build binaries
run : |
make minikube-linux-amd64
make e2e-linux-amd64
- name: install gopogh
run: |
cd /tmp
GO111MODULE="on" go get github.com/medyagh/gopogh@v0.0.17 || true
cd -
- name: run integration test
run: |
mkdir -p /tmp/testhome
MINIKUBE_HOME=/tmp/testhome sudo -E ./out/e2e-linux-amd64 -minikube-start-args=--vm-driver=podman -expected-default-driver= -test.timeout=70m -test.v -binary=out/minikube-linux-amd64 2>&1 | tee ./report/testout.txt
- name: generate gopogh report
run: |
export PATH=${PATH}:`go env GOPATH`/bin
go tool test2json -t < ./report/testout.txt > ./report/testout.json || true
gopogh -in ./report/testout.json -out ./report/testout.html -name "docker ubuntu" -repo github.com/kubernetes/minikube/ || true
- uses: actions/upload-artifact@v1
with:
name: podman_on_ubuntu_latest_report
path: report
3 changes: 1 addition & 2 deletions cmd/minikube/cmd/docker-env.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ import (
"github.com/pkg/errors"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"k8s.io/minikube/pkg/drivers/kic"
"k8s.io/minikube/pkg/drivers/kic/oci"
"k8s.io/minikube/pkg/minikube/cluster"
"k8s.io/minikube/pkg/minikube/config"
Expand Down Expand Up @@ -269,7 +268,7 @@ func dockerEnvVars(ec DockerEnvConfig) (map[string]string, error) {
if err != nil {
return nil, errors.Wrapf(err, "get hostbind port for %d", constants.DockerDaemonPort)
}
env[constants.DockerCertPathEnv] = dockerURL(kic.DefaultBindIPV4, port)
env[constants.DockerCertPathEnv] = dockerURL(oci.DefaultBindIPV4, port)
}
return env, nil
}
Expand Down
20 changes: 11 additions & 9 deletions cmd/minikube/cmd/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -744,11 +744,11 @@ func validateUser(drvName string) {

useForce := viper.GetBool(force)

if driver.BareMetal(drvName) && u.Uid != "0" && !useForce {
if driver.NeedsRoot(drvName) && u.Uid != "0" && !useForce {
exit.WithCodeT(exit.Permissions, `The "{{.driver_name}}" driver requires root privileges. Please run minikube using 'sudo minikube --vm-driver={{.driver_name}}'.`, out.V{"driver_name": drvName})
}

if driver.BareMetal(drvName) || u.Uid != "0" {
if driver.NeedsRoot(drvName) || u.Uid != "0" {
return
}

Expand Down Expand Up @@ -812,16 +812,18 @@ func validateFlags(cmd *cobra.Command, drvName string) {
validateDiskSize()
validateMemorySize()

if driver.BareMetal(drvName) {
if viper.GetString(config.MachineProfile) != constants.DefaultMachineName {
exit.WithCodeT(exit.Config, "The 'none' driver does not support multiple profiles: https://minikube.sigs.k8s.io/docs/reference/drivers/none/")
}

if !driver.HasResourceLimits(drvName) { // both podman and none need root and they both cant specify resources
if cmd.Flags().Changed(cpus) {
out.WarningT("The 'none' driver does not respect the --cpus flag")
out.WarningT("The '{{.name}}' driver does not respect the --cpus flag", out.V{"name": drvName})
}
if cmd.Flags().Changed(memory) {
out.WarningT("The 'none' driver does not respect the --memory flag")
out.WarningT("The '{{.name}}' driver does not respect the --memory flag", out.V{"name": drvName})
}
}

if driver.BareMetal(drvName) {
if viper.GetString(config.MachineProfile) != constants.DefaultMachineName {
exit.WithCodeT(exit.Config, "The '{{.name}} driver does not support multiple profiles: https://minikube.sigs.k8s.io/docs/reference/drivers/none/", out.V{"name": drvName})
}

runtime := viper.GetString(containerRuntime)
Expand Down
11 changes: 8 additions & 3 deletions hack/jenkins/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,13 @@
readonly TEST_ROOT="${HOME}/minikube-integration"
readonly TEST_HOME="${TEST_ROOT}/${OS_ARCH}-${VM_DRIVER}-${MINIKUBE_LOCATION}-$$-${COMMIT}"
export GOPATH="$HOME/go"
export KUBECONFIG="${TEST_HOME}/kubeconfig"
export PATH=$PATH:"/usr/local/bin/:/usr/local/go/bin/:$GOPATH/bin"

# installing golang so we could do go get for gopogh
sudo ./installers/check_install_golang.sh "1.13.4" "/usr/local" || true


echo ">> Starting at $(date)"
echo ""
echo "arch: ${OS_ARCH}"
Expand All @@ -42,6 +47,7 @@ echo "uptime: $(uptime)"
# Setting KUBECONFIG prevents the version ceck from erroring out due to permission issues
echo "kubectl: $(env KUBECONFIG=${TEST_HOME} kubectl version --client --short=true)"
echo "docker: $(docker version --format '{{ .Client.Version }}')"
echo "podman: $(sudo podman version --format '{{.Version}}' || true)"
echo "go: $(go version || true)"


Expand Down Expand Up @@ -235,7 +241,6 @@ cleanup_stale_routes || true

mkdir -p "${TEST_HOME}"
export MINIKUBE_HOME="${TEST_HOME}/.minikube"
export KUBECONFIG="${TEST_HOME}/kubeconfig"


# Build the gvisor image so that we can integration test changes to pkg/gvisor
Expand Down Expand Up @@ -319,7 +324,7 @@ touch "${JSON_OUT}"
echo ">> Running go test2json"
go tool test2json -t < "${TEST_OUT}" > "${JSON_OUT}" || true
echo ">> Installing gopogh"
cd /tmp
cd $(mktemp -d)
GO111MODULE="on" go get -u github.com/medyagh/gopogh@v0.0.17 || true
cd -
echo ">> Running gopogh"
Expand Down Expand Up @@ -347,7 +352,7 @@ fi

echo ">> Cleaning up after ourselves ..."
${SUDO_PREFIX}${MINIKUBE_BIN} tunnel --cleanup || true
${SUDO_PREFIX}${MINIKUBE_BIN} delete --all >/dev/null 2>/dev/null || true
${SUDO_PREFIX}${MINIKUBE_BIN} delete --all --purge >/dev/null 2>/dev/null || true
cleanup_stale_routes || true

${SUDO_PREFIX} rm -Rf "${MINIKUBE_HOME}" || true
Expand Down
3 changes: 3 additions & 0 deletions hack/jenkins/linux_integration_tests_docker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,7 @@ JOB_NAME="Docker_Linux"
mkdir -p cron && gsutil -qm rsync "gs://minikube-builds/${MINIKUBE_LOCATION}/cron" cron || echo "FAILED TO GET CRON FILES"
sudo install cron/cleanup_and_reboot_Linux.sh /etc/cron.hourly/cleanup_and_reboot || echo "FAILED TO INSTALL CLEANUP"

# removing possible left over docker containers from previous runs
docker rm -f -v $(docker ps -aq) >/dev/null 2>&1 || true

source ./common.sh
4 changes: 2 additions & 2 deletions hack/jenkins/linux_integration_tests_none.sh
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ docker rm -f $(docker ps -aq) >/dev/null 2>&1 || true
sudo rm -rf /data/*
# Cleanup old Kubernetes configs
sudo rm -rf /etc/kubernetes/*
# Cleanup old minikube files
sudo rm -rf /var/lib/minikube/*
sudo rm -rf /var/lib/minikube/*

# Stop any leftover kubelets
systemctl is-active --quiet kubelet \
&& echo "stopping kubelet" \
Expand Down
41 changes: 41 additions & 0 deletions hack/jenkins/linux_integration_tests_podman.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/bin/bash

# Copyright 2019 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


# This script runs the integration tests on a Linux machine for the KVM Driver

# The script expects the following env variables:
# MINIKUBE_LOCATION: GIT_COMMIT from upstream build.
# COMMIT: Actual commit ID from upstream build
# EXTRA_BUILD_ARGS (optional): Extra args to be passed into the minikube integrations tests
# access_token: The Github API access token. Injected by the Jenkins credential provider.

set -e

OS_ARCH="linux-amd64"
VM_DRIVER="podman"
JOB_NAME="Podman_Linux"

mkdir -p cron && gsutil -qm rsync "gs://minikube-builds/${MINIKUBE_LOCATION}/cron" cron || echo "FAILED TO GET CRON FILES"
sudo install cron/cleanup_and_reboot_Linux.sh /etc/cron.hourly/cleanup_and_reboot || echo "FAILED TO INSTALL CLEANUP"
SUDO_PREFIX="sudo -E "

EXTRA_ARGS="--container-runtime=containerd"

# remove possible left over podman containers
sudo podman rm -f -v $(sudo podman ps -aq) || true

source ./common.sh
1 change: 1 addition & 0 deletions hack/jenkins/minikube_set_pending.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ jobs=(
'KVM_Linux'
'none_Linux'
'Docker_Linux'
'Podman_Linux'
)

# retry_github_status provides reliable github status updates
Expand Down
6 changes: 4 additions & 2 deletions pkg/addons/kubectl.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,13 @@ limitations under the License.
package addons

import (
"fmt"
"os/exec"
"path"

"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/constants"
"k8s.io/minikube/pkg/minikube/vmpath"
)

var (
Expand All @@ -41,7 +43,7 @@ func kubectlCommand(profile string, files []string, enable bool) (*exec.Cmd, err
kubectlAction = "delete"
}

args := []string{"KUBECONFIG=/var/lib/minikube/kubeconfig", kubectlBinary, kubectlAction}
args := []string{fmt.Sprintf("KUBECONFIG=%s", path.Join(vmpath.GuestPersistentDir, "kubeconfig")), kubectlBinary, kubectlAction}
for _, f := range files {
args = append(args, []string{"-f", f}...)
}
Expand All @@ -63,5 +65,5 @@ func kubernetesVersion(profile string) (string, error) {
}

func kubectlBinaryPath(version string) string {
return path.Join("/var/lib/minikube/binaries", version, "kubectl")
return path.Join(vmpath.GuestPersistentDir, "binaries", version, "kubectl")
}
16 changes: 8 additions & 8 deletions pkg/drivers/kic/kic.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ import (
"k8s.io/minikube/pkg/minikube/constants"
)

// Driver represents a kic driver https://minikube.sigs.k8s.io/docs/reference/drivers/kic/
// Driver represents a kic driver https://minikube.sigs.k8s.io/docs/reference/drivers/docker
type Driver struct {
*drivers.BaseDriver
*pkgdrivers.CommonDriver
Expand Down Expand Up @@ -76,15 +76,15 @@ func (d *Driver) Create() error {

// control plane specific options
params.PortMappings = append(params.PortMappings, oci.PortMapping{
ListenAddress: DefaultBindIPV4,
ListenAddress: oci.DefaultBindIPV4,
ContainerPort: constants.APIServerPort,
},
oci.PortMapping{
ListenAddress: DefaultBindIPV4,
ListenAddress: oci.DefaultBindIPV4,
ContainerPort: constants.SSHPort,
},
oci.PortMapping{
ListenAddress: DefaultBindIPV4,
ListenAddress: oci.DefaultBindIPV4,
ContainerPort: constants.DockerDaemonPort,
},
)
Expand Down Expand Up @@ -138,12 +138,12 @@ func (d *Driver) GetIP() (string, error) {

// GetExternalIP returns an IP which is accissble from outside
func (d *Driver) GetExternalIP() (string, error) {
return DefaultBindIPV4, nil
return oci.DefaultBindIPV4, nil
}

// GetSSHHostname returns hostname for use with ssh
func (d *Driver) GetSSHHostname() (string, error) {
return DefaultBindIPV4, nil
return oci.DefaultBindIPV4, nil
}

// GetSSHPort returns port for use with ssh
Expand Down Expand Up @@ -186,7 +186,7 @@ func (d *Driver) GetState() (state.State, error) {

cmd := exec.Command(d.NodeConfig.OCIBinary, "inspect", "-f", "{{.State.Status}}", d.MachineName)
out, err := cmd.CombinedOutput()
o := strings.Trim(string(out), "\n")
o := strings.TrimSpace(string(out))
if err != nil {
return state.Error, errors.Wrapf(err, "get container %s status", d.MachineName)
}
Expand Down Expand Up @@ -222,7 +222,7 @@ func (d *Driver) Remove() error {
}
cmd := exec.Command(d.NodeConfig.OCIBinary, "rm", "-f", "-v", d.MachineName)
o, err := cmd.CombinedOutput()
out := strings.Trim(string(o), "\n")
out := strings.TrimSpace(string(o))
if err != nil {
if strings.Contains(out, "is already in progress") {
log.Warnf("Docker engine is stuck. please restart docker daemon on your computer.", d.MachineName)
Expand Down
Loading