Able to quick create a HA cluster by kube-up.sh centos provider by Shawyeok · Pull Request #39462 · kubernetes/kubernetes

Shawyeok · 2017-01-05T09:03:43Z

Make kube-up.sh centos provider support quick create a HA cluster, as I said above #39430, it's more flexible than kops or kubeadm for some people in a limited network region.

I'm new to k8s dev, so if this pull request need to change, please let me know.

Added support for creating HA clusters for centos using kube-up.sh.

k8s-ci-robot · 2017-01-05T09:03:44Z

Hi @Shawyeok. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with @k8s-bot ok to test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Details

If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot · 2017-01-05T09:03:45Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA.

Once you've signed, please reply here (e.g. "I signed it!") and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.

Details

If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-reviewable · 2017-01-05T09:03:51Z

This change is

Shawyeok · 2017-01-05T09:24:55Z

I signed it!

Shawyeok · 2017-01-05T09:38:50Z

@xiaoping378 Would you mind take a look about this PR?

foxish · 2017-01-05T18:31:12Z

/cc @kubernetes/sig-cluster-lifecycle-misc

roberthbailey · 2017-01-05T18:44:30Z

It looks like your PR only affect the centos provider. If that was the intent, please update the PR description and the release note to make it clear to reviewers and users. Thanks!

Shawyeok · 2017-01-05T18:52:37Z

@roberthbailey OK, I've changed that. Maybe I should add support for ubuntu provider and others later.

roberthbailey · 2017-01-05T18:56:06Z

@Shawyeok - doing it in multiple PRs sounds like the best path forward.

Shawyeok · 2017-01-05T19:03:12Z

@roberthbailey Yeah, I think so. If a PR is too large, it's hard to review or find bug in it.

Shawyeok · 2017-01-06T06:34:23Z

@roberthbailey @zmerlynn I added two more commits to fix a problem. What should I do to make this PR move forward?

Shawyeok · 2017-01-09T01:52:09Z

/lgtm

k8s-ci-robot · 2017-01-09T01:52:10Z

@Shawyeok: you can't LGTM your own PR.

Details

In response to this comment:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Shawyeok · 2017-01-09T02:04:58Z

@mikedanese @jbeda @eparis Would you mind take a look about this PR?

roberthbailey · 2017-01-09T05:56:59Z

Please squash your commits.

Shawyeok · 2017-01-09T06:35:21Z

@roberthbailey OK, I got squashed my commits.

jszczepkowski · 2017-01-09T15:25:20Z

@Shawyeok
How did you check this PR? Is it working?

Shawyeok · 2017-01-09T15:46:39Z

@jszczepkowski I'm check the PR for quick create a HA cluster using centos provider. e.g. code below will crate a HA k8s cluster (replicated api servers, etcd cluster, leader elect)

#!/bin/bash

# file: cluster/env.sh
# define your master machines in comma separate: <user0>@<ip0> <user1>@<ip1> <user2>@<ip2>
export MASTERS="centos@172.31.1.222 centos@172.31.15.8 centos@172.31.3.78"
# define your node machines in comma separate: <user0>@<ip0> <user1>@<ip1> <user2>@<ip2>
export NODES="centos@172.31.2.202 centos@172.31.11.188"

export KUBERNETES_PROVIDER=centos

export CERT_GROUP=ec2-user

# ssh-agent
eval `ssh-agent -s` && ssh-add ~/.ssh/<my-private-key>.pem

$ cd cluster
$ ./kube-up.sh

As I said above #39430 , kubeadm and kops doesn't work for a limited network region, but kube-xxx.sh is more flexible, we just need to prepare machines(don't care about gce or aws or azure, even in a local network.)

justinsb · 2017-01-09T15:50:21Z

We would like to make kops work (of course), and I'm sure team-kubeadm would love to make kubeadm work also. For kops, it's Route53, right? If you have time it would be great to know if the following are supported in AWS cn-north:

Route53 Public Zones (I think no)
Route53 Private Zones (?)
Internet-Facing ELBs
Internal ELBs

Shawyeok · 2017-01-09T16:02:55Z

@justinsb Route53 doesn't serve on aws cn-north region, and ELB works well. But as I knew, route53 is requirement of kops. I'm looking forward to kops and kubeadm works well in china region and don't need too many hack. Thanks very much for your services.

jszczepkowski · 2017-01-10T15:08:52Z

+
+# By default, etcd cluster will use etcd-discovery
+#   https://github.com/coreos/etcd/blob/master/Documentation/op-guide/clustering.md#etcd-discovery
+export NUM_ETCD_NODES=${NUM_MASTERS}


Variables should be in the following format: "${VALUE}"
So, here you should have: NUM_ETCD_NODES="${NUM_MASTERS}"
Please, update also other places.

zmerlynn · 2017-01-25T00:50:51Z

/approve
/lgtm

Fix: cannot get default master advertise address correctly Set default value of NUM_MASTERS and NUM_NODES by MASTERS and NODES themself Code cleanup and documented Using runtime reconfiguration for etcd cluster instead of etcd discovery Add exceptions for verify-flags

Shawyeok · 2017-01-25T15:39:29Z

@zmerlynn I've missed run hack/verify-flags-underscore.py -e > hack/verify-flags/exceptions.txt again after changed cluster/centos/config-default.sh.

Change of cluster/centos/config-default.sh:

-etcd_servers="${prefix}http://$master_ip:2379"
+etcd_servers="${prefix}http://${master_ip}:2379"

Sorry for delay.

Shawyeok · 2017-01-25T16:04:09Z

@zmerlynn Now all checks were passed, need lgtm and approved again.

zmerlynn · 2017-01-25T17:22:49Z

/approve
/lgtm

k8s-github-robot · 2017-01-25T23:55:10Z

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2017-01-26T00:40:57Z

Automatic merge from submit-queue

strugglingyouth · 2017-02-21T08:05:59Z

@Shawyeok

I see the official said 1.5 version integrated HA, but it seems only in gce use, we use the centos, so I see your PR, So I tested it，download the k8s 1.5.2 source, I will cluster/centos directory file replacement for you to modify, and then use kube-up to start the test, but failed, the following is my steps:

And first,config the cluster/centos/config-default.sh file.Then,Manually download kubernetes-client-linux-amd64.tar.gz kubernetes-server-linux-amd64.tar.gz easy-rsa.tar.gz packages and placed in the specified directory.

export KUBERNETES_PROVIDER=centos

bash -x ./kube-up.sh

>  .........
>
>  + local host=centos@xxxx
>  + shift
>  + ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oLogLevel=ERROR -C -t centos@xxx 'mkdir -p ~/kube_temp;                    sudo mkdir -p /opt/kubernetes/bin;                    sudo mkdir -p /opt/kubernetes/cfg'
>  + kube-scp centos@xxx './../cluster/../cluster/centos/ca-cert ./../cluster/../cluster/centos/binaries/master ./../cluster/../cluster/centos/master ./../cluster/../cluster/centos/config-default.sh ./../cluster/../cluster/centos/util.sh' '~/kube_temp'
>  + local host=centos@xxx
>  + src=($2)
>  + local src
>  + local 'dst=~/kube_temp'
>  + scp -r -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oLogLevel=ERROR -C ./../cluster/../cluster/centos/ca-cert ./../cluster/../cluster/centos/binaries/master ./../cluster/../cluster/centos/master ./../cluster/../cluster/centos/config-default.sh ./../cluster/../cluster/centos/util.sh 'centos@xxxxx:~/kube_temp'
>
>  ./../cluster/../cluster/centos/ca-cert: No such file or directory
>
>  scheduler.sh                                                              100% 1601     1.6KB/s   00:00
>  controller-manager.sh                                                     100% 2126     2.1KB/s   00:00
>  apiserver.sh                                                              100% 4439     4.3KB/s   00:00
>  etcd.sh                                                                   100% 2219     2.2KB/s   00:00
>  apiserver.sh                                                              100% 4424     4.3KB/s   00:00
>  controller-manager.sh                                                     100% 2237     2.2KB/s   00:00
>  etcd.sh                                                                   100% 2299     2.3KB/s   00:00
>  flannel.sh                                                                100% 1758     1.7KB/s   00:00
>  post-etcd.sh                                                              100%  853     0.8KB/s   00:00
>  scheduler.sh                                                              100% 1708     1.7KB/s   00:00
>  config-default.sh                                                         100% 4753     4.6KB/s   00:00
>  util.sh                                                                   100%   11KB  10.5KB/s   00:00
>  + kill 10478
>

And then it's over.

I would like to ask you this installation method is correct?

Shawyeok · 2017-02-21T08:49:35Z

The problem is cluster/centos/ca-cert: No such file or directory, the certificates are required for kube-apiserver.

strugglingyouth · 2017-02-22T04:29:50Z

@Shawyeok
Now there is no configuration is completed, has been stuck in the service start process.

+ scp -r -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oLogLevel=ERROR -C ./cluster/../cluster/../cluster/centos/binaries/node ./cluster/../cluster/../cluster/centos/node ./cluster/../cluster/../cluster/centos/config-default.sh ./cluster/../cluster/../cluster/centos/util.sh 'centos@xxxx:~/kube_temp'
flannel.sh                                                                         100% 1881     1.8KB/s   00:00
proxy.sh                                                                           100% 1717     1.7KB/s   00:00
kubelet.sh                                                                         100% 2455     2.4KB/s   00:00
docker.sh                                                                          100% 1405     1.4KB/s   00:00
remove-docker0.sh                                                                  100%  850     0.8KB/s   00:00
mk-docker-opts.sh                                                                  100% 2590     2.5KB/s   00:00
mk-docker-opts.sh                                                                  100% 2590     2.5KB/s   00:00
remove-docker0.sh                                                                  100%  850     0.8KB/s   00:00
docker.sh                                                                          100% 1402     1.4KB/s   00:00
flannel.sh                                                                         100% 1881     1.8KB/s   00:00
kubelet.sh                                                                         100% 2737     2.7KB/s   00:00
proxy.sh                                                                           100% 1719     1.7KB/s   00:00
config-default.sh                                                                  100% 4753     4.6KB/s   00:00
util.sh                                                                            100%   11KB  10.5KB/s   00:00
+ kube-ssh centos@xxxx '     rm -rf /opt/kubernetes/bin;     sudo cp -r ~/kube_temp/node/bin /opt/kubernetes;     sudo chmod -R +x /opt/kubernetes/bin;     sudo ln -s /opt/kubernetes/bin/* /usr/local/bin/;     sudo bash ~/kube_temp/node/scripts/flannel.sh http://xxxx:2379 172.16.0.0/16;     sudo bash ~/kube_temp/node/scripts/docker.sh "";     sudo bash ~/kube_temp/node/scripts/kubelet.sh xxxx xxx 192.168.3.100 cluster.local;     sudo bash ~/kube_temp/node/scripts/proxy.sh xxxx'
+ local host=centos@xxxx
+ shift
+ ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oLogLevel=ERROR -C -t centos@xxxx '     rm -rf /opt/kubernetes/bin;     sudo cp -r ~/kube_temp/node/bin /opt/kubernetes;     sudo chmod -R +x /opt/kubernetes/bin;     sudo ln -s /opt/kubernetes/bin/* /usr/local/bin/;     sudo bash ~/kube_temp/node/scripts/flannel.sh http://xxxx:2379 172.16.0.0/16;     sudo bash ~/kube_temp/node/scripts/docker.sh "";     sudo bash ~/kube_temp/node/scripts/kubelet.sh xxxx xxxx 192.168.3.100 cluster.local;     sudo bash ~/kube_temp/node/scripts/proxy.sh xxxx'

And stop in here.I found that there is a problem with the docker start.Can you leave a contact, like qq, or a tutorial for my reference.

Shawyeok · 2017-02-22T05:04:45Z

Would you mind provide some logs of docker service?

journalctl -xf -u docker

Shawyeok · 2017-02-22T05:22:13Z

There is a docker dependencies problems in centos:

error while loading shared libraries: libltdl.so.7: cannot open shared object file: No such file or directory

It cause by a runtime lib which docker depend is missing. You can resolve it as below:

yum provides libltdl.so.7
yum install -y libtool-ltdl

strugglingyouth · 2017-02-22T08:15:18Z

@Shawyeok
Although it is not successful, but I would like to know what you are the principle of how to do the handover between master and elections, the trouble to talk briefly, thank you very much.

Shawyeok · 2017-02-22T17:47:30Z

Actually, I don't understand your question. If you want to know about the principle of master-elected, you should take a look about raft.

@jszczepkowski

Automatic merge from submit-queue Centos provider: generate SSL certificates for etcd cluster. **What this PR does / why we need it**: Support secure etcd cluster for centos provider, generate SSL certificates for etcd in default. Running it w/o SSL is exposing cluster data to everyone and is not recommended. [#39462](#39462 (comment)) /cc @jszczepkowski @zmerlynn **Release note**: ```release-note Support secure etcd cluster for centos provider. ```

Automatic merge from submit-queue Able to quick create a HA cluster by kube-up.sh centos provider Make `kube-up.sh` `centos provider` support quick create a HA cluster, as I said above [kubernetes#39430](kubernetes#39430), it's more flexible than `kops` or `kubeadm` for some people in a limited network region. I'm new to k8s dev, so if this pull request need to change, please let me know. ```release-note Added support for creating HA clusters for centos using kube-up.sh. ```

@jszczepkowski

…tcd-cluster Automatic merge from submit-queue Centos provider: generate SSL certificates for etcd cluster. **What this PR does / why we need it**: Support secure etcd cluster for centos provider, generate SSL certificates for etcd in default. Running it w/o SSL is exposing cluster data to everyone and is not recommended. [kubernetes#39462](kubernetes#39462 (comment)) /cc @jszczepkowski @zmerlynn **Release note**: ```release-note Support secure etcd cluster for centos provider. ```

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jan 5, 2017

k8s-github-robot assigned zmerlynn Jan 5, 2017

k8s-github-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jan 5, 2017

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jan 5, 2017

roberthbailey assigned jszczepkowski Jan 5, 2017

Shawyeok changed the title ~~make kube-up.sh support quick create a HA cluster~~ Able to quick create a HA cluster by kube-up.sh centos provider Jan 5, 2017

Shawyeok force-pushed the master branch from d212f0c to e5abc09 Compare January 9, 2017 06:31

jszczepkowski reviewed Jan 10, 2017

View reviewed changes

k8s-github-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 25, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 25, 2017

k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 25, 2017

Shawyeok force-pushed the master branch from 81d9788 to 78170bf Compare January 25, 2017 14:54

k8s-github-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 25, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 25, 2017

k8s-github-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 25, 2017

k8s-github-robot merged commit 75c6990 into kubernetes:master Jan 26, 2017

Shawyeok mentioned this pull request Mar 13, 2017

Centos provider: generate SSL certificates for etcd cluster. #42994

Merged

Uh oh!

Conversation

Shawyeok commented Jan 5, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Jan 5, 2017

Uh oh!

k8s-ci-robot commented Jan 5, 2017

Uh oh!

k8s-reviewable commented Jan 5, 2017

Uh oh!

Shawyeok commented Jan 5, 2017

Uh oh!

Shawyeok commented Jan 5, 2017

Uh oh!

foxish commented Jan 5, 2017

Uh oh!

roberthbailey commented Jan 5, 2017

Uh oh!

Shawyeok commented Jan 5, 2017

Uh oh!

roberthbailey commented Jan 5, 2017

Uh oh!

Shawyeok commented Jan 5, 2017

Uh oh!

Shawyeok commented Jan 6, 2017

Uh oh!

Shawyeok commented Jan 9, 2017

Uh oh!

k8s-ci-robot commented Jan 9, 2017

Uh oh!

Shawyeok commented Jan 9, 2017

Uh oh!

roberthbailey commented Jan 9, 2017

Uh oh!

Shawyeok commented Jan 9, 2017

Uh oh!

jszczepkowski commented Jan 9, 2017

Uh oh!

Shawyeok commented Jan 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justinsb commented Jan 9, 2017

Uh oh!

Shawyeok commented Jan 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jszczepkowski Jan 10, 2017

Choose a reason for hiding this comment

Uh oh!

zmerlynn commented Jan 25, 2017

Uh oh!

Shawyeok commented Jan 25, 2017

Uh oh!

Shawyeok commented Jan 25, 2017

Uh oh!

zmerlynn commented Jan 25, 2017

Uh oh!

k8s-github-robot commented Jan 25, 2017

Uh oh!

k8s-github-robot commented Jan 26, 2017

Uh oh!

strugglingyouth commented Feb 21, 2017

Uh oh!

Shawyeok commented Feb 21, 2017

Uh oh!

strugglingyouth commented Feb 22, 2017

Uh oh!

Shawyeok commented Feb 22, 2017

Uh oh!

Shawyeok commented Feb 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

strugglingyouth commented Feb 22, 2017

Uh oh!

Shawyeok commented Feb 22, 2017

Uh oh!

Reviewers

Assignees

Labels

Shawyeok commented Jan 5, 2017 •

edited

Loading

Shawyeok commented Jan 9, 2017 •

edited

Loading

Shawyeok commented Jan 9, 2017 •

edited

Loading

Shawyeok commented Feb 22, 2017 •

edited

Loading