openshift start node is deprecated and --config no longer works

[adelton]

Which section(s) is the issue in?

https://docs.okd.io/latest/install_config/master_node_configuration.html

What needs fixing?

The text says

The openshift start command and its subcommands (master to launch a master server and node to launch a node server) ...

and

To make this easier, the configuration files can be created with the --write-config option and then used with the --config option.

However, running openshift start node --config /etc/origin/node1/node-config.yaml command which used to be working for many releases now fails with

Command "start" is deprecated, This command will be replaced by the hypershift and hyperkube binaries for starting individual components.
Error: unknown flag: --config


Available Commands:
  etcd        Launch etcd server
  master      Launch a master
  network     Launch node network

Use "openshift <command> --help" for more information about a given command.

So it seems that openshift start node should not longer be recommended and alternative documentation for starting node is needed. Also, the mention of using --config seems no longer correct, as it's no longer recognized at all.

[GangChenTFS]

@adelton do you have find solution for this issue? would you please share your solution? or you have other way to install and configure openshift OKD. thanks in advance

[adelton]

No, I sadly don't know how to solve the problem, short of using openshift-ansible.

[hdonati]

No news ?
It is impossible to upgrade OKD cluster 3.10 to 3.11 with freshly delivered Centos7 RPMs from repo http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin/
systemctl start origin-node
fails because of line in /etc/systemd/system/origin-node.service :
ExecStart=/usr/bin/openshift start node --config=${CONFIG_FILE} $OPTIONS
after upgrade
Is there another way to start a node ???

[ricardoferreiracosta08]

On latest version, v3.11.0, theopenshift start nodefunctionality and openshift start have been removed - the Kubelet must now be started directly #20344, #20717

• By using the Kubelet directly we make nodes easier to manage and more consistent with the upstream.
• Future releases will remove other parts of openshift start master.

Reference: https://github.com/openshift/origin/releases/tag/v3.11.0

[adelton]

Thanks, @ricardoferreiracosta08. Could the documentation at https://docs.okd.io/latest/install_config/master_node_configuration.html be updated to reflect that change?

[adelton]

When I replace

openshift start node --config /etc/origin/node-$(hostname)/node-config.yaml

with

hyperkube kubelet $( /usr/bin/openshift-node-config --config=/etc/origin/node-$(hostname)/node-config.yaml )

the process ends up with reporting

W0611 10:26:29.656136   27448 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d
E0611 10:26:29.656244   27448 kubelet.go:2101] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

The /etc/cni/net.d is empty, while with openshift start node it has /etc/cni/net.d/80-openshift-network.conf containing

{
  "cniVersion": "0.2.0",
  "name": "openshift-sdn",
  "type": "openshift-sdn"
}

@ricardoferreiracosta08, would you know how to force hyperkube kubelet to also configure the CNI?

[krumware]

Can someone confirm whether or not these instructions are supposed to work with 3.11? We are also seeing the same issue.
https://docs.okd.io/latest/getting_started/administrators.html#downloading-the-binary

Edit: These new commands will not work in 3.11 per the comments above.
In similar style to the official docs that are not updated at time of writing. Instead of

sudo ./openshift start

use

sudo ./hyperkube kubelet

There are still issues running so additional instructions may be missing.
We are seeing the repeating output of

I0627 11:20:41.220818    2561 kubelet_node_status.go:269] Setting node annotation to enable volume controller attach/detach
E0627 11:20:50.728101    2561 kubelet_network.go:117] Failed to ensure rule to drop packet marked by KUBE-MARK-DROP in filter chain KUBE-FIREWALL: error appending rule: exit status 1: iptables: No chain/target/match by that name.
I0627 11:20:51.240828    2561 kubelet_node_status.go:269] Setting node annotation to enable volume controller attach/detach

[TuranTimur]

On latest version, v3.11.0, theopenshift start nodefunctionality and openshift start have been removed - the Kubelet must now be started directly #20344, #20717

* By using the Kubelet directly we make nodes easier to manage and more consistent with the upstream.

* Future releases will remove other parts of `openshift start master`.

Reference: https://github.com/openshift/origin/releases/tag/v3.11.0

Can you please provide document?

[adelton]

For OpenShift 3.11, replacing

openshift start node --config /etc/origin/node-$(hostname)/node-config.yaml

with

openshift start network --config=/etc/origin/node-$(hostname)/node-config.yaml --kubeconfig=/etc/origin/node-$(hostname)/node.kubeconfig
hyperkube kubelet $( /usr/bin/openshift-node-config --config=/etc/origin/node-$(hostname)/node-config.yaml )

seems to give me functionally equivalent setup, at least for origin-* packages on Fedora 29 and Fedora 30.

[adelton]

The above approach also work for OCP 3.11.

[adellape]

Fixed via #21268.