Proposal TCE Concierge (installer)

[miclettej]

Abstract

We believe one of the large areas of friction for TCE end users is getting the Tanzu CLI installed. Getting the Tanzu CLI installed is defined as:

Installing the Tanzu CLI:

• Installed to $PATH
• Installing CLI Plugins:
• Installed to ${XDG_DATA_HOME}/tanzu-cli
• Verifying Tanzu CLI and plugins are up to date and compatible with each other

Proposal

To achieve increased success rate when installing Tanzu CLI, we propose the introduction of the following deliverable:

A standalone Concierge (installer) for the Tanzu CLI - simple executable which launches a user interface that checks for an existing Tanzu CLI installation, allows for new installation of Tanzu CLI and verifies Tanzu CLI installation status

• UI: Install CLI and Default CLI Plugins
• UI: Verify CLI success of installation

Concierge

The Concierge should consist of a simple executable which launches a user interface that checks for an existing Tanzu CLI installation, allows for new installation of Tanzu CLI and verifies Tanzu CLI installation status. The Concierge UI is to be built on web technologies embedded in an Electron or similar style framework for cross-operating system distribution. This allows the Concierge to be delivered and run as an executable to provide a quality user experience while maintaining the simplicity of a native installer interface. The intent is to provide a seamless experience across host operating systems while allowing for flexibility for expansion of installer options or configurations in future releases.

First time users will be invited to install the Tanzu CLI with a single-click experience in the Concierge UI. Automation of installing the Tanzu CLI and plugins will be completed as far as technical feasibility is allowed. Users who complete the installation or those who already have the Tanzu CLI installed will be invited to launch the Tanzu UI with the click of a button.

For environments that are unable to run the GUI installer, users may continue to install the CLI via package managers or manual download and installation.

Post-MVP considerations:

• The Tanzu CLI installer may include configurability so that “flavors” of the installer may be pre-defined, which would result in installation of a specific set of CLI plugins.
• Post-installation management of CLI plugins (E.G. I need the Pinniped plugin, help me install or update it).
• Ability to detect new versions of the CLI and prompt for upgrade.
• Ability to check for the presence of kubectl and docker, perhaps assisting with those installations within the Installer
• Ability to check any other known issues the community has encountered in significant numbers

Concierge Download

The release process should build the Concierge for each of our supported platforms. Those Concierge binaries will need to be signed to prevent unsigned executable warnings from the operating system.

The installation package will be published along with the GitHub release artifacts.

Concierge Workflow

The Concierge will follow these steps when launched:

• Check for existence of tanzu CLI
  If present, provide options to:
  launch UI (once available); or
  perform reinstall; or
  perform uninstall
• Perform basic prerequisite checks
  Minimum CPU and memory
  Docker present
  Other required commands installed (kubectl, etc)
  Check for internet connectivity and, if airgapped, let the user know what they may need to install
• Allow user to select installation location (TBD, Windows installers typically allow this, but we may want to enforce a standard location on the PATH for the binary to be installed. Plugin location is not configurable)
• Perform installation
  Place tanzu CLI in PATH
  %PROGRAM_FILES%/tanzu on Windows
  /usr/local/bin/tanzu on others
  Remove existing plugin cache if present to force rediscovery of plugins
  Install plugin bundle
  Ensure package TCE package repos have been configured
• On completion, summarize installation state and provide suggestions for next steps (launch UI)

Reinstall workflow

Reinstallation will consist of performing the same steps as an installation, forcing the overwrite of any installed executable files.

Uninstall workflow

Uninstall steps are:

Delete the tanzu binary and all installed plugins
Delete the plugin cache
Provide summary results of the uninstall

Note: the uninstall process will not remove any local configuration metadata (files in ~/.config/tanzu) as they may be wanted in the case of a reinstall.

Project Motivation

Several GitHub issues and slack threads have been generated pointing to areas of friction for users. A key focal point for the Tanzu CLI installer and Tanzu UI UX design lies in analyzing these areas of friction and generating solutions. Many of those GitHub issues related to failures during bootstrapping, installing and getting started with TCE are as follows:

GitHub Issues:

Simplifying UX
Provide a quick path for impatient people who just want to see something happen. #2182
The tanzu management-cluster create command fails with "invalid provider name" error #2128
Error Running configure-tce.sh OSX #2543
Install.bat falsely reports an error #2381
Installation on OSX has excessive warnings #2540
Space in username causes install.bat to give errors on Windows #2140

Sanity Checks
The install.sh script fails if user doesn't have sudo access. #1647
Docker installation: "Failed to invoke API on cluster” #2200
error when retrieving non-admin workload cluster kubeconfig #2199
Installation of Chocolatey package appears to hang on tanzu init #2132
docker based mgmt cluster: failed to create provider object cert-manager.io/v1alpha2 #2127
creating docker clusters behind corporate firewall with ssl dpi fails #1395
Can't create a cluster due to OS image not detected #2793
What should I do next？ #2706
Kickstart UI not booting up on creating management cluster UI #2318
unable to create bootstrap cluster: failed to create kind cluster tkg-kind- #2138

[kartiklunkad26]

How does this installer tie to the installer to creating clusters. Will they be separate user experiences?

[stmcginnis]

How does this installer tie to the installer to creating clusters. Will they be separate user experiences?

This proposal is related to installing the tanzu CLI and related TCE plugins so that you can then deploy clusters. So the difference between "install" and "deploy" here is key.

[dvonthenen]

For the UI experience, you should be able to use hack/release/install.sh/.bat as a blueprint for landing things where they need to in your implementation. It should be noted that this workflow changes from release to release and we would need to keep on top of this.

[joshrosso]

We'll email the mailing list today to notify we're ready for comments.

Core engineering will review this proposal in depth next week.

Thanks!

[jorgemoralespou]

@miclettej having read what's written in the description (I guess that's the proposal), some comments/questions that would need further clarification:

• What's the installation mechanism of the concierge? Will they be produced as .exe/.msi, .dmg, .pkg/.deb/.rpm so that a typical OS installation process happen?
• Once Concierge is installed, is it assumed to be always up and running or is it to be executed on demand?
• Will it provide information about my installation, or just go through the install/update of Tanzu (for TCE) install?

Apart from these, I like what I read :-D

[miclettej]

@miclettej having read what's written in the description (I guess that's the proposal), some comments/questions that would need further clarification:

• What's the installation mechanism of the concierge? Will they be produced as .exe/.msi, .dmg, .pkg/.deb/.rpm so that a typical OS installation process happen?
• Once Concierge is installed, is it assumed to be always up and running or is it to be executed on demand?
• Will it provide information about my installation, or just go through the install/update of Tanzu (for TCE) install?

Apart from these, I like what I read :-D

Hello @jorgemoralespou thank you for the questions.

Yes the installation mechanism/deliverables will be a set of executables for supported operating systems (dmg, exe for certain in MVP).

The concierge is intended to be executed on demand for MVP. I think I know where you're going with this question ;-) Further down the road map I think we may revisit the idea of merging a concierge and cluster creation UI into a single native app, but many steps we would have to take to get there.

It will provide some information about any current installation that it discovers already in place, as well as some details about a new installation if it is performing one.

@swalner-vmware is leading this project, so I will defer to him for additional info on supported operating systems for MVP and installation details.

[jorgemoralespou]

Thanks, my question was more focused on "documenting on the proposal why a native app that needs to be installer vs just an installer for every version". It's more about the rationale of why we're doing things, so people reading the proposal don't wonder why this approach as I initially did. I totally get it, I would just want it to be better described in the proposal ;-)

[dvonthenen]

Installing the Tanzu CLI:

Installed to $PATH
Installing CLI Plugins:
Installed to ${XDG_DATA_HOME}/tanzu-cli
Verifying Tanzu CLI and plugins are up to date and compatible with each other

This is probably the root of the problem here... this is definitely not a guarantee and in fact, we know this changes between each and every release. Trying to dissect this a second time sounds extremely problematic and also inserts another person into yet another packaging and installation process and then have a the UI be updated to match.

We currently have 2 mechanisms in which we allow for a quick and easy install:

• choco (windows)
• brew (mac)

If users are really having problems with installation, we should just provide a wrapper for the existing process and just create MSI, DMG, and RPM/DEBs which can all be done with existing process.

In the future, there is work in progress for the OCI registry to just download bits.

[jpmcb]

TLDR: I'm unsure I see a need for this beyond something that a single command would fufill. However, I'm definately coming from the perspective of a heavy CLI user. Do we have research or user feedback that they want to use a GUI to install the tools?

There have definitely been problems with users installing the CLI and it's binaries. But if the CLI, with alot of the context aware work done by the tanzu-framework team, was able to automatically know which plugins it needed, download them, and keep itself in a good working order, doesn't that eliminate most of the need for a higher order gui installer?

---

verifies Tanzu CLI installation status

I really like the idea of something that verifies an existing install. Something that can look in my $PATH and say "Yes, you have a correct install of TCE". Something that comes to mind is Neo-Vim's :checkhealth command which checks if it is installed properly and it's plugins are configured ok. Maybe something like this exists as it's own command on tanzu?

---

embedded in an Electron or similar style framework for cross-operating system distribution.

What are the release implications of delivering something like this?

[dvonthenen]

There was also mention (internally) this can be cross built on macos only which is going to be very difficult downstream during windows signing. These are some internal details which we can't make public but would be gladly talk about offline.

[swalner-vmware]

Installing the Tanzu CLI:
Installed to $PATH
Installing CLI Plugins:
Installed to ${XDG_DATA_HOME}/tanzu-cli
Verifying Tanzu CLI and plugins are up to date and compatible with each other

This is probably the root of the problem here... this is definitely not a guarantee and in fact, we know this changes between each and every release. Trying to dissect this a second time sounds extremely problematic and also inserts another person into yet another packaging and installation process and then have a the UI be updated to match.
We currently have 2 mechanisms in which we allow for a quick and easy install:
choco (windows)
brew (mac)
If users are really having problems with installation, we should just provide a wrapper for the existing process and just create MSI, DMG, and RPM/DEBs which can all be done with existing process.
In the future, there is work in progress for the OCI registry to just download bits.

@dvonthenen , sorry I'm a bit late to the party; my GitHub notifications do not seem to be firing correctly.
We have gotten feedback that the CLI install is a source of friction for many users. I can appreciate that to many of us it seems simple to use choco or brew, but apparently mileage varies on that. So it feels worthwhile to create a UI that eases the pain point.
You also mention that the installation process itself changes, so we'd need to keep the UI in sync with those changes. Very true that they would need to be in sync (a release and its installation process). We'll see what level of churn there is in the overall process that would require updates. For example, with the current imagining, adding another plugin would not require a change to the Concierge (the build process would simply include the plugin, and the Concierge would install all plugins included). However, other changes would be breaking, and we'd need to update the Concierge to be able to do the new install. Hopefully the level of change to the Concierge would be similar (or smaller) to the current change required in install.sh.
But I get the point that it's another dependency, another moving part. Is it worth the effort?
I think so, but reasonable minds may differ on that point. :)
(I think @jpmcb agrees that a UI may be overkill.)

[swalner-vmware]

@jpmcb wrote:

I really like the idea of something that verifies an existing install. Something that can look in my $PATH and say "Yes, you have a correct install of TCE". Something that comes to mind is Neo-Vim's :checkhealth command which checks if it is installed properly and it's plugins are configured ok. Maybe something like this exists as it's own command on tanzu?

I think there are many opportunities for helping the user out with the CLI installation and configuration.
For example, warning a user that they have a TKG installation if they're trying to install TCE. Or even just telling the user where their binary is and what edition it is. Potentially detecting that there is a more recent release and downloading and installing that for the user with a single click. Our MVP proposal does not include these more "fancy" options, but it feels like value that can be added over time, if the project moves forward.
I don't think that tanzu itself has any self-diagnostics (beyond reporting the build information), but I may be wrong on that.

[swalner-vmware]

@dvonthenen wrote:

There was also mention (internally) this can be cross built on macos only which is going to be very difficult downstream during windows signing. These are some internal details which we can't make public but would be gladly talk about offline.

Thanks, David. It's certainly disappointing that the tool we have in mind for the packaging has some limitations in terms of being able to build cross-platform. You're correct that our current candidate can build all platforms (Mac, Win, Linux) on MacOS only. I'm assuming that if the proposal is approved, we'll work our way through those problems (either with this tool or another). Thanks for the offer for talking it over; I expect we'll take you up on that if/when we move forward.

[joshrosso]

After learning some of the build complications and other technical considerations, we're going to look at getting some of this work into tanzu's root CLI. For now we're going to table this proposal and revisit it at a later date.