Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): use kubernetes javascript client instead of invoking kubctl CLI #673

Merged
merged 39 commits into from
Jan 24, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
bfd72cf
feat: implement Kubectl class using kubernetes javascript client
leninmehedy Jan 15, 2024
df76d00
feat: implement copy file to container
leninmehedy Jan 15, 2024
f90450c
cleanup
leninmehedy Jan 15, 2024
66a4817
feat: implement hasDir and hasFile
leninmehedy Jan 15, 2024
cade2bc
fix: remove unused args
leninmehedy Jan 15, 2024
33913a9
fix: rewrite async function of a promise with polling
leninmehedy Jan 15, 2024
aad0af9
add timeout arg
leninmehedy Jan 15, 2024
7e03a6e
remove incorrect docs
leninmehedy Jan 15, 2024
fc26a01
fix: use exec status to exit from polling and code cleanup
leninmehedy Jan 15, 2024
2eddad2
feat: implement port forward and polling utility function
leninmehedy Jan 16, 2024
d51cdfa
fix: refactor config manager to contain the config
leninmehedy Jan 17, 2024
02fdaa8
fix: remove unnecessary polling from exec call and cache configManage…
leninmehedy Jan 17, 2024
82a80f4
feat: implement wait for pod and add check for copy methods
leninmehedy Jan 17, 2024
f24f9d3
remove unused import
leninmehedy Jan 18, 2024
1415260
fix: replace old kubectl with kubectl2 in chart command
leninmehedy Jan 19, 2024
26dc63c
fix: use Kubect2 for node commands
leninmehedy Jan 19, 2024
75f3c63
test: fix tests for missing kubect2
leninmehedy Jan 19, 2024
6a4226c
fix: do not require kind and cluster creation functionalities
leninmehedy Jan 22, 2024
ec8dd5c
fix: split platform code copy and extract into separate steps for bet…
leninmehedy Jan 22, 2024
b663d57
fix: get pod IP and copy files checks
leninmehedy Jan 22, 2024
214daba
fix: use default namespace for cluster setup
leninmehedy Jan 22, 2024
12c94ab
fix node setup
leninmehedy Jan 22, 2024
90b0f4d
fix: check for namespace
leninmehedy Jan 22, 2024
1a5981b
fix: e2e tests and CI pipelline
leninmehedy Jan 22, 2024
2b3f9a3
fix: e2e tests
leninmehedy Jan 23, 2024
505ab8d
fix: check for pod readiness individually to show more info to the user
leninmehedy Jan 23, 2024
644c0e0
add setup script for e2e test
leninmehedy Jan 23, 2024
468b0b0
Merge branch 'main' into 605-kubernetes-javascript-client
leninmehedy Jan 24, 2024
5a9b96c
fix: make waitForPod non-blocking
leninmehedy Jan 24, 2024
af5e7a4
fix: do not poll after copyToPod or copyFromPod
leninmehedy Jan 24, 2024
7387dff
fix: add clone method for Kubectl2
leninmehedy Jan 24, 2024
1194025
fix: remove kubectl usage
leninmehedy Jan 24, 2024
5ba143c
fix: waitForPod
leninmehedy Jan 24, 2024
acd277f
fix: relay command
leninmehedy Jan 24, 2024
a911703
fix: rename Kubectl2 to K8
leninmehedy Jan 24, 2024
1c8ee25
fix test
leninmehedy Jan 24, 2024
6956df2
rename test setup script
leninmehedy Jan 24, 2024
e9555a2
fix: allow namespace to be created during chart install
leninmehedy Jan 24, 2024
0933869
Merge branch 'main' into 605-kubernetes-javascript-client
leninmehedy Jan 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: do not require kind and cluster creation functionalities
User is expected to bring their own kubernetes cluster

Signed-off-by: Lenin Mehedy <lenin.mehedy@swirldslabs.com>
  • Loading branch information
leninmehedy committed Jan 22, 2024
commit 6a4226c2371b39f87adfa972de4940c81db89c85
2 changes: 0 additions & 2 deletions fullstack-network-manager/src/commands/base.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ export class BaseCommand extends ShellRunner {
if (!opts || !opts.chartManager) throw new Error('An instance of core/ChartManager is required')
if (!opts || !opts.configManager) throw new Error('An instance of core/ConfigManager is required')
if (!opts || !opts.depManager) throw new Error('An instance of core/DependencyManager is required')
if (!opts || !opts.clusterManager) throw new Error('An instance of core/ClusterManager is required')

super(opts.logger)

Expand All @@ -34,6 +33,5 @@ export class BaseCommand extends ShellRunner {
this.chartManager = opts.chartManager
this.configManager = opts.configManager
this.depManager = opts.depManager
this.clusterManager = opts.clusterManager
}
}
26 changes: 13 additions & 13 deletions fullstack-network-manager/src/commands/chart.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -59,18 +59,18 @@ export class ChartCommand extends BaseCommand {

async prepareConfig (task, argv) {
this.configManager.load(argv)
const namespace = this.configManager.flagValue(flags.namespace)
const nodeIds = this.configManager.flagValue(flags.nodeIDs)
const chartDir = this.configManager.flagValue(flags.chartDirectory)
const valuesFile = this.configManager.flagValue(flags.valuesFile)
const deployMirrorNode = this.configManager.flagValue(flags.deployMirrorNode)
const deployExplorer = this.configManager.flagValue(flags.deployHederaExplorer)
const enableTls = this.configManager.flagValue(flags.enableTls)
const tlsClusterIssuerName = this.configManager.flagValue(flags.tlsClusterIssuerName)
const tlsClusterIssuerNamespace = this.configManager.flagValue(flags.tlsClusterIssuerNamespace)
const enableHederaExplorerTls = this.configManager.flagValue(flags.enableHederaExplorerTls)
const acmeClusterIssuer = this.configManager.flagValue(flags.acmeClusterIssuer)
const selfSignedClusterIssuer = this.configManager.flagValue(flags.selfSignedClusterIssuer)
const namespace = this.configManager.getFlag(flags.namespace)
const nodeIds = this.configManager.getFlag(flags.nodeIDs)
const chartDir = this.configManager.getFlag(flags.chartDirectory)
const valuesFile = this.configManager.getFlag(flags.valuesFile)
const deployMirrorNode = this.configManager.getFlag(flags.deployMirrorNode)
const deployExplorer = this.configManager.getFlag(flags.deployHederaExplorer)
const enableTls = this.configManager.getFlag(flags.enableTls)
const tlsClusterIssuerName = this.configManager.getFlag(flags.tlsClusterIssuerName)
const tlsClusterIssuerNamespace = this.configManager.getFlag(flags.tlsClusterIssuerNamespace)
const enableHederaExplorerTls = this.configManager.getFlag(flags.enableHederaExplorerTls)
const acmeClusterIssuer = this.configManager.getFlag(flags.acmeClusterIssuer)
const selfSignedClusterIssuer = this.configManager.getFlag(flags.selfSignedClusterIssuer)

// prompt if values are missing and create a config object
const config = {
Expand Down Expand Up @@ -154,7 +154,7 @@ export class ChartCommand extends BaseCommand {
title: 'Initialize',
task: async (ctx, task) => {
self.configManager.load(argv)
const namespace = self.configManager.flagValue(flags.namespace)
const namespace = self.configManager.getFlag(flags.namespace)
ctx.config = {
namespace: await prompts.promptNamespaceArg(task, namespace)
}
Expand Down
250 changes: 20 additions & 230 deletions fullstack-network-manager/src/commands/cluster.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -11,36 +11,18 @@ import * as prompts from './prompts.mjs'
*/
export class ClusterCommand extends BaseCommand {
async showClusterList () {
this.logger.showList('Clusters', await this.clusterManager.getClusters())
this.logger.showList('Clusters', await this.kubectl2.getClusters())
return true
}

/**
* List available namespaces
* @returns {Promise<string[]>}
*/
async getNameSpaces () {
try {
return await this.kubectl.getNamespace('--no-headers', '-o name')
} catch (e) {
this.logger.showUserError(e)
}

return []
}

/**
* Get cluster-info for the given cluster name
* @param argv arguments containing cluster name
* @returns {Promise<boolean>}
*/
async getClusterInfo (argv) {
async getClusterInfo () {
try {
const clusterName = argv.clusterName
const output = await this.clusterManager.getClusterInfo(clusterName)

this.logger.showUser(chalk.green(`\nCluster information (${clusterName})\n---------------------------------------`))
output.forEach(line => this.logger.showUser(line))
const cluster = this.kubectl2.getKubeConfig().getCurrentCluster()
this.logger.showJSON(`Cluster Information (${cluster.name})`, cluster)
this.logger.showUser('\n')
return true
} catch (e) {
Expand All @@ -50,31 +32,6 @@ export class ClusterCommand extends BaseCommand {
return false
}

async createNamespace (argv) {
try {
const namespace = argv.namespace
const namespaces = await this.getNameSpaces()
this.logger.showUser(chalk.cyan('> checking namespace:'), chalk.yellow(`${namespace}`))
if (!namespaces.includes(`namespace/${namespace}`)) {
this.logger.showUser(chalk.cyan('> creating namespace:'), chalk.yellow(`${namespace} ...`))
await this.kubectl.createNamespace(namespace)
this.logger.showUser(chalk.green('OK'), `namespace '${namespace}' is created`)
} else {
this.logger.showUser(chalk.green('OK'), `namespace '${namespace}' already exists`)
}

await this.kubectl.config(`set-context --current --namespace="${namespace}"`)

this.logger.showList('Namespaces', await this.getNameSpaces())

return true
} catch (e) {
this.logger.showUserError(e)
}

return false
}

/**
* Show list of installed chart
* @param namespace
Expand All @@ -83,123 +40,6 @@ export class ClusterCommand extends BaseCommand {
this.logger.showList('Installed Charts', await this.chartManager.getInstalledCharts(namespace))
}

/**
* Create a cluster
* @param argv command arguments
* @returns {Promise<boolean>}
*/
async create (argv) {
const self = this

const tasks = new Listr([
{
title: 'Initialize',
task: async (ctx, task) => {
self.configManager.load(argv)

// get existing choices
ctx.clusters = await self.clusterManager.getClusters()

// extract config values
const clusterName = self.configManager.flagValue(flags.clusterName)
const namespace = self.configManager.flagValue(flags.namespace)

ctx.config = {
clusterName: await prompts.promptClusterNameArg(task, clusterName),
namespace: await prompts.promptNamespaceArg(task, namespace)
}
}
},
{
title: 'Create cluster',
task: async (ctx, _) => {
const clusterName = ctx.config.clusterName
ctx.clusters = await self.clusterManager.getClusters()
if (!ctx.clusters.includes(clusterName)) {
await self.clusterManager.createCluster(clusterName)
await self.kubectl.get('--raw=\'/healthz?verbose\'')
}
}
},
{
title: 'Create namespace',
task: async (ctx, _) => {
const namespace = ctx.config.namespace
ctx.namespaces = await self.getNameSpaces()
if (!ctx.namespaces.includes(`namespace/${namespace}`)) {
await self.kubectl.createNamespace(namespace)
}

await this.kubectl.config(`set-context --current --namespace="${namespace}"`)

// display info
ctx.namespaces = await self.getNameSpaces()
ctx.kubeContexts = await self.kubectl.config('get-contexts --no-headers | awk \'{print $2 " [" $NF "]"}\'')
ctx.clusters = await self.clusterManager.getClusters()
self.logger.showList('Namespaces', await ctx.namespaces)
self.logger.showList('Clusters', await ctx.clusters)
self.logger.showList('Kubernetes Contexts', ctx.kubeContexts)
}
}
], {
concurrent: false,
rendererOptions: constants.LISTR_DEFAULT_RENDERER_OPTION
})

try {
await tasks.run()
} catch (e) {
throw new FullstackTestingError('Error on cluster create', e)
}

return true
}

/**
* Delete a cluster
* @param argv
* @returns {Promise<boolean>}
*/
async delete (argv) {
const self = this

const tasks = new Listr([
{
title: 'Initialize',
task: async (ctx, task) => {
self.configManager.load(argv)
const clusterName = self.configManager.flagValue(flags.clusterName)

// get existing choices
ctx.clusters = await self.clusterManager.getClusters()

ctx.config = {
clusterName: await prompts.promptSelectClusterNameArg(task, clusterName, ctx.clusters)
}
}
},
{
title: 'Delete cluster',
task: async (ctx, _) => {
await this.clusterManager.deleteCluster(ctx.config.clusterName)
self.logger.showList('Clusters', await self.clusterManager.getClusters())
},
skip: (ctx, _) => !ctx.clusters.includes(ctx.config.clusterName)
}
], {
concurrent: false,
rendererOptions: constants.LISTR_DEFAULT_RENDERER_OPTION
})

try {
await tasks.run()
} catch (e) {
throw new FullstackTestingError('Error on cluster reset', e)
}

return true
}

/**
* Setup cluster with shared components
* @param argv
Expand All @@ -212,31 +52,22 @@ export class ClusterCommand extends BaseCommand {
{
title: 'Initialize',
task: async (ctx, task) => {
const kubeConfig = await self.clusterManager.getKubeConfig()
if (!kubeConfig['current-context']) {
throw new FullstackTestingError('kubectl context is not set, set context by running: kubectl config use-context <context-name>')
}

self.configManager.load(argv)

// extract config values
const clusterName = self.configManager.flagValue(flags.clusterName)
const namespace = self.configManager.flagValue(flags.namespace)
const chartDir = self.configManager.flagValue(flags.chartDirectory)
const deployPrometheusStack = self.configManager.flagValue(flags.deployPrometheusStack)
const deployMinio = self.configManager.flagValue(flags.deployMinio)
const deployEnvoyGateway = self.configManager.flagValue(flags.deployEnvoyGateway)
const deployCertManager = self.configManager.flagValue(flags.deployCertManager)
const deployCertManagerCrds = self.configManager.flagValue(flags.deployCertManagerCrds)

// get existing choices
const clusters = await self.clusterManager.getClusters()
const namespaces = await self.kubectl.getNamespace('--no-headers', '-o name')
const clusterName = self.configManager.getFlag(flags.clusterName)
const namespace = 'default' // we always install the shared component in default namespace
const chartDir = self.configManager.getFlag(flags.chartDirectory)
const deployPrometheusStack = self.configManager.getFlag(flags.deployPrometheusStack)
const deployMinio = self.configManager.getFlag(flags.deployMinio)
const deployEnvoyGateway = self.configManager.getFlag(flags.deployEnvoyGateway)
const deployCertManager = self.configManager.getFlag(flags.deployCertManager)
const deployCertManagerCrds = self.configManager.getFlag(flags.deployCertManagerCrds)

// prompt if inputs are empty and set it in the context
ctx.config = {
clusterName: await prompts.promptSelectClusterNameArg(task, clusterName, clusters),
namespace: await prompts.promptSelectNamespaceArg(task, namespace, namespaces),
clusterName,
namespace,
chartDir: await prompts.promptChartDir(task, chartDir),
deployPrometheusStack: await prompts.promptDeployPrometheusStack(task, deployPrometheusStack),
deployMinio: await prompts.promptDeployMinio(task, deployMinio),
Expand All @@ -247,8 +78,6 @@ export class ClusterCommand extends BaseCommand {

self.logger.debug('Prepare ctx.config', { config: ctx.config, argv })

// set current context based on cluster and namespace
await self.clusterManager.setContext(ctx.config.clusterName, ctx.config.namespace)
ctx.isChartInstalled = await this.chartManager.isChartInstalled(ctx.config.namespace, constants.CHART_FST_SETUP_NAME)
}
},
Expand Down Expand Up @@ -321,16 +150,12 @@ export class ClusterCommand extends BaseCommand {
title: 'Initialize',
task: async (ctx, task) => {
self.configManager.load(argv)
const clusterName = self.configManager.flagValue(flags.clusterName)
const namespace = self.configManager.flagValue(flags.namespace)

// get existing choices
const clusters = await self.clusterManager.getClusters()
const namespaces = await self.kubectl.getNamespace('--no-headers', '-o name')
const clusterName = self.configManager.getFlag(flags.clusterName)
const namespace = self.configManager.getFlag(flags.namespace)

ctx.config = {
clusterName: await prompts.promptSelectClusterNameArg(task, clusterName, clusters),
namespace: await prompts.promptSelectNamespaceArg(task, namespace, namespaces)
clusterName,
namespace
}

ctx.isChartInstalled = await this.chartManager.isChartInstalled(ctx.config.namespace, constants.CHART_FST_SETUP_NAME)
Expand Down Expand Up @@ -366,46 +191,12 @@ export class ClusterCommand extends BaseCommand {
static getCommandDefinition (clusterCmd) {
return {
command: 'cluster',
desc: 'Manage FST cluster',
desc: 'Manage cluster',
builder: yargs => {
return yargs
.command({
command: 'create',
desc: 'Create a cluster',
builder: y => flags.setCommandFlags(y, flags.clusterName, flags.namespace),
handler: argv => {
clusterCmd.logger.debug("==== Running 'cluster create' ===", { argv })

clusterCmd.create(argv).then(r => {
clusterCmd.logger.debug('==== Finished running `cluster create`====')

if (!r) process.exit(1)
}).catch(err => {
clusterCmd.logger.showUserError(err)
process.exit(1)
})
}
})
.command({
command: 'delete',
desc: 'Delete a cluster',
builder: y => flags.setCommandFlags(y, flags.clusterName),
handler: argv => {
clusterCmd.logger.debug("==== Running 'cluster delete' ===", { argv })

clusterCmd.delete(argv).then(r => {
clusterCmd.logger.debug('==== Finished running `cluster delete`====')

if (!r) process.exit(1)
}).catch(err => {
clusterCmd.logger.showUserError(err)
process.exit(1)
})
}
})
.command({
command: 'list',
desc: 'List all clusters',
desc: 'List all available clusters',
handler: argv => {
clusterCmd.logger.debug("==== Running 'cluster list' ===", { argv })

Expand All @@ -422,7 +213,6 @@ export class ClusterCommand extends BaseCommand {
.command({
command: 'info',
desc: 'Get cluster info',
builder: y => flags.setCommandFlags(y, flags.clusterName),
handler: argv => {
clusterCmd.logger.debug("==== Running 'cluster info' ===", { argv })
clusterCmd.getClusterInfo(argv).then(r => {
Expand Down
4 changes: 2 additions & 2 deletions fullstack-network-manager/src/commands/flags.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ export const clusterName = {
name: 'cluster-name',
definition: {
describe: 'Cluster name',
default: core.constants.CLUSTER_NAME,
default: '',
alias: 'c',
type: 'string'
}
Expand All @@ -37,7 +37,7 @@ export const namespace = {
name: 'namespace',
definition: {
describe: 'Namespace',
default: core.constants.NAMESPACE_NAME,
default: '',
alias: 'n',
type: 'string'
}
Expand Down
Loading