Skip to content

Latest commit

 

History

History
222 lines (174 loc) · 7.68 KB

README.md

File metadata and controls

222 lines (174 loc) · 7.68 KB
Raw

local-kubernetes-playground

Software engineers at ETHR previously used a variation of the following project as a playground for software development, automation testing, research, and for demonstrating proof-of-concepts.

This playground was the motivation behind establishing x-ethr and its related open-source repositories.

Important

The following project requires an expansive amount of knowledge around development, kubernetes, and overall systems. While the guide can be followed step-by-step to produce a fully functioning cluster, there are requirements that would otherwise be challenging for beginners to 1. understand, 2. setup, 3. debug.

If requirements are correctly met, the entirety of this project can be deployed in under five minutes by simply following the usage section.

Users of local-kubernetes-playground will involve themselves in the following disciplines:

  • Software Engineering
  • DevOps
  • Systems Administration
  • GitOps
  • Databases
  • Security

Example

The Playground's Deployed Service Mesh

istio-example

Requirements

Important

Usage, requirements, and documentation was vetted on a Mac Studio, M1 Max 2022 on MacOS, Sonoma 14.5. Other systems are likely subject to incompatibilities.

System
Optional(s)

Usage

Note

During the first minute or two, there may be a few warnings that surface. Due to Kubernetes reconciliation, all errors should resolve by minute three or four.

  1. Install kind. 
    go install sigs.k8s.io/kind@latest
sudo install "$(go env --json | jq -r ".GOPATH")/bin/kind" /usr/local/bin
  2. Create a cluster via kind. 
    kind create cluster --config "configuration.yaml"
kubectl config set-context "$(printf "%s-kind" "kind")"
  3. Unable node(s). 
    kubectl label node kind-control-plane node.kubernetes.io/exclude-from-external-load-balancers-
  4. Setup a local load-balancer (within its own private terminal session). 
    go install sigs.k8s.io/cloud-provider-kind@latest
sudo install "$(go env --json | jq -r ".GOPATH")/bin/cloud-provider-kind" /usr/local/bin
cloud-provider-kind -v 9
  5. Verify connectivity to the cluster.
    • If using OpenLens, select the kind-kind context.
  6. Bootstrap. 
    flux bootstrap github --repository "https://github.com/x-ethr/cluster-management" \
    --owner "x-ethr" \
    --private "false" \
    --personal "false" \
    --path "clusters/local" \
    --verbose
  7. Sync local cluster repository's vendors. 
    git submodule update --remote --recursive
  8. Add kustomization.yaml to new cluster directory (only applicable during first-time cluster setup). 
    cat << EOF > ./vendors/cluster-management/clusters/local/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: []
EOF
  9. Optionally, update the Kustomization.flux-system.spec.interval (changes each time a local cluster is bootstrapped).
  10. Push local changes to vendors submodules. 
    git submodule foreach "git add . && git commit --message \"Git Submodule Update(s)\" && git push -u origin HEAD:main"
  11. Start the local registry. 
    bash ./scripts/registry.bash
  12. Wait for the various resources to reconcile successfully.

Service-Mesh

The following command will port-forward the gateway's configured port 80 and expose it on localhost:8080.

kubectl port-forward --namespace development services/gateway 8080:80
Network Traffic

In order to view tracing and network traffic, issue the following command(s):

for i in $(seq 1 250); do
    curl "http://localhost:8080/v1/test-service-1"
    curl "http://localhost:8080/v1/test-service-2"
    curl "http://localhost:8080/v1/test-service-2/alpha"
    
    curl "http://localhost:8080/v1/authentication"
done
Kiali

The following command will expose the kiali service and open a browser to its dashboard.

istioctl dashboard kiali
Tracing (Jaeger)

The following command will expose the jaeger service and open a browser to its dashboard.

istioctl dashboard jaeger
Istio & istoctl

Useful istoctl command(s)

kubectl -n istio-system logs --since=1h istiod-6bc5bc58b4-wvhmc --follow
Redis

Useful kubectl command(s)

Logging

kubectl --namespace caching logs --since=10m services/redis --follow

Useful redis-cli command(s)

redis-cli

Add consumer to consumer group

xadd demo-stream * name john email jdoe@test.com
xadd demo-stream * tom tom@test.com

Contributions

Please see the Contributing Guide file for additional details.

Debugging

Ingress LB Address
kubectl get --namespace istio-system svc/ingress-gateway -o=jsonpath='{.status.loadBalancer.ingress[0].ip}'
(upstream_reset_before_response_started{connection_termination})

Restart the Istio API Gateway Deployment

External Reference(s)