Skip to content

Commit d8b85ec

Browse files
hughsaundershughsaunders
committed
Initialise bash-lib repo
This commit adds the structure of the libraries and a few funcions. Most of the code in this commit is geared towards making sure this repo stays tested and documented in future. The following things are checked: * All functions are documented * All functions are tested * All scripts are linted See Readme for a list of the functions that are included, and for more information on testing with BATS.
1 parent a620e57 commit d8b85ec

35 files changed

+1025
-2
lines changed

‎.gitmodules

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
[submodule "test-utils/bats"]
2+
path = test-utils/bats
3+
url = https://github.com/bats-core/bats
4+
[submodule "test-utils/bats-assert-1"]
5+
path = test-utils/bats-assert-1
6+
url = https://github.com/jasonkarns/bats-assert-1
7+
[submodule "test-utils/bats-support"]
8+
path = test-utils/bats-support
9+
url = https://github.com/ztombol/bats-support

‎Jenkinsfile

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#!/usr/bin/env groovy
2+
3+
pipeline {
4+
agent { label 'executor-v2' }
5+
6+
options {
7+
timestamps()
8+
buildDiscarder(logRotator(numToKeepStr: '30'))
9+
}
10+
11+
triggers {
12+
cron(getDailyCronString())
13+
}
14+
15+
environment {
16+
BATS_OUTPUT_FORMAT="junit"
17+
}
18+
19+
stages {
20+
stage('Bash Linting') {
21+
environment {
22+
BATS_SUITE="BATS-Lint"
23+
}
24+
steps {
25+
sh './tests-for-this-repo/run-bats-tests.sh tests-for-this-repo/lint.bats'
26+
}
27+
}
28+
29+
stage('Python Linting') {
30+
steps {
31+
sh './tests-for-this-repo/python-lint.sh'
32+
}
33+
}
34+
35+
stage('Bash Tests') {
36+
environment {
37+
BATS_SUITE="BATS-Tests"
38+
}
39+
steps {
40+
sh './tests-for-this-repo/run-bats-tests.sh $(ls tests-for-this-repo/*.bats|grep -v lint.bats)'
41+
}
42+
}
43+
}
44+
45+
post {
46+
always {
47+
junit '*-junit.xml'
48+
cleanupAndNotify(currentBuild.currentResult)
49+
}
50+
}
51+
}

‎README.md

Lines changed: 197 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,197 @@
1-
# bashlib
2-
Common bash functions for use in test pipelines
1+
# bash-lib
2+
```
3+
_______________ _______________
4+
.' .' .|
5+
.' .' .' |
6+
.'_______________.'______________ .' |
7+
| ___ _____ ___ || ___ _____ ___ | |
8+
||_=_|__=__|_=_||||_=_|__=__|_=_|| |
9+
______||_____===_____||||_____===_____|| | __________
10+
.' ||_____===_____||||_____===_____|| .' .'|
11+
.' ||_____===_____||||_____===_____|| .' .' |
12+
.'___________|_______________||_______________|.'__________.' |
13+
|.----------.|.-----___-----.||.-----___-----.|| |_____.----------.
14+
|] |||_____________||||_____________||| .' [ |
15+
|| ||.-----___-----.||.-----___-----.||.' | |
16+
|| |||_____________||||_____________|||==========| |
17+
|| ||.-----___-----.||.-----___-----.|| |_____| |
18+
|] o|||_____________||||_____________||| .' [ 'o|
19+
|| ||.-----___-----.||.-----___-----.||.' | |
20+
|| ||| ||||_____________|||==========| |
21+
|| ||| |||.-----___-----.|| |_____| |
22+
|] ||| |||| ||| .' [ |
23+
||__________|||_____________||||_____________|||.'________|__________|
24+
''----------'''------------------------------'''----------''
25+
(o)LGB (o)
26+
```
27+
28+
The place to store functions that are used in pipelines for multiple repos.
29+
30+
Please add whatever is useful to you, but keep it tidy so its still useful to everyone else :)
31+
32+
## Usage
33+
Firstly acquire a clone of bash-lib:
34+
1. If submodules are acceptable in your workflow, I recommend adding a submodule `git submodule add git@github.com:conjurinc/bash-lib bash-lib`
35+
1. Otherise you can use emulate submodules with a bash function eg:<sup>[1](#footnote-1)</sup>
36+
```bash
37+
function init_bash_lib(){
38+
# shellcheck disable=SC2086,SC2046
39+
d="$(cd $(dirname ${BASH_SOURCE[0]}); pwd)"
40+
ref_path="${d}/.bash_lib_ref"
41+
pushd ${d}
42+
git clone --recurse-submodules git@github.com:conjurinc/bash-lib
43+
pushd bash-lib
44+
if [[ -e "${ref_path}" ]]
45+
then
46+
git checkout "$(cat ${ref_path})"
47+
else
48+
git rev-parse HEAD > "${ref_path}"
49+
echo "Please commit ${ref_path} to ensure a consistent version is used"
50+
fi
51+
. init.sh
52+
popd
53+
popd
54+
}
55+
```
56+
57+
It is highly important that however you acquire bash-lib, you always **use it at
58+
a pinned verison**. Hopefully bash-lib will end up widely used, and we do not
59+
want to break every project when updating bash-lib. Each project must update the pin
60+
at a time that suits them to avoid unexpected breakages.
61+
62+
Once you have bash-lib cloned in your project, you source two things:
63+
64+
1. Source `bash-lib/init.sh`. This ensures submodules are initalised and sets the BASH_LIB env var to the absolute path to the bash-lib dir. This makes it easy to source libraries from other scripts.
65+
1. Source `${BASH_LIB}/lib-name/lib.sh` for any libraries you are interested in.
66+
67+
You are now ready to use bash-lib functions :)
68+
69+
## Structure
70+
The `/init.sh` script sets up everything required to use the library, most
71+
importantly the `BASH_LIB` variable which gives the absolute path to the root
72+
of the library and should be used for sourcing the modules.
73+
74+
The repo is organised into libraries, each library is a directory that has a
75+
lib.sh file. Sourcing the lib.sh for a library should expose all the functions
76+
that library offers. The lib.sh file may source or reference other supporting
77+
files within it's directory.
78+
79+
```
80+
/init.sh
81+
/lib-name/
82+
lib.sh # main library file
83+
supporting-file.yaml # a supporting file
84+
/tests-for-this-repo # self tests for this repo.
85+
```
86+
## Style Guide
87+
Follow the [google shell style guide](https://google.github.io/styleguide/shell.xml#Naming_Conventions).
88+
TL;DR:
89+
1. Use snake_case function and variable names
90+
1. Use `function` when declaring functions.
91+
92+
93+
## Contents
94+
95+
<!-- html table due to markdown's lack of support for lists within tables -->
96+
<table>
97+
<thead>
98+
<tr>
99+
<th>Library</th>
100+
<th>Description</th>
101+
<th>Functions</th>
102+
</tr>
103+
</thead>
104+
<tbody>
105+
<tr>
106+
<td><a href="filehandling/lib.sh">filehandling</a></td>
107+
<td>Functions relating to file and path handling
108+
<td>
109+
<ol>
110+
<li> abs_path: Ensure a path is absolute</li>
111+
</ol>
112+
</td>
113+
</tr>
114+
<tr>
115+
<td><a href="git/lib.sh">git</a></td>
116+
<td>Git helpers</td>
117+
<td>
118+
<ol>
119+
<li>repo_root: find the root of the current git repo</li>
120+
<li>all_files_in_repo: list files tracked by git</li>
121+
</ol>
122+
</td>
123+
</tr>
124+
<td><a href="helpers/lib.sh">helpers</a></td>
125+
<td>Bash scripting helpers</td>
126+
<td>
127+
<ol>
128+
<li>die: print message and exit 1</li>
129+
<li>spushd/spopd: safe verisons of pushd & popd that call die if the push/pop fails, they also drop stdout. </li>
130+
</ol>
131+
</td>
132+
</tr>
133+
<tr>
134+
<td><a href="k8s/lib.sh">k8s</a></td>
135+
<td>Utils for connecting to K8s</td>
136+
<td>
137+
<ol>
138+
<li>build_gke_image: Build docker image for running kubectl commands against GKE</li>
139+
<li>delete_gke_image: Delete image from GKE</li>
140+
<li>run_docker_gke_command: Run command in gke-utils container, already authenticated to k8s cluster.</li>
141+
</ol>
142+
</td>
143+
</tr>
144+
<tr>
145+
<td><a href="logging/lib.sh">logging</a></td>
146+
<td>Helpers related to login</td>
147+
<td>
148+
<ol>
149+
<li>announce: echo message in ascii banner to distinguish it from other log messages.</li>
150+
</ol>
151+
</td>
152+
</tr>
153+
<tr>
154+
<td><a href="test-utils/lib.sh">test-utils</a></td>
155+
<td>Helpers for executing tests</td>
156+
<td>
157+
<ol>
158+
<li>shellcheck_script: execute shellcheck against a script, uses docker.</li>
159+
<li>find_scripts: find git tracked files with .sh extension</li>
160+
<li>tap2junit: Convert a subset of <a href="http://testanything.org/">TAP</a> to JUnit XML. Retains logs for errors</li>
161+
</ol>
162+
</td>
163+
</tr>
164+
</tbody>
165+
</table>
166+
167+
## Testing
168+
Tests are written using [BATS](https://github.com/bats-core/bats). Each lib should have a `lib-name.bats` file in [tests-for-this-repo](/tests-for-this-repo).
169+
Asserts are provided by [bats-assert-1](https://github.com/jasonkarns/bats-assert-1). The value in these is that they provide useful debugging output when the assertion fails, eg expected x got y.
170+
171+
Example:
172+
```bash
173+
# source support and assert libraries
174+
. "${BASH_LIB}/test-utils/bats-support/load.bash"
175+
. "${BASH_LIB}/test-utils/bats-assert-1/load.bash"
176+
177+
# source the library under test
178+
. "${BASH_LIB}/git/lib.sh"
179+
180+
# define a test that calls a library function
181+
@test "it does the thing" {
182+
some_prep_work
183+
# run is a wrapper that catches failures so that assertsions can be run,
184+
# otherwise the test would immediately fail.
185+
run does_the_thing
186+
assert_success
187+
assert_output "thing done"
188+
}
189+
```
190+
191+
Test fixtures should go in /tests-for-this-repo/fixtures/lib-name.
192+
193+
194+
### Footnotes
195+
<a name="footnote-1">1</a>: Early Readers/Reviewers may notice that the
196+
submodule alternative clone function fails. This is because most of the
197+
content hasn't been merged to master yet (its still in the initial_libs branch)

‎filehandling/lib.sh

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
#!/bin/bash
2+
3+
: "${BASH_LIB:?BASH_LIB must be set. Please source bash-lib/init.sh before other scripts from bash-lib.}"
4+
. "${BASH_LIB}/helpers/lib.sh"
5+
6+
#https://stackoverflow.com/a/23002317
7+
function abs_path() {
8+
# generate absolute path from relative path
9+
# $1 : relative filename
10+
# return : absolute path
11+
if [ -d "$1" ]; then
12+
# dir
13+
(spushd "$1"; pwd)
14+
elif [ -f "$1" ]; then
15+
# file
16+
if [[ $1 = /* ]]; then
17+
echo "$1"
18+
elif [[ $1 == */* ]]; then
19+
echo "$(spushd "${1%/*}"; pwd)/${1##*/}"
20+
else
21+
echo "$(pwd)/$1"
22+
fi
23+
fi
24+
}

‎git/lib.sh

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/bin/bash
2+
3+
: "${BASH_LIB:?BASH_LIB must be set. Please source bash-lib/init.sh before other scripts from bash-lib.}"
4+
5+
# Get the top level of a git repo
6+
function repo_root(){
7+
git rev-parse --show-toplevel
8+
}
9+
10+
# List files tracked by git
11+
function all_files_in_repo(){
12+
git ls-tree -r HEAD --name-only
13+
}

‎helpers/lib.sh

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/bash
2+
3+
: "${BASH_LIB:?BASH_LIB must be set. Please source bash-lib/init.sh before other scripts from bash-lib.}"
4+
5+
function die(){
6+
echo "${@}"
7+
exit 1
8+
}
9+
10+
#safe pushd
11+
function spushd(){
12+
pushd "${1}" >/dev/null || die "pushd ${1} failed :("
13+
}
14+
15+
#safe popd
16+
function spopd(){
17+
popd >/dev/null || die "popd failed :("
18+
}

‎init.sh

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
#!/bin/bash
2+
3+
## Initialisation Functions for the
4+
## Conjurinc Bash Library
5+
6+
# Shell Otions
7+
set -euo pipefail
8+
9+
# This script should be sourced before any of
10+
# the other scripts in this repo. Other scripts
11+
# make use of ${BASH_LIB} to find each other.
12+
13+
# Get the relative path to the repo root
14+
# shellcheck disable=SC2086
15+
BASH_LIB_RELATIVE="$(dirname ${BASH_SOURCE[0]})"
16+
17+
# Must be set in order to load the filehandling
18+
# module. Will be updated when abs_path is available.
19+
BASH_LIB="${BASH_LIB_RELATIVE}"
20+
21+
# Load the filehandling module for the abspath
22+
# function
23+
. "${BASH_LIB_RELATIVE}/filehandling/lib.sh"
24+
25+
# Export the absolute path
26+
# shellcheck disable=SC2086
27+
BASH_LIB="$(abs_path ${BASH_LIB_RELATIVE})"
28+
export BASH_LIB
29+
30+
. "${BASH_LIB}/helpers/lib.sh"
31+
32+
# Update Submodules
33+
spushd "${BASH_LIB}"
34+
git submodule update --init --recursive
35+
spopd
36+
37+
export BATS_CMD="${BASH_LIB}/test-utils/bats/bin/bats"

‎k8s/Dockerfile

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
FROM google/cloud-sdk
2+
3+
RUN mkdir -p /src /scripts
4+
WORKDIR /src
5+
COPY platform_login.sh /scripts
6+
7+
# Install Docker client
8+
RUN apt-get update -y && \
9+
apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common wget && \
10+
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg | apt-key add - && \
11+
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable" && \
12+
apt-get update && \
13+
apt-get install -y docker-ce && \
14+
rm -rf /var/lib/apt/lists/*
15+
16+
# Install kubectl CLI
17+
ARG KUBECTL_CLI_URL
18+
RUN wget -O /usr/local/bin/kubectl ${KUBECTL_CLI_URL:-https://storage.googleapis.com/kubernetes-release/release/v1.7.6/bin/linux/amd64/kubectl} && \
19+
chmod +x /usr/local/bin/kubectl

0 commit comments

Comments
 (0)