Skip to content

Commit

Permalink
Merge pull request #2 from iranzo/add-and-run-precommit
Browse files Browse the repository at this point in the history
Add pre-commit configuration and execute over repo files
  • Loading branch information
romfreiman authored Jun 28, 2020
2 parents a67338e + 63a5433 commit 5812d7f
Show file tree
Hide file tree
Showing 29 changed files with 975 additions and 466 deletions.
5 changes: 5 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[flake8]
ignore = E203, E266, E501, W503, F403, F401, E402, E722, C901
max-line-length = 79
max-complexity = 18
select = B,C,E,F,W,T4,B9
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,3 @@ bm-inventory
*__pycache__*
minikube
*.log

53 changes: 53 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
fail_fast: true
repos:
- repo: meta
hooks:
- id: check-useless-excludes
- repo: https://github.com/prettier/prettier
rev: 2.0.2
hooks:
- id: prettier
files: \.(css|js|md|markdown|json)
- repo: https://github.com/python/black
rev: 19.10b0
hooks:
- id: black
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.5.0
hooks:
- id: check-added-large-files
- id: check-ast
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-json
- id: check-merge-conflict
- id: check-symlinks
- id: check-xml
- id: check-yaml
args: [--unsafe]
- id: end-of-file-fixer
- id: fix-encoding-pragma
- id: forbid-new-submodules
- id: requirements-txt-fixer
- id: sort-simple-yaml
- id: trailing-whitespace
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.5.0
hooks:
- id: flake8
- repo: https://github.com/pecigonzalo/pre-commit-shfmt.git
rev: 9ee28e3f14556aa88dd5255f2e091d1d2f607bb7
hooks:
- id: shell-fmt
args:
- --indent=4
- repo: https://github.com/asottile/blacken-docs
rev: v1.6.0
hooks:
- id: blacken-docs
# - repo: https://github.com/hcodes/yaspeller.git
# rev: v6.1.0
# hooks:
# - id: yaspeller
# files: ".md"
# types: [markdown]
82 changes: 82 additions & 0 deletions .yaspeller.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
{
"ignoreUrls": true,
"findRepeatWords": true,
"maxRequests": 5,
"ignoreDigits": true,
"lang": "en",
"dictionary": [
"Ansible",
"baremetal",
"BareMetalHost",
"CI",
"CIDR",
"ConfigMap",
"DCO",
"dhcp",
"dhcpd",
"dnsmasq",
"endpoint",
"filesystem",
"filesystems",
"GA",
"GitHub",
"hostname",
"hostnames",
"iDRAC",
"iLO",
"IPI",
"IPMI",
"KNI",
"machineset",
"Markdown",
"nameserver",
"NICs",
"OCP",
"openshift",
"OpenShift",
"orchestrator",
"pingable",
"playbook",
"playbooks",
"podman",
"provisioner",
"PRs",
"PXE",
"RHCOS",
"RHEL",
"routable",
"subzone",
"Unreachable",
"VIPs",
"VLAN",
"VM",
"YAML",
"DPDK",
"PTP",
"Kubernetes",
"virtualization",
"NMstate",
"Hugepages",
"NFV",
"BM",
"SR-IOV",
"CNV",
"Netlify",
"SCTP",
"sexualized",
"asciidoc",
"FQDN",
"MachineConfig",
"VMs",
"NTP",
"QEMU",
"kubelet",
"ansible",
"endfor",
"Jekyll",
"ipi",
"versioned",
"unversioned",
"devprev"
]
}
2 changes: 1 addition & 1 deletion Dockerfile.test-infra
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ rm -f /lib/systemd/system/anaconda.target.wants/*

RUN wget `wget https://www.terraform.io/downloads.html -q -O - | grep -oP "(https://releases.hashicorp.com/terraform/.*linux_amd64\.zip)(?=\")" | head -n 1` && unzip terraform*.zip -d /usr/bin/ && rm -rf terraform*.zip

RUN wget https://dl.google.com/go/go1.13.4.linux-amd64.tar.gz && tar -C /usr/local -xf go1.13.4.linux-amd64.tar.gz && rm -f go1.13.4.linux-amd64.tar.gz
RUN wget https://dl.google.com/go/go1.13.4.linux-amd64.tar.gz && tar -C /usr/local -xf go1.13.4.linux-amd64.tar.gz && rm -f go1.13.4.linux-amd64.tar.gz
ENV PATH /usr/local/go/bin:$PATH
RUN mkdir -p ~/.terraform.d/plugins
RUN go get -v -u github.com/dmacvicar/terraform-provider-libvirt && cd ~/.terraform.d/plugins && go build -a -v github.com/dmacvicar/terraform-provider-libvirt && rm -rf ~/go
Expand Down
65 changes: 52 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,49 @@
# Test-Infra

This project deploys the OpenShift Assisted Installer in Minikube and spawns libvirt VMs that represent bare metal hosts.

# Prerequisites

- CentOS 8 or RHEL 8 host
- File system that supports d_type
- Ideally on a bare metal host with at least 64G of RAM.
- Run as a user with passwordless sudo access or be ready to enter sudo password for prepare phase.
- Get a valid pull secret (JSON string) from [redhat.com](https://cloud.redhat.com/openshift/install/pull-secret) if you want to test the installation (not needed for testing only the discovery flow). Export it as

```bash
export PULL_SECRET='<pull secret JSON>'
```

# Instructions


## Host preparation

On the bare metal host:

```bash
dnf install -y git make
cd /home/test # don't do it on /root it will breaks build image mounts and fail to run
git clone https://github.com/tsorya/test-infra.git
```

When using this infra for the first time on a host, run:

```bash
make create_full_environment
```

This will install required packages, configure libvirt, pull relevant Docker images, and start Minikube.

## Usage

There are different options to use test-infra, which can be found in the makefile.

## Full flow cases

The following is a list of stages that will be run:

1. Start Minikube if not started yet
1. Deploy services for assisted deployment on Minikube
1. Deploy services for assisted deployment on Minikube
1. Create cluster in bm-inventory service
1. Download ISO image
1. Spawn required number of VMs from downloaded ISO with parameters that can be configured by OS env (check makefile)
Expand All @@ -46,86 +56,107 @@ The following is a list of stages that will be run:
1. Verifying cluster is in state "installed"
1. Download kubeconfig to build/kubeconfig


**Note**: Please make sure no previous cluster is running before running a new one (it will rewrite its build files).

### Run full flow with install

To run the full flow, including installation:

```bash
make run_full_flow_with_install
```

Or to run it together with create_full_environment (requires sudo password):
````bash

```bash
make all
````
```

### Run full flow without install

To run the flow without the installation stage:

```bash
make run_full_flow
```

### Run only deploy nodes (without pre deploy of all assisted service)

```bash
make deploy_nodes or make deploy_nodes_with_install
```

### Redeploy nodes

```bash
make redeploy_nodes or make redeploy_nodes_with_install
```

### Redeploy with assisted services

```bash
make redeploy_all or make redeploy_all_with_install
```

## Cleaning

Cleaning test-infra environment.

### Clean all include minikube

```bash
make destroy
```

### Clean nodes only

```bash
make destroy_nodes
```

### Delete all virsh resources

Sometimes you may need to delete all libvirt resources

```bash
make delete_all_virsh_resources
```

### Install cluster

Install cluster after nodes were deployed. Can take ClusterId as os env

```bash
make install_cluster
make install_cluster
```

### Create cluster and download ISO

```bash
make download_iso
```

### deploy_bm_inventory and Create cluster and download ISO

```bash
make download_iso_for_remote_use
```

### start_minikube and Deploy UI and and open port forwarding on port 6008, allows to connect to it from browser

```bash
make deploy_ui
```

### Kill all open port forwarding commands, will be part of destroy target

```bash
make kill_all_port_forwardings
```


## OS parameters used for configurations
~~~~

```
BMI_BRANCH bm-inventory branch to use, default: master
IMAGE path to ISO to spawn VM with, if set vms will be spawn with this iso without creating cluster
NUM_MASTERS number of VMs to spawn as masters, default: 3
Expand All @@ -149,42 +180,50 @@ INSTALLER_IMAGE: assisted-installer image to use, will update bm-inventory co
SERVICE: bm-inventory image to use
DEPLOY_TAG: the tag to be used for all images (bm-inventory, assisted-installer, agent, etc) this will override any other os params
~~~~
```

## Test bm-inventory image

```bash
make redeploy_all SERVICE=<image to test>
or
or
export PULL_SECRET='<pull secret JSON>'; make redeploy_all_with_install SERVICE=<image to test>
```

## Test agent image

```bash
make redeploy_all AGENT_DOCKER_IMAGE=<image to test>
make redeploy_all AGENT_DOCKER_IMAGE=<image to test>
or
make redeploy_all_with_install AGENT_DOCKER_IMAGE=<image to test>
```

## Test installer image

```bash
make redeploy_all INSTALLER_IMAGE=<image to test>
make redeploy_all INSTALLER_IMAGE=<image to test>
or
export PULL_SECRET='<pull secret JSON>'; make redeploy_all_with_install INSTALLER_IMAGE=<image to test>
```

## Test installer, bm-inventory and agent images in the same flow

```bash
make redeploy_all INSTALLER_IMAGE=<image to test> AGENT_DOCKER_IMAGE=<image to test> SERVICE=<image to test>
or
or
export PULL_SECRET='<pull secret JSON>'; make redeploy_all_with_install INSTALLER_IMAGE=<image to test> AGENT_DOCKER_IMAGE=<image to test> SERVICE=<image to test>
```

# Test infra image

## Create and push new image will create new bm-inventory client, build new image and push image

```bash
make build_and_push_image IMAGE_NAME=<your full image path> IMAGE_TAG=<default is latest>
```

## Use new image, will pull image from hub, check that image is public, if tag is not latest update skipper YAML

```bash
make image_build IMAGE_NAME=<your image> IMAGE_TAG=<default is latest>
```
Loading

0 comments on commit 5812d7f

Please sign in to comment.