Provisioning support (theforeman#295)

* [WIP] Provisioning support

The idea here is that you can run the playbook on any old box:

  ansible-playbook -l centos7-katello-p4-nightly playbooks/katello_provisioning.yml

README.md

@@ -11,6 +11,7 @@ Forklift provides tools to create Foreman/Katello environments for development,
  * [Production Environments](docs/production.md)
  * [Development Environments](docs/development.md)
  * [Testing Environments](docs/testing.md)
+ * [Provisioning environment](docs/provision.md)
  * [Troubleshooting](docs/troubleshooting.md)
 
 ## Using Forklift

config/base_boxes.yaml

@@ -74,6 +74,17 @@ boxes:
     ansible:
       group: 'server'
 
+  centos7-provision-nightly:
+    box: centos7
+    memory: 8096
+    ansible:
+      playbook:
+          - 'playbooks/katello.yml'
+          - 'playbooks/katello_provisioning.yml'
+      group: 'server'
+      variables:
+        puppet_repositories_version: 4
+
   centos7-katello-p4-nightly:
     box: centos7
     ansible:

docs/provision.md

@@ -0,0 +1,53 @@
+# Katello provisioning setup
+
+A role is included which will configure an all-on-one provisioning environment
+with Katello. It uses nested libvirt to make your virtual machine a hypervisor
+capable of running it's own VM's.  It also sets up a private NAT network on the
+host.
+
+## How to configure
+
+1. Enable nested virt on your phyiscal box.  See http://www.rdoxenham.com/?p=275
+comments for more details.  Essentially, add this line to kvm-intel.conf and
+reboot:
+
+        echo "options kvm-intel nested=1" | sudo tee /etc/modprobe.d/kvm-intel.conf
+
+2. Build a box
+
+    - Option 1: Use the `centos7-provisioning-nightly` box.
+
+    - Option 2: Use an existing katello box (e.g. centos7-katello-p4-nightly) and run the provisioning playbook (it takes a while, as it syncs (on-demand) centos7, puppet 4).   ***Note*: If you are using puppet 4, you need to increase the ram on the box, to something like 8096 otherwise candlepin crashes with OOM.**
+        `ansible-playbook -l centos7-katello-p4-nightly playbooks/katello_provisioning.yml`
+
+4. Login and create a compute profile, because this isn't possible with hammer or the API.
+
+   - Click Infrastructure / Compute Resources
+   - Click "libvirt"
+   - Click Compute profiles
+   - Click 2-Medium
+        - increse ram to 1024MB (required for centos 7)
+        - change network type to NAT, network name = provision
+   - Click Submit
+
+5.  Configure Activation Key
+
+  - Content/ Activation Keys
+  - Assign all available subscriptions to the activation key
+
+6. Configure / Host groups
+
+    - Edit Forklift CentOS 7
+    - Set Compute profile to be "2-Medium"
+    - Assign the `CentOS 7` activation key to the host group
+
+7. You're good to go! Let's provision a box!
+
+     - Click Hosts/ New Host
+     - Fill in:
+                - org, location, host group
+                - deploy on = libvirt
+
+     DONE! Click submit :tada:
+
+7. If you want to view the console while it boots, make sure to trust the CA certificate in your browser, it's hosted at https://centos7-katello-nightly.example.com/pub/katello-server-ca.crt, and you'll need to make sure you're accessing the katello via it's proper hostname (add an entry to /etc/hosts)

playbooks/katello_provisioning.yml

@@ -0,0 +1,5 @@
+- hosts: all
+  become: true
+  roles:
+    - libvirt_setup
+    - katello_provisioning

playbooks/roles/katello_provisioning/defaults/main.yml

@@ -0,0 +1,7 @@
+---
+katello_provisioning_hammer: hammer -u admin -p changeme
+katello_provisioning_organization: Default Organization
+katello_provisioning_organization_label: "{{ katello_provisioning_organization|regex_replace('\\s', '_') }}"
+katello_provisioning_location: Default Location
+katello_provisioning_foreman: True
+katello_provisioning_katello: True

playbooks/roles/katello_provisioning/tasks/foreman.yml

@@ -0,0 +1,131 @@
+---
+- name: 'install libvirt gem'
+  yum: name=rubygem-ruby-libvirt state=present
+
+- name: 'install foreman-libvirt'
+  yum: name=foreman-libvirt state=present
+
+- name: 'enable provisioning with foreman-installer'
+  shell: >
+    foreman-installer
+    --disable-system-checks
+    --foreman-proxy-dns true
+    --foreman-proxy-dns-provider libvirt
+    --foreman-proxy-dns-interface virbr1
+    --foreman-proxy-dns-zone example.com
+    --foreman-proxy-dns-forwarders 8.8.8.8
+    --foreman-proxy-dns-reverse 73.168.192.in-addr.arpa
+    --foreman-proxy-dhcp true
+    --foreman-proxy-dhcp-provider libvirt
+    --foreman-proxy-libvirt-network provision
+    --foreman-proxy-dhcp-interface virbr1
+    --foreman-proxy-dhcp-range "192.168.73.2 192.168.73.100"
+    --foreman-proxy-dhcp-gateway 192.168.73.1
+    --foreman-proxy-dhcp-nameservers 192.168.73.1
+    --foreman-proxy-tftp-servername 192.168.73.1
+    --foreman-proxy-tftp true
+    --foreman-proxy-tftp-managed false
+
+- name: 'restart foreman-proxy'
+  service: name=foreman-proxy state=restarted
+
+- name: 'restart foreman'
+  service: name=httpd state=restarted
+
+- name: 'refresh features'
+  shell: >
+    {{ katello_provisioning_hammer }} proxy refresh-features --id {{ katello_provisioning_smart_proxy.Id }}
+
+# Compute Resource
+- name: 'find compute resource'
+  shell: >
+    {{ katello_provisioning_hammer }} compute-resource info --name "libvirt"
+  register: katello_provisioning_compute_resource
+  ignore_errors: True
+
+- name: 'create compute resource'
+  shell: >
+    {{ katello_provisioning_hammer }} compute-resource create
+    --name "libvirt"
+    --url "qemu:///system"
+    --provider libvirt
+    --organizations "{{ katello_provisioning_organization }}"
+    --locations "{{ katello_provisioning_location }}"
+  when: katello_provisioning_compute_resource.stderr.find('not found') != -1
+
+# Domain
+- name: 'find domain'
+  shell: >
+    {{ katello_provisioning_hammer }} domain info --name "example.com"
+  register: katello_provisioning_domain
+  ignore_errors: True
+
+- name: 'create domain'
+  shell: >
+    {{ katello_provisioning_hammer }} domain create
+    --name example.com
+    --dns-id {{ katello_provisioning_smart_proxy.Id }}
+    --organizations "{{ katello_provisioning_organization }}"
+    --locations "{{ katello_provisioning_location }}"
+  when: katello_provisioning_domain.stderr.find('not found') != -1
+
+- name: 'update domain'  # it may have been automatically created by puppet if katello reports first
+  shell: >
+    {{ katello_provisioning_hammer }} domain update
+    --name example.com
+    --dns-id {{ katello_provisioning_smart_proxy.Id }}
+    --organizations "{{ katello_provisioning_organization }}"
+    --locations "{{ katello_provisioning_location }}"
+
+# Subnet
+- name: 'find subnet'
+  shell: >
+    {{ katello_provisioning_hammer }} subnet info --name "192.168.73.0/24"
+  register: katello_provisioning_subnet
+  ignore_errors: True
+
+- name: 'create subnet'
+  shell: >
+    {{ katello_provisioning_hammer }} subnet create
+    --name "192.168.73.0/24"
+    --dhcp-id {{ katello_provisioning_smart_proxy.Id }}
+    --dns-id {{ katello_provisioning_smart_proxy.Id }}
+    --tftp-id {{ katello_provisioning_smart_proxy.Id }}
+    --domains example.com
+    --from 192.168.73.2
+    --to 192.168.73.100
+    --network 192.168.73.0
+    --mask 255.255.255.0
+    --ipam DHCP
+    --gateway 192.168.73.1
+    --dns-primary 192.168.73.1
+    --organizations "{{ katello_provisioning_organization }}"
+    --locations "{{ katello_provisioning_location }}"
+  when: katello_provisioning_subnet.stderr.find('not found') != -1
+
+# Puppet
+- name: 'find environment'
+  shell: >
+    {{ katello_provisioning_hammer }} environment info --name "production"
+  register: katello_provisioning_environment
+  ignore_errors: True
+
+- name: 'create environment'
+  shell: >
+    {{ katello_provisioning_hammer }} environment create
+    --name production
+    --organizations "{{ katello_provisioning_organization }}"
+    --locations "{{ katello_provisioning_location }}"
+  when: katello_provisioning_environment.stderr.find('not found') != -1
+
+- name: 'update environment'  # it may have been automatically created by puppet if katello reports first
+  shell: >
+    {{ katello_provisioning_hammer }} environment update
+    --name production
+    --organizations "{{ katello_provisioning_organization }}"
+    --locations "{{ katello_provisioning_location }}"
+
+# query local nameservers http://projects.theforeman.org/issues/13419
+- name: 'query local nameservers'
+  shell: >
+    {{ katello_provisioning_hammer }} settings set --name query_local_nameservers --value true

playbooks/roles/katello_provisioning/tasks/katello.yml

@@ -0,0 +1,163 @@
+---
+# Product
+- name: 'find product'
+  shell: >
+    {{ katello_provisioning_hammer }} product info
+    --name "CentOS"
+    --organization "{{ katello_provisioning_organization }}"
+  register: katello_provisioning_product
+  ignore_errors: True
+
+- name: 'create centos product'
+  shell: >
+    {{ katello_provisioning_hammer }} product create
+    --name "CentOS"
+    --organization "{{ katello_provisioning_organization }}"
+  when: katello_provisioning_product.stderr.find('not found') != -1
+
+# CentOS
+- name: 'find centos repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository info
+    --name "CentOS 7"
+    --product "CentOS"
+    --organization "{{ katello_provisioning_organization }}"
+  register: katello_provisioning_repo
+  ignore_errors: True
+
+- name: 'create centos repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository create
+    --product="CentOS"
+    --content-type="yum"
+    --name "CentOS 7"
+    --url http://mirror.centos.org/centos/7/os/x86_64
+    --download-policy on_demand
+    --organization "{{ katello_provisioning_organization }}"
+  when: katello_provisioning_repo.stderr.find('not found') != -1
+
+- name: 'sync the centos repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository synchronize
+    --name "CentOS 7"
+    --product "CentOS"
+    --organization "{{ katello_provisioning_organization }}"
+
+# Katello Agent
+- name: 'find agent repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository info
+    --name "Katello Agent"
+    --product "CentOS"
+    --organization "{{ katello_provisioning_organization }}"
+  register: katello_provisioning_repo
+  ignore_errors: True
+
+- name: 'create agent repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository create
+    --product="CentOS"
+    --content-type="yum"
+    --name "Katello Agent"
+    --url https://fedorapeople.org/groups/katello/releases/yum/nightly/client/el7/x86_64/
+    --download-policy on_demand
+    --organization "{{ katello_provisioning_organization }}"
+  when: katello_provisioning_repo.stderr.find('not found') != -1
+
+- name: 'sync the agent repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository synchronize
+    --name "Katello Agent"
+    --product "CentOS"
+    --organization "{{ katello_provisioning_organization }}"
+
+# Puppet 4
+- name: 'find puppet4 repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository info
+    --name "Puppet 4"
+    --product "CentOS"
+    --organization "{{ katello_provisioning_organization }}"
+  register: katello_provisioning_repo
+  ignore_errors: True
+
+- name: 'create puppet4 repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository create
+    --product="CentOS"
+    --content-type="yum"
+    --name "Puppet 4"
+    --url http://yum.puppetlabs.com/el/7/PC1/x86_64
+    --download-policy on_demand
+    --organization "{{ katello_provisioning_organization }}"
+  when: katello_provisioning_repo.stderr.find('not found') != -1
+
+- name: 'sync the puppet4 repo'
+  shell: >
+    {{ katello_provisioning_hammer }} repository synchronize
+    --name "Puppet 4"
+    --product "CentOS"
+    --organization "{{ katello_provisioning_organization }}"
+
+# Activation key
+- name: 'find activation key'
+  shell: >
+    {{ katello_provisioning_hammer }} activation-key info
+    --name "CentOS 7"
+    --organization "{{ katello_provisioning_organization }}"
+  register: katello_provisioning_activation_key
+  ignore_errors: True
+
+- name: 'create activation key'
+  shell: >
+    {{ katello_provisioning_hammer }} activation-key create
+    --organization "{{ katello_provisioning_organization }}"
+    --name="CentOS 7"
+    --content-view="Default Organization View"
+    --lifecycle-environment="Library"
+    --unlimited-hosts
+  when: katello_provisioning_activation_key.stderr.find('not found') != -1
+
+- name: 'medium env should be in locs'
+  shell: >
+    {{ katello_provisioning_hammer }} medium update
+    --name "{{ katello_provisioning_organization_label }}/Library/CentOS/CentOS_7"
+    --organizations "{{ katello_provisioning_organization }}"
+    --locations "{{ katello_provisioning_location }}"
+
+# Host group
+- name: 'find host group'
+  shell: >
+    {{ katello_provisioning_hammer }} hostgroup info
+    --name "Forklift CentOS 7"
+  register: katello_provisioning_hg
+  ignore_errors: True
+
+- name: 'create a host group'
+  shell: >
+    {{ katello_provisioning_hammer }} hostgroup create
+    --name "Forklift CentOS 7"
+    --architecture x86_64
+    --content-view "Default Organization View"
+    --domain example.com
+    --lifecycle-environment Library
+    --operatingsystem "CentOS 7"
+    --root-pass changeme
+    --organizations "{{ katello_provisioning_organization }}"
+    --query-organization "{{ katello_provisioning_organization }}"
+    --partition-table "Kickstart default"
+    --medium "{{ katello_provisioning_organization_label }}/Library/CentOS/CentOS_7"
+    --content-source-id {{ katello_provisioning_smart_proxy.Id }}
+    --environment production
+    --puppet-ca-proxy-id {{ katello_provisioning_smart_proxy.Id }}
+    --puppet-proxy-id {{ katello_provisioning_smart_proxy.Id }}
+    --subnet "192.168.73.0/24"
+    --locations "{{ katello_provisioning_location }}"
+  when: katello_provisioning_hg.stderr.find('not found') != -1
+
+- name: 'set enable-puppet4 parameter'
+  shell: >
+      {{ katello_provisioning_hammer }} hostgroup set-parameter
+      --hostgroup "Forklift CentOS 7"
+      --name enable-puppet4
+      --value true