Chef Compliance demo & office hour

Demo some latest and greatest Chef Compliance things!

Audit Cookbook updates

Initially the audit cookbook supported reporting 3 ways:

• Directly to Compliance
• To Compliance via Chef Server + Compliance integration
• Directly to Visibility

Some recent reporting additions:

• moved from converge phase to running as report handler (NEW!)
• json file (NEW!)
• Multiple reporting endpoints (NEW!)
• fetch profiles from Automate and report to Visibility through Chef Server proxy, without any changes to client.rb (NEW!)

Store profiles in Automate

There is now an option to store Inspec profiles via a new asset store in Automate! Version Requirement:

• automate 0.6.6

Using the Automate asset store

Enable the feature in Automate

To enable profile asset storage:

# delivery.rb
compliance_profiles['enable'] = true

inspec cli

$ inspec compliance help
# this is using the --dctoken (data collector token from chef-server.rb)
$ inspec compliance login_automate https://automate-server.test --insecure true --user admin --dctoken 93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506 --ent brewinc
# this is using the --usertoken (Automate user token)
$ inspec compliance login_automate https://automate-server.test --insecure true --user admin --usertoken 30284287ee4054ff8807fb63d9cbd1c506= --ent brewinc
$ inspec compliance upload /Users/jmiller/Devel/compliance-profiles/ssh.tar.gz

Full example: https://gist.github.com/jeremymv2/cb34e6dfcad040b1cad50636d256b44e

Automate API

$ curl -X POST "https://automate-server.test/compliance/profiles/jmiller" \
-H "chef-delivery-enterprise: brewinc" -H "chef-delivery-user: jmiller" \
-H "chef-delivery-token: tzwlbWMtgBC0lo6sxkAYKSShxSJEohnU7IAE4NCUGCg=" \
--form "file=@/Users/jmiller/Devel/compliance-profiles/ssh.tar.gz" -k -D -

HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Server: openresty
Date: Mon, 05 Dec 2016 17:52:30 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Connection: keep-alive
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, X-Requested-With
Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE, PATCH
Access-Control-Allow-Origin:

Full example: https://gist.github.com/jeremymv2/a1cf41bbd2d3a5250e3cf97131efa1a5

Thoughts on Compliance Server

Question: "With the advent of Automate Compliance profile storage, why even bother with Chef Compliance server?"

Answer: "At the moment, the Compliance server is the only option with a UI for managing profiles and running remote ssh scans for non-chef clients. However, the Automate Compliance Profile UI will be added soon as well as other functionality, so Chef Compliance would be a better choice only for non-Chef ecosystems."

Testing out the NEW

Let's examine what it takes to run inspec scan of a node migrating

From this:

• Audit Cookbook 2.x
• Profiles Stored on Compliance Server
• Reports sent directly to Visibility (requires client.rb settings for data_collector)

To this:

• Audit Cookbook 2.x
• Profiles Stored in Automate (Requires >= 0.6.6)
• Inspec Scan Reports + Converge data sent to Visibility with NO client.rb data_collector settings required (requires Chef Client >= 12.16.42 and Chef Server >= 12.11.0)

Assumption: 'linux' and 'ssh' profiles have been uploaded to Automate under user 'jmiller' per example above.

We will define our Audit profile settings via a cookbook wrapper of the audit cookbook, called audit_wrapper

DEPRECATION WARNING: If you're still using version Audit Cookbook 1.x you may have something like:

# wrapper cookbook
default['audit']['collector'] = 'chef-visibility'
default['audit']['server'] = 'https://compliance-server.test/api'
default['audit']['refresh_token'] = '2/-YL_ht4owKI1WzczoDDXNhluoZl9Va8nEHpQyBF0w7OFCIa__RZ5vYrcfe5TB_ypycUeFN7BNVhs_4A5HgSvAw=='
default['audit']['profiles'] =
  {
    'base/linux' => true
  }

Reporting with data_collector defined in client.rb

The fetcher attribute was introduced in Audit 2.0.0 Also, in 2.x profiles are now an array of hashes

The attributes below will fetch profiles via Compliance Server, post reports to Visibility and requires the Chef Server Integration with Compliance Server

# wrapper cookbook
default['audit']['collector'] = 'chef-visibility'
default['audit']['fetcher'] = 'chef-server'
default['audit']['profiles'] = [
  {
    'name' => 'linux',
    'compliance' => 'base/linux'
  }
]

In the scenario above, the client node MUST be configured with the data_collector for ingesting data into Automate

# client.rb
data_collector['server_url'] = 'https://automate-server.test/data-collector/v0/'
data_collector['token'] = '93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506'
...

No more data_collector in client.rb!

In this scenario, the audit cookbook now requires minimum attribute settings. The configuration below will fetch profiles from Automate and report through the Chef Server, without requiring any data_collector settings in the client.rb

# wrapper cookbook
default['audit']['collector'] = 'chef-server-visibility'
default['audit']['profiles'] = [
  {
    'name' => 'linux',
    'compliance' => 'jmiller/linux'
  }
]

No more client side data_collector settings to manage!

# client.rb
# data_collector['server_url'] = 'https://automate-server.test/data-collector/v0/'
# data_collector['token'] = '93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506'

However, you must configure the Chef Server. Note: root_url is used on Chef Server, not server_url (which is used on client side in client.rb) Note: You MUST specify both the root_url AND token.

# chef-server.rb
data_collector['root_url'] = 'https://automate-server.test/data-collector/v0/'
data_collector['token'] = '93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506'
profiles['root_url'] = 'https://automate-server.test'

Meta Profiles

Reference: http://lollyrock.com/articles/chef-compliance-meta-profiles/

Meta profiles are out! A meta profile is an overlay or a collection of multiple profiles.

Some Uses-cases are:

• a deviation from CIS benchmarks
• collection of all CIS profiles that apply to your infrastructure (eg. a company-wide profile)

A meta profile contains all its dependencies as vendored/bundled profiles so that it can run inside an Air Gapped Environment without requirement for Internet access.

An example (https://github.com/chris-rock/acme-inspec-profile):

$ cat acme-inspec-profile/inspec.yml
name: acme-inspec-profile
title: Meta profile for Acme Inc
maintainer: Christoph Hartmann
copyright: Christoph Hartmann
copyright_email: chris@lollyrock.com
license: Apache 2.0
summary: This profile collects all compliance and security related requirements for Acme Inc.
version: 0.1.0
depends:
  - name: linux-patch-benchmark
    git: https://github.com/dev-sec/linux-patch-benchmark.git
  - name: windows-patch-benchmark
    git: https://github.com/dev-sec/windows-patch-benchmark.git
  - name: os-hardening
    git: https://github.com/dev-sec/tests-os-hardening.git
  - name: ssh-hardening
    git: https://github.com/dev-sec/tests-ssh-hardening.git
  - name: ssl-benchmark
    git: https://github.com/dev-sec/ssl-benchmark.git

Dependency locks:

$ cat acme-inspec-profile/inspec.lock
---
lockfile_version: 1
depends:
- name: linux-patch-benchmark
  resolved_source:
    git: https://github.com/dev-sec/linux-patch-benchmark.git
    ref: d53030317b711f36fa2fde9e18170ce6b4eaacf2
  version_constraints: ">= 0"
- name: windows-patch-benchmark
  resolved_source:
    git: https://github.com/dev-sec/windows-patch-benchmark.git
    ref: c183d08eb25638e7f5eac97e521640ea314c8e3d
  version_constraints: ">= 0"
- name: os-hardening
  resolved_source:
    git: https://github.com/dev-sec/tests-os-hardening.git
    ref: da3a1b6ce8a845d6818152a824e123c2445c355f
  version_constraints: ">= 0"
- name: ssh-hardening
  resolved_source:
    git: https://github.com/dev-sec/tests-ssh-hardening.git
    ref: 75754b9b3fe45c601f0fa0036b01c61c8b8e26d9
  version_constraints: ">= 0"
- name: ssl-benchmark
  resolved_source:
    git: https://github.com/dev-sec/ssl-benchmark.git
    ref: e17486c864434c818f96ca13edd2c5a420100a45
  version_constraints: ">= 0"

Example controls:

$ cat acme-inspec-profile/controls/example.rb
# encoding: utf-8
# copyright: 2015, The Authors
# license: All rights reserved

# import full profile
include_controls 'hardening/ssh-hardening'

# select only individual controls
include_controls 'ssl-benchmark' do
  control "tls1.2"
end

# inspec knows that it cannot run Windows tests on Linux
include_controls 'windows-patch-benchmark'

Additional Resources

• managing Chef Client client.rb settings: https://github.com/chef-cookbooks/chef-client
• upgrading Chef Client: https://github.com/chef-cookbooks/omnibus_updater