Intro to the ChefDK

Intro to the ChefDK UUASC Meet-Up Ned Harris

Ned Harris Solutions Architect, CHEF Twitter: @nedward777 Email: [email protected] Linkden: nedward777

Agenda 1. Quick intro to Chef 2. Explore the ChefDK 3. Build some Infrastructure 4. Q&A

Quick Intro to Chef

A Taste of Chef Chef is a large set of tools that are able to be used on multiple platforms and in numerous configurations. We will have time to only explore some of its most fundamental pieces.

So why would I want to use Chef? …

Infrastructure is Complicated

Hard Style vs. Soft Style

Force things to be simpler

Accept the complexity and learn to manage it

Managing Complexity SSH, make with the typey typey

Managing Complexity SSH, make with the typey typey Keep notes in ~/server.txt

Managing Complexity SSH, make with the typey typey Keep notes in ~/server.txt Move notes to the wiki

Managing Complexity SSH, make with the typey typey Keep notes in ~/server.txt Move notes to the wiki Custom scripts (setup.sh)

Managing Complexity SSH, make with the typey typey Keep notes in ~/server.txt Move notes to the wiki Custom scripts (setup.sh) Golden Images

Gold is Heavy!

Typical Infrastructure Graphite

Nagios

Jboss App Memcache Postgres Slaves Postgres Master

New Compliance Mandate! Graphite

Nagios

Jboss App

•  Move SSH off port 22 •  Lets put it on 2022

Memcache Postgres Slaves Postgres Master

6 Golden Image Updates Graphite

1

2

Nagios

3

Jboss App 4

Memcache

5

Postgres Slaves

•  edit /etc/ssh/sshd_config

6

Postgres Master

12 Instance Replacements Graphite 3

•  Delete, launch •  Repeat •  Typically manually

1

2

Nagios

4

5

6

8

9

Memcache

10

11

Postgres Slaves

12

Postgres Master

7

Jboss App

Done in Maintenance Windows Graphite 3

•  Don’t break anything! •  Bob just got fired =(

1

2

Nagios

4

5

6

8

9

Memcache

5 10

11

Postgres Slaves

12 12

Postgres Master

7

Jboss App

Different IP Addresses? Graphite

Nagios

Jboss App Memcache Postgres Slaves

•  Invalid configs!

Postgres Master

Managing Complexity SSH, make with the typey typey Keep notes in ~/server.txt Move notes to the wiki Custom scripts (setup.sh) Golden Images Policy-driven configuration management

Enter Chef… Chef is an automation framework that enables Infrastructure as Code Chef leverages reusable definitions to automate desired state Chef is API driven Chef supports Linux variants, Unix variants, AIX and Windows, all as first class citizens.

The Chef Software Platform Management Console

High Availability and Replication

Analytics Platform

Chef Client Nodes

Chef Server

Cookbook and Policy Authoring

Chef Development Kit

Data Center

Test-Driven Infrastructure

The Cloud

Building Blocks

Building Blocks: What is a Resource? A Resource is a system state you define Example: Package installed, state of a service, configuration file existing You declare what the state of the resource is Chef automatically determine HOW that state is achieved resource one

package "apache2"

Test and Repair Resources follow a test and repair model

package "vim"


package "vim"

Is vim installed?


package "vim"

Is vim installed? Yes


package "vim"

Is vim installed? Yes Done


package "vim"


No


package "vim"


No Install it


package "vim"


No Install it

Resources – Test and Repair Resources follow a test and repair model Resource currently in the desired state? (test) Yes – Do nothing No – Bring the resource into the desired state (repair)

Building Blocks: What is a Recipe? A recipe is a collection of Resources Resources are executed in the order they are listed

Recipe - a collection of resources package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode "0644" notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => :true action [:enable, :start] end

Recipes – Order Matters package "haproxy" do

Resources are applied in order

action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode "0644" notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => :true action [:enable, :start] end







Building Blocks: What is a Cookbook? A cookbook is a set of recipes A cookbook is a defined set of items and different outcomes that you expect to address A cookbook could have a recipe to install apache2/ httpd but also another set of recipes to activate modules required.

./attributes ./attributes/default.rb ./CHANGELOG.md ./metadata.rb ./README.md ./recipes ./recipes/application.rb ./recipes/balancer.rb ./recipes/database.rb ./recipes/default.rb ./recipes/webserver.rb ./templates ./templates/default ./templates/default/mysite.conf.erb

chef-client chef-client is an executable performs all actions required to bring the node into the desired state typically run on a regular basis daemon cron Windows service

Included with ChefDK

chef-client applying policies

chef-client applying policies chef-client



Test & Repair Apply the policy

chef-client applying policies repeatedly chef-client



Test & Repair Apply the policy chef-client






chef-client modes In conjunction with a Chef Server Local mode (no Chef Server)

Dynamic Data Gathered via Ohai "os": "linux", "os_version": "2.6.32-431.el6.x86_64", "lsb": { }, "platform": "centos", "platform_version": "6.5", "platform_family": "rhel", "virtualization": { "systems": { "vbox": "guest" }, "system": "vbox", "role": "guest" },

Accessing Dynamic Data node

ipaddress

platform

Represented in a recipe

node[virtualization][system]

virtualization

system

Accessing Dynamic Data node

ipaddress

platform

virtualization

system Create a list of all machines running centos

centosmachines = search("node",”platform:centos”)

So when this… Graphite

Nagios


…becomes this… Graphite

Nagios


…this can happen automatically! Graphite

Nagios


Count the resources Graphite

Nagios Jboss App Memcache Postgres Slaves

•  12+ resource changes for 1 node addition

•  •  •  •  •  •  •  •  •  •  •  • 

Load balancer config Nagios host ping Nagios host ssh Nagios host HTTP Nagios host app health Graphite CPU Graphite Memory Graphite Disk Graphite SNMP Memcache firewall Postgres firewall Postgres authZ config

Questions?

My Tool Kit •  •  •  •  • 

ChefDK Git Virtual Box Vagrant Sublime Text

ChefDK The omnibus installer is used to set up the Chef development kit on a workstation, including the chef-client itself, an embedded version of Ruby, RubyGems, OpenSSL, key-value stores, parsers, libraries, command line utilities, and community tools such as Kitchen, Berkshelf, and ChefSpec. https://downloads.chef.io/chef-dk/

ChefDK: In the Box First Class Support on Windows, Linux, and OSX for the entire suite of Chef development tools ●  Test Kitchen: Virtualized testing harness ●  Berkshelf: Dependency solver ●  Chef-Vault: Secrets management ●  Rubocop / Foodcritic: Code linting ●  Chefspec: In-memory Unit Testing ●  Chef.bin: New wrapper binary to tie it all together, with new extensible cookbook generators.

git Git is a distributed revision control system with an emphasis on speed, data integrity, and support for distributed, non-linear workflows. Git was initially designed and developed by Linus Torvalds for Linux kernel development in 2005, and has since become the most widely adopted version control system for software development. http://git-scm.com/downloads

VirtualBox Desktop Virtualization Main appealing quality… Its free

https://www.virtualbox.org/

Vagrant Vagrant is a tool for building complete development environments. With an easy-touse workflow and focus on automation.

https://www.vagrantup.com/

Sublime Text Sublime Text is a sophisticated text editor for code, markup and prose. You'll love the slick user interface, extraordinary features and amazing performance. http://www.sublimetext.com

Let’s stand up some Infrastructure!

Test Kitchen Test harness to execute code on one or more platforms Driver plugins to allow your code to run on various cloud and virtualization providers Includes support for many testing frameworks Included with ChefDK

Test Matrix Two operating systems

ubuntu-12.04 centos-6.4

Test Matrix Two operating systems One recipe

default ubuntu-12.04

apache::default

centos-6.4

apache::default

Test Matrix Two operating systems Two recipes

default

ssl

ubuntu-12.04

apache::default apache::ssl

centos-6.4


Test Matrix Three operating systems Two recipes

default

ssl

ubuntu-12.04


centos-6.4


ubuntu-14.04


Configuring the Kitchen OPEN IN EDITOR: apache/.kitchen.yml --driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] attributes:

SAVE FILE!

.kitchen.yml driver - virtualization or cloud provider

--driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] attributes:

.kitchen.yml provisioner application to configure the node


.kitchen.yml platforms - target operating systems


.kitchen.yml suites - target configurations


.kitchen.yml default ubuntu-12.04

apache::default

centos-6.4

apache::default

--driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4

suites: - name: default run_list: - recipe[apache::default]

.kitchen.yml default

ssl

ubuntu-12.04


centos-6.4


--driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] - name: ssl run_list: - recipe[apache::ssl]

.kitchen.yml default

ssl

ubuntu-12.04


centos-6.4


ubuntu-14.04


--driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 - name: ubuntu-14.04 suites: - name: default run_list: - recipe[apache::default] - name: ssl run_list: - recipe[apache::ssl]

.kitchen.yml The configuration file for your Test Kitchen driver – virtualization or cloud provider provisioner – application to configure the node platforms – target operating systems suites – target configurations

kitchen create

kitchen login

kitchen login

[chef@ip-172-31-44-173 apache]$ kitchen login

kitchen login

ssh [chef@ip-172-31-44-173 apache]$ kitchen login

kitchen login

[kitchen@5379d310dc59 ~]$

ssh [chef@ip-172-31-44-173 apache]$ kitchen login

kitchen converge Install Chef Upload cookbooks Apply the run_list

Serverspec Write tests to verify your servers Not dependent on Chef Defines many resource types package, service, user, etc.

Works well with Test Kitchen http://serverspec.org/

Default location for tests Test Kitchen will look in the test/ integration directory for test-related files

Suite subdirectory The next level subdirectory will match the suite name.

test/ └── integration └── default └── serverspec └── default_spec.rb suites: - name: default run_list: recipe[apache::default]

Suite subdirectory The next level subdirectory will match the suite name.


Busser subdirectory Test Kitchen utilizes bussers to manage test plugins. We’ll be using the serverspec plugin


Generic Expectation Form describe "" do it "" do expect(thing).to eq result end end

The Chef workflow

Create new skeleton cookbook.

Create a VM environment for cookbook development.

Write/debug cookbook recipes (iterative step).

Perform acceptance tests.

Deploy to production.

Further Resources

What do I do now? To learn more you are going to need more practice and more resources. It takes a village and a couple of web servers. OBJECTIVE: q  Provide you with resources that you can read q  Provide you with resources that you can watch q  Provide you with resources that you can listen q  Provide you with resources that you can attend

The slides from this workshop. http://bit.ly/1MuhF9e

docs.chef.io

Learning Chef A Guide to Configuration Management and Automation

Customizing Chef Getting the Most Out of Your Infrastructure Automation

learnchef.com Activity focused learning for those new to Chef.

What do I do now?

To learn more you are going to need more practice and more resources. It takes a village and a couple of web servers. OBJECTIVE: ü  Provide you with resources that you can read q  Provide you with resources that you can watch q  Provide you with resources that you can listen q  Provide you with resources that you can attend

Youtube Channel •  ChefConf Talks •  Training Videos

https://www.youtube.com/user/getchef/playlists

foodfightshow.org Food Fight is a bi-weekly podcast for the Chef community. We bring together the smartest people in the Chef community and the broader DevOps world to discuss the thorniest issues in system administration.

What do I do now?

To learn more you are going to need more practice and more resources. It takes a village and a couple of web servers. OBJECTIVE: ü  Provide you with resources that you can read ü  Provide you with resources that you can watch q  Provide you with resources that you can listen q  Provide you with resources that you can attend

theshipshow.com The Ship Show is a twice-monthly podcast, featuring discussion on everything from build engineering to devops to release management, plus interviews, new tools and techniques, and reviews.

foodfightshow.org Food Fight is a bi-weekly podcast for the Chef community. We bring together the smartest people in the Chef community and the broader DevOps world to discuss the thorniest issues in system administration.

What do I do now?

To learn more you are going to need more practice and more resources. It takes a village and a couple of web servers. OBJECTIVE: ü  Provide you with resources that you can read ü  Provide you with resources that you can watch ü  Provide you with resources that you can listen q  Provide you with resources that you can attend

Chef Developers' IRC Meeting Join members of the Chef Community in a meeting for Chef Developers where we’ll discuss the future of the Chef project and other things pertinent to the community. irc.freenode.net#chef-hacking https://github.com/chef/chef-community-irc-meetings

Chef Community Summit The Chef Community will gather for two days of open space sessions and brainstorm on Chef best practices. The Chef Community Summit is a facilitated Open Space event. The participants of the summit propose topics, organize an agenda, and discuss and work on the ideas that are most important to the community.

https://www.chef.io/summit/

ChefConf It’s the gathering of hundreds of Chef community members. We get together to learn about the latest and greatest in the industry (both the hows and the whys), as well as exchange ideas, brainstorm solutions, and give hugs, which has become the calling card of the DevOps community, and the Chef community in particular.

https://www.chef.io/chefconf/

Discussion What questions can we answer for you?