Container Images. â An image is a stack of Read-Only file ... horizontal auto-scaling naming and ... kubectl run my-ng
@briandorsey #kubernetes #GOTOber
1
Kubernetes Changing the way we think and talk about computing
GOTO Berlin - December 2015
@briandorsey #kubernetes #GOTOber
2
What is this talk? Container
@briandorsey #kubernetes #GOTOber
3
Brian Dorsey Developer Advocate - Google Cloud platform +BrianDorsey @briandorsey
@briandorsey #kubernetes #GOTOber
4
Containers? Yes/No if yes GOTO slide 7; if no GOTO slide 15;
@briandorsey #kubernetes #GOTOber
5
Containers
The Old Way: Shared Machines No isolation app
app
app
app
No namespacing Common libs Highly coupled apps and OS
libs kernel Bare Metal
@briandorsey #kubernetes #GOTOber
7
The Old Way: Virtual Machines Some isolation app
app
libs
libs
kernel
kernel
app
app
libs
libs
kernel
kernel
Expensive and inefficient Still highly coupled to the guest OS Hard to manage
Hypervisor
@briandorsey #kubernetes #GOTOber
8
The New Way: Containers App specific isolation Lightweight & efficient
app
app
libs
libs
app
app
libs
libs
Independent of the host Linux distribution … Lots of containers to manage kernel Hypervisor
@briandorsey #kubernetes #GOTOber
9
Container Images ●
An image is a stack of Read-Only file system layers.
●
Usual process: ○ build ○ push to repository ○ pull to execution host ○ start container from image
App PHP & Apache Libs Debian
@briandorsey #kubernetes #GOTOber
10
Image Layers Read / Write
●
●
A container is a process ○ started with kernel restrictions ○ a stack of shared Read-Only file system layers ○ plus a process specific ReadWrite layer
Read / Write
Read / Write
App PHP & Apache Libs Debian
Every new container gets a new Read-Write later. All containers from the same image start from exactly the same state! @briandorsey #kubernetes #GOTOber
11
Mounting Host Directories ●
●
●
It's possible to mount host directories into a container's filesystem. These are mutable and do outlive the container.
Read / Write
host dir
App PHP & Apache Libs
They're only available on that host.
Debian
@briandorsey #kubernetes #GOTOber
12
Why containers? • Performance • Repeatability • Quality of service • Accounting • Portability A fundamentally different way of managing applications
@briandorsey #kubernetes #GOTOber
Images by Connie Zhou 13
Containers are awesome! Let’s run lots of them!
@briandorsey #kubernetes #GOTOber
14
Kubernetes
PaaS
Virtual Machines Physical Computers @briandorsey #kubernetes #GOTOber
16
PaaS Container Clusters Virtual Machines Physical Computers @briandorsey #kubernetes #GOTOber
17
Kubernetes Greek for “Helmsman”; also the root of the words “governor” and “cybernetic” • Runs and manages containers • Inspired and informed by Google’s experiences and internal systems • Supports multiple cloud and bare-metal environments • Supports multiple container runtimes • 100% Open source, written in Go Manage applications, not machines @briandorsey #kubernetes #GOTOber
18
Everything at Google runs in containers: • • • •
Gmail, Web Search, Maps, ... MapReduce, batch, ... GFS, Colossus, ... Even Google’s Cloud Platform: VMs run in containers!
@briandorsey #kubernetes #GOTOber
19
Everything at Google runs in containers: • • • •
Gmail, Web Search, Maps, ... MapReduce, batch, ... GFS, Colossus, ... Even Google’s Cloud Platform: VMs run in containers!
We launch over 2 billion containers per week @briandorsey #kubernetes #GOTOber
20
A toolkit for running distributed systems in production co-locating helper processes
naming and discovery
mounting storage systems
load balancing
distributing secrets
rolling updates
application health checking
resource monitoring
replicating application instances
log access and ingestion
horizontal auto-scaling
support for introspection and debugging
@briandorsey #kubernetes #GOTOber
21
Start with a Cluster Laptop to high-availability multi-node cluster Hosted or self managed On-Premise or Cloud Bare Metal or Virtual Machines Most OSes (inc. RedHat Atomic, Fedora, CentOS) Or just a bunch of Raspberry PIs Many options, See Matrix for details Kubernetes Cluster Matrix: http://bit.ly/1MmhpMW @briandorsey #kubernetes #GOTOber
22
Start with a Cluster Kubernetes Master Scheduler
API Server
Kubelet
Kubelet
Kubelet
@briandorsey #kubernetes #GOTOber
23
Setting up a cluster ●
Choose a platform: GCE, AWS, Azure, Rackspace, Ubuntu, Juju … ○ Then run: export KUBERNETES_PROVIDER=; curl -sS https://get.k8s.io | bash
● ● ●
Or choose a distro such as RedHat Atomic, CoreOS Tectonic, Mirantis Murano (OpenStack), Mesos Or use a recipes for bare metal configurations for Centos, Fedora, etc Use a hosted option such as Google Container Engine
@briandorsey #kubernetes #GOTOber
24
Deploy containers $ kubectl run my-nginx --image=nginx --replicas=2 --port=80
@briandorsey #kubernetes #GOTOber
25
@briandorsey #kubernetes #GOTOber
26
A pod of whales containers Consumers
The atom of scheduling for containers An application specific logical host Hosts containers and volumes Each has its own routable IP address (no NAT)
Web Server
Ephemeral • Pods are functionally identical and therefore ephemeral and replaceable
Pod
@briandorsey #kubernetes #GOTOber
Volume
27
Pods Can be used to group multiple containers & shared volumes
Consumers
git Repo
Containers within a pod are tightly coupled Shared namespaces • Containers in a pod share IP, port and IPC namespaces • Containers in a pod talk to each other through localhost
Git Synchronizer
Pod
@briandorsey #kubernetes #GOTOber
Node.js App Container
Volume
28
Pod Networking (across nodes) Pods have IPs which are routable 10.1.2.106
Pods can reach each other without NAT Even across nodes
10.1.2.0/24
No Brokering of Port Numbers These are fundamental requirements
10.1.1.211
10.1.1.2
10.1.1.0/24
Many solutions Flannel, Weave, OpenVSwitch, Cloud Provider
10.1.3.45 10.1.3.17
10.1.3.0/24 10.1.3.0/24
@briandorsey #kubernetes #GOTOber
29
Create a service $ kubectl expose rc my-nginx --port=80 --type=LoadBalancer
@briandorsey #kubernetes #GOTOber
30
Services Client
A logical grouping of pods that perform the same function • grouped by label selector VIP
Load balances incoming requests across constituent pods
Service Label selector: type = FE
Choice of pod is random but supports session affinity (ClientIP) Gets a stable virtual IP and port • also a DNS name
Pod Container type = FE
@briandorsey #kubernetes #GOTOber
Pod Container type = FE
Pod Container Container type = FE 31
Labels ← These are important Dashboard
Dashboard Pod
show: type = FE type = FE
Pod Pod
Pod Pod
frontend
frontend
type = FE
show: version = v2
version = v2
version = v2
Behavior ● ● ●
Benefits
Metadata with semantic meaning Membership identifier The only Grouping Mechanism
➔ ➔ ➔
Allow for intent of many users (e.g. dashboards) Build higher level systems … Queryable by Selectors
@briandorsey #kubernetes #GOTOber
32
Replication Controllers Replication Replication Controller Controller #pods = 2 version = v1
Pod
version= v1
Pod Pod
Pod Pod
frontend
frontend
version = v1
version = v2
Replication Controller #pods = 1 version = v2 show: version = v2
show: version = v2
● ● ●
Behavior
Keeps Pods running Gives direct control of Pod #s Grouped by Label Selector
Benefits ➔ ➔ ➔
Recreates Pods, maintains desired state Fine-grained control for scaling Standard grouping semantics
@briandorsey #kubernetes #GOTOber
33
Replication Controller
Replication Controllers
Replication Controller
Canonical example of control loops Have one job: ensure N copies of a pod ● ● ●
if too few, start new ones if too many, kill some group == selector
Replicated pods are fungible ●
-
Name = “backend” Selector = {“name”: “backend”} Template = { ... } NumReplicas = 4 Start 1 more
How many?
3
How many?
OK
4
No implied order or identity API Server
@briandorsey #kubernetes #GOTOber
34
Managing Deployments
Scale $ kubectl scale rc my-nginx --replicas=5
@briandorsey #kubernetes #GOTOber
36
Scaling Example Replication Replication Controller Controller version = v1 type = FE 1 4 #pods = 2
Pod
Pod Pod
Pod Pod
frontend
frontend
Pod
version= v1
version = v1
version = v1
version = v1
type = FE
type = FE
type = FE
type = FE
show: version = v2
Service Service name = frontend Label selectors: version = 1.0 Label selector: type = Frontend type = FE @briandorsey #kubernetes #GOTOber
37
Canary Replication Replication Controller Controller version = v1 type = BE #pods = 2
Pod
Pod Pod
Pod Pod
frontend
frontend
version= v1
version = v1
version = v2
type = BE
type = BE
type = BE
show: version = v2
Replication Replication Controller Controller version = v2 type = BE #pods = 1
show: version = v2
Service Service name = backend Label selectors: version = 1.0 Label selector: type = Frontend type = BE @briandorsey #kubernetes #GOTOber
38
Rolling Update $ kubectl rolling-update frontend --image=frontend:v2
@briandorsey #kubernetes #GOTOber
39
Rolling Update Replication Replication Controller Controller version = v1 type = BE #pods = 2
Pod
Pod Pod
Pod
frontend version= v1
version = v2 v1
version = v2
type = BE
type = BE
type = BE
show: version = v2
Replication Controller version = v2 type = BE #pods = 2
show: version = v2
Service Service name = backend Label selectors: version = 1.0 Label selector: type = Frontend type = BE @briandorsey #kubernetes #GOTOber
40
Autoscale $ kubectl autoscale rc frontend --min=1 --max=20
@briandorsey #kubernetes #GOTOber
41
Pod Horizontal Autoscaling Beta (1.1) Replication Replication Controller Controller name=locust role=worker 2 1 #pods = 4
Pod
Pod Pod
Pod Pod
frontend
frontend
Pod
name=locust
name=locust
name=locust
name=locust
role=worker
role=worker
role=worker
role=worker
show: version = v2 70% CPU
Scale CPU Target% = 50
40% CPU
Heapster >< 50% CPU
@briandorsey #kubernetes #GOTOber
42
Managing State
I still have questions about state!
Database
In a cluster of ephemeral containers
Application state must exist outside of the container
@briandorsey #kubernetes #GOTOber
44
Volumes Bound to the Pod that encloses it Look like Directories to Containers What and where they are determined by Volume Type Many Volume options EmptyDir HostPath nfs, iSCSI (and similar services) Cloud Provider Block Storage
Pod
@briandorsey #kubernetes #GOTOber
45
Outside the Cluster
Database
e.g.: MySQL managed by DBAs or managed cloud services
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
@briandorsey #kubernetes #GOTOber
46
Adapt to run in the Cluster App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
Database
e.g.: MySQL runs in a pod and mounts a filesystem provided by the cluster
@briandorsey #kubernetes #GOTOber
47
Cluster Native App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
App Pod
ex: run Cassandra or Riak inside the cluster @briandorsey #kubernetes #GOTOber
48
Cluster native - MySQL on Vitess Open source MySQL scaling solution Vitess has been serving all YouTube database traffic since 2011 Replication, dynamic sharding, caching and more Designed for a distributed, containerized world Kubernetes configs included
http://vitess.io/
@briandorsey #kubernetes #GOTOber
49
Secrets Problem: how to grant a pod access to a secured something? ● don’t put secrets in the container image! 12-factor says: config comes from the environment ● Kubernetes is the environment
Pod
Secret
node
Manage secrets via the Kubernetes API Inject them as virtual volumes into Pods ● late-binding ● tmpfs - never touches disk @briandorsey #kubernetes #GOTOber
App Pod
50
Wrap-up
Kubernetes status & plans Open sourced in June, 2014 v1.0 in July, 2015, v1.1 in November 2015 Google Container Engine (GKE) ● hosted Kubernetes - don’t think about cluster setup ● GA in August, 2015
PaaSes: ● RedHat OpenShift, Deis, Stratos
Distros: ● CoreOS Tectonic, Mirantis Murano (OpenStack), RedHat Atomic, Mesos
Working towards a 1.2 release @briandorsey #kubernetes #GOTOber
52
Google Container Engine (GA) -- Demo Managed Kubernetes (Kubernetes v1.1) Manages Kubernetes master uptime Manages Updates Cluster Resize via Managed Instance Groups Cluster Node Autoscaling Centralized Logging Google Cloud VPN support
@briandorsey #kubernetes #GOTOber
53
Kubernetes is Open Source We want your help! http://kubernetes.io https://github.com/GoogleCloudPlatform/kubernetes Slack: #kubernetes-users @kubernetesio
@briandorsey #kubernetes #GOTOber
54
PaaS
Yo
ap ur p?
Container Clusters Virtual Machines Physical Computers @briandorsey #kubernetes #GOTOber
55
Tweet questions afterwards to: @briandorsey Slides: goo.gl/NI1GaM
Questions @briandorsey #kubernetes #GOTOber
56
@briandorsey #kubernetes #GOTOber
57