blog-post-getting-rails-on-kubernetes

https://engineering.adwerx.com/rails-on-kubernetes-8cd4940eacbe#.s3ufpxe0w

kubernetes-open-questions

Kubernetes master may also be a worker node and have pods scheduled to it.

Some commands

blog-post-getting-rails-on-kubernetes#some-commands

kubectl get pods

kubectl describe pod [pod name] (also shows last event)

kubectl create -f ./some/path

kubectl apply -f ./some/path (to make changes)

kubectl port-forward [pod name] 3606:3606 (get access to it on your local machine)

kubectl delete job deploy-tasks

kubectl describe job deploy-tasks

kubectl attach (TODO - what is this?)

blog-post-getting-rails-on-kubernetes#some-commands

livenessProbe and readynessProbe for health checks

In a deployment, nodeSelector will give you an affinity for a specific node-pool.

There is a load balancer service.

HorizontalPodAutoscaler

Within that file, there is an apiVersion: extensions/v1beta1

He creates a template that he pipes into kubectl (adds .tmpl suffix to the file). For that he uses envsubst in the gettext package.

In his script he uses jsonpath.

Collecting metrics

blog-post-getting-rails-on-kubernetes#dd-agent-and-daemon-set

dd-agent kubernetes-docs-deamon-set spec - sends health of every node, pod, and kube-system service to Datadog

blog-post-getting-rails-on-kubernetes#dd-agent-and-daemon-set

References

blog-post-lessons-learned-from-one-year-kubernetes

• addons
  • At 8:35 DNS and UI are addons (#)
• API Server
  • AKA apiserver
  • TODO - does this run in the cluster itself?
  • a RESTful API - has swagger support
  • "a number of processes that implement the Kubernetes API" - kubernetes-docs-high-availability
  • installed using the kubelet - kubernetes-docs-high-availability
  • Only the apiserver should communicate with etcd/store, and not other components (scheduler, kubelet, etc.). (#)
  • In contrast to Omega, which exposes the store directly to trusted control-plane components, state in Kubernetes is accessed exclusively through a domain-specific REST API that applies higher-level versioning, validation, semantics, and policy, in support of a more diverse array of clients. (#)
  • the centralization makes it easy to enforce common semantics, invariants, and policies (#)
• annotation
  • TODO
  • from kubernetes-kubectl-commands "Update the annotations on a resource"
  • for non-identifying information... information would not be used for object selection and therefore doesn't belong in labels
  • annotations... "can be used to communicate application structure" (#)
  • "annotations can be set by the container itself or other actors in the management system" (#)
• autoscale (verb)
  • Things that can be autoscaled: deployment, replica set, or replication controller
• backend
  • TODO from kubernetes-docs-services
• claim
  • AKA Persistent Volume Claim
  • AKA PVC
  • A PersistentVolumeClaim (PVC) is a request for storage by a developer. It is similar to a pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific storage size and access modes such as read and read-write. (#)
• cluster
  • From kubernetes-docs-ingress "A group of nodes firewalled from the internet, that are the primary compute resources managed by Kubernetes."
  • From kubernetes-code-readme "a set of physical or virtual machines and other infrastructure resources used by Kubernetes to run your applications. Kubernetes can run anywhere! See the Getting Started Guides for instructions for a variety of services."
• cluster add-on
  • TODO
• cluster IP
  • At 37:50 - The default option is exposing the service port as a cluster IP (#)
  • At 48:50 - An intranet-only service is called a cluster ip (#)
• cluster manager
  • from kubernetes-kubectl-commands "kubectl controls the Kubernetes cluster manager"
• cluster network
  • from kubernetes-docs-ingress "A set of links, logical or physical, that facilitate communication within a cluster according to the Kubernetes networking model. Examples of a Cluster network include Overlays such as flannel or SDNs such as OVS."
• component
  • TODO - unclear
  • in kubectl get cs (get component statuses) it gives statuses for etcd, the scheduler, and the controller manager
  • ... still working on a few things, like making kubelet itself (all our components, really) run within containers (#)
  • "The master is the machine where the control plane components run, including etcd (the cluster database) and the API server (which the kubectl CLI communicates with). All of these components run in pods started by kubelet." (#)
• config maps
  • "provides mechanisms to inject containers with configuration data while keeping containers agnostic of Kubernetes" (#)
  • can be used to store fine-grained information like individual properties or coarse-grained information like entire config files or JSON blobs (#)
  • is similar to secrets, but designed to more conveniently support working with strings that do not contain sensitive information. (#)
  • Both users and system components may store configuration data in ConfigMap (#)
• control plane
  • The Kubernetes control plane is split into a set of components. Currently they all run on a single master node, but that is expected to change soon in order to support high-availability clusters. These components work together to provide a unified view of the cluster. (#)
  • "The master is the machine where the control plane components run, including etcd (the cluster database) and the API server (which the kubectl CLI communicates with). All of these components run in pods started by kubelet." (#)
• controller manager
  • A small binary that lives on the "master side of the house"
  • Responsible for doing health maintenance.
  • modifies cluster state, and can get a lease-lock from the API Server to ensure single actor modifying cluster state - kubernetes-docs-high-availability
• CNI - Container Network Interface
  • TODO
• daemon set
  • kubernetes-docs-deamon-set
• dashboard
  • TODO
  • kubectl cluster-info gives kubernetes-dashboard is running at https://172.17.4.99:443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
• declaritive primitives
  • TODO - Same as the (maybe called templates or manifests) that declare this stuff?
  • robust declarative primitives for maintaining the desired state requested by the user. We see these primitives as the main value added by Kubernetes. Self-healing mechanisms, such as auto-restarting, re-scheduling, and replicating containers require active controllers, not just imperative orchestration. (#)
• deployment
  • includes a replication controller
  • includes the setup for rollouts
• definition
  • At 52:42 - definition seems a bit like manifest (#)
• data volume
  • Independent of the lifespan of the containers
• desired state
  • The declarative statement about what you'd like the world to look like
• edge router
  • From kubernetes-docs-ingress "A router that enforces the firewall policy for your cluster. This could be a gateway managed by a cloudprovider or a physical piece of hardware."
• endpoint
  • TODO
  • Can be internal or external
  • Can also be non-Kubernetes
• etcd
  • AKA clustered etcd
  • A fault-tolerant key/value datastore.
  • Clustered etcd replicates your storage to all master instances in your cluster - kubernetes-docs-high-availability
  • Also a lock manager (TODO used that way in Kubernetes? Was used that way for the CoreOS competitor to Kubernetes)
  • can check status with kubectl get cs (component statuses)
• expose (verb)
  • Can take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service.
• extensions resources
  • things like DaemonSets, Deployments, HorizontalPodAutoscalers, Ingress, Jobs and ReplicaSets (are enabled by default)
• features
  • TODO - what is this?
  • In kubernetes-kubectl-commands it says: "Set specific features on objects". Also, what are objects? Same as resources?
• fluentd
  • unified logger
  • At 8:35 fluentd runs on the master node (#)
• frontend
  • TODO from kubernetes-docs-services
• Heapster
  • Container Cluster Monitoring and Performance Analysis
  • kubectl cluster-info gives "Heapster is running at https://172.17.4.99:443/api/v1/proxy/namespaces/kube-system/services/heapster"
• Init containers
  • For containers on a single pod, there are init-containers which run serially and in order prior to running the main container (#)
• Interprocess communication (IPC) namespace
  • TODO - is one way pod is isolated
• IP addresses
  • From kubernetes-docs-services - "Each pod gets an IP addresses, but it cannot be relied upon to be stable over time."
  • From video-kubernetes-architecture - Each container in the pod shares the same IP address, and considers the other containers to be on localhost.
  • At 24:30 - A Cluster IP is available only within the cluster. (#)
  • we decided that Kubernetes would allocate an IP address per pod, thus aligning network identity (IP address) with application identity (#)
• job
  • creates a pod for a specific purpose, then removes that pod when it's done
• kubeadm
  • Installs Kubernetes on bare metal
• kubectl
  • The command-line client that talks to the API server
  • from kubernetes-kubectl-commands "kubectl controls the Kubernetes cluster manager"
• KubeDNS
  • DNS is a built-in service launched automatically using the addon manager cluster add-on
  • Kubernetes DNS schedules a DNS Pod and Service on the cluster, and configures the kubelets to tell individual containers to use the DNS Service's IP to resolve DNS names.
  • kubectl cluster-info gives "KubeDNS is running at https://172.17.4.99:443/api/v1/proxy/namespaces/kube-system/services/kube-dns"
• kubelet
  • A small daemon that lives on the "machines" themselves.
  • node agents (kubelet) (#)
  • is used on each of the worker nodes, and can also be used on the master node, as described in kubernetes-docs-high-availability
  • When the scheduler tells it that its desired state is to be running X containers, it has a reconciliation loop that attempts to get it to that state.
  • installs the API Server
  • At 10:00 the kubelet reports health to the master (#)
  • ... still working on a few things, like making kubelet itself (all our components, really) run within containers (#)
  • "While the kubelet plays an important role in a Kubernetes cluster, it also works well in standalone mode — outside of a Kubernetes cluster." (TODO - why?) (#)
  • My understanding is that the kubelet was added to CoreOS directly (#) (is pre-installed in the base node OS (TODO what is their name for that?))
  • "The master is the machine where the control plane components run, including etcd (the cluster database) and the API server (which the kubectl CLI communicates with). All of these components run in pods started by kubelet." (#)
• kube-proxy
  • lives on each node
  • core networking component of Kubernetes
  • manages the ip tables on each node
  • also exposes services
  • related to, or possibly the same as service-proxy (TODO?)
  • kube-proxy can do simple TCP and UDP stream forwarding (round robin) across a set of backends. (#)
  • At 48:10 - if you access a service at the right port, but wrong node, kube proxy will bounce it to the right node (#)
• kube-system pods
  • Based on kubectl cluster-info, the following reside at ...v1/proxy/namespaces/kube-system/...
    • heapster
    • KubeDNS
    • dashboard
• label
  • An explicitly documented name and identifier (#)
  • label (DNS_LABEL): An alphanumeric (a-z, and 0-9) string, with a maximum length of 63 characters, with the '-' character allowed anywhere except the first or last character, suitable for use as a hostname or segment in a domain name. (#)
  • Key/Value pairs that are useful for organizing your things
  • form the basis of two API objects that will be talked about in talk-technical-overview-of-kubernetes after 12 minutes (TODO ensure this gets updated)
  • From kubernetes-code-readme Labels are used to organize and select groups of objects based on key:value pairs.
  • associate one kubernetes object with another
• label key
  • label keys have two segments: an optional prefix and name
  • TODO - I don't understand what the optional prefix is for
• label selector
  • the core grouping primitive in Kubernetes.
  • two types of selectors: equality-based and set-based (TODO, what is the difference?)
  • Label selectors are the grouping mechanism in Kubernetes, and define the scope of all management operations that can span multiple entities. (#)
• lease-lock
  • provided by the API Server to ensure single actor modifying cluster state (like controller manager or scheduler)
• load balancer
  • want one between each layer of your application
  • TODO - special names for these load balancers?
• manifest
  • The file that's passed to kubectl create that creates one or many resources, for example.
  • At 52:42 - definition seems a bit like manifest (#)
• maps
  • "Labels selectors for both objects are defined in json or yaml files using maps, and only equality-based requirement selectors are supported:" (#)
• master
  • TODO -
  • AKA master node (or nodes in an HA setup). Contrast this with worker
  • kubectl cluster-info gives "Kubernetes master is running at https://172.17.4.99:443"
  • At 8:35 master /can/ run containers, but usually doesn't. (#)
  • At 30:54 - When you run kubectl get nodes if it says SchedulingDisabled on the master node, it will not run containers (#)
  • Node controller is a component in Kubernetes master which manages Node objects. (#)
• minikube
  • a tool that makes it easy to run Kubernetes locally. It runs Kubernetes in a VM on your local machine.
• minion
  • the deprecated name for a node
• network namespace
  • TODO - is one way pod is isolated
• name
  • An explicitly documented name and identifier (#)
  • Name: A non-empty string guaranteed to be unique within a given scope at a particular time; used in resource URLs; provided by clients at creation time and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish distinct entities, and reference particular entities across operations. (#)
  • http://kubernetes.io/docs/user-guide/identifiers/
  • client-provided.
  • only one object of a given kind can have a given name at a time (i.e., they are spatially unique).
  • used to refer to an object in a resource URL, such as /api/v1/pods/some-name
• namespaces
  • TODO - there are a bunch of these, but I don't understand them yet
  • by default, everything is in the default namespace
  • At 6:20 - namespaces... different ones for environments: test/staging/production or for tenants (#)
  • From video-kubernetes-architecture - Process ID (PID) namespace
  • From video-kubernetes-architecture - Network namespace
  • From video-kubernetes-architecture - Interprocess communication (IPC) namespace
  • From video-kubernetes-architecture - Unix Timesharing (UTS) namespace
  • kubectl --namespace=demos get events (#)
• node
  • from kubernetes-kubectl-commands "Update the taints on one or more nodes"
  • from kubernetes-docs-ingress "A single virtual or physical machine in a Kubernetes cluster."
  • node is a worker machine in Kubernetes, previously known as minion. (#)
• node affinity
  • TODO - kubernetes-docs-assigning-pods-to-nodes
• node controller
  • Node controller is a component in Kubernetes master which manages Node objects. (#)
  • node controller receiving heartbeats (#) (TODO - who sends these?)
  • kubernetes-docs-node#node-controller-assigns-cidr-block

assigning a CIDR block to the Node when it is registered

• node pools
  • TODO
• node port
  • At 24:40 - A Node Port is an external service. Called node port because every node in the cluster makes the service available on the same port. (#)
  • At 46:15 - the node ports fall within a specific range (#)
  • At 48:10 - if you access a service at the right port, but wrong node, kube proxy will bounce it to the right node (#)
• object
  • TODO - same as resource?
  • seems to be referred to much more often than resources.
  • every Kubernetes object has three basic fields in its description: Object Metadata, Specification (or Spec), and Status. (#)
• persistent volume
  • attached to the node prior to running the pod itself on the node
  • "Unlike volumes, persistent volumes are not associated with any specific pod or containers when they are created" (#)
  • networked storage resource of the cluster that has been provisioned by an administrator. It is a resource in the cluster just like a node is a cluster resource. Are volume plugins like Volumes, but they have a distinct lifecycle that is independent of any individual pod that consumes the PV. (#)
  • Types: Are implemented as plugins. Kubernetes supports popular backends and distributed file systems including Amazon EBS, GCE Persistent Disks, Cinder, Azure File System, NFS, iSCSI, Gluster, and Ceph among other types. (#)
• pet sets
  • Pet Sets, an evolving concept in Kubernetes for running highly available stateful workloads. (#)
  • Pet Sets, soon to be replaced by Stateful Sets in Kubernetes v1.5, give you deterministic order of initialization of pods. (#)
• pod
  • The atomic unit of a kubernetes cluster
  • A collection of one of more containers that work well together
  • from kubernetes-code-readme "a colocated group of application containers with shared volumes. They're the smallest deployable units that can be created, scheduled, and managed with Kubernetes. Pods can be created individually, but it's recommended that you use a replication controller even if creating a single pod."
  • A set of data-volumes
  • namespaces
  • labels
  • The atomic unit of scheduling. "It doesn't make any sense for those two containers to land on different machines" talk-technical-overview-of-kubernetes
  • can be connected together on localhost (different containers on a pod can all see each other because they share a network namespace)
  • From kubernetes-docs-services - "Each pod gets an IP addresses, but it cannot be relied upon to be stable over time."
  • From video-kubernetes-architecture - "Alternative to VM with multiple processes"
  • From video-kubernetes-architecture - Each pod is isolated by Process ID (PID) namespace, Network namespace, Interprocess communication (IPC) namespace, Unix Timesharing (UTS) namespace
  • At 18:20 "Every pod is completely autonomous. There is no notion of an 'application' that brings multiple pods together" (#)
  • always runs an application container inside a top-level pod, even if the pod contains a single container (#)
  • it's easy to add a new small support service, because it operates in the private execution environment provided by its own container (#)
  • we decided that Kubernetes would allocate an IP address per pod, thus aligning network identity (IP address) with application identity (#)
• podmaster
  • TODO
  • name from diagram in kubernetes-docs-high-availability on master
• port name
  • An explicitly documented name and identifier (#)
  • port name (IANA_SVC_NAME): An alphanumeric (a-z, and 0-9) string, with a maximum length of 15 characters, with the '-' character allowed anywhere except the first or the last character or adjacent to another '-' character, it must contain at least a (a-z) character. (#)
• Process ID (PID) namespace
  • TODO - is one way pod is isolated
• Prometheus
  • A way of monitoring a Kubernetes cluster
• resource
  • TODO - seems to be a pretty generic term. What can it refer to?
  • Allow idempotent creation of API resources (#148) and enforcement of space-uniqueness of singleton objects. (#) (In same doc other things are referred to as objects)
• reconciliation loop
  • Name from talk-technical-overview-of-kubernetes at 14:20
  • Same loop can be used for create, update, and delete
• replica set
  • run X copies of a pod
  • handle pod failures (health checks) - TODO, what actually does this?
• replication controller
  • combination of a template (a desired state template), some labels, and a number (like I want three)
  • From kubernetes-code-readme "manage the lifecycle of pods. They ensure that a specified number of pods are running at any given time, by creating or killing pods as required."
• scheduler
  • A small binary that lives on the "master side of the house"
  • Responsible for scheduling pods onto machines
  • modifies cluster state, and can get a lease-lock from the API Server to ensure single actor modifying cluster state - kubernetes-docs-high-availability
  • The scheduler binds unscheduled pods to nodes via the /binding API. The scheduler is pluggable, and we expect to support multiple cluster schedulers and even user-provided schedulers in the future. (#)
  • The scheduler needs to take into account individual and collective resource requirements, quality of service requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, deadlines, and so on (#)
• schedulable
  • A node-level attribute that determines whether pods will be scheduled onto the node.
  • cordon marks a node as unschedulable
  • uncordon marks a node as schedulable
  • drain removes pods from a node and marks is as unschedulable.
• secret
  • exposed by data volume or ENV vars
  • managed by the API Server
• selector
  • TODO (it's in the spec section in the yml file - what is that yml file called?)
  • same as label selector, or are there additional types of selectors?
• service
  • From kubernetes-code-readme "Services provide a single, stable name and address for a set of pods. They act as basic load balancers."
  • From kubernetes-docs-services "A Kubernetes Service is an abstraction which defines a logical set of Pods and a policy by which to access them."
  • Get a known, static, fixed IP address
  • They get a DNS entry that's associated with that known, static, fixed IP address
  • The IP address only exists in the routing tables in the machines in the cluster (otherwise it's fake)
  • Has a name, some ports, and a label query.
  • The load balancer sends traffic to the dynamic set of pods that match the label query
  • TODO in expose, it says you can expose a service as a new Kubernetes Service. What does that mean?
  • from kubernetes-docs-ingress "A Kubernetes Service that identifies a set of pods using label selectors. Unless mentioned otherwise, Services are assumed to have virtual IPs only routable within the cluster network."
  • At 22:15 an abstraction to define a logical set of pods, bound by a policy to access them. (#)
  • At 22:15 services are exposed through internal and external endpoints (#)
  • At 22:15 can also point to a non-kubernetes endpoint through a Virtual-IP-Bridge (#)
  • At 22:15 supports TCP and UDP (#)
  • Interfaces with kube-proxy to manipulate IP tables (#)
  • At 39:40 - there is a default kubernetes service when you do kubectl get svc (#)
  • At 51:50 - If you scale a service by using a replication controller, the service will route to the right nodes using a load balancing scheme (#)
  • simple load balancing including session affinity
• service port
  • The port specified in the spec of a service manifest.
  • At 37:50 - The default option is exposing the service port as a cluster IP (#)
• service proxy
  • that lives on the "machines" themselves
  • provides load balancing
• spec
  • TODO the spec section within the manifest.
• Stateful sets
  • Pet Sets, soon to be replaced by Stateful Sets in Kubernetes v1.5, give you deterministic order of initialization of pods. (#)
• storage class
  • Enables administrators to create multiple tiers or classes of storage they offer. Different classes map to different levels of quality-of-service, backup policies, and arbitrary policies as defined by the administrators. (#)
• storage pools
  • TODO - blog-post-running-stateful-application-in-kubernetes
• subdomain
  • An explicitly documented name and identifier (#)
  • subdomain (DNS_SUBDOMAIN): One or more lowercase rfc1035/rfc1123 labels separated by '.' with a maximum length of 253 characters. (#)
• supervisord
  • At 8:35 supervisord runs docker and kubelet (#)
  • At 10:20, supervisord ensures that docker and kublet remain running (#)
• taint
  • TODO - what is this?
  • from kubernetes-kubectl-commands "Update the taints on one or more nodes"
• template
  • TODO
• UID
  • An explicitly documented name and identifier (#)
  • UID: A non-empty, opaque, system-generated value guaranteed to be unique in time and space; intended to distinguish between historical occurrences of similar entities. (#)
• UUID
  • An explicitly documented name and identifier (#)
  • universally unique identifier (UUID): A 128 bit generated value that is extremely unlikely to collide across time and space and requires no central coordination. (#)
  • In the case of Kubernetes, a UUID is one type of UID
• Unix Timesharing (UTS) namespace
  • TODO - is one way pod is isolated
• Virtual-IP-Bridge
  • TODO - a way that services can point to non-Kubernetes endpoints.
• worker
  • AKA worker node
  • At 12:25 the node is also called a minion or a slave (TODO is that only the worker node?) (#)

Referring Pages