Operator Patter in Kubernetes

The Operator Pattern

The Operator pattern is an extension of the Kubernetes control plane that manages custom resources and automates operational tasks. It’s designed to encode human operational knowledge into software, making it easier to manage complex, stateful applications.

Here’s a diagram illustrating the Operator pattern:

In this pattern:

1. Custom Resources define the desired state of your application.
2. The Operator watches for changes to these Custom Resources.
3. When a change is detected, the Operator’s reconciliation loop is triggered.
4. The Operator reads the desired state from the Custom Resource and compares it to the actual state of the system.
5. If there’s a discrepancy, the Operator takes action to align the actual state with the desired state. This might involve:
   • Creating, updating, or deleting Kubernetes resources
   • Making API calls to external systems
   • Performing complex deployment or scaling operations

Code Generation for Operators

The Go code and CustomResourceDefinition (CRD) for an Operator are typically generated using specialized tools. The two most common tools for this purpose are the Operator SDK and Kubebuilder.

Using Operator SDK

The Operator SDK, part of the Operator Framework, provides a set of tools to build, test, and package Operators. Here’s a typical workflow:

1. Install the Operator SDK CLI.

2. Create a new Operator project:

   operator-sdk init --domain example.com --repo github.com/example/my-operator
   

3. Create a new API (Custom Resource):

   operator-sdk create api --group myapp --version v1 --kind MyApp
   

4. This command generates:
   • A Go type definition for your Custom Resource in api/v1/myapp_types.go
   • A controller in controllers/myapp_controller.go
   • A CustomResourceDefinition manifest in config/crd/bases/

5. Modify the generated files to define your Custom Resource’s spec and status, and implement your controller’s reconciliation logic.

6. Generate the CRD manifest:

   make manifests
   

Using Kubebuilder

Kubebuilder is another popular tool for building Kubernetes APIs and controllers. The process is similar:

1. Install Kubebuilder.

2. Create a new project:

   kubebuilder init --domain example.com
   

3. Create a new API:

   kubebuilder create api --group myapp --version v1 --kind MyApp
   

4. This generates similar files to the Operator SDK.

5. Modify the generated files as needed.

6. Generate the CRD manifest:

   make manifests
   

Both tools use code generation to create boilerplate code and Kubernetes manifests, allowing you to focus on implementing your Operator’s business logic.

CustomResourceDefinition Generation

The CustomResourceDefinition (CRD) is typically generated from the Go struct that defines your Custom Resource. Both the Operator SDK and Kubebuilder use controller-gen, a tool from the controller-tools project, to generate CRDs.

The process works like this:

1. You define your Custom Resource as a Go struct with special marker comments.

2. The // +kubebuilder: comments provide metadata about your Custom Resource, including validation rules.

3. When you run make manifests, controller-gen reads these comments and generates a CRD YAML file.

Here’s an example of a Go struct with marker comments:

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"

// MyApp is the Schema for the myapps API
type MyApp struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   MyAppSpec   `json:"spec,omitempty"`
    Status MyAppStatus `json:"status,omitempty"`
}

// MyAppSpec defines the desired state of MyApp
type MyAppSpec struct {
    // +kubebuilder:validation:Minimum=0
    // +kubebuilder:validation:Maximum=100
    // +kubebuilder:validation:ExclusiveMaximum=false
    Size int32 `json:"size"`
}

// MyAppStatus defines the observed state of MyApp
type MyAppStatus struct {
    Nodes []string `json:"nodes"`
}

This approach ensures that your Go code and CRD remain in sync, reducing the chance of errors and making it easier to evolve your API over time.

Conclusion

The Operator pattern, along with tools like Operator SDK and Kubebuilder, has revolutionized how we build extensions for Kubernetes. By leveraging code generation and the power of Custom Resources, we can create sophisticated, application-specific controllers that encode operational knowledge directly into our Kubernetes clusters.

As you embark on building your own Operators, remember that the true power lies not just in the technical implementation, but in the domain-specific knowledge you embed into your Operator. This is what transforms a simple controller into a powerful tool for managing complex applications at scale.