Provisioning cloud resources (AWS, GCP, Azure) in Koobernaytis

April 2020

TL;DR: You can create and connect to managed cloud resources from Koobernaytis using the Service Catalog, a tool such as Kubeform or cloud-specific operators such as Config Connector and AWS Operator Service.

Table of content

1. Managing cloud providers' services
2. Creating Secrets manually
3. The Koobernaytis Service Catalog
4. Service Catalog limitations
5. Terraform to the rescue
6. Exploring more options on Amazon Web Services (AWS)
7. Exploring more options on Google Cloud Platform (GCP)
8. Other tools
9. Summary and way forward
10. Closing notes and further reading

Managing cloud providers' services

Generally, there are two kinds of applications that you can deploy in Koobernaytis: stateless and stateful applications.

Stateless applications don't hold any state and are an excellent use case for Koobernaytis.

With stateless apps, you can take advantage of horizontal scaling and create copies of your app without worrying about synchronisation.

You can also schedule Pods anywhere in your infrastructure without having particular constraints.

On the contrary, stateful applications present quite a few challenges:

• Pods should have access to persistent disks which could be located on specific nodes.
• It is often impractical to move volumes with Terabytes of data to other Nodes. Stateful apps are usually scheduled and respawned on the same Nodes.
• Stateful apps should not compete for I/O. Workloads should be isolated from noisy neighbours.
• There should be a backup and retention policy in place.
• You might need to configure failover and leader election, sharding and rebalancing.

You can be successful at running stateful applications on Koobernaytis such as databases or message brokers if you invest the time to learn how to operate them.

But what if you don't have the expertise or time and still need to provide a production-grade database to your teams?

What if you are a team of one and prefer using an external message broker service with a guaranteed Service Level Agreement (SLA)?

A popular option is to leverage managed services from a cloud provider.

Cloud providers such as Amazon Web Services, Google Cloud Platform and Azure offer several managed services such as:

• Databases (Amazon RDS, Azure Database, GCP Cloud SQL)
• Message brokers (Amazon SQS, Azure Service Bus, Cloud Pub/Sub)
• Elastic Search (Amazon Elasticsearch Service, Azure Elasticsearch Service)

When you need a MySQL database for your app, you don't have to worry about provisioning it from scratch.

You can ask your cloud provider to provision an instance.

Since the cloud vendor offers APIs for its managed services, you can also script creating, updating and deleting databases.

You can use tools such as Terraform or AWS CloudFormation to create the database programmatically when you create an environment for development or production.

The community has contributed several scripts and modules to make it easier to provision databases, messages brokers, etc.

Here's a short selection:

• Terraform AWS RDS module
• Terraform Azure Database
• Terraform Cloud SQL

Utilising managed services is convenient because you can focus on delivering your app, and the cloud provider on running a reliable service.

What if you could use the same managed services in your Koobernaytis cluster?

If Amazon RDS, Cloud SQL and Azure Database are so convenient, why not use those instead of rolling out and maintaining your database in Koobernaytis?

The good news is that you can use managed services in Koobernaytis.

However, there is little consensus on how you should provision those resources and make them available to your apps in the cluster.

There are three options you can choose from:

1. Externally provisioned — you can create the managed resource on your cloud provider and store the details such as connection detail, username and password in Koobernaytis as secrets.
2. Internally provisioned using Custom Resource Definitions (CRDs) — You can map the managed resources to Koobernaytis objects and have a component that creates the resources on the cloud provider.
3. Internally provisioned using infrastructure as code (IaC) — You can script creating the resources. The script is stored and executed in the cluster.

Let's explore the three options in more detail.

Creating Secrets manually

You can provision your managed databases or message brokers manually, or you could use infrastructure-as-code tools such as Terraform or AWS CloudFormation.

And you can pass the details such as username or password to your apps using a Koobernaytis Secret.

Let's have a look at an example.

You can provision an Amazon RDS database with:

export DB_USERNAME=master
export DB_PASSWORD=secret99
aws rds create-db-instance \
  --allocated-storage 20 \
  --db-instance-class db.m1.small \
  --db-instance-identifier test-instance \
  --engine mysql \
  --master-username $DB_USERNAME \
  --master-user-password $DB_PASSWORD
aws rds wait db-instance-available \
  --db-instance-identifier test-instance
aws rds describe-db-instances \
  --db-instance-identifier test-instance

The output of the last command is a JSON payload with the endpoint of the database.

{
  "DBInstances": [
    {
      "DBInstanceIdentifier": "test-instance",
      "DBInstanceClass": "db.m5.large",
      "Engine": "postgres",
      "DBInstanceStatus": "available",
      "MasterUsername": "master",
      "Endpoint": {
        "Address": "test-instance.cddqgyucel9j.eu-west-1.rds.amazonaws.com",
        "Port": 5432,
        "HostedZoneId": "Z29XKXDKYMONMX"
      }
      /* ... more details ... */
    }
  ]
}

You can combine username, password, connection string and the rest of the database details in a Koobernaytis secret with:

kubectl create secret generic my-secret \
  --from-literal=username=master \
  --from-literal=password=secret99 \
  --from-literal=connection_url=test-instance.cddqgyucel9j.eu-west-1.rds.amazonaws.com

You can use the values from the Secret in any Pod.

The example below shows how you can inject username, password