How to configure OSCAR in Google Kubernetes Engine — A Step-by-Step Guide

Setup OSCAR in a Kubernetes cluster along with OpenFaaS.

Photo by Avi Richards on Unsplash

OSCAR is an open-source platform to support the Functions as a Service (FaaS) computing model for file-processing applications maintained by the GRyCAP community. This article aims to guide the installation process of OSCAR in a cluster managed by Google Kubernetes Engine.

Prerequisites:

• Access to Google Kubernetes Engine with a billing account.
• Docker
• A beginner’s level understanding of Kubernetes
• GCloud CLI installed and configured with your Google account
• Properly installed and configured Go
• A beginner’s level understanding of containers and container orchestrators
• The Kubernetes command-line tool (kubectl)
• Helm
• Arkade
• A domain name / DNS server (E.g. Cloudflare)

Architecture:

source: https://grycap.github.io/oscar/#introduction

Create a Kubernetes cluster in GKE

In this article, we are focusing on the Google Kubernetes Engine. This can be also configured in a local cluster or in other Kubernetes providers such as EC3.

As the first step, we need to create a fresh cluster with the image type as ubuntu.

Cluster configuration

Once the cluster is created, configure your local machine terminal with the cluster in GKE as shown in the below image.

Then configure role-based access control for the newly created cluster.

kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin \
--user $(gcloud config get-value account)

Deploy Nginx Controller on Kubernetes cluster

NGINX Controller is an application that runs in a cluster and configures an HTTP load balancer according to Ingress resources. In our case, this will be used to reach the backends (Minio, OSCAR, etc) via the internet.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/cloud/deploy.yaml

Create Namespaces of OSCAR

The next step is to create the namespaces relevant to OSCAR. Where the related deployments and services of OSCAR will fall under this.

kubectl apply -f https://raw.githubusercontent.com/grycap/oscar/master/deploy/yaml/oscar-namespaces.yaml

Deploy OpenFaaS

The easiest way to deploy OpenFaaS in your Kubernetes cluster is through arkade. A Helm chart for OpenFaaS is also available. The choice is yours :). However, make sure that you create the operator in OpenFaaS and set the functionNamespace as oscar-svc which is very important. Since the functions are now managed by OSCAR we are setting the function namespace as oscar-svc. OSCAR communicates with the OpenFaaS Kubernetes controller via the OpenFaaS operator. You can find more information about this here.

arkade install openfaas --load-balancer --operator --set functionNamespace=oscar-svc

Create Storage for Minio

Before deploying Minio we need to allocate storage for it. Using the below YAML files this can be achieved. Feel free to change the values as needed.

kubectl apply -f https://gist.githubusercontent.com/rajitha1998/557e88c7da18c80537aefe285d69c244/raw/a89ab6f0d022f225e1577835b048d71f8ec0db18/storage-class.yml
kubectl apply -f https://gist.githubusercontent.com/rajitha1998/e730e7ca278d5711efd0fccc080ad174/raw/4c3a45a552d31772a7db74734bbdd64052c377b5/storage.yml

Deploy Minio from Helm Chart

Minio is high-performance, Kubernetes-native object storage built for the demands of the hybrid cloud (similar to Amazon S3). when deploying Minio make sure to set the storage space created for it. The default memory per pod is set to 4 Gigabytes in default. Which is a huge amount for some users. Therefore make sure to change it as the resources you have. In this case, we are setting it to 256 Megabytes. The access key and the secret key will be used to access Minio from the frontend and also by OSCAR to communicate with each other.

helm repo add minio https://helm.min.io
helm install --namespace oscar minio minio/minio --set accessKey=minio --set secretKey=abcd --set persistence.existingClaim=lightnig --set resources.requests.memory=256Mi

Deploy NFS Server Provisioner

NFS server provisioner is required for the creation of ReadWriteMany PersistentVolumes in the Kubernetes cluster. This is required by the OSCAR services in order to mount storage volumes with the FaaS Supervisor inside the Kubernetes Jobs.

helm repo add kvaps https://kvaps.github.io/charts
helm install nfs-server-provisioner kvaps/nfs-server-provisioner

Deploy OSCAR from Helm Chart

Now we can deploy OSCAR from the provided Helm chart. Make sure to set “serverlessBackend” as OpenFaaS. Therefore, it will use OpenFaaS components.

helm repo add grycap https://grycap.github.io/helm-charts/
helm install — namespace=oscar oscar grycap/oscar — set authPass=abcd — set service.type=ClusterIP — set createIngress=true — set volume.storageClassName=nfs — set minIO.endpoint=https://xxx.xxx.tech — set minIO.TLSVerify=false — set minIO.accessKey=minio — set minIO.secretKey=abcd — set serverlessBackend=openfaas

Once this is deployed, your cluster should look like this:

P.S: If you prefer to use the same ingress created for OSCAR for Minio, the ingress file should be updated as below before deploying OSCAR.

Configuring Domain Name Servers

Add A Name Records pointing to the IP of ingress of OSCAR. Once the domain name servers are updated, you can access the front-end of OSCAR and Minio through the relevant hosts.

Congratulations 🥳!! You have now successfully deployed OSCAR in the Google Kubernetes Engine.

Special thanks to Sebas Risco who supported me with this. Hope this helps to configure OSCAR on your Kubernetes cluster with a proper understanding of each step :)

Cheers 🥂 !!