This tutorial is a first step towards deploying GroIMP on a compute cluster. This at the moment is more of a prove of concept than a established way of using GroIMP.

The idea is to have several multiple pods running the GroIMP API server GroLink and one pod or job that is scheduling tasks to the GroLink pods. This one pod/job could later be a web service or a deployment job, but for this tutorial its going to be just a small python terminal-Pod we can connect to and run our script by hand. This script than uses python multiprocessing to send API calls to the different GroLink pods in parallel and than collecting the results to a csv file.

The Python script will create a small data set for a sensitivity analysis of a very simple model and execute the model.

To run a Model we need a model, but since this is not about modelling or statistics or analysis this model can be very very simple. It grows a tree with branches of a first order and print the crown radius after every step. The radius is hereby depending on the angle and the length vactor (lenV), which are the values we are going to test in our analysis.

The idea of the sensitivity analysis using GroLink is explained in this tutorial.

//model.rgg
import parameters.*;
 
module Bud(int order,float len) extends Sphere(0.1);
 
protected void init ()
[
	Axiom ==> Bud(0,1);
]
 
 
public void run ()
[
	Bud(0,len) ==> F (len) 
		[RL(parameters.angle) F(len*parameters.lenV) Bud(1,len*parameters.lenV/2)]
		RH(90)[RL(parameters.angle) F(len*parameters.lenV) Bud(1,len*parameters.lenV/2)]
		RH(90)[RL(parameters.angle) F(len*parameters.lenV) Bud(1,len*parameters.lenV/2)]
		RH(90)[RL(parameters.angle) F(len*parameters.lenV) Bud(1,len*parameters.lenV/2)]
	Bud(0,len*parameters.lenV);
	Bud(1,len) ==> F (len) Bud(1,len*parameters.lenV);
	{getRadius();}
]
 
 
public void getRadius(){
	println(max(location((*Bud*)).x)+0.2);
 
}

// param/parameters.rgg
static float lenV=0.9;
static float angle=45;

To do the following a Kubernetes cluster and installed version of kubectl is required. I would suggest for the beginning to use a virtual cluster like minikube so if something goes wrong its not that bad.

We start by creating the namespace “grolinktutorial” on the cluster with the following command:

kubectl create namespace grolinktutorial

The namespace defines which nodes we want to address or what roles are used. In our case the only additional role we need is that a pod can see other pods (so our terminal node can find the GroLink nodes)

This role is defined as following and shoud be stored in role.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: grolinktutorial
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

And additionally we need a ClusterRoleBinding that binds who has this role, in our case we just bind it to default because this is the simplest. The following shoud be stored in rolebinding.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: podreaderbinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: pod-reader
subjects:
  - kind: ServiceAccount
    name: default
    namespace: grolinktutorial

Now we can apply them using kubectl in a command line:

kubectl apply -f role.yaml
kubectl apply -f rolebinding.yaml

For the Grolink pods we can create a deployment that uses the GroIMP default docker image from GitLab and executes the GroLINK api on it with -a api. For the beginning we only create 3 replicas since we are working with a simulated cluster. An we define a matchLabel so we can later figgure out which pods are actually running the API.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: groimp-grolink
  namespace: grolinktutorial
  labels:
    app: grolink
spec:
  replicas: 3
  selector:
    matchLabels:
      app: grolink
  template:
    metadata:
      labels:
        app: grolink
    spec:
      containers:
      - name: grolink
        image: registry.gitlab.com/grogra/groimp:latest
        args: ["-a", "api"]

This can be executed for a file similar to the role and the Binding:

kubectl apply -f grolinkDeply.yml

For our terminal pod we are not very picky we just need a pod that runs for ever and can execute python code. Therefore we can just use the default python image and let it run the embedded web server(we don't need this server at all, but if the node is not busy is dies).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: terminal
  namespace: grolinktutorial
  labels:
    app: terminal
spec:
  replicas: 1
  selector:
    matchLabels:
      app: terminal
  template:
    metadata:
      labels:
        app: terminal
    spec:
      containers:
      - name: terminal
        image: python
        args: ["python","-m", "http.server"]

We need to also deply this:

kubectl apply -f terminalDeply.yml

To test if all pods are running as we want them to we can just list all of them for our name space:

kubectl get pods --namespace grolinktutorial

Now we can start for the first time running code on our little setup. To test if every thing is working lets open a workbench on each pod, list the files in it and close it. To find all the Pods we use the python kr8s library on our namespace with a selector that checks if the app of this node is GgroLink. Then we can use the IP addresses of the pods with the GroPy library.

from GroPy import GroPy
import kr8s
podIPs=[]
selector = {'app': 'grolink'}
for podS in kr8s.get("pods", namespace="grolinktutorial", label_selector=selector):
    print(podS.status.podIP)
    podIPs.append(podS.status.podIP)
 
for i in podIPs:
    link = GroPy.GroLink("http://"+i+":58081/api/")
    wb = link.createWB().run().read()
    print(wb.listFiles().run().read())
    wb.close()

To run this on the terminal pod we have to first get the name of the terminal pod using the command

kubectl get pods --namespace grolinktutorial

In this list one name should start with “terminal-”, this is the one we want.

Now we can use kubectl to copy the python file with the code from above on this pod:

kubectl -n grolinktutorial cp run.py terminal-<second-part-of-the-name>:/

and run it with

kubectl -n grolinktutorial exec  terminal-<second-part-of-the-name> python run.py