Random   •   Archives   •   RSS   •   About   •   Contact

Selenium grid on Kubernetes

This post continues from where we left off on the Minikube guide. If you do not already have a Kubernetes cluster, you should read that first.

Selenium Grid allows you to build a cluster of Selenium nodes. Today we will create a Selenium cluster with 1 hub and 4 nodes on Kubernetes.

Selenium Hub

Let's launch the hub:

kubectl get pods
kubectl run selenium-hub --image selenium/hub:2.53.1 --port 4444
kubectl get pods

To access the deployment externally, we need to expose it:

kubectl get services
kubectl expose deployment selenium-hub --type=NodePort
kubectl get services

My deployment was exposed here:

http://192.168.99.100:31136/grid/console

To find out where your deployment was exposed:

minikube service selenium-hub --url

You may open this URI in a web browser.

Selenium Hub Overlearning

You can skip this section or try it for extra credit.

We may use kubectl exec for container introspection:

kubectl exec selenium-hub-3216163580-7pqtx -- ps aux

As you can see we have a java process running

kubectl exec selenium-hub-3216163580-7pqtx -- ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
seluser      1  0.0  0.1  18044  2688 ?        Ss   17:20   0:00 /bin/bash /opt/bin/entry_point.sh
seluser      7  0.1  3.1 2928868 64864 ?       Sl   17:20   0:13 java -jar /opt/selenium/selenium-server-standalone.jar -role hub -hubConfig /opt/selenium/config.json
seluser     87  0.0  0.1  34424  2856 ?        Rs   20:54   0:00 ps aux

We may also look at the config file and test the service internally:

# inspect the selenium config file.
kubectl exec selenium-hub-3216163580-7pqtx -- cat /opt/selenium/config.json

# see if selenium is really listening on port 4444.
kubectl exec selenium-hub-3216163580-7pqtx -- wget 127.0.0.1:4444 -O -

You can even shell into the container:

kubectl exec -it elenium-grid-3216163580-7pqtx -- /bin/bash

Exit out of the container and lets setup some Selenium Nodes!

Selenium Node

Lets spin up a Selenium Chrome node:

kubectl get pods
kubectl run selenium-node-chrome --image selenium/node-chrome:2.53.1 --env="HUB_PORT_4444_TCP_ADDR=selenium-hub" --env="HUB_PORT_4444_TCP_PORT=4444"
kubectl get pods

Kubernetes will use service discovery to resolve selenium-hub to the service (pods) running the hub!

If you refresh the hub browser window, you should see a connected Chrome Node, like this:

Selenium Hub with one connected Chrome Node.

Selenium Node Overlearning

You can skip this section or try it for extra credit.

The first time I tried to launch a Selenium node and I had trouble.

I ran this:

kubectl get pods
kubectl run selenium-node-chrome --image selenium/node-chrome:2.53.1
kubectl get pods

The new pod went into status CrashLoopBackOff:

NAME                                    READY     STATUS             RESTARTS   AGE
selenium-grid-3216163580-7pqtx          1/1       Running            1          3d
selenium-node-chrome-4019562870-mcpfg   0/1       CrashLoopBackOff   6          6m

To troubleshoot, I used the following commands:

kubectl describe pod selenium-node-chrome

This command allowed me to review the Kubernetes level logs. Everything seemed healthy so next I looked at the Docker level logs:

kubectl logs selenium-node-chrome-4019562870-mcpfg
Not linked with a running Hub container

Ok, the error Not linked with a running Hub container appears when Selenium node cannot find the hub.

Docker has a --link flag to link containers together, Kubernetes doesn't have this. After some research, it seems --link manages ENV vars.

You can see the environment vars of a pod using this command:

kubectl exec selenium-hub-3216163580-7pqtx -- printenv

I learned that the selinum-node-chrome docker image expects some ENV vars and if it doesn't get them, it goes into a crash loop.

I reached out over IRC in the #kubernetes and #selenium channels to ask about the ENV vars needed. A really helpful user named smccarthy linked me to this:

https://github.com/kubernetes/kubernetes/tree/master/examples/selenium

Apparently one of the example Kubernetes clusters is a Selenium Grid setup!

Looking over the example, I found the ENV vars that the selenium-node containers expect: HUB_PORT_4444_TCP_ADDR and HUB_PORT_4444_TCP_PORT

Man, why would they put the port (4444) in the key?

Anyways, we pass these key/values when creating the container like this:

kubectl run selenium-node-chrome --image selenium/node-chrome:2.53.1 --env="HUB_PORT_4444_TCP_ADDR=selenium-hub" --env="HUB_PORT_4444_TCP_PORT=4444"

Selenium Scale

Now we can scale up and down the Selenium Grid cluster. Lets scale the deployment to 4 replica node-chrome pods.

kubectl get pods
kubectl scale deployment selenium-node-chrome --replicas=4
kubectl get pods

Finally, if you refresh the hub browser window, you should see 4 connected Chrome nodes!

Selenium Hub with four connected Chrome Nodes.

If you liked this post, leave me a message in the comments!




Want comments on your site?

Remarkbox — is a free SaaS comment service which embeds into your pages to keep the conversation in the same place as your contentr. It works everywhere, even static HTML sites like this one!

Remarks: Selenium grid on Kubernetes

© Russell Ballestrini.