This section is primarily targeted at people developing the Dataverse application on a Kubernetes platform, run CI jobs or similar.
Dataverse Development Usage¶
Images on Docker Hub are meant for production usage or quick demos. When developing Dataverse, testing a new feature not yet shipped in a release or running integration tests you have a need to deploy all moving parts into a (more or less) ephemeral environment.
Please prepare your environment first:
While you can build images manually (see below) and deploy manually, it will be much easier to let Skaffold take care of it.
skaffold run or
skaffold dev from the root of the project will
build, tag and deploy for you.
For a deeper insight, read docs at https://skaffold.dev/docs.
STOP. You did preparation for this, did ya? Go ahead.
Now lets no more time, and create our initial deployment, already using your checked out branch of Dataverse. Choose between traditional Glassfish or Payara based development images by activating via profile:
skaffold run -p init # --- or --- skaffold run -p init -p payara
(Between us: this will simply deploy the dev persona and the bootstrap job for you.)
While you are waiting for the deployment to finish (see A word on waiting for more), think about how you will access your cluster. Your options:
skaffold dev(see below!), you can add
--port-forward. See also port forward docs. This is currently not possible with
When using Minikube, see Make Dataverse reachable via browser
When using KinD, easiest way forward is
Ingressis also possible, see upstream doc.
When using k3d, easiest way to go is by exposing the ingress
Example workflow for local development¶
You might choose not to use
skaffold dev, as build and deploy times are long.
Using it means every saved file will trigger the deploy chain, which is pretty
Currently only using a local cluster with
kind is supported by this
project when using Skaffold. See preparing cluster.
Build development images manually¶
In most cases, you will want to let tools automatically build new images for you. See below for your options.
Simply use Docker or other build tool again (almost like above, but different path):
docker build -t iqss/dataverse-k8s:test --pull -f docker/dataverse-k8s/glassfish-dev/Dockerfile . docker build -t iqss/dataverse-k8s:payara-test --pull -f docker/dataverse-k8s/payara-dev/Dockerfile . docker build -t iqss/solr-k8s:test docker/solr-k8s
Currently there is no Solr development image. This is likely to change.
A word on waiting¶
Be aware that initial builds and deployments take lots of time due to cold caches. Recurring builds and deployments will be much faster, although you will still suffer from Glassfish WAR deployment times.
Deployment times of Dataverse are still a nightmare as of writing (Jan. 2020). For any change to the codebase, you most likely will have to wait about five to ten minutes to redeploy from compilation to reload of webpage.
This is due to a really big WAR file (which needs to be loaded into the cluster) and being stuck on old technology in combination with a tremendous monolith.
YOU HAVE BEEN WARNED.
Typically, when there is no change to
pom.xml and build caches are warmed up,
building will take about 1 minute,
loading images into cluster about 45 secs,
deploying to Kubernetes about 30 secs
and Glassfish startup + WAR deployment about 3 minutes.
How about some coffee?