Building CogStack

Owned by Lukasz Roguski (Unlicensed)
Last updated: Mar 24, 2023 by Tom Searle

Overview

CogStack Pipeline application is built and distributed as a single Java JAR archive. The Java application can be either:

We usually emphasise using the docker-based deployment and using our official docker image with CogStack Pipeline application, so that one does not need to worry about the dependencies to build and run CogStack.


Info

Starting with version 1.3.0 we have updated the build scripts to use the newest Java SDK 11.

Building using an older version of JDK should be possible, but requires manual changes to the gradle build script (please see below).

On this page :



Building locally

CogStack Pipeline can be build locally possibly on any system with installed Java SDK. The provided building script will take care of downloading all the necessary packages resolving the components dependencies.

Prerequisites

Java SE Development Kit in version >= 11.0 is required to be installed on the system. The most commonly used JDK distributions are:


Note

Please note that with the change of licensing from Oracle coming with the new version 11 (requiring commercial license for production and for support) we emphasise using the OpenJDK build variant. More information about the licensing can be read >here<.

In our Docker images we are also now using a base image with OpenJDK.


Info

When building locally and trying to run acceptance and integration tests, some additional dependencies need to be satisfied. Apache Tika conversion tool relies on third-party software which needs to be installed on the system. These are TesseractOCR (for extracting the text from image data) and ImageMagick (for conversions between image formats).


Building

CogStack Pipeline application can be build locally from source from the official CogStack Github repository. The application can be build using Gradle Build Tool wrapper which is available in the CogStack bundle – to build it, just type in the main directory:

./gradlew bootJar


The final build JAR archive with all the JAVA dependencies will be placed in ./build/libs/ folder.


Info

There are also other build options available to run using the Gradle Build wrapper (such as compiling and running unit tests, etc), but they won't be covered here.





Building in TravisCI

CogStack Pipeline uses TravisCI continuous integration service for building and testing the code from the official GitHub repository. The Travis build passes if building the JAR file is successful and all the unit, acceptance and integration tests pass. The builds and their statuses are available on the official TravisCI CogStack Pipeline site. 

The Travis build task (with the platform, environment and dependencies definitions) is defined in .travis.yml file available in the CogStack main directory.

The script that is launched to build and test the application is travis_gradle_build.sh.





Building in Docker cloud

CogStack Pipeline

CogStack Pipeline application is also build automatically in the Docker Hub which will be available in the generated Docker images. The official CogStack Docker Hub repository is under organisation cogstacksystems and there are also available other custom images of microservices used inside the CogStack ecosystem (v1).

CogStack Pipeline image builds are triggered by push notifications in the CogStack Github repository – there are two different image versions available correspondingly:

  • cogstack-pipeline:latest is build from the master branch (stable),
  • cogstack-pipeline:dev-latest is build from the dev branch (development).

There are also available images with tags for strict versioning:

  • cogstack-pipeline:1.2.1
  • cogstack-pipeline:1.3.0

The Dockerfile used to build these images is available in the main CogStack pipeline directory.


Note

It is always highly recommended to use the build from the master branch, being the most stable one. However, for accessing new features and bug fixes one should use the build from the dev branch. The changes from the dev branch are regularly merged into into the master.


CogStack Pipeline + GATE

Apart from the main CogStack Pipeline image, there is also available a Pipeline image with bundled GATE Embedded 8.4.1. It can be easily used to run NLP applications using GATE NLP suite without the need to worry about installing and configuring GATE suite locally.

This image is based on the core CogStack Pipeline image. It comes available with versions:

  • cogstack-pipeline-gate:latest is build from the master branch (stable),
  • cogstack-pipeline-gate:dev-latest is build from the dev branch (development).

There are also available images with tags for strict versioning:

  • cogstack-pipeline-gate:1.3.0

The Dockerfile used to build these images is available in dockerfiles/gate directory.


Note

Apart from the CogStack Pipeline application, in Docker Hub are being build custom images of the microservices used inside CogStack Ecosystem, such as Fluentd logging driver.