Integration Tests for SMX4 with Python

Integration tests and unit tests are important for project quality. Unit tests usually are well suited for developer to verify his changes in runtime. On the other hand, integration tests, are for target user to verify that project's features in the way he interacts with project, work properly. In this article, I will show how to automate integration tests for ServiceMix 4 using SoapUI testrunner and a simple python script.
The idea is to spawn ServiceMix 4 Karaf console and interact with it using python expect library. During this interaction, SoapUI testrunner script is invoked in order to run SoapUI tests.
First, we need to grab SMX4_DIR and SOAPUI_DIR environment variables in our script, like this:

SMX4_DIR=os.getenv("SMX4_DIR")
SOAPUI_DIR=os.getenv("SOAPUI_DIR")

This way, we can invoke later our script using following shell command:

SMX4_DIR=/some/path SOAPUI_DIR=/some/other/path ./our-python-script

Then, we need to spawn ServiceMix 4 console by using python expect library:

import pexpect
import time
import sys
child = pexpect.spawn("bin/servicemix")
child.logfile = sys.stdout
child.expect("karaf.*>")
time.sleep(3)

Here, we set logfile to stdout in order to see our automated interaction with ServiceMix console. Then we need to wait for ServiceMix console command prompt, which would mean console is ready. Additionally, we need to wait a few seconds to avoid problems with running commands too early (which is a kind of small bug in ServiceMix). Then, we can install our features, which we want to test. This example starts Apache HISE test bundle, which loads also Apache HISE engine from dependencies.

child.sendline("features:addUrl mvn:org.apache.hise/hise-karaf/0.3.0-SNAPSHOT/xml/features");
child.expect("karaf.*>")
child.sendline("features:install hise-h2-test-example-osgi")
child.expect("karaf.*>")

Next, we need to wait until the feature is properly started. ServiceMix 4 OSGi container initializes bundles in background, so it's not enough to wait for command prompt to have it started (there doesn't seem to exist a "wait-until-started" console command). So we grep in a loop over installed bundles and see if status is started. In this example, we do 30 retries every second and fail our integration test script after this period, by raising exception.

child.sendline("features:addUrl mvn:org.apache.hise/hise-karaf/0.3.0-SNAPSHOT/xml/features");
rep=0
while True:
    child.sendline("osgi:list|grep -i hise-test-example-osgi")
    l=child.readline()
    l=child.readline()
    if re.match(".*Started", l) != None:
        break
    time.sleep(1)
    child.expect("karaf.*>")
    rep=rep+1
    if rep>30:
        raise Exception("Bundle not installed")

Next, we need to run SoapUI testrunner in order to execute test cases. We need to implement syscall method in order to fail integration tests if SoapUI testrunner completes with fault (non-zero exit code).

import os
def syscall(c):
    if os.system(c) != 0:
        raise Exception("Sys call failed: " + c)

syscall(SOAPUI_DIR + "/bin/testrunner.sh -f results hise-soapui-project.xml")

At the end, we can exit gracefully from ServiceMix console by using shutdown command, like this:

child.sendline("shutdown")

And that's it. Full code of integration test script is available in Apache HISE sources, from Apache repository http://svn.apache.org/repos/asf/incubator/hise/trunk/itest/itest.