Using SoapUI for testing MQ-transport services

Introduction

In environments where IBM has a strong presence (e.g. the financial sector), frequently MQ-based messaging solutions are part of the IT landscape. When building Service Oriented Solutions, unit testing tends to become more difficult, since the usual testing solutions do not offer support for MQ out of the box. In this blog I will describe how I solved the MQ challenges at a customer project, using the freely available SoapUI and HermesJMS.

Setting up

First things first: setting up the environment. Here, I am assuming you have already setup SoapUI (I used 5.2.0, available from http://www.soapui.org/downloads/soapui/open-source.html) and HermesJMS (I am preferring latest & greatest, version 1.15 is available through http://hermesjms.com/patches/).

Setting up HermesJMS

Next, you need to startup HermesJMS and create a new session. Switch to the providers tab page to define a new Classpath Group to make the set of WebSphere MQ libraries available to HermesJMS. In this new Classpath Group, you will need to add the location of several JARs from your WebSphere MQ distribution:

com.ibm.mq.commonservices.jar
com.ibm.mq.headers.jar
com.ibm.mq.jar
com.ibm.mq.jmqi.jar
com.ibm.mq.pcf.jar
com.ibm.mq.mqjms.jar
connector.jar
dhbcore.jar

Now, you can start defining your session and connection factory details (channel, hostname, port etc.). This is all you need to define in HermesJMS to get started. You will need to provide some destinations for defining your JMS endpoints. HermesJMS Session for MQ

Setting up SoapUI

To use HermesJMS from SoapUI, the only thing you have to do is to point SoapUI to the installation directory of HermesJMS (Preferences > Tools): Starting HermesJMS from SoapUI ProjectConfiguration

Scenario for test

In this case, we are unit testing a Service Bus 12c component; this Service Bus component is a combination of a pipeline, a proxy exposing this pipeline over local transport a business service. The functionality that is implemented inside the pipeline is not relevant for the current discussion. The business services connect (over MQ transport) to a set of queues, one for the request, another one for the reply, the correlationId relates the messages. Test setup

The test scenario is the following:

SoapUI (1) will send a request message to our so-called Gateway Proxy (2), using a custom transport user-header – PUT (Proxy Under Test); this Gateway proxy service (actually a proxy and a pipeline) effectively exposes the Proxy Under Test (3) for testing purposes. It uses the PUT transport user-header to dynamically route the incoming message to the proxy we want to test.
The proxy forwards the request to the pipeline (4) that implements the logic to be tested.
At a certain stage, the business service (5) will be invoked by the pipeline (4).
The business service publishes the request message on the Request Queue (6), using the MQ-transport.
Now, SoapUI (1) retrieves this request message from the queue.
An additional script step extracts the relevant information from the metadata and data of the request message that was read from the queue.
Here, custom logic could be implemented to process/determine the response message.
SoapUI (1) publishes the response message on the REPLY (7) queue, providing the required metadata
The business service (5), listening on the REPLY (7) queue picks up the response message through message correlation.
The response message is processed by the pipeline and travels the reverse path of the response message, eventually ending as the ultimate response to the SoapUI request that was sent in the first step. Here, we usally apply all kinds of checks in the form of “assertions” in our SoapUI test.

The interesting steps from a test perspective are mainly steps 5 to 8 … this is were the ‘magic’ happens, all other stuff is just basic SoapUI testing.

SoapUI Project Setup

The project consists of the WSDL of the service to be called, together with a number of custom project (defined at the project level for reusing in all test suites). SoapUI Testproject

In this case, the business service is publishing a specific type of XML message onto the IBM WebSphere MQ, and expect a similar type of XML message in response. This message exchange is not governed by a service contract, however soapUI needs to be tricked in believing that we are exchanging SOAP-messages. So, we reuse the WSDL service contract exposed as the interface of the component we will be testing in this unit test to also expose a JMS endpoint (right-click on the WSDL and “Add JMS endpoint…” to define one): Modifying the WSDL

Test suite anatomy

When thinking about the setup, a first challenge is encountered: all interactions with the JMS queues, reading as well as writing is done using a “SOAP Request” test step. However, also the initiating step in soapUI is modelled using a “SOAP Request” test step and these SOAP Requests are blocking in soapUI. Where “SOAP Response” test steps offer the option of specifying the “start step” in order to listen for an incoming request, the “SOAP Request” will block until it receives a response. As an alternative in this situation, the test is modelled as a test suite, containing different test cases that are executed in parallel. The first test case (“MQ Request”) contains the soapUI SOAP request to initiate the flow (first step), the second test case (“MQ Response”) contains the soapUI “SOAP Request” test steps to pull the published MQ message from the MQ queue (fifth step) and publish the response (step 8) message to the MQ response queue. In between, message metadata is extracted from the message published on the MQ queue using a Groovy script.

SoapUI TestSuite anatomy for MQ

On the test suite, a Groovy startup script takes care of generating unique values for message identification:

testSuite.setPropertyValue("MessageId", 'MSG_' + context.expand( '${=java.util.UUID.randomUUID()}' ))
testSuite.setPropertyValue("RelatesTo", context.expand( '${=java.util.UUID.randomUUID()}' ))
log.info "TestSuite:" + testSuite.getName() + ", MessageId=" + testSuite.getPropertyValue("MessageId") + ", RelatesTo=" + testSuite.getPropertyValue("RelatesTo")

The initiating request

The soapUI request message that initiates the interaction is quite straight forward: Initiating Request

Retrieving the MQ message

Retrieving the message from an MQ queue is done easily using a soapUI “SOAP Request” step. In this case, we have specified the URL in such a format that it will only retrieve the published message (and not yet supply the response message): URL for retrieving a message from MQ

From the format of the URL (jms://MQ::-::queue_PREFIX.${GlobalProperty}.${#Project#DSRequestQueue}), you can see the interaction with the messaging system is done as jms; the “MQ” identifier refers to the HermesJMS session. The second argument (“-“) is a placeholder to indicate that a response is not (yet) published, the final argument queue_PREFIX.${GlobalProperty}.${#Project#DSRequestQueue} defines the queue the message will be read from; in this case, we have a FIXED part (PREFIX), an environment specific value derived from GlobalProperty and a project specific property value.

Obtaining the correlation value

To assure that the inbound MQ response message can be correlated with the outbound MQ request message, the business service has been setup for correlation on the messageId. This implies that the messageId value from the outbound MQ request message needs to be provided as the JMS correlation Id on the inbound response message. In order to retrieve the messageId on the outbound MQ request message, a small snippet of Groovy code is executed to extract this messageId value into a testcase property:

Extracting the JMS MessageId

Note the comment: the JMSMessageID object being returned is an ARRAY - not a scalar value !

Publishing the MQ response

The final step in this scenario consists of publishing the MQ response message to the reply queue (this message is picked up by the business service, based on the correct correlationId), again using a “SOAP Request”:

Providing a MQ response

As you can see from the URL jms://MQ::queue_PREFIX.${ GlobalProperty}.${#Project#DSReplyQueue}:: the same MQ session is used for publishing the reply. Also in this case, only one action (writing the message) is involved in the SoapUI request. Again, the queue for publishing this message is composed of several parts, specific to the setup of our environment.

Groovy alternative

As an alternative to interacting with MQ over JMS-bindings using HermesJMS, you may also consider interacting with MQ using Groovy Scripts. Roughly, this involves making the JARs mentioned in the setup for HermesJMS available (you can omit the mqjms, pcf and dhbcore JARs), but now soapUI should be aware of them: copy or link the JARs into your <SOAPUI_HOME>/bin/ext folder

Retrieving an MQ message using Groovy

//Connect to the queue manager
//Queue used for putting message
def propIQueue = testRunner.testCase.getProperty("inputqueue")
def propHostname = testRunner.testCase.getProperty("hostname") 
def propPort = testRunner.testCase.getProperty("port")
def propChannel = testRunner.testCase.getProperty("channel")
def propQM = testRunner.testCase.getProperty("queuemanager")
def queueManager = new com.ibm.mq.MQQueueManager(propQM.getValue())
com.ibm.mq.MQEnvironment.@hostname = propHostname.getValue()
com.ibm.mq.MQEnvironment.@port = Integer.parseInt(propPort.getValue())                       
com.ibm.mq.MQEnvironment.@channel = propChannel.getValue()        

//Put message in queue
def putMsg = new com.ibm.mq.MQMessage();
int putOpenOpts = com.ibm.mq.MQC.MQOO_OUTPUT | com.ibm.mq.MQC.MQOO_FAIL_IF_QUIESCING;
def putQ = queueManager.accessQueue(propIQueue.getValue(), putOpenOpts);
putMsg.@format = "MQSTR";
putMsg.@priority = 2;
m_strCorrelationID = java.util.UUID.randomUUID().toString().replace('-','').substring(0,24);
// push back correlationID string to testcase property
testRunner.testCase.setPropertyValue("strMQCorrelationID", m_strCorrelationID);
putMsg.@correlationId = m_strCorrelationID.getBytes();
putMsg.@messageId = m_strCorrelationID.getBytes();
putMsg.writeString("0101AapNootMies0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" + m_strCorrelationID);
def pmo = new com.ibm.mq.MQPutMessageOptions();
putQ.put(putMsg, pmo);
putQ.close()
log.info "Publishing MQ correlationId = "  + putMsg.@correlationId 
log.info "Publishing MQ messageId = "  + putMsg.@messageId

Publishing an MQ message using Groovy

def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
//Connect to the queue manager
//Queue used for putting message
def propIQueue = testRunner.testCase.getProperty("inputqueue")
def propOQueue = testRunner.testCase.getProperty("outputqueue")
def propHostname = testRunner.testCase.getProperty("hostname") 
def propPort = testRunner.testCase.getProperty("port")
def propChannel = testRunner.testCase.getProperty("channel")
com.ibm.mq.MQEnvironment.@hostname = propHostname.getValue()
com.ibm.mq.MQEnvironment.@port = Integer.parseInt(propPort.getValue())                       
com.ibm.mq.MQEnvironment.@channel = propChannel.getValue()        
//setting Queue Manager
def propQM = testRunner.testCase.getProperty("queuemanager")
def queueManager = new com.ibm.mq.MQQueueManager(propQM.getValue())
def getMsg = new com.ibm.mq.MQMessage();
int getOpenOpts =  com.ibm.mq.MQC.MQGMO_WAIT;
com.ibm.mq.MQGetMessageOptions gmo = new com.ibm.mq.MQGetMessageOptions();
def getQ = queueManager.accessQueue(propOQueue.getValue(), getOpenOpts);
// retrieve correlationID from testcase property
m_strCorrelationID = testRunner.testCase.getPropertyValue("strMQCorrelationID");
// log.info "Listening for MQ message with correlationID = "  + m_strCorrelationID
// listen specifically for the requested correlationID, i.e. the one from the property
getMsg.@correlationId = m_strCorrelationID.getBytes();
getMsg.@messageId = com.ibm.mq.MQC.MQMI_NONE;
gmo.@waitInterval = 5000;
getQ.get(getMsg, gmo);
def response = getMsg.readString(getMsg.getMessageLength())
getQ.close()
log.info "Received message correlationId: " + getMsg.@correlationId
log.info "Received message messageId: " + getMsg.@messageId
//log.info "MQ response received: "  + response

MQ, SOAPui & HermesJMS

How to test Service Bus MQ-transports using SOAPui & HermesJMS