About

Jadira is the home for Sousan and Chris Pheby's open source projects. These are reusable open source Java modules that provide first class solutions using the most effective current JEE technologies.

Search
Tag Cloud
...
Login
« Eclipse Mars | Main | Spring, Hibernate and JTA - A Better Integration »
Monday
Apr212014

Batching Transactions for Performance with Spring

We all know that JTA is slow, right? Not so fast, a key issue with JTA in a messaging environment is that we are applying a transaction once per message. The theme of this post is the implementation of transactional batching, another problem I've faced and resolved many times over the course of about a decade.

Significant performance benefits can be realised with many messaging providers such as ActiveMQ and WebSphereMQ and these can be achieved without sacrificing the reliability of the end application.

For a bit of context here's an extract from an existing article on this subject:

Should you forgo JMSTemplate and batch everything? Well, as with pretty much anything, it depends.

JMSTemplate is simple to configure and use – it also ties in nicely to the various Spring abstractions (TransactionManagers come to mind). There are certainly some upsides to getting down to the JMS API, however they come at a cost of time coding and losing those hooks; which you might regret later.

Batching itself is a funny one, in that it’s one of those things that could be easily misused. Transactions are not there for improving performance. They exist as a logical construct for making atomic changes. If you batch up messages in memory to send to the broker for a performance upshot that are unrelated; and your container goes down before you’ve had a chance to commit; you have violated the contract of “this business action has been completed”, and you lost your messages. Not something you want for important, change-the-world messages.

If you have messages that are relatively unimportant that won’t affect the correct operation of your system, there are better options available to you for improving performance such as dropping persistence.

If, however, you have related messages tied to a single operation, then perhaps getting down to the JMS API and batching these together under a single transaction is something you might want to consider. The key here is ensuring that you don’t lose the real meaning of transaction.

Jakub Korab, Batching JMS messages for performance; not so fast

Now for the surprising part - you don't have to forgo Spring's JMSTemplate and DefaultMessageListenerContainer to get transactional batching. Jadira JMS introduces specialisations of these classes that support transactional batching within the familiar Spring programming model.

Transactional Batching with BatchedMessageListenerContainer

BatchedMessageListenerContainer customises DefaultMessageListenerContainer to support transactional batching. What this means in practice is that up to n messages can be configured to be read and processed within each transaction. You can use this class as a drop-in replacement for DefaultMessageListenerContainer.

It does provide some additional properties to configure. Firstly you should wire in a transactionManager instance using the existing setter. Secondly, the batchSize property should be configured to an appropriate value (it defaults to 150). Two additional properties - retryMitigation and concludeBatchOnRedeliveredMessage - control error mitigation and attempt to reduce the batch size when errors are encountered. It is recommended to leave these enabled unless you are certain they will not be required.

With the container configured you are now ready to use it. An important factor to be aware of is that you should avoid rollback for errors that can be identified and handled appropriately within the application. For example, a badly formed message should be delivered to an error queue, rather than relying on rollback. This will help ensure one bad message doesn't roll back the entire application. Jadira provides a base class you can extend in your Message Driven Pojos to assist with this (see below).

If you are interested in how this class works, be aware that the key functionality lies in a customised implementation of doReceiveAndExecute. Here is the key code segment:

            messages = new ArrayList<Message>();

            message = receiveMessage(consumer);
            if (message != null) {

                messages.add(message);
                ...
            } ...

            int count = 0;

            ...
            while ((message != null) && (++count < maxMessagesPerTransaction) ...) {
               
                message = receiveMessageNoWait(consumer);

                if (message != null) {
                    messages.add(message);
                    ...
                } ...
            }

Transactional Batching with BatchedJMSTemplate

BatchedMessageListenerContainer is good for the event driven style, but in certain circumstances you want access to a polling batching capability. For this purpose, BatchedJmsTemplate is also supplied. This class extends JmsTemplate and adds additional methods equivalent to the existing methods but that perform batch operations. A setter allows configuration of the batch size - this can also be specified on a message basis. Like BatchedMessageListenerContainer the transaction manager should be injected into the class.

AbstractMessageDriven

AbstractMessageDriven is a base class that can be used when implementing MessageDrivenPojos (MDPs). It provides an error handling pattern that allows messages in error to be routed direct to an error queue with error information and JMS Headers preserved.

Simply subclass implementing doOnMessage (rather than onMessage) and inject in a JmsTemplate for delivering messages to an error queue. To trigger delivery to an error queue, simply throw Fatal JmsException from within your doOnMessage method.

The stack trace and exception message will be preserved in the message properties cause_exceptionMessage and cause_exceptionStackTrace. Other JMS Headers and JMS Properties will be preserved with the prefix 'original_' - the CorrelationID and Priority will remain on the message. The error payload will be the original message that caused the error.

This class works particularly well in conjunction with the BatchedMessageListenerContainer and BatchedJmsTemplate where it mitigates the likelihood of rollback from errors in message processing.

These classes are in GitHub - I welcome your feedback as ever.

Unit Testing with ActiveMQ

By combining ActiveMQ and JBoss Transactions we can develop message driven code with fully transactional behaviour - out of container.

The approach fits very well with the JPA configuration described in the previous blog post - they make perfect partners. Here's an example of the necessary Spring configuration:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:beans="http://www.springframework.org/schema/beans" xmlns:amq="http://activemq.apache.org/schema/core"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
                        http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://activemq.apache.org/schema/core
                        http://activemq.apache.org/schema/core/activemq-core.xsd
                        http://www.springframework.org/schema/tx
                        http://www.springframework.org/schema/tx/spring-tx.xsd">

    <!-- Transaction Manager to use -->
    <bean id="jbossTransactionManager"
        class="com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionManagerImple">
        <property name="transactionTimeout" value="300" />
    </bean>

    <bean id="transactionManager"
        class="org.springframework.transaction.jta.JtaTransactionManager">
        <property name="transactionManager">
            <ref bean="jbossTransactionManager" />
        </property>
        <property name="allowCustomIsolationLevels" value="true" />
    </bean>

    <!-- Embedded ActiveMQ Broker -->
    <amq:broker id="activeMQBroker" useJmx="false" persistent="false">
        <amq:transportConnectors>
            <amq:transportConnector uri="tcp://localhost:43232" />
        </amq:transportConnectors>
    </amq:broker>

    <bean id="activeMQConnectionFactory" class="org.apache.activemq.ActiveMQXAConnectionFactory"
        depends-on="activeMQBroker">
        <property name="brokerURL"
            value="tcp://localhost:43232?jms.redeliveryPolicy.maximumRedeliveries=1" />
    </bean>

    <bean id="jmsConnectionFactory"
        class="org.springframework.jms.connection.CachingConnectionFactory">
        <property name="targetConnectionFactory" ref="activeMQConnectionFactory" />
        <property name="sessionCacheSize" value="2" />
    </bean>

    <!-- ActiveMQ destinations to use -->
    <amq:queue id="testDestination" physicalName="org.jadira.jms.test" />

    <!-- Example Listener -->
    <bean id="testMessageListener" class="org.jadira.jms.container.TestMessageListener" />

    <!-- The Container -->
    <bean id="messageListenerContainer"
        class="org.jadira.jms.container.BatchedMessageListenerContainer">
        <property name="connectionFactory" ref="jmsConnectionFactory" />
        <property name="destinationName" value="testDestination" />
        <property name="messageListener" ref="testMessageListener" />
        <property name="transactionManager" ref="transactionManager" />
        <property name="exceptionListener" ref="jmsConnectionFactory" />
        <property name="concurrency" value="2-5" />
        <property name="maxMessagesPerTransaction" value="100" />
    </bean>

</beans>

PrintView Printer Friendly Version

EmailEmail Article to Friend

References (2)

References allow you to track sources for this article, as well as articles that were written in response to this article.

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>