Odtnhj

Summary - Message loss during systematic shutdown of application.

I have a application written in spring integration and I am consuming requests from external systems using 'jms:message-driven-channel-adapter'. This is the configuration of the channel adapter -

<jms:message-driven-channel-adapter id="InboundAdapter" destination="InQueue"
 connection-factory="connectionFactory"
 channel="responseChannel"
 error-channel="errorChannel"
 acknowledge="transacted"
 receive-timeout="50000"
 auto-startup="true"/>

my response channel looks like this -

 <int:chain id="processorChain" input-channel="responseChannel">
 <int:service-activator method="doProcess" ref="inputProcessor" />

 <int:router id="inputRouter" method="route">
 <int:mapping value="REQUIRED" channel="builderChannel"/>
 <int:mapping value="NOT_REQUIRED" channel="TerminateChannel"/>
 <bean id="Router" class="xxx.xxx.Router">
 </bean>
 </int:router>
 </int:chain>

Now when i perform 'kill -9 pid', then I see that the message is rolled back to the queue and it is all good. But when I am performing 'kill pid', than the message is lost somewhere. I have enabled teh jms logs and I can see that JMS listener is waiting for the message in transit to complete before closing the JMS consumer but still I don't the message rolling back to my queue. This is the logs snippet that I see in my logs

trace: 15.11.18 15:42:46.619 [Thread-10] DEBUG org.springframework.jms.listener.DefaultMessageListenerContainer - Waiting for shutdown of message listener invokers
trace: 15.11.18 15:42:46.619 [Thread-10] DEBUG org.springframework.jms.listener.DefaultMessageListenerContainer - Still waiting for shutdown of 1 message listener invokers (iteration 0)

After this logs, it is calling the service activators which are defined in the response channel.

Can someone please shed some light on above behavior, I was expecting that when we issue kill command than all messages which are in transit should be rolled back to the queue, instead of this, it is trying to call the activators defined with in channel and after calling last component which is router, it just finishes.

Any help on this topic will be really helpful!!

After some more debugging and banging my head with the system I found the exact scenario where it is happening. Let me put it thru to see if it makes sense. When a systematic shutdown is triggered, JMS is waiting for the messages in transit to finish before stopping the application, till this point everything is good. Currently this chain is being executed -

<int:chain input-channel="inputchain">
 <int:transformer id="xxx" method="transform">
 <bean class="xxx" />
 </int:transformer>

 <int:service-activator id="xxx" method="doProcess">
 <bean class="xx">
 <constructor-arg ref="xxx"/>
 </bean>
 </int:service-activator>

 <int:service-activator id="xxx" ref="rulesProcessor" method="doProcess"/>
 <int:service-activator id="xxx" ref="xxx" method="doProcess"/>

 <!-- Existing Flow Continues -->
 <int:router id="xxxRequiredRouter" method="xxxRequired">
 <int:mapping value="Required" channel="firstChannel"/>
 <int:mapping value="NotRequired" channel="secondChannel"/>
 <bean id="xxxRouter" class="xxx.Router" />
 </bean>
 </int:router>

</int:chain>

<int:chain input-channel="secondChannel">

 some logic

</int:chain>

So the thread finishes this chain and exits gracefully. It doesn't call my next chain 'secondChannel'. My transaction boundary doesn't finish on this chain 'inputchain', it finishes at the end of secondChannel. So I have not comnmitted this transction to database as transaction boundary is at the end of next chain, so this is not available in DB and application is thinking that chain execution is finished so it is complete, so it is not rolled back to teh queue as well. So in teh end I don't have this message in my database and it is not in queue.

So is this the case that only the chain which is being executed when shutdown was triggered will finish and it will not delegate processing to subsequent chains?

A SIGTERM kill will wait for the thread to complete its work.

Eventually the container will interrupt it, but that will only help if it's doing something that's interruptible.