Wednesday, May 7, 2014

Increase Integration Server JVM memory

I am aware this is very basic thing and might not be useful for lot of people. Just sharing it with people in case some one is stuck.

First the recommend settings for JVM memory are as per below.

below are the three different parameters which controls memory allocated to JVM

JAVA_MAX_MEM=4096M
JAVA_MIN_MEM=2048M
JAVA_MAX_PERM_SIZE=1024M



Recommendation is MIN_MEM should be 50% of MAX_MEM and MAX_PERM_SIZE should be set to 50% MIN_MEM. example above clarifies this.

Now in order to change memory setting follow below steps.

  1. Stop Integration Server.
  2. Go to installation directory\IntegrationServer\bin
  3. edit setenv.bat or setenv.sh as per your Operation System. .bat for windows and .sh for linux\Unix
  4. save file.
After file save follow below steps as applicable to your Operating System

For Windows
  1. Go to SoftwareAG\IntegrationServer\support\win32 folder
  2. Run installSvc.bat with unreg as option. It will look like "installSvc.bat unreg"
  3.  Verify if service is removed from windows services.
  4. Now run installSvc.bat without any option.
  5. Verify if service is created.
  6. Start Integration server from services.
  7. Verify memory in Statistic page in IS admin to confirm if new memory values are reflected.
For Unix\Linux
No need to follow steps like windows. just start up Integration server and new values should be reflected.

Hope this will be helpful for few.

Do leave feedback in comment section.

Monday, July 18, 2011

MQ Listener Issue [ADA.600.1007] - Not able to consume data from MQ

Few days back we faced a very stupid issue in UAT environment.

Whenever source application was putting data to MQ, data was getting consumed but nothing was coming into webMethods server. There used to be no data left in MQ however we were not able to see anything for listener document and no instance for interface was getting triggered.

While this we were able to see errors for the listener in server error logs. Below is the error message which we were seeing in logs.


ERROR - Mq_Adapter.listeners:receive  java.lang.NullPointerException at com.wm.adapter.wmmqadapter.connection.wmMQMessage.getJMSHeader(wmMQMessage.java:1634)
 Stack trace data ... 0f969420-a307-11e0-b119-e96436d8b712  NULL  0f969420-a307-11e0-b119-e96436d8b712
Mq_Adapter.listeners:receive  [ADA.600.1007] 1007
null
 Stack trace data ... 0f969420-a307-11e0-b119-e96436d8b712  NULL  0f969420-a307-11e0-b119-e96436d8b712
2011-06-30 07:08:48 EDT    java.lang.NullPointerException at com.wm.adapter.wmmqadapter.connection.wmMQMessage.getJMSHeader(wmMQMessage.java:1634)
 Stack trace data ... 0f969420-a307-11e0-b119-e96436d8b712  0f969420-a307-11e0-b119-e96436d8b712  54854c50-a309-11e0-826d-c487ee2f6cd0
2011-06-30 07:08:48 EDT    [ADA.600.1007] 1007
null
 Stack trace data ... 0f969420-a307-11e0-b119-e96436d8b712  0f969420-a307-11e0-b119-e96436d8b712  54854c50-a309-11e0-826d-c487ee2f6cd0
2011-06-30 07:08:47 EDT    java.lang.NullPointerException at com.wm.adapter.wmmqadapter.connection.wmMQMessage.getJMSHeader(wmMQMessage.java:1634)
 Stack trace data ... 0f969420-a307-11e0-b119-e96436d8b712  0f969420-a307-11e0-b119-e96436d8b712  538baa10-a309-11e0-826c-fdb750e73640
2011-06-30 07:08:47 EDT    [ADA.600.1007] 1007
null
 Stack trace data ... 0f969420-a307-11e0-b119-e96436d8b712  0f969420-a307-11e0-b119-e96436d8b712  538baa10-a309-11e0-826c-fdb750e73640
Mq_Adapter.listeners:receive  java.lang.NullPointerException at com.wm.adapter.wmmqadapter.connection.wmMQMessage.getJMSHeader(wmMQMessage.java:1634)

For troubleshooting purpose, we created a dummy put service in webMethods which was pushing the same external message to same mq to which external application was trying to put. To our surprise when message was put through dummy service it was consumed well and everything was getting processed in webMethods absolutely fine.

Also that messages from external application was getting processed fine in DEV environment. Problem was environment specific. that ruled out any issue in our code as it was working fine in DEV.

We started looking about the configuration and difference between DEV/UAT environment. at last the issue which we found was that MQ adapter fix was missing in our UAT environment. We recently updated to wM8 from 712 and Fix Patch MQS_6-5_Fix8 was not applied to the new UAT servers. Becuase of this fix data from external application was not getting through. though it was not happening with all the interface.

webMethods do behave very weird some time.

Next time if you also facing the same kind of issue, please check if you have got all latest fixes from SAG.

PS - If you also have faced the similar issue, please do share your experience with me in comment section.

Thanks. Comments and suggestions are most welcome.

Tuesday, June 7, 2011

Reason to avoid appendToDocumentList

Whenever I see usage of appendToDocumentList in any flow service for creating output document list, I wonder if we can achieve it through some other way e.g. Output array option of loop, java service loop or putting index of incrementing variable inside loop.

Also the question used to arise what are the benefits of using other methods over appendToDocumentList. There is a set of reaction which you get when one talks about the usage of appendToDocumentList e.g.
  • Performance is badly hit with appendToDocumentList.
  • Stay away from appendToDocumentList and use PSUtilities service addToList.
To find out the reason to avoid a particular service which is provided by SAG, I did a little brain storming session with the geniuses out there.

First let me put the alternates of appendToDocList.
  • Explicit Loop: If the size of output document list to be created is same as input document list use output array option in loop.
  • Sometime there are conditional mappings and output document list can be different from the input one. In that case try and use PSUtilities.list:addToList. Create a copy of this service in your package and use it.
  • Implicit Loop: for simple lists of the same size, you may want to link them directly in a MAP step and let the IS handle the looping implicitly.
  • Java Loop: Write a java service for looping logic.
Question Arises Why are we trying to avoid appendToDocumentList at all. It’s SAG provided utility and there should be some solid reason behind this. I got a very informative answer from Mr Percio (castropb) in wmusers.

Reason to avoid appendToDocumentList
Percio said - For large lists, appendToDocumentList performs poorly because of the way it is implemented. Every time you call appendToDocumentList, it basically creates a brand new list with size equal to plus 1 (assuming you're appending one item). It then copies all the items from the original list to the new list and puts the appended item at the end of the new list. This frequent memory reallocation and copying of data is what gives you the performance hit.

When you use an output array, output array is allocated once with the same size as the input array so you don't run into this problem.

I ran a test for a customer a few years ago to compare different methods of mapping a source list to a target list involving large lists (up to 100,000 items), and at the time, they ranked as follows from fastest to slowest:

1. Java Loop: looping done through a Java service.

2. Implicit Loop: for simple lists of the same size, you may want to link them directly in a MAP step and let the IS handle looping implicitly. Works when the variable names inside a doc list are identical.

3. Explicit Loop: using a LOOP step and its Output Array.

4. Append to Array List: similar to append to document list except that an array list is used in the background so there's no reallocation and copying of data. It is important to set an appropriate initial size to maximize performance.

5. Append to Document List: using the WmPublic service.

6. Dynamic Index: using a LOOP step without specifying Output Array and mapping to a specific item in the output list using a variable index

Note: Time taken to copy the lists grew linearly as the size of the list grew for method 1 to 4 and grew exponentially for method 5,6.

One new question arises after reading "appendToDocumentList performs poorly for larger lists" is what is a large list? It depends on the physical resources available to the IS and complexity of your mappings, etc.

From the analysis given by Percio, it is very clear that one should avoid appendToDocumentList.

Another Perspective
I also got an interesting answer from reamon. He mentioned

“Key for all of the options listed in Percio's excellent summary--test which approach works for your situation. Do *not* assume one approach will be faster than another.

In recent tests of my own with appendToDocumentList (though not with the number of elements Percio used) I found that performance had improved dramatically from the same tests I had done a few years ago. The JVM version and settings being used undoubtedly will have a big impact on this performance.

As Percio noted, there is no one answer. So the key is to try out different approaches until you get the performance your integration needs.”

Thank you Percio and Rob Eamon(reamon) for your valuable inputs.

Conclusion
I personally avoid appendToDocumentList because of performance and to shorten the code size.

As already said in post, before coming to conclusion of using or not using append service please make sure that you have tested all your scenarios and use the one which is best suitable for your integration.

Hope this post will help you guys. Comments/discussions are most welcome.

Appreciate if you can provide feedback in comment section or by clicking on reaction check box. Thanks

Wednesday, May 25, 2011

Trigger Processing Mode

Hey Guys,

Today I am going to post a doubt to discuss. I have faced this question a number of time and i don't find me giving a very satisfactory answer.

Question is :-


What is the difference/similarities between IS Trigger processing

  1. If mode is set to serial
  2. Or if mode is set to Concurrent but with Maximum Thread set to 1.
Something which i found out is if we have concurrent with max thread as 1, then this will indicate to Broker that it can hand out documents to multiple IS instances at the same time without any document order maintained.

Please post your thoughts in comment section.

Happy Learning :)

Tuesday, February 23, 2010

Transformer Inside Loop

Most of us have this misconception that transformers can not be used inside loop. well i had this misconception. Recently i got insight into this and got clarification that transformer can be used inside loop.

"Transformers can be used in a loop. What they do not do is accept a list when a single item is expected. The error message generated in this case is completely misleading. If you receive this error, you're passing a list instead of a single item. Pass a single item and the error will disappear."

Thanks Reamon for the clarification.