"OutOfMemory" memory leak tips

When stress testing Enterprise software, we might encounter "OutOfMemory" exceptions.

This post talks about:

1. Memory leaks and Memory Analyzing tools
2. What are some tips for managing memory leaks ?

1.  Memory leaks and Memory Analyzing tools

Your starting point could be your server logs.  The server logs might have "OutOfMemory" exceptions in it.  Also, you could use a Memory analyzer tool to delve deep into your software and detect memory leaks.

Some of the Memory analyzing tools are given here:

• JProbe
• JProfiler
• Eclipse MAT
• jconsole, jmap

2.  What are some tips for managing memory leaks ?

For our discussion here, let's consider a J2EE Application server (e.g.: JBoss, IBM WebSphere, Oracle WebLogic, etc.,) as an example.  Assume the Application server is configured in cluster mode.

1. Bench marking:  To begin with, we need some reference Memory footprints which would serve as bench marking data points for our analysis.  We can start with at least 3 use cases with varying stress loads - LIGHT, MODERATE, HEAVY.  Based on these 3 data points, we can extrapolate and guesstimate the required minimum memory for the various loads.
2. -Xms and -Xmx: 
1. The JVM command line options -Xms and -Xmx to start the Application Server instances could be increased.  To begin with, the -Xmx could be increased to approximately 70% (this is just a thumb rule; based on trial and error, this threshold can be played with appropriately for your use case) of the maximum allowed by the system.  This should at least delay the occurrence of "OutOfMemory".  This is a measurable indicator.
2. When a service call is made in a cluster environment (with many server instances), we can not predict how calls are getting routed.  The routing algorithm could be round-robin, least busy, etc.  (My understanding is, this routing algorithm is chosen during the configuration of cluster environment).  Based on that, the service call gets routed to an instance of server which handles the request.  When we see "OutOfMemory" issues in the server logs, the first thing to be done is to correlate the request with server instance.  That would lead to the server instance which is starving for memory.  Memory needs to be pumped into that server instance.  To be consistent, we can bump the JVM command line options -Xms and -Xmx to the same value for every server instance.
3. References of objects which are no longer needed (ie, suppose the object has gone out of scope of the control) could be initialized to null.  This would indicate the Garbage Collector that that particular object is no longer needed and so its associated memory can be relinquished back to the heap.
4. Database Tier:  
1. When querying from DB tables, if you are using resultSet, the resultSet has a configurable parameter "fetchSize".  Play with that value and see if that helps.
2. If possible, consider using cursors for querying DB tables.  Lets take a typical table query, "select * from <Table>".  This is a commonly used full table scan query on all tables.  If resultSets are being used, this could return a ginormous resultSet object.  This resultSet is transformed into other objects (like JAXB objects, etc.,) and passed around to the upper tiers.  This resultSet would hold up lots of memory.  Using cursors, we could fine tune it, and deal it on demand, thereby consuming less memory.
5. EJB:  This is with respect to EJB technologies.  
1. The number of EJB pool instances configured for each EJB could be bumped down or up.  Even if this does not make a direct impact, this would not harm.  
2. Similarly, if JMS Queues or Topics are used, then you can play with the Flow control config parameters.  Bump down or up the pool instance counts and see if there is an impact.
3. Suppose you have a Stateless Session Bean in your application which supports methods that are responsible for fetching all rows from a DB table in a bulk fashion.  If the DB table is populated with tons and tons of rows, then trying to fetch all rows in one call could lead to "OutOfMemory" issue.  Instead of trying to fetch all rows in one call, you can consider a Stateful Session Bean approach, and use the Iterator pattern for fetching rows in a batch fashion from the DB table.  The Stateful Session Bean can be programmed to remember the number of rows fetched each time.     
6. XML Parsing:  (DOM, SAX, StAX, TrAX)  If possible, pick the optimal XML parser for the application.  DOM consumes more memory as it loads up the entire object tree.

SemTech Berlin 2012

My tech talk in SemTech Berlin 2012 went really well. The talk was also received well by the audience.

Here is the deck ppt of "Semantic Cloud Governance":
http://www.slideshare.net/arivolit/semantic-cloud-governance

Semantic Cloud Governance

Recently, I got an invite to speak in the SemTech Berlin 2012 conference. The topic of my speech would be "Semantic Cloud Governance".

The complete schedule of the conference is available at:

http://semtechbizberlin2012.semanticweb.com/agenda.cfm?confid=66&scheduleDay=PRINT

Here is the abstract link for "Semantic Cloud Governance":

http://semtechbizberlin2012.semanticweb.com/sessionPop.cfm?confid=66&proposalid=4463

In the subsequent posts, I'll write more about my findings in this topic!

See you!

Oracle Edition Based Redefinition question ...

I was playing with Oracle's EBR feature using Oracle 11.2.x. I added a new dummy column in a table using the "ALTER TABLE table_name ADD (column_name, dataType)" SQL statement. I also made the necessary required changes in the editioning view - namely adding the above mentioned column in the editioning view.

While testing, what I discovered was, the newly added column was always showing up in the first position (ie, column number is one) in the new editioning view. I am not sure if this is the expected behaviour in EBR. If you notice, the "ALTER TABLE table_name ADD (column_name, dataType)" SQL statement, would always add the new column as the last column in the table. I expected a similar thing in the new editioning view.

If I know the answer, will update this post!

SemTech UK 2011 conference in London, UK

Here is the Deck PPT of our speech titled Semantic SOA Governance, BPM and Complex Event Processing:

http://www.slideshare.net/arivolit/semantic-soa-governance-bpm-and-complex-event-processing


I attended many sessions as well. I'll write about them in a subsequent post.

Semantic SOA Governance, BPM and Comple Event Processing

I would like to let you know about an upcoming open source conference, the SemTech UK 2011. This conference would be held in London, UK between Sept 26-27, 2011. My ex-colleague, Keshava Rangarajan and I received an invitation to speak in the conference. The topic of our speech is "Semantic SOA Governance, BPM and Complex Event processing". Our topic falls under the "Innovative Products & Solutions" track.

For the complete events itinerary, please advert to:
http://semtechbizuk2011.semanticweb.com/programDetails.cfm?ptype=K&optionID=257&pgid=4

Here is the link to our abstract:
http://semtechuk2011.semanticweb.com/sessionPop.cfm?confid=63&proposalid=4309

Earlier Keshava Rangarajan and I have spoken in "HP Software Universe 2008" at Las Vegas representing Oracle. The topic was "Semantic SOA Governance". This time we are extending that topic, and additionally we would be covering BPM and Complex Event Processing as well.

Briefly, here are some of the challenges that we are intending to cover:

Challenges:
• How do we intelligently create, publish, search, discover, consume, manage, meter, monitor, govern and report on SOA services such that it delivers superior value to organizations?
• How can we deploy semantic technologies to solve this problem effectively ?


I intend to write more on this. Also, I'll write about the entire conference, other speeches, interesting ideas, etc. in the subsequent posts!

Semantic SOA Governance, Cloud Governance for enterprise vs public domain problems

I have been doing some research on using Semantics in SOA Governance and Cloud Governance for quite some time.

So far, I had only been considering Enterprise domain problems when we think of Governance solutions. I came across the following two links:

1) Cloud Governance:
http://www.theinfoboom.com/articles/why-clouds-needs-service-governance/

2) HP Service Catalog:
https://coral.saas.hp.com/coral/web/service-catalog/

These are interesting areas.

Now, if we open up our scope and consider public domain problems as well, along with the traditional enterprise domain problems, the significance would be great! But one thing to be aware of is, opening up the services infrastructure to public infrastructure would definitely change the scale of the problem, which brings its own set of challenges, but they can be solved.

Besides there is another angle. This is the mobile domain. How about the Twitter and Facebook ?

Basically, we would be very curious to know about what others think about our company, product, or our blog post or article that we put out. The users write reviews, and it would add great value to have an ontological structure and persist them in repositories. By this way, we would have covered the whole nine yards from the Taxonomy to the Folksonomy when we include the feedback reviews of users!

The potentials are great which demands attention and can not be overlooked!