Convergence Performance Tuning

Skip to end of metadata
Go to start of metadata

This documentation applies to Communications Suite 6 Update 1 at the time of its release. The most up-to-date documentation is available at the Communications Suite Documentation Home.

Tuning Application Server to Enhance Convergence Performance

Contributed by Russ Petruzzelli
Russ Petruzzelli is a Sun QA engineer currently conducting performance tests on Convergence.

This document contains the following sections:

Convergence Performance Tuning Overview

Recent advances in storage, servers, and Java have affected how one tunes web containers for middleware. There are systems with multi-threaded chips having 32 effective processors, operating systems with virtualized containers like Solaris zones, and file systems like ZFS that can spread files out over many disks. Java 1.6 can automatically adjust itself based on dynamic conditions. The tuning options available are many, and you must choose what works for you.

The tuning guidance presented here offers options to examine and configure. However, these options do not address specific hardware configurations and are not guaranteed to improve performance for any particular hardware configuration, performance load, or type of load on your system.

Try out the options and tips that apply to your deployment, test their impact on performance, and tweak the option values as needed.

Performance Tuning for Application Server 9.1 Update 1

The Convergence application itself is a java application bundled into a war file that runs inside a web container. Currently, the supported web container is the Sun Java Application Server 9.1 Update 1. This document describes how to optimize the Application Server environment to allow Convergence to deliver the best possible performance.

For information about how the Convergence components are deployed to Application Server, see Introduction to the Convergence Service.

In general, you can follow the tuning information in the Sun Java System Application Server 9.1 Performance Tuning Guide. Additional references are listed at the end of this document.

We recommend that you use the Application Server's administration browser interface rather than directly editing the domain.xml file to make the changes described below. The changes do not take effect until the domain instance has been restarted. Hardware is assumed to have at least 2 CPUs and 2G of memory (a Server-Class machine).

Make the modifications described below to the Application Server domain/config/port in which Convergence is running. The operating system in these examples is Solaris 10.

Tuning Application Server Config Parameters

This section contains the following topics:

Tuning Parameters for HTTP Service-Request Processing

On the Request Processing tab in the HTTP Service page tune the following HTTP Request Processing settings. (To navigate to this page, in the Application Server Admin Console, select Configuration > HTTP Service.)

  • Thread Count: 40 ## General Range: 20-80
  • Initial Thread Count: 8
  • Thread Increment: 2
  • Request Timeout: 20 (Change this setting from the default value of 30 seconds.)
  • Buffer Length: 16384 (Change this setting from the default of 8192 kilobytes.)
Here is good information from Scott Oaks's blog:

The request threads run HTTP requests. You want "just enough" of those: enough to keep the machine busy, but not so many that they compete for CPU resources – if they compete for CPU resources, then your throughput will suffer greatly. Too many request processing threads is often a big performance problem...
How many is "just enough"? It depends, of course – in a case where HTTP requests don't use any external resource and are hence CPU bound, you want only as many HTTP request processing threads as you have CPUs on the machine. But if the HTTP request makes a database call (even indirectly, like by using a JPA entity), the request will block while waiting for the database, and you could profitably run another thread. So this takes some trial and error, but start with the same number of threads as you have CPU and increase them until you no longer see an improvement in throughput.

Since Convergence communicates extensively with back-end messaging and calendar resources request blocking could be an issue. You will need to monitor your own deployment and adjust the Thread Count accordingly.

Tuning Parameters for the HTTP Listener

Increase the HTTP listener acceptor-threads. The default value is: acceptor-threads="1".

In the HTTP Service section of the Application Server Admin Console, on the listener for the port for Convergence (such as 8080):
Start with a value of 2, monitor the performance, and consider increasing to 4 or more.

To configure this setting, select the following items in the Application Server Admin Console: Configuration > HTTP Service > HTTP Listeners > Listener1.

(HTTP-listener-1 is assumed to be in use for Convergence.)

Take these steps:

  • Increase the acceptor-threads value to the number of CPUs on the system. (This has not been tested beyond 4.)
  • If you have only one interface (NIC), change the default 0.0.0.0 IP address to your IP for the host. For example: 192.18.75.103.

Configuring Application Server to Compress Files Sent to the Client

You can improve server response times by reducing the size of the HTTP response. One way is to compress file size by using the gzip utility. Not all files should be compressed (for example, images and some PDF files), but many files are compatible with this method.

To decide if this technique is right for you, search the Web for "gzip best practices" to find out more information. If you choose to implement this practice, realize that the server does more work to compress files into gzip format, which might impact the server's scalability under heavy loads.

To compress files sent to the client by using the Application Server Administration Console:

  1. In a browser, connect to the Application Server Administration Console.
    GZIP is turned on by adding additional properties to the http-listener element.
  2. In the Administration Console, navigate to the HTTP Listeners:
    Configuration >> HTTP Service >> HTTP Listeners
  3. Add the following properties the httplistener for Sun Convergence (normally httplistener1):
    compression on
    compressionMinSize 1000
    compressableMimeType "text/html,text/xml,text/css,text/javascript,text/json"
    

Alternatively, you can use the command line to compress files sent to the client by running the following commands:

asadmin set server.http-service.http-listener.http-listener-1.property.compression=on
asadmin set server.http-service.http-listener.http-listener-1.property.compressionMinSize=1000
asadmin set server.http-service.http-listener.http-listener-1.property.compressableMimeType="text/html,text/xml,text/css,text/javascript,text/json"
Note
While Gzip compression improves the response times for individual users, it does create extra work on the server and may impact scalability.

For more infomation, see http-listener in the Sun Java System Application Server 9.1 Administration Reference.

Enhancing Browser-Side Caching of Static Files

This section explains how to use Expires headers to enable long-term caching of static files in the browser.

When this feature is implemented, Application Server includes the Expires header in the HTTP response. The Expires header allows files cached in the browser to remain in cache for the time specified in the ExpiresFilter.class file.

Task Summary

To enable Expires headers, you must perform these tasks:

  • Modify the Convergence application's web-configuration file deployed to the Application Server's domain directory
  • Install a new class file

To Enable Expires Headers

The following steps assume that Convergence is configured in the Application Server's domain1 directory. (By default, the Application Server's domain directory on Solaris is /opt/SUNWappserver/domains/domain1).

Take these steps:

  1. Edit the ./domains/domain1/config/default-web.xml file. Add the following lines to the file:

    Insert the new filter and filter-mapping at the top level, directly under <web-app>. For example:
  2. Create a subdirectory named iwc under domains/domain1/lib/classes directory.
  3. Copy the ExpiresFilter.class) to the directory domains/domain1/lib/classes/iwc.
  4. Restart the Application Server.
The ExpiresFilter.class file provided has a two-year expiration time set. The Java source code is provided if you desire to set your own time interval. Modify the line c.set(c.get(Calendar.YEAR)+2, c.get(Calendar.MONTH) as desired.
The ExpiresFilter.class file and Java source code provided here have been copied from the original files and modified. This code is not supported or maintained by Sun Microsystems, Inc. For more information about the history of this filter, visit the author's blog here.

Tuning the JVM Options

This section contains the following topics:

Activating the Garbage Collection Log

This log has negligible impact on server performance and provides valuable debugging and performance history data.

Add the following entry, using your own path. For example:

<jvm-options>-Xloggc:/opt/SUNWappserver/domains/domain1/logs/gclog</jvm-options>

Note: This log is overwritten each time the server is restarted.

Invoking the Java HotSpot Server VM

Make sure that the JVM options in the domain.xml file for the Application Server instance specify -server, not -client:

<jvm-options>-server</jvm-options>

Server Class machines are defined as having at least 2 CPUs and 2 GB of memory.

Remove the -client option if present and add the -server option. You can verify what mode the server actually started with by running:

grep 'HotSpot' server.log"*,

This will show either ...Client VM... or ...Server VM.....

Activating the 64-bit JVM Mode

To configure and activate a 64-bit JVM, take these steps:

  1. On Solaris, you can verify that the operating system kernel is running in 64-bit mode by running:
    /usr/bin/isainfo -kv
    
  2. If needed, download and install the 64-bit jvm files on the JVM instance used by the Application Server on the machine. Verify the 64-bit files are available by running:
    "/<appserver_java_dir>/java -d64 -version"
    
  3. On the Application Server, replace the JVM option, -server (or -client), with -d64
Note
While running in a 64-bit JVM will increase the capacity available for the heap, it does not generally increase throughput. Some experienced users have found better results staying under the 4G limit, using a 32 bit JVM, and using multiple application server instances or Solaris 10 zones to scale horizontally. The JDK 1.6_11-and-below 64-bit-JVM imposes a 15-20% performance penalty (over the 32bit version). JDK1.6_12 and OpenJDK1.7 have resolved this.

Tuning the JVM Heap Size

In the Application Server Admin Console, under Common Tasks, select Application Server >> JVM Settings Tab >> JVM Options Sub-Tab >> Add/Modify the options...

The heap size option is: -XmxNNNNm.

Generally, you increase this up to nearly the available memory on your machine, leaving plenty of memory for the operating system and any other applications running on the machine. "Plenty" depends on your needs, but a safe value to set aside for the OS is around 300-500MB. Remember that both the PermGen Space and any Native Threads use memory over and above what you reserve for the JVM heap.

For example, you could set the heap size option to

<jvm-options>-Xmx2048m</jvm-options>
Note
JVMs running in 32-bit mode are limited to 4GB of memory. Convergence is installed (but not activated) with the full 64-bit JVM files.)

Setting Garbage Collection Algorithms

To increase the stability and predictability of the heap size and the ratios of its configuration, you can explicitly set the following parameters:

  • -XX:+UseParallelGC. --This parameter is used by default on a machine qualifying as Server Class. This default collector is sufficient.
  • -XX:+UseParallelOldGC. --This parameter is off by default. This statement makes the tenured generation run GC in parallel, too.
  • -XX:-UseAdaptiveSizePolicy. --Turn off GC ergonomics. Note the minus sign in this statement. Specify min and max values explicitly. See the summary below.
  • -XX:NewRatio=1 -- Optimize the Young Generation Size. Using a ratio (as opposed to setting a numerical size with NewSize) allows for the maximum possible young generation size relative to the overall heap, no matter your MaxHeap size.

    Tests show that most of the objects created for Convergence are short-lived, thus benefiting from a larger young generation size.

    The NewRatio means {New:Old}. So, when NewRatio=1, then new:old = 1:1. Therefore, the young generation size = 1/2 of the total Java heap. The young generation size can never be larger than half the overall heap because - in the worst case - all the young generation space could be promoted to the old generation. Therefore, the old generation must be at least as large as the young generation size.

    For more information about the NewRatio option, refer to: http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp. Monitor your own heap usage with the Jconsole monitor.

Setting the Permanent Generation Size

Be aware that MaxPermSize may need to be increased. But start with the default:

<jvm-options>-XX:MaxPermSize=192m</jvm-options>

Use a tool such as jconsole to determine how best to optimize your own system. For more details, see Miscellaneous Tips, below.

Tuning the JVM RMI GC Interval Parameters

It is better if full Garbage Collections (GCs) on the Java heap do not occur frequently and are not called explicitly. It is best to let the JVM decide when to do full garbage collections.

Unfortunately, the Application Server has a couple of JVM options for RMI applications that invoke full GCs often. If you are not running any applications using RMI, you should increase the rmi.dgc... values, or configure them never to occur.

The default intervals are:

  • Before JDK 1.6: 1 minute
  • JDK 1.6: 1 hour

These intervals are increased to 10 hours:

<jvm-options>-Dsun.rmi.dgc.server.gcInterval=36000000</jvm-options>
<jvm-options>-Dsun.rmi.dgc.client.gcInterval=36000000</jvm-options>
Note
Removing them entirely from the domain.xml file will cause the full GCs to occur at the default interval.

You can also use either of the following two options to prevent the full GCs invoked for RMI:

  • Disable explicit GC by adding:
<jvm-options>-XX:+DisableExplicitGC</jvm-options>
  • Use the 1.6 JVM and set:
-XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent

ExplicitGCInvokesConcurrent is only available in JVM 1.6+.

Ramifications of Disabling Explicit GCs

You should also consider the ramifications of disabling explicit GCs. When another application is connecting to the Application Server with RMI, memory for objects in the Application Server's heap will not be released and the calling application will not be able to release the reference to that object, thus possibly causing memory overflow on the other application.

Sample List of JVM Options

The following list is a sample section of the domain.xml file's JVM options:

<jvm-options>-server</jvm-options>
<jvm-options>-XX:+DisableExplicitGC</jvm-options>
<jvm-options>-XX:+UseParallelGC</jvm-options>
<jvm-options>-XX:+UseParallelOldGC</jvm-options>
<jvm-options>-XX:-UseAdaptiveSizePolicy</jvm-options>
<jvm-options>-Xms1024M -Xmx1024M</jvm-options>
<jvm-options>-XX:NewRatio=1</jvm-options>
<jvm-options>-XX:PermSize=192M</jvm-options>
<jvm-options>-XX:MaxPermSize=192M</jvm-options>
<jvm-options>-Xloggc:/opt/SUNWappserver/domains/domain1/logs/gclog</jvm-options>

Miscellaneous Performance Tuning Tips

Class Data Sharing

Class data sharing (CDS) is a new feature in J2SE 5.0. CDS applies only when the "Java HotSpot Client VM" is used. Since we recommend using the "Java HotSpot Server VM," this feature does not apply.

Inspect Settings

Inspect your settings with the following commands:

To see all java processes running on your machine:

jps -mlvV

To view your settings in effect for the JVM for the Application Server:

jmap -heap <java-process-id for the app server>.
Monitoring the JVM

JConsole is a built-in JVM monitoring tool in Java 1.5 and higher. On the SUT, set the display variable to your local machine and run the following command: jconsole

For more information, read the following articles:

UseConcMarkSweepGC

The intrepid system administrator may want to consider using UseConcMarkSweepGC instead of UseParallelGC. For details, see Tuning Garbage Collection with the 5.0 Java Virtual Machine.

New GC Algorithm

Look out for a new GC algorithm in Java 1.7 called G1. It is estimated to be released in 2009.

Additional Tuning Tips

You can find detailed guidance and additional performance tuning tips in these documents:

Additional References

Labels:
convergence convergence Delete
guide guide Delete
performance performance Delete
tuning tuning Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

Sign up or Log in to add a comment or watch this page.


The individuals who post here are part of the extended Oracle community and they might not be employed or in any way formally affiliated with Oracle. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Oracle nor any other party necessarily agrees with them.