Open Source Software Technical Articles

Want the Best of the Wazi Blogs Delivered Directly to your Inbox?

Subscribe to Wazi by Email

Your email:

Connect with Us!

Current Articles | RSS Feed RSS Feed

How to Run Multiple Instances of Tomcat on a Single Server

  
  
  

Sometimes you want to host applications that cannot run on the same instance of Tomcat because of some incompatibility between the required classes of the two applications, or you might want to deploy the same application multiple times, perhaps for different customers, and you cannot do that on a single instance of Tomcat. Don't despair – in those situations, you can run multiple instances of Tomcat on a single GNU/Linux server. Each will use the same binary, but turn to different areas for data.



In this guide I'll run Tomcat on a CentOS 6 server and create two new Tomcat instances – one to run Solr and one for Jopr. Regardless of which GNU/Linux distribution you use or which Java applications you run in Tomcat, the principles are the same as those shown here.




You can install the latest Tomcat release on CentOS with the command yum install tomcat6. Tomcat relies on settings defined in certain environment variables. To run multiple instances, you have to understand the role of two variables in particular:



$CATALINA_HOME represents the root of the Tomcat installation; in CentOS the default value is /usr/share/tomcat6. Under the root you'll find two directories:




    • lib contains all libraries used for Tomcat. Any library present here is visible to all Tomcat instances.

    • bin contains shell scripts to start and stop Tomcat. You can create a modified version in /etc/init.d/ to start and stop the instances.




$CATALINA_BASE is used when Tomcat is configured to run multiple instances. If defined, Tomcat calculates all relative references for files in the following directories on the basis of the value set for CATALINA_BASE instead of CATALINA_HOME:




    • conf contains configuration files and related DTDs. The most important file located here is server.xml.

    • logs holds log and output files.

    • webapps is where you put the applications.

    • work – Tomcat translates and converts any JavaServer Pages (JSP) into servlets and stores them here.

    • temp is used for temporary files.

By default on CentOS $CATALINA_BASE has no value, so all the directories are related to $CATALINA_HOME. We want to change this, and to do so we'll work with the script that starts and stops Tomcat, but first, let's create the skeleton of the directory structure that we want to use to store the information from our Tomcats.


19a98812-f823-48dc-841e-bf029c63c6d7

Tomcat Directory Structure


I like to keep all my Tomcat instances in an easy-to-reach directory with a standard structure. I use /srv/tomcat/instance name. The following commands prepare your directories, populate the conf directory, and change the ownership of the files to the tomcat user:




for i in solr jopr
do
mkdir -p /srv/tomcat/${i}/logs
mkdir /srv/tomcat/${i}/webapps
mkdir /srv/tomcat/${i}/work
mkdir /srv/tomcat/${i}/temp
cp -R /usr/share/tomcat6/conf/ /srv/tomcat/${i}
done
chown -R tomcat. /srv/tomcat/


Tomcat Init Script and Configuration Files



By default the Tomcat init script is located in /etc/init.d/tomcat6. It looks for a config file in /etc/syscinfig/tomcat6 in which you can define $CATALINA_BASE and other things specific to a single instance. To be ready to have two new instances you need to create two new configuration files for the instances and make two links to /etc/init.d/tomcat6. The names of the scripts in /etc/init.d and /etc/sysconfig must be the same; you can use the name of the instances, in this case solr and jopr:




cp /etc/sysconfig/tomcat6 /etc/sysconfig/solr
cp /etc/sysconfig/tomcat6 /etc/sysconfig/jopr
ln -s /etc/init.d/tomcat6 /etc/init.d/solr
ln -s /etc/init.d/tomcat6 /etc/init.d/jopr


Now edit the configuration file for each instance and set the parameters for it. For example, for solr, set these variables:




CATALINA_BASE="/srv/tomcat/solr"
CATALINA_PID="/var/run/solr.pid"
CONNECTOR_PORT="8180"
TOMCAT_LOG="/srv/tomcat/solr/logs/catalina.out"


Each instance of Tomcat must be configured to have different values for these variables, or you may get errors or anomalies. In this file you can also set the JAVA_OPTS variable, so you could decide to give 512MB of RAM to instance Solr and 1024MB of RAM to instance Jopr.



The last thing to set up is the main Tomcat configuration file, $CATALINA_BASE/conf/server.xml. In our example we have one for each instance:




/srv/tomcat/solr/conf/server.xml
/srv/tomcat/jopr/conf/server.xml


In this file you should customize the ports used by each instance. I suggest incrementing the port numbers by 100 for each instance number. For example, instance number 1 would have 8180 as the http port and instance number 2 would have 8280 as http port. If you keep a strict policy on port assignment, administration is much easier.



Here's a list of the ports to change:




    • Shutdown port is used for shutting down Tomcat. When we call the shutdown.sh script, it sends a signal to shutdown port. If the Tomcat Java process receives the signal, it cleans up and exits.

    • HTTP port is the actual port that exposes the application to an outside client via HTTP.

    • ajp (Apache JServ Protocol) port may be used by a web server (such as the Apache httpd server) to communicate with Tomcat. This port is also used if you set up a load-balanced server.

    • Redirect port – If the Connector supports non-SSL requests and a request is received for which a security constraint requires SSL, Catalina will automatically redirect the request to this port.



Let's Go!



Once you've saved your changes in the configuration files, you're all done. You can put your applications in the $CATALINA_BASE/webapps directory and start the two instances; start the solr instance with the command /etc/init.d/solr start and the jopr instance with /etc/init.d/jopr start. They will run independently and you can start and stop them as you like, or start them automatically at boot time.



We've only set up two instances here, but you can add as many instances as you like; your only constraint will be the amount of RAM on your CentOS server.



A Final Note on Class Loaders



If you have an environment with multiple Tomcats, or multiple web apps on the same Tomcat, or a mix of the two, you also need to understand the role of Tomcat class loaders.



The Java Classloaders are the part of the Java Runtime Environment that dynamically loads Java classes into the Java Virtual Machine. Multiple class loaders enable a developer to partition the JVM in such a way that you can have multiple, different definitions of the same class loaded.



If you're running just a single instance of Tomcat, you can't go wrong; you have just one application, so it's not so important where the classes are loaded. This changes if you have multiple Tomcats running and you want to have, for example, class A loaded only on Tomcat A.



When Tomcat is started, it creates a set of class loaders that are organized into the following parent-child relationships, where the parent class loader is above the child class loader:




Bootstrap
|
System
|
Common
/ \
Webapp1 Webapp2 ...



Normally, application classes should not be placed in the common class loader, because classes that are here are visible to both Tomcat internally and to all web applications. The locations searched by this class loader are defined by the common.loader property in $CATALINA_BASE/conf/catalina.properties. The default setting searches the following locations in order:




    • unpacked classes and resources in $CATALINA_BASE/lib

    • AR files in $CATALINA_BASE/lib

    • unpacked classes and resources in $CATALINA_HOME/lib

    • JAR files in $CATALINA_HOME/lib



Given that, you can use the directory $CATALINA_BASE/lib as a repository of jar files and classes that must be loaded before the applications, but only from one Tomcat, such as JDBC drivers for databases or classes for LDAP servers.




This work is licensed under a Creative Commons Attribution 3.0 Unported License
Creative Commons License.


This work is licensed under a Creative Commons Attribution 3.0 Unported License
Creative Commons License.

Comments

I created two instances, prod and stag, followed all these steps, modified parameters in /etc/sysconfig/prod and /etc/sysconfig/stag, and CATALINA_BASE/conf/server.xml for these instances. When start /etc/init.d/prod, I notice it still points CATALINA_BASE to that one specified in /etc/init.d/tomcat6, /usr/share/tomcat6. How to debug where it goes wrong? 
 
Thank you!
Posted @ Friday, July 27, 2012 4:46 PM by Charlie
I have just worked through this using Tomcat7 from the jpackage project and it doesn't work. 
 
The problem is that although the init script follows the Red Hat way of doing things with /etc/sysconfig etc, it then calls another script /usr/sbin/dtomcat7 which goes and reads the the main configue (/etc/tomcat7/tomcat7.conf there by completely ignoring everything in the init script!
Posted @ Wednesday, August 22, 2012 10:30 PM by Mike
That worked a treat, many thanks for the great post. THere was one additional step that I needed to do on Red Hat Linux, create a directory in /var/log for each instance: 
 
mkdir /var/log/solr 
mkdir /var/log/jopr
Posted @ Monday, October 22, 2012 2:54 AM by gareth
I have written a Java based portable utility for automating the above steps for creating multiple tomcat instances.
Posted @ Monday, January 07, 2013 4:04 AM by Java Experience
@Mike For Tomcat 7 on Red Hat, go to /usr/sbin/dtomcat7 and comment out the part where the script is loading /etc/tomcat7/tomcat7.conf. 
 
Worked for me.
Posted @ Tuesday, January 29, 2013 8:32 AM by Panos
Doesn't work for me with yum install. There seems to be something pointing back to the original (CATALINA_HOME) config. When I start the new instance (appsrv1) I get port already in use errors or some permission denied error because the tomcat6.pid file doesn't exist(depending on if the $CATALINA_BASE instance is running or not).
Posted @ Tuesday, February 12, 2013 10:00 AM by tj
Can we make two Tomcat Servers part of cluster?
Posted @ Wednesday, February 13, 2013 2:58 AM by java67
good
Posted @ Wednesday, April 17, 2013 6:15 AM by deepak
Great post. Definitely will be back for more.
Posted @ Monday, April 29, 2013 12:50 PM by Paul
Is there a parameter that limits the number of tomcat instances on a the server. 
I have installed three instances and at a time only two of them work.
Posted @ Thursday, May 23, 2013 12:52 AM by Shivani
nice, works good for me using tomcat downloaded binaries, ie: not via yum. Now only if there was an init script that could start them all up or stop them all...I have 7 instances!
Posted @ Wednesday, October 16, 2013 6:23 PM by Isaac
I followed the steps, but i get this error: 
$ cat /srv/tomcat/solr/logs/catalina.out  
/usr/sbin/tomcat6: line 30: /srv/tomcat/solr/logs/catalina.out: Permission denied 
 
I described the exact details here: 
http://superuser.com/questions/674860/running-multiple-instances-of-tomcat-on-centos 
 
what can be the problem?
Posted @ Tuesday, November 12, 2013 5:24 AM by David Portabella
This is a nice setup, I have written a script simillar to this to automate things. 
 
Unfortunally RedHat does not provide support to multiple instances setup!  
Posted @ Thursday, August 28, 2014 2:22 PM by Felipe Ferreira
Post Comment
Name
 *
Email
 *
Website (optional)
Comment
 *

Allowed tags: <a> link, <b> bold, <i> italics