Multi-Tier Clusters

Today most of projects are build upon a composite structure that has multiple different APIs. Since many of them is unaware of the others and has its of implementation and dependency , every one of them should be have its own configuration which will make them work as a single application.

This is the hard point in fact "To bundle them in single application"

An Example:

in a sample medium size application like AppFuse would be using Hibernate/JPA, Acegi,SpringFramework,JasperReports, JSF, Tomcat/Jetty and so on. it is all good even perfect when you are run in a single JVM. but if you want to build a cluster for only a load balancing you should configure every one by one.


In other world you be building a multi tier cluster for each APIs , which would be harder rhan implemation for some old fashioned singleton type application (a type of application which has lots of static variable and caches)

your applcation would be looking like this.


what should be your success criteria

performance : cluster should not slow down the application in a critical point
transaction: any rollback should be noticed
failover: in case of any instance fail over any other one can replace it.
seamless: the cluster mode should be seamless. should be hidden behind proper interfaces


today most open and commercial API support clusters. but in most cases you would implement a yous own schema for your own needs . before start you should read some articles about "Distributed Application" , you can use any distributable caching, maps, or JMS bridges between your components

Load Balance and Cluster Setup for Tomcat via Apache 2.2

for updated version please visit

http://www.altuure.com/2007/12/03/complete-apache-22-ajp-load-balance-via-mod_proxy/

Hello, here I will try to describe a sample configuration of a multiple tomcats and one APACHE for load balancing and identical tomcat server instances. These tomcat instances will be identical since they serve the same application 'medya'. This sample is based on opensuse 10.1 Remastered with apache 2.2 and tomcat 5.5.20. It’s done for a server with 16GB RAM and 4 CPUs and jvm only enable 2 GB heap size

main purpose is using same tomcat-home for diffrent tomcat-bases for identically same tomcat instances

Part I: Tomcat Cluster and Linux Service definitions

First download and extract the tomcat 5.5.x to /usr/tomcat55 this will the unique tomcat home for our tomcat instances. Same tomcat instance count will be 3 (s1,s2,s3). Then replace server.xml with the same file at the attachment. This file contains simple variable configuration like

<Server port="${tomcat.server.port}1" shutdown="SHUTDOWN"> .... <Connector port="${tomcat.server.port}2 ...

variable are given as external jvm parameters for the same configuration file.

tomcat.base: standard tomcat variable for common base jars files names and shell scripts.
tomcat.server.port: used in the configuration file as port prefix
${tomcat.server.port}1: Shutdown port
${tomcat.server.port}2: http connector port
${tomcat.server.port}3: ajp connector

eg:if tomcat.server.port:771 then shutdown port will be 7711, http will be 7712,ajp will be 7713

tomcat.server.name: tomcat server name for load balancing and cluster

and sample tomcat cluster configuration

then create a directory for first tomcat base. /opt/tb55/s1 and link file system to base tomcat instance I do this sample on a single computer and it can be done over a network since the files are only used at startup for very limit size. and copy same base for more

mkdir /opt/tb55
mkdir /opt/tb55/s1
ln /usr/tomcat55/conf /opt/tb55/s1/conf -s
ln /usr/tomcat55/shared /opt/tb55s1/shared -s
mkdir /opt/tb55/s1/logs
mkdir /opt/tb55/s1/temp
cp /usr/tomcat55/webapps /opt/tb55/s1/webapps -R
cp /opt/tb55/s1 /opt/tb55/s2 -R
cp /opt/tb55/s1 /opt/tb55/s3 -R

Up to we build single tomcat home and three tomcat base. these instances have same library with different applcation base(webapps),log and temp directories.

Part II: Defining and starting tomcats as linux service in a cluster structure.

copy the service.sh to /usr/tomcat55/bin it will be standard startup script. this script is based upon the script at link. and copy startup script tb-s1,tb-s2,tb-s3 to /etc/init.d for service definition.

content of tb-s1 (tomcat 1 service script)

TOMCAT_USER=tomcat
PP=771
export CATALINA_NAME=s1
export CATALINA_PORT=`echo $PP`1
export CATALINA_BASE=/opt/tb55/$CATALINA_NAME
export CATALINA_HOME=/usr/tomcat55
export CATALINA_TMPDIR=$CATALINA_BASE/temp
export CATALINA_NAME=s1
export CATALINA_LOCK=$CATALINA_BASE/$CATALINA_NAME.lock
export CATALINA_PID=$CATALINA_BASE/$CATALINA_NAME.lock
export JAVA_OPTS="-Duser.language=en -Dtomcat.server.port=$PP -Dtomcat.server.name=$CATALINA_NAME -Dcom.sun.management.jmxremote"
export JAVA_HOME=/usr/java/jdk1.5.0_05
chown $TOMCAT_USER:$TOMCAT_USER $CATALINA_HOME -Rh
chown $TOMCAT_USER:$TOMCAT_USER $CATALINA_BASE -hR
chown $TOMCAT_USER:$TOMCAT_USER $CATALINA_TMPDIR -Rh
su $TOMCAT_USER -m -c "/usr/tomcat55/bin/service.sh $1"

As you see this contains variables for server.xml and jvm options you add or remove yours. this service will be run as a system user tomcat.

now you can add this service as system service

chkconfig --add tb-s1
chkconfig --add tb-s2
chkconfig --add tb-s3

Part III: Defining AJP connector ,proxy and load balancer for APACHE

put mod_proxy_balancer.conf to /etc/apache2/conf.d

this file contains all configuration for

proxy_ajp_module: for tomcat ajp connector used by proxy module
proxy_module: used by apache to support for proxy
proxy_balancer_module: used to load balance application context over different tomcat instances

LoadModule proxy_module /usr/lib/apache2/mod_proxy.so
LoadModule proxy_balancer_module /usr/lib/apache2/mod_proxy_balancer.so
LoadModule proxy_ajp_module /usr/lib/apache2/mod_proxy_ajp.so
<Location /balancer-manager>
SetHandler balancer-manager
</Location>
<Proxy balancer://medyaCluster>
BalancerMember ajp://localhost:7713 route=s1
BalancerMember ajp://localhost:7723 route=s2
BalancerMember ajp://localhost:7733 route=s3
</Proxy>
<Location /medya>
ProxyPass balancer://medyaCluster/medya stickysession=JSESSIONID
</Location>

and restart apache2 service

Part IV: Deploying Sample application

simple put you application folder to /home/tomcat/medya

and out medya.xml to /usr/tomcat55/conf/Catalina/localhost/medya.xml

<Context docBase="${user.home}/medya" distributable="true" path="/medya"></Context>

I hope you enjoy your application :) .

Sorry for formating :(

tomcatsetup.tar.gz

for updated version please visit

http://www.altuure.com/2007/12/03/complete-apache-22-ajp-load-balance-via-mod_proxy/