I’ve been taking a break from posting for the last couple of weeks. I was starting to get a bit run down, and feel like burn out was about to set in. The kind of blog posts I do take quite a bit of time, both in terms of the technical background work and the time to write and proof read the posts. Balancing that with work, plus personal projects, family life something had to take a break, and it’s not going to be work or family life 
Anyhow, I’m back with another 5 minute guide. This time, how to set up clustering with Apache Web Server and Apache Tomcat.
For the purposes of the rest of this article, when I say “Apache” I mean the web server, and when I say “Tomcat” I mean Tomcat.
There are pretty much two ways to set up basic clustering, which use two different Apache modules. The architecture for both, is the same. Apache sits in front of the Tomcat nodes and acts as a load balancer.
Traffic is passed between Apache and Tomcat(s) using the binary AJP 1.3 protocol. The two modules are mod_jk and mod_proxy.
mod_jk stands for “jakarta” the original project under which Tomcat was developed. It is the older way of setting this up, but still has some advantages.
mod_proxy is a newer and more generic way of setting this up. The rest of this guide will focus on mod_proxy, since it ships “out of the box” with newer versions of Apache.
You should be able to follow this guide by downloading Apache and Tomcat default distributions and following the steps. No funny business required.
Clustering Background
You can cluster at the request or session level. Request level means that each request may go to a different node – this is the ideal since the traffic would be balanced across all nodes, and if a node goes down, the user has no idea. Unfortunately this requires session replication between all nodes, not just of HttpSession, but ANY session state. For the purposes of this article I’m going to describe Session level clustering, since it is simpler to set up, and works regardless of the dynamics of your application.
……. After all we only have 5 minutes!
Session level clustering means if your application is one that requires a login or other forms of session-state, and one or more your Tomcat nodes goes down, on their next request, the user will be asked to log in again, since they will hit a different node which does not have any stored session data for the user.
This is still an improvement on a non-clustered environment where, if your node goes down, you have no application at all!
And we still get the benefits of load balancing across nodes, which allows us to scale our application out horizontally across many machines.
Anyhow without further ado, let’s get into the how-to.
Setting Up The Nodes
In most situations you would be deploying the nodes on physically separate machines, but in this example we will set them up on a single machine, but on different ports. This allows us to easily test this configuration.
Nothing much changes for the physically separate set up – just the Hostnames of the nodes as you would expect.
Oh and I’m working on Windows – but aside from the installation of Apache and Tomcat nothing is different between platforms since the configuration files are standard on all platforms.
- Download Tomcat .ZIP distribution, e.g.
- We’ll use a folder to install all this stuff in. Let’s say it’s “C:\cluster” for the purposes of the article.
- Unzip the Tomcat distro twice, into two folders -
C:\cluster\tomcat-node-1
C:\cluster\tomcat-node-2
- Start up each of the nodes, using the bin/startup.bat / bin/startup.sh scripts. Ensure they start. If they don’t you may need to point Tomcat to the JDK installation on your machine.
- Open up the server.xml configuration on
c:\cluster\tomcat-node-1\conf\server.xml
- There are two places we need to (potentially) configure -
The first line is the connector for the AJP protocol. The “port” attribute is the important part here. We will leave this one as is, but for our second (or subsequent) Tomcat nodes, we will need to change it to a different value.The second part is the “engine” element. The “jvmRoute” attribute has to be added – this configures the name of this node in the cluster. The “jvmRoute” must be unique across all your nodes. For our purposes we will use “node1″ and “node2″ for our two node cluster.
- This step is optional, but for production configs, you may want to remove the HTTP connector for Tomcat – that’s one less port to secure, and you don’t need it for the cluster to operate. Comment out the following lines of the server.xml -
- Now repeat this for C:\cluster\tomcat-node-2\conf\server.xml
Change the jvmRoute to “node2″ and the AJP connector port to “8019″.
We’re done with Tomcat. Start each node up, and ensure it still works.
Setting Up The Apache Cluster
Okay, this is the important part.
- Download and install Apache HTTP Server.
Use the custom option to install it into C:\cluster\apache2.2
- Now open up c:\cluster\apache2.2\conf\httpd.conf in your favourite text editor.
- Firstly, we need to uncomment the following lines (delete the ‘#’) -
These enable the necessary mod_proxy modules in Apache. - Finally, go to the end of the file, and add the following:
<Proxy balancer://testcluster stickysession=JSESSIONID> BalancerMember ajp://127.0.0.1:8009 min=10 max=100 route=node1 loadfactor=1 BalancerMember ajp://127.0.0.1:8019 min=20 max=200 route=node2 loadfactor=1 </Proxy> ProxyPass /examples balancer://testcluster/examples
The above is the actual clustering configuration.
The first section configures a load balancer across our two nodes. The loadfactor can be modified to send more traffic to one or the other node. i.e. how much load can this member handle compared to the others?
This allows you to balance effectively if you have multiple servers which have different hardware profiles.
Note also the “route” setting which must match the names of the “jvmRoutes” in the Tomcat server.xml for each node. This in conjunction with the “stickysession” setting is key for a Tomcat cluster, as this configures the session management. It tells mod_proxy to look for the node’s route in the given session cookie to determine which node that session is using. This allows all requests from a given client to go to the node which is holding the session state for the client.
The ProxyPass line configures the actual URL from Apache to the load balanced cluster. You may want this to be “/”
e.g. “ProxyPass /balancer://testcluster/”
In our case we’re just configuring the Tomcat /examples application for our test. - Save it, and restart your Apache server.
Test It Out
With your Apache server running you should be able to go to http://localhost/examples
You should get a 503 error page as per below -
This is because both Tomcat nodes are down.
Start up node1 (c:\cluster\tomcat-node-1\bin\startup) and reload http://localhost/examples
You should see the examples application from the default Tomcat installation -
Shut down node1, and then start up node2. Repeat the test. You should see the same page as above. We have transparently moved from node1 to node2 since node1 went down.
Start both nodes up and your cluster is now working.
You’re done!
Optional: Set Up Apache Balancer Manager
mod_proxy has an additional “balancer manager” component which provides a nice web interface to the load balanced cluster. It’s worthwhile setting this up if you want to remotely administer / monitor the cluster.
To do so is easy -
- Add the following to the bottom of your C:\cluster\apache2.2\conf\httpd.conf
<Location /balancer-manager> SetHandler balancer-manager AuthType Basic AuthName "Balancer Manager" AuthUserFile "C:/cluster/apache2.2/conf/.htpasswd" Require valid-user </Location>
This configures the balancer manager at http://localhost/balancer-manager
- We need to create a password file to secure it. At the command prompt you can use -
c:\cluster\apache2.2\bin\htpasswd -c c:\cluster\apache2.2\conf\.htpasswd admin
Then set a password when prompted. This password would be used by the balancer-manager URL to authenticate.
Restart your Apache web server, and go to http://localhost/balancer-manager
You should be prompted for a username/password as you set before, and see the balancer manager tool as below:
Related posts:
Nice article, but as always with Tomcat – it needs Apache HTTPD server to get balanced. I haven’t found *one* article that describes how to do the clustering (for production – not just for playing) with *pure Java*, i.e. only with Tomcat( maybe + other Java based software). E.g. Resin can do clustering by itself, and other containers too, but not so with Tomcat
.
Nice post, thanks for sharing.
Thanks. Useful article
Nice article…its more useful.
Could you please do same on JBOSS?
Thanks a lot for this wel organized guide. Really helps me, I have bookmarked it.
Truly a very interesting and informative post.
Thanks for sharing. will be busy trying this out..
Nice Article !
It is extremely high quality article. I want to draw attention of those readers who are new to Apache Web Server. Please do not forget to change the port of your Apache Webserver (which runs on port 80 by-default). This article also helped in this case
(http://stackoverflow.com/questions/195641/windows-could-not-start-the-apache2-on-local-computer-problem)
Once my webserver get started then it is rally a 5 minute task.
Thanks a lot Richard !
Thanks, great article. Clear and to the point.
The single apache instance is a SPOF.
You should extend this configuration to eliminate the single point of failure (SPOF).
A nice configuration could be using a combination of wackamole and round robin dns.
See this article for more details:
http://wiki.directi.com/display/DEV/The Compendium of Load Balancing Strategies.
Great article by the way.
Really Very very good notes compiled at one location
Guys…. please help i’m getting the following error at last step
Forbidden
You don’t have permission to access /balancer-manager/ on this server.
I am not able to create the htpasswd file in the conf folder so i have created inside bin folder and copied into the conf folder still not able to access the balancer manager.
The error you’re getting just means that you have failed Apache’s authentication check for the URL.
This could be either because your login details do not match the ones in your .htpasswd file, or Apache may not be able to see the file, either because it is not where Apache expects (i.e. a config error) or you may have a permissions problem (can Apache’s running user actually read the .htpasswd file?)
Can’t really say anymore on the info above.
Hi Richard,
Thanks for replying me. i have resolved this error by using “Allow from localhost” inside the Location tag. as this is a try out i’ll look into this authentication problem later. Your tutorial on clustering is extremely superb. Have u posted any tutorial on load balancing and session replication?
Thanks Richard [You Rocks]…..
@Gerald, glad you have managed to workaround the issue. I have a partially complete article on session replication done, but not ready as yet. Should be posted sometime in the coming months
Dear Richard,
I was breaking my heads in Tomcat Clustering with many websites for a very long time, after going through your post just once I’m able to complete it in less than 5 minutes. Its an excellent tutorial . Excellent easy article. Do u have any similar posts on Load Balancing and Session replication or can u point me to any tutorials on this topic. Anyway Thanks for this great post. Keep up the good work going.
God Bless.
Nice and crisp; very useful.
Hi
I followed your guide and it helped me a lot, thank you very much!
I do have one problem tho and I think it’s session related.
The application that I’m trying to run logs in to a database. When I only have one tomcat started it works perfectly(either one) but when both are started I can’t get past the log in screen. When using the Apache port. (http://localhost/application/) Runs on port 80
If I log in through the separate tomcat URLs, both started: ( (http://localhost:8085/application/) & (http://localhost:8086/application/) I can log in perfectly.
Please help
@Rian
Check the “jvmRoute” setting in Tomcat matches the “route” setting in Apache, and the Proxy section in Apache has the “stickysession=JSESSIONID” in the declaration.
If that’s all correct then you may have an application specific issue.
Nice article. If u don’t mind, could you please do same on request level clustering?
Thanks for a helpful article!
I ran into a minor issues with jsessionid parameters, cookies and Apache wicket. Perhaps this is obvious to others, but it seems important that the source and destination paths be the same. In the above example, you use “/examples” and “balancer://testcluster/examples” whereas I’m not sure if it works properly with sessions and backend processing if you mapped “/myredirect” to “balancer://testcluster/examples”
What about SHUTDOWN port 8005 . Will it be same ? As per my knowledge we can’t start second tomcat if it’s shutdown port is 8005 and 8005 is already occupied.
Changing the shutdown port of second tomcat runs smoothly, But i am facing one problem. When i stop one tomcat, In balancer-manager it still shows ‘OK’ instead of ‘err’. I have cleared cookies etc. Still it is showing status ‘OK’.
btw. this article has saved a lot of time and it did what i was looking since couple of weeks.
Thanks a lot.
skd
ProxyPass /examples balancer://testcluster/examples
i am not getting testcluster here.plz help me.
ProxyPass /examples balancer://testcluster/examples
i am confused for the purpose of testcluster
Excellent article. I was looking for one that gives a no nonsense step by step account of how to set up clustering with Tomcat Apache and I found it.
Thanks Richard, Very nice tutorial about tomcat clusters, I was looking for it for 1 week, really saved my time both reading and implementing
Big thanks for your tutorial Richard..
i foolwed the instructions ..everythin wokrked fine on windows…i have to do the same setup in ubuntu..i found out that by default mod_proxu is not available in apache for ubuntu…i tried reverse_proxy but did not work..can you please share some ideas on how to proceed?
Hi,
I have the cluster running,
When one server is down, request is redirected to the other node by apache.
But the Session Data is lost.(I am using the simple session jsp, which displays session info and object)
Is it suppose to work like this or the session objects replicated in this ?
nicely done and it works…good starting point to expand on…
@kannan
You have configured it correctly. The above config only does load balancing, not transparent failover.
For that you need to configure session replication between your Tomcat instances. You also need to be sure that your app doesn’t store anything that is not Serializable safely in the HttpSession.
For many use-cases having the user restart their session if an app-server happens to fail is an ok trade-off to make and it simplifies the infrastructure quite a bit.
Thanks much. Hakuna Matata…