JBoss AS7 Clustering Using mod_cluster and http 2.4 (Part 1)
This article is part one of a multi-part blog series that will explain how to set up a JBoss Cluster using httpd 2.4, JBoss mod_cluster, and JBoss AS 7. In this article I will explain how to build mod_cluster to work with Apache’s httpd, 2.4.3 specifically, and how to configure httpd to use mod_cluster.
I am writing this article because the latest version of mod_cluster available is 1.2.0.Final, which was built in February 2012 and only works with httpd 2.2.8+. I want to use mod_cluster with httpd 2.4.3.
My environment is CentOS 6.3 x86_64 with the Development Tools installed. While getting everything working, I suggest disabling selinux and iptables. They can easily get in the way and make it very difficult to troubleshoot. At the end of this article, I’ll explain how to configure them once you have things working correctly.
First, pull the latest code for httpd, apr, and apr-util. The most current versions at the time of this article are apr 1.4.6, apr-util 1.5.1, and httpd 2.4.3. One additional package that I needed to install was pcre-devel.
Now, start to build and install the packages. I’ll keep this part as brief as possible without skipping any important steps. Some of the parameters may not be necessary, but here is how I did it:
tar xzfv apr-1.4.6.tar.gz
./configure --prefix=/usr/local/httpd --datadir=/usr/local/httpd/var --enable-shared
sudo make install
tar xzfv apr-util-1.5.1.tar.gz
./configure --prefix=/usr/local/httpd --infodir=/usr/local/httpd/info --mandir=/usr/local/httpd/man --sysconfdir=/usr/local/httpd/conf --datadir=/usr/local/httpd/var --with-apr=/usr/local/httpd
sudo make install
tar xzfv httpd-2.4.3.tar.gz
./configure --prefix=/usr/local/httpd --enable-mpms-shared=all --with-mpm=event --enable-static-support --enable-mods-shared=reallyall --enable-proxy=shared --enable-proxy-ajp=shared --enable-proxy-balancer=shared --enable-proxy-connect=shared --enable-proxy-ftp=shared --enable-proxy-http=shared --with-apr=/usr/local/httpd --with-apr-util=/usr/local/httpd
sudo make install
Phew! Now that we have our base httpd installed, we can get to the important part of building and configuring mod_cluster. Because the latest release of mod_cluster doesn’t work with httpd 2.4.3, I chose to pull the mod_cluster code base directly from the master branch on github. I tried to build a few of the early tags, hoping that since they were marked as “Final” they would be more stable, but they all failed one or more tests.
git clone https://github.com/modcluster/mod_cluster.git
Now that mod_cluster is built successfully and passed all of its tests, we need to build the native modules that will be installed with httpd. I’ve chosen to put these modules in a separate folder rather than putting them into the httpd/modules folder. It’s just a matter of personal preference. From the base folder of the mod_cluster that you just cloned from git:
sudo mkdir /usr/local/mod_cluster
sudo cp mod_advertise.so /usr/local/mod_cluster/
sudo cp mod_manager.so /usr/local/mod_cluster/
sudo cp mod_proxy_cluster.so /usr/local/mod_cluster/
./configure --with-apxs=/usr/local/httpd/bin/apxs make
sudo cp mod_slotmem.so /usr/local/mod_cluster/
Next, we need to configure the mod_cluster modules to work with httpd. I chose to create a separate mod_cluster.conf file and put it in conf/extra but you could just as easily include these settings within the httpd.conf file. Here are the contents of my conf/extra/mod_cluster.conf:
LoadModule advertise_module ../mod_cluster/mod_advertise.so
LoadModule manager_module ../mod_cluster/mod_manager.so
LoadModule proxy_cluster_module ../mod_cluster/mod_proxy_cluster.so
LoadModule slotmem_module ../mod_cluster/mod_slotmem.soListen 10.0.1.100:1528
Require ip 10.0.1
Require ip 10.0.1
Obviously, you need to change your IP address to match your own, and if you want to, you may use a different port number. We also need to make a few changes to the conf/httpd.conf file to finish. We need mod_proxy.so and mod_proxy_ajp.so for mod_cluster to work correctly, so uncomment or add the following two lines:
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_module modules/mod_proxy_ajp.so
Finally, we need to make sure to include our mod_cluster.conf, so add the following line:
That should be it to get mod_cluster installed and configured in httpd 2.4.3. After we start up the httpd server we should be able to access the mod_cluster-manager via http://10.0.1.100:1528/mod_cluster-manager. Of course, you won’t see anything there yet; that will have to wait until next time.
Remember, at the beginning of this article I promised I’d explain how to configure selinux and iptable to play nice with mod_cluster. Below is how you would configure both on your httpd server.
To configure selinux, first install policycoreutils-python. Then, go into the directory where you cloned mod_cluster and then into the native/selinux folder. Once there you need to run the following:
make -f /usr/share/selinux/devel/Makefile
sudo semodule -i mod_cluster.pp
sudo semanage port -a -t http_port_t -p udp 23364
sudo semanage port -a -t http_port_t -p tcp 1528
For iptables, put in a few rules to allow the multicast traffic for mod_cluster, and JBoss as well as udp/tcp traffic for node to node communications. Here are the rules I used; yours may differ slightly depending on your current iptables setup:
sudo iptables -I INPUT 1 -p udp -d 18.104.22.168/24 -j ACCEPT -m comment --comment "mod_cluster multicast udp traffic"
sudo iptables -I INPUT 2 -p udp -d 22.214.171.124/4 -j ACCEPT -m comment --comment "clustered JBoss multicast udp traffic"
sudo iptables -I INPUT 3 -p udp -s 10.0.1.0/24 -j ACCEPT -m comment --comment "node to node udp traffic"
sudo iptables -I INPUT 4 -p tcp -s 10.0.1.0/24 -j ACCEPT -m comment --comment "node to node tcp traffic"
sudo iptables -I INPUT 5 -p tcp --dport 1528 -s 10.0.1.0/24 -j ACCEPT -m comment --comment "mod_cluster-manager traffic"
sudo /etc/init.d/iptables save
Now, iptables and selinux should be configured correctly to allow mod_cluster and httpd to communicate freely with the other servers.
One final note: if you chose a port other than 1528, you’ll need to make the same change to your iptables and selinux configurations.
Next time, I’ll go through configuring JBoss to use mod_cluster.
This work is licensed under a Creative Commons Attribution 3.0 Unported License