Hazelcast – Setting up multi node instances
Given that Hazelcast is a distributed IMDG and typically is set up on multiple machines, it requires access to the internal/external network. The most important use-case being discovery of Hazelcast nodes within a cluster.
Hazelcast requires the following ports −
-
1 inbound port to receive pings/data from other Hazelcast nodes/clients
-
n number of outbound ports which are required to send ping/data to other members of the cluster.
This node discovery happens in few ways −
-
Multicast
-
TCP/IP
-
Amazon EC2 auto discovery
Of this, we will look at Multicast and TCP/IP
Multicast
Multicast joining mechanism is enabled by default. is a way of communication form in which message is transmitted to all the nodes in a group. And this is what Hazelcast uses to discover other members of the cluster. All the examples that we have looked at earlier use multicast to discover members.
Example
Let’s now explicitly turn it on. Save the following in hazelcast-multicast.xml
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd" xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <network> <join> <multicast enabled="true" /> </join> </network> </hazelcast>
And then, let us execute the following −
java -Dhazelcast.config=hazelcast-multicast.xml -cp .targetdemo-0.0.1- SNAPSHOT.jar com.example.demo.XMLConfigLoadExample
Output
In the output, we notice the following lines from Hazelcast which effectively means that multicast joiner is used to discover the members.
Jan 30, 2021 5:26:15 PM com.hazelcast.instance.Node INFO: [localhost]:5701 [dev] [3.12.12] Creating MulticastJoiner
Multicast, by default, accepts communication from all the machines in the multicast group. This may be a security concern and that is why typically, on-premise, multicast communication is firewalled. So, while multicast is good for development work, in production, it is best to use TCP/IP based discovery.
TCP/IP
Due to the drawbacks stated for Multicast, TCP/IP is the preferred way for communication. In case of TCP/IP, a member can connect to only known/listed members.
Example
Let’s use TCP/IP for discovery mechanisms. Save the following in hazelcast-tcp.xml
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd" xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <network> <join> <multicast enabled="false" /> <tcp-ip enabled="true"> <members>localhost</members> </tcp-ip> </join> </network> </hazelcast>
And then, let’s execute the following command −
java -Dhazelcast.config=hazelcast-tcp.xml -cp .targetdemo-0.0.1-SNAPSHOT.jar com.example.demo.XMLConfigLoadExample
Output
The output is following −
INFO: [localhost]:5701 [dev] [3.12.12] Creating TcpIpJoiner Jan 30, 2021 8:09:29 PM com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
The above output shows that TCP/IP joiner was use to join two members.
And if you execute following command on two different shells −
java ''-Dhazelcast.config=hazelcast-tcp.xml'' -cp .targetdemo-0.0.1-SNAPSHOT.jar com.example.demo.MultiInstanceHazelcastExample
We see the following output −
Members {size:2, ver:2} [ Member [localhost]:5701 - 62eedeae-2701-4df0-843c-7c3655e16b0f Member [localhost]:5702 - 859c1b46-06e6-495a-8565-7320f7738dd1 this ]
The above output means that the nodes were able to join using TCP/IP and both are using localhost as the IP address.
Note that we can specify more IPs or the machine names (which would be resolved by DNS) in the XML configuration file.
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd" xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <network> <join> <multicast enabled="false" /> <tcp-ip enabled="true"> <members>machine1, machine2....</members> </tcp-ip> </join> </network> </hazelcast>