Pathfinder Core PRO clustering protects against hardware failures. Clustering allows you to connect multiple networked Pathfinder Core PRO systems so that they automatically and dynamically share configuration data and states. They also monitor each other and take over event processing if one system fails.
As you prepare to configure your cluster:
Begin with a single Pathfinder Core PRO.
Create the routers and add other required events and components.
Take a backup of your configuration and download it to your local computer. See the section on Backup/Restore for more information. This ensures you can get back to your current operational state if you make a mistake when setting up your cluster.
Click System > Clusters in the navigation bar.

Four buttons are available at the top of the Clusters page: Create, Join, Leave, and Manual Sync, followed by the Advanced link. Below them is the Floating IP Addresses list.
Note: Leave and Manual Sync are disabled until the system is part of a cluster.
Creating a Cluster
First, click Create to create the cluster. The system will ask you to create a cluster administration password:

This password is important because it is used throughout the cluster so that the Pathfinder Core PRO units can log in to each other and synchronize data. When you provide this password, the system will create a special ClusterAdmin user for this purpose. You can see this user when you view System > Users, and the system issues a severe warning if you try to change the password of that user, because it must be identical across all systems.
The system is now part of a one-node cluster, shown as a card on the Clusters page.
Joining a Cluster
Now add the second system to the cluster. The second system’s configuration will be overwritten during this process. If the system is new, make sure you first assign an IP address (see the Quick Start for your platform). The IP must have network access to the first unit in the cluster, so pay attention to the network configuration. You do not need to enable Livewire discovery on this second system.
On the second system, browse to System > Clusters, and then click the Join button. The system will ask for the IP address of any Pathfinder Core PRO system that is already a part of the cluster:

Click Join. The system then asks for the cluster administration password you provided when you created the cluster:

The system now shows a Joining Cluster progress message box while it joins the cluster.

If the systems use encrypted (TLS) cluster communication and the second system does not yet trust the first system's certificate, a Trust Certificate dialog appears. Click Trust and try the join again. See Security Settings and Clustering in the System article.
The join process tells the first system that the new node is joining the cluster. The second system then asks the first system to generate a special backup. It then downloads and restores that backup and reboots. A countdown is shown while it restores and reboots:

At that point, the cluster should be created, and you should be able to see both systems on the Clusters page.

Each node in the cluster has a card:
Item | Description |
|---|---|
Name | The hostname of the node. |
LOCAL | Shows which of the systems you are currently logged in to. |
ONLINE / OFFLINE | Whether the node can currently be reached by the cluster. |
EVENT SYSTEM | Marks the node that is currently running the event system (Logic Flows, Timers and so on). See Events and Timers below. |
IP | The IP address of the node, and the SapV2 port used for cluster communication. |
Node State | The state of the Pathfinder Core PRO system in the cluster. |
Event System | Yes on the node running the event system. |
Certificate | Whether this node trusts the other node's certificate (OK), for encrypted cluster communication. A failure here is a link that opens the Trust Certificate dialog. |
SYNC | The progress of a synchronization in progress. |
× | Ejects that node from the cluster (not shown on the local node's card). See Ejecting a Node below. |
The same cluster seen from the second system shows that node as LOCAL, with the event system still on the first:

A successfully synchronized cluster should show all node states on the web page of all systems as SyncNewerChangesComplete. If your cluster does not reach that state within a few minutes after a restart, contact Telos support to help determine why the cluster is not synchronizing properly.
Repeat this procedure for each additional Pathfinder Core PRO system you want to add to the cluster.
Once the cluster is created, changes to the configuration and state information are automatically synchronized across the Pathfinder Core PRO systems in the cluster. Additionally, if one of the Pathfinder Core PRO systems fails, one of the other systems begins processing the events. See the section on Events and Timers for more information.
Manual Sync
If there is some doubt that the cluster is properly synchronized, you can force it to synchronize. To do this, browse to the system that is incorrect (out of sync), select System > Clusters, and click the manual sync button. The system will ask for confirmation:

Click Sync. The system then requests a backup from the other (known good) system, downloads and restores that backup, and then reboots. This procedure is usually performed during a Telos support session.
Leaving a Cluster
To leave a cluster, first make sure that the system is connected to other nodes in the cluster. Then click System > Clusters and click the Leave button. The system will ask for confirmation:

Click Leave. The system reverts to a stand-alone system. After the system leaves a cluster, its configuration is still the same as it was before it left. Quickly remove the system from the network, reset it to factory defaults, or both. Otherwise, the system may start trying to process events that the rest of the cluster is also trying to process.
Ejecting a Node
The × button on a node's card should only be used in the rare situation where you need to forcefully remove a non-functioning node from the cluster. The system warns you first:

The preferred method is to go to the node to be removed and click the Leave button. Forcefully ejecting a node can cause problems if the node comes back online. An ejected node needs a restart, and it no longer shares the cluster's license (see Clustering and Licensing below).
Advanced Settings
Click Advanced to open the cluster's advanced settings:

Override Primary chooses which node runs the event system. Normally (None) the cluster decides, as described in Events and Timers below. Choosing a node makes that node run the event system whenever it is online, for example to keep events on the unit in the main equipment room. Use it with care, and set it back to None when it is no longer needed.
Floating IP Addresses
A floating IP address belongs to the cluster rather than to one node. It moves to whichever node should answer on it, so devices and users can always reach the cluster at the same address. The Pathfinder Core PRO GPIO node can also advertise itself on a floating IP (see Virtual GPIO Node in the Devices article).
Click Add + in the Floating IP Addresses list to add one:

Field | Description |
|---|---|
Id | A unique number for the floating IP. |
Floating IP address | The address. It must be in the same network range as the selected interface, and must not be used by anything else. |
Network interface | Livewire or Office. |
Host priority follows cluster | When selected, the address follows the node running the event system. |
Host priority list | When Host priority follows cluster is not selected: the cluster nodes' addresses on this interface, comma-separated, in the order in which they should take the floating IP. |
Click Apply to store the floating IP. It is synchronized to every node in the cluster.
Clustering and Licensing
A cluster shares its license, and a backup node that has no license of its own runs on its primary's license. See Licensing in a Cluster in the Licensing article.
Events and Timers
Some elements of a Pathfinder Core PRO system should only execute on a single system at a time, so that a cluster does not request duplicate changes from a device. Examples are Logic Flows and Timers. These always execute on the system that is currently active and has the lowest IP address. Heartbeats are constantly sent and monitored between the systems so that if a system stops responding, the system with the next lowest IP address takes over those responsibilities. The node currently doing this work is marked EVENT SYSTEM on the Clusters page. To make a particular node take precedence instead of the lowest address, use Override Primary (see Advanced Settings above).
Clustering and SapV2
SapV2 is the protocol used for both cluster communications and internal communications. This protocol is available on port 9600 and is described in more detail in the appendix of this manual. If the cluster does not seem to be working properly, this protocol can provide a great deal of information about the clustering process to help solve the problem.
GPIO Node Clustering
The Pathfinder Core PRO GPIO Node in a cluster is synchronized as well. Port additions, removals, and configurations applied to either Pathfinder Core PRO in the cluster are replicated to the other Pathfinder Core PRO in the cluster.
Pin closure states will also be replicated in most cases. If the port has been configured with a multicast GPIO channel or snake routing assignment (IP/port), replication becomes a bit more nuanced. Closures that are directly tripped will still be replicated. However, closures that are sensed from the multicast or unicast snake (GPIs from the other snake port) will not be replicated. This is because it is assumed that both cluster nodes are monitoring those changes directly from the third piece of equipment and will pick up their states directly from the other end of the snake. Therefore, cluster replication of that state would be redundant and could lead to race conditions.
When dealing with unicast snake mode (IP/port assignments), there are some additional clustering nuances to be aware of. These are easiest to explain with an example.
When you route a GPIO node to the Pathfinder Core PRO GPIO port, the address field in the GPIO port on both cluster nodes is filled in as IP address/port. For example:
System | Livewire IP address |
|---|---|
Pathfinder Core PRO A | 172.16.1.241 |
Pathfinder Core PRO B | 172.16.1.242 |
GPIO XNode | 172.16.1.85 |
If you route (using the Pathfinder Core PRO GPIO Router) the XNode GPIO source 1 to the Pathfinder Core PRO A or B destination 1, the destination port on both Pathfinder Core PROs will look like:
CFG GPO 1 SRCA:"172.16.1.85/1"In this example, both Pathfinder Core PROs are monitoring the GPI changes on port 1 of XNode at 172.16.1.85 and making matching changes on their GPO pins. This is straightforward.
However, if we route either of the Pathfinder Core PRO servers to the XNode GPIO, things are more complicated. We can only apply one IP Address to the SRCA field on the XNode destination. Therefore, Pathfinder Core PRO will use the Axia Livewire address of whichever Pathfinder Core PRO in the cluster currently has its event system active. In the example above, Pathfinder Core PRO A would typically be the active server in the cluster. When you make the route change on either Pathfinder Core PRO, the system will use the active server IP address on the XNode destination. The XNode destination would look like:
CFG GPO 1 SRCA:"172.16.1.241/1"If the Pathfinder Core PRO A server fails or is shut down, the B server loops through its routes, finds every destination that Pathfinder Core PRO A’s GPIO ports were routed to, and switches them to Pathfinder Core PRO B’s external Livewire IP address.
A fails
XNode port gets reassigned:
CFG GPO 1 SRCA:"172.16.1.242/1"If A comes back online, the port switches back to the A server when the event system comes online again.
Be aware that the XNode GPIO clears its pin states to all high each time a route change happens and then reassigns them to the new values based on the new GPIs. This means that if a snake route is held low on the XNode based on a low GPI on Pathfinder Core PRO, during a cluster failover those pins might flicker to high and then back to low. The flip is usually almost instantaneous.
This should only cause an issue if you have a Pathfinder Core PRO GPI that is unicast snake routed to an XNode (or other Axia GPIO device) GPO and that closure is both normally held low and you have an activity (such as an automation advance) that happens on the flip from high to low and you fail over to the secondary node in the cluster. This is similar to what would happen if a device supplying that same closure were to be restarted. This is not an issue for multicast snake routing.
To avoid this, advertise the GPIO node on a floating IP (see Floating IP Addresses above). The IP route then does not change. Instead, the IP address used for the external routes moves between the Pathfinder Core PRO devices.
We strongly recommend that you test this functionality to fully understand how it works. In most cases, it should be seamless.