[[JGroups_Subsystem]]
= JGroups Subsystem

ifdef::env-github[]
:tip-caption: :bulb:
:note-caption: :information_source:
:important-caption: :heavy_exclamation_mark:
:caution-caption: :fire:
:warning-caption: :warning:
endif::[]

[[jgroups-purpose]]
== Purpose

The JGroups subsystem provides group communication support for HA services in the form of JGroups channels.

Named channel instances permit application peers in a cluster to communicate as a group and in such a way that the communication satisfies defined properties (e.g. reliable, ordered,failure-sensitive).
Communication properties are configurable for each channel and are defined by the protocol stack used to create the channel.
Protocol stacks consist of a base transport layer (used to transport messages around the cluster) together with a user-defined,
ordered stack of protocol layers,
where each protocol layer supports a given communication property.

The JGroups subsystem provides the following features:

* definition of named protocol stacks and channels
* specification of a stack for each channel and a default channel
* viewing of runtime metrics associated with channels

In the following sections, we describe the JGroups subsystem.

[IMPORTANT]

JGroups channels are created transparently as part of the clustering functionality.
For example, on clustered application deployment, channels will be created behind the scenes to support clustered features such as session replication or transmission of SSO contexts around the cluster.

[[jgroups-configuration-example]]
== Configuration example

[IMPORTANT]

For up-to-date information about subsystem configuration options see the
link:feature-pack/doc/reference/subsystem/jgroups/index.html[/subsystem=jgroups model reference].

Following is the default JGroups subsystem configuration.
We shall use this example to explain the meaning of the various elements and attributes.

[source,xml,options="nowrap"]
----
<subsystem xmlns="urn:jboss:domain:jgroups:9.0">
    <channels default="ee">
        <channel name="ee" stack="udp" cluster="ejb"/>
    </channels>
    <stacks>
        <stack name="udp">
            <transport type="UDP" socket-binding="jgroups-udp"/>
            <protocol type="RED"/>
            <protocol type="PING"/>
            <protocol type="MERGE3"/>
            <socket-protocol type="FD_SOCK2" socket-binding="jgroups-udp-fd"/>
            <protocol type="FD_ALL3"/>
            <protocol type="VERIFY_SUSPECT2"/>
            <protocol type="pbcast.NAKACK2"/>
            <protocol type="UNICAST3"/>
            <protocol type="pbcast.STABLE"/>
            <protocol type="pbcast.GMS"/>
            <protocol type="UFC"/>
            <protocol type="MFC"/>
            <protocol type="FRAG4"/>
        </stack>
        <stack name="tcp">
            <transport type="TCP" socket-binding="jgroups-tcp"/>
            <protocol type="RED"/>
            <socket-protocol type="MPING" socket-binding="jgroups-mping"/>
            <protocol type="MERGE3"/>
            <protocol type="FD_ALL3"/>
            <protocol type="VERIFY_SUSPECT2"/>
            <protocol type="pbcast.NAKACK2"/>
            <protocol type="UNICAST3"/>
            <protocol type="pbcast.STABLE"/>
            <protocol type="pbcast.GMS"/>
            <protocol type="UFC"/>
            <protocol type="MFC"/>
            <protocol type="FRAG4"/>
        </stack>
    </stacks>
</subsystem>
----

[IMPORTANT]

The XML schema for the subsystem, describing all valid elements and attributes,
can be found in the WildFly distribution,
in the `docs/schema` directory.

[[subsystem]]
=== <subsystem>

This element is used to configure the subsystem within a WildFly configuration profile.

* `xmlns` specifies the XML namespace of the JGroups subsystem and, in particular, its version.

[[jgroups-subsystem-channels]]
=== <channels>

This element is used to configure JGroups channels and the default channel to use.

* `default` is used to specify a default channel for the JGroups subsystem.
This default channel will be used whenever a channel is required but no channel is specified.

[[jgroups-subsystem-channel]]
==== <channel>

This element configures individual JGroups channels with attributes:

* `name` defines the name of this channel that can be referenced or looked up.
* `stack` defines the stack configuration to use for this channel.
* `cluster` defines the logical cluster name shared across the cluster members.

[[stack]]
=== <stack>

This element containing a sequence of protocols is used to configure a JGroups protocol stack.

* `name` This attribute is used to specify the name of the stack.

[[jgroups-transport]]
=== <transport>

This element is used to configure the transport layer (required) of the protocol stack with attributes:

* `type` specifies the transport type (e.g. `UDP`, `TCP`, `TCP_NIO2`)
* `module` defines a JBoss Modules module to load the protocol class from (defaults to `org.jgroups`)
* `socket-binding` references a defined socket binding in the server profile.
It is used when JGroups needs to create general sockets internally.
* `diagnostics-socket-binding` references a defined socket binding in the server profile.
It is used when JGroups needs to create sockets for use with the diagnostics program.
For more about the use of diagnostics,
see the JGroups documentation for `probe.sh` tool.
* `thread-pool` configures the thread pool options, such as keepalive-time, min-threads and max-threads.
* `site` defines a site (data centre) ID for this node.
* `rack` defines a rack (server rack) ID for this node.
* `machine` defines a machine (host) is for this node.

[IMPORTANT]
Attributes site, rack and machine IDs are used by the Infinispan topology-aware consistent hash function,
which, when using dist mode, prevents dist mode replicas from being stored on the same host, rack or site.

[[property]]
==== <property>

This element is used to configure a transport or protocol property.

* `name` This attribute specifies the name of the protocol property.
The value is provided as text for the property element.

[[protocol]]
=== <protocol>

This element is used to configure a (non-transport) protocol layer in the JGroups stack.
Protocol layers are ordered within the stack.

* `type` specifies the name of the JGroups protocol implementation (e.g. `MPING`, `pbcast.GMS`),
with the default package prefix `org.jgroups.protocols` removed.

[[socket-protocol]]
=== <socket-protocol>

Extension of the generic protocol with addition of:

* `socket-binding` references a defined socket binding in the server profile.
It is used when JGroups needs to create general sockets internally for this protocol instance.
Used by `MPING`, `FD_SOCK`, etc.

[[socket-discovery-protocol]]
=== <socket-discovery-protocol>

Extension of the generic protocol with addition of:

* `socket-binding` references a list of socket bindings for a protocol used for initial discovery.
Used by `TCPPING`.

[[jdbc-protocol]]
=== <jdbc-protocol>

* `data-source` reference for JDBC protocols to be used instead of connection and JNDI lookup properties.
Used by `JDBC_PING`.

[[relay]]
=== <relay>

This element is used to configure the RELAY protocol for a JGroups stack.
RELAY is a protocol which provides cross-site replication between defined sites (data centres).
In the RELAY protocol, defined sites specify the names of remote sites (backup sites) to which their data should be backed up.
Channels are defined between sites to permit the RELAY protocol to transport the data from the current site to a backup site.

* `site` specifies the name of the current site.
Site names can be referenced elsewhere
(e.g. in the JGroups remote-site configuration elements, as well as backup configuration elements in the Infinispan subsystem)

[[remote-site]]
==== <remote-site>

This element is used to configure a remote site for the RELAY protocol with attributes:

* `name` specifies the name of the remote site to which this configuration applies.
* `stack` specifies a JGroups protocol stack to use for communication between this site and the remote site.
* `cluster` specifies the name of the JGroups channel to use for communication between this site and the remote site.

[[jgroups-use-cases]]
== Use Cases

In many cases, channels will be configured via XML as in the example above, so that the channels will be available upon server startup.
However, channels may also be added, removed or have their configurations modified.
The recommended way to do this is by making use of the WildFly management API command-line interface (CLI).
In this section, we present some key use cases for the managing the JGroups subsystem.

The key use cases covered are:

* adding a stack
* adding a protocol to an existing stack
* configuring a property of a protocol

[IMPORTANT]
The WildFly management API command-line interface (CLI) itself can be
used to provide extensive information on the attributes and commands
available in the JGroups subsystem interface used in these examples.

[[add-a-stack]]
=== Creating a new stack

To create a new stack it must be created using a `batch` to satisfy capability references checks.

[source,options="nowrap"]
----
batch
/subsystem=jgroups/stack=mystack:add
/subsystem=jgroups/stack=mystack/transport=TCP:add(socket-binding=jgroups-tcp)
/subsystem=jgroups/stack=mystack/protocol=RED:add
/subsystem=jgroups/stack=mystack/protocol=MPING:add(socket-binding=jgroups-mping)
/subsystem=jgroups/stack=mystack/protocol=MERGE3:add
/subsystem=jgroups/stack=mystack/protocol=FD_ALL3:add
/subsystem=jgroups/stack=mystack/protocol=VERIFY_SUSPECT2:add
/subsystem=jgroups/stack=mystack/protocol=pbcast.NAKACK2:add
/subsystem=jgroups/stack=mystack/protocol=UNICAST3:add
/subsystem=jgroups/stack=mystack/protocol=pbcast.STABLE:add
/subsystem=jgroups/stack=mystack/protocol=pbcast.GMS:add
/subsystem=jgroups/stack=mystack/protocol=UFC:add
/subsystem=jgroups/stack=mystack/protocol=MFC:add
/subsystem=jgroups/stack=mystack/protocol=FRAG4:add
run-batch
----

The default `ee` channel can be then reconfigured to use this stack with the following operation:

[source,options="nowrap"]
----
/subsystem=jgroups/channel=ee:write-attribute(name=stack, value=mystack)
----

[[add-a-protocol-to-a-stack]]
=== Adding a protocol to a stack

While the order of management resources in WildFly is typically not a concern,
ordering is essential when configuring JGroups protocols.
Protocols can be inserted into existing stacks at any point using `add-index` attribute of the `add` protocol.
Following is an example of adding the `RED` protocol which always operates at the top of the stack,
hence it is added at index of zero.

[source,options="nowrap"]
----
/subsystem=jgroups/stack=udp/protocol=RED:add(add-index=0)
----

Analogously, a protocol can be removed from the protocol referencing its name:

[source,options="nowrap"]
----
/subsystem=jgroups/stack=udp/protocol=RED:remove
----

[[add-a-property-to-a-protocol]]
=== Configuring a property of protocol

Most JGroups protocols support configuration options that are not exposed as management attributed within the WildFly subsystem.
These can be configured using properties.
Following is an example of fine-tuning the `RED` protocol's `min_treshold` to increase from default to `0.6`.

[source,options="nowrap"]
----
/subsystem=jgroups/stack=udp/protocol=RED:map-put(name=properties, key=min_threshold, value=0.6)
----
