blob: e43d94dca42ace6805074c444100d07df53fac65 [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 2.0.18">
<link rel="icon" type="image/png" href="images/favicon.png">
<title>Messaging Concepts</title>
<link rel="stylesheet" href="css/asciidoctor.css">
<link rel="stylesheet" href="css/font-awesome.css">
</head>
<body class="book toc2 toc-left">
<div id="header">
<h1>Messaging Concepts</h1>
<div id="toc" class="toc2">
<div id="toctitle"><a href="index.html">User Manual for 2.33.0</a></div>
<ul class="sectlevel1">
<li><a href="#general-concepts">1. General Concepts</a></li>
<li><a href="#messaging-styles">2. Messaging styles</a>
<ul class="sectlevel2">
<li><a href="#point-to-point">2.1. Point-to-Point</a></li>
<li><a href="#publish-subscribe">2.2. Publish-Subscribe</a></li>
</ul>
</li>
<li><a href="#delivery-guarantees">3. Delivery guarantees</a></li>
<li><a href="#transactions">4. Transactions</a></li>
<li><a href="#durability">5. Durability</a></li>
<li><a href="#messaging-apis">6. Messaging APIs</a>
<ul class="sectlevel2">
<li><a href="#jms-jakarta-messaging">6.1. JMS &amp; Jakarta Messaging</a></li>
<li><a href="#system-specific-apis">6.2. System specific APIs</a></li>
</ul>
</li>
<li><a href="#high-availability">7. High Availability</a></li>
<li><a href="#clusters">8. Clusters</a></li>
<li><a href="#bridges-and-routing">9. Bridges and routing</a></li>
</ul>
</div>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>Apache ActiveMQ Artemis is an asynchronous messaging system, an example of <a href="https://en.wikipedia.org/wiki/Message-oriented_middleware">Message Oriented Middleware</a>, we&#8217;ll just call them messaging systems in the remainder of this book.</p>
</div>
<div class="paragraph">
<p>We&#8217;ll first present a brief overview of what kind of things messaging systems do, where they&#8217;re useful and the kind of concepts you&#8217;ll hear about in the messaging world.</p>
</div>
<div class="paragraph">
<p>If you&#8217;re already familiar with what a messaging system is and what it&#8217;s capable of, then you can skip this chapter.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="general-concepts"><a class="anchor" href="#general-concepts"></a><a class="link" href="#general-concepts">1. General Concepts</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Messaging systems allow you to loosely couple heterogeneous systems together, whilst typically providing reliability, transactions and many other features.</p>
</div>
<div class="paragraph">
<p>Unlike systems based on a <a href="https://en.wikipedia.org/wiki/Remote_procedure_call">Remote Procedure Call</a> (RPC) pattern, messaging systems primarily use an asynchronous message passing pattern with no tight relationship between requests and responses.
Most messaging systems also support a request-response mode but this is not a primary feature of messaging systems.</p>
</div>
<div class="paragraph">
<p>Designing systems to be asynchronous from end-to-end allows you to really take advantage of your hardware resources, minimizing the amount of threads blocking on IO operations, and to use your network bandwidth to its full capacity.
With an RPC approach you have to wait for a response for each request you make so are limited by the network round trip time, or <em>latency</em> of your network.
With an asynchronous system you can pipeline flows of messages in different directions, so are limited by the network <em>bandwidth</em> not the latency.
This typically allows you to create much higher performance applications.</p>
</div>
<div class="paragraph">
<p>Messaging systems decouple the senders of messages from the consumers of messages.
The senders and consumers of messages are completely independent and know nothing of each other.
This allows you to create flexible, loosely coupled systems.</p>
</div>
<div class="paragraph">
<p>Often, large enterprises use a messaging system to implement a message bus which loosely couples heterogeneous systems together.
Message buses often form the core of an <a href="https://en.wikipedia.org/wiki/Enterprise_service_bus">Enterprise Service Bus</a> (ESB).
Using a message bus to de-couple disparate systems can allow the system to grow and adapt more easily.
It also allows more flexibility to add new systems or retire old ones since they don&#8217;t have brittle dependencies on each other.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="messaging-styles"><a class="anchor" href="#messaging-styles"></a><a class="link" href="#messaging-styles">2. Messaging styles</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Messaging systems normally support two main styles of asynchronous messaging: <a href="https://en.wikipedia.org/wiki/Message_queue">message queue</a> messaging (also known as <em>point-to-point messaging</em>) and <a href="https://en.wikipedia.org/wiki/Publish_subscribe">publish subscribe</a> messaging.
We&#8217;ll summarise them briefly here:</p>
</div>
<div class="sect2">
<h3 id="point-to-point"><a class="anchor" href="#point-to-point"></a><a class="link" href="#point-to-point">2.1. Point-to-Point</a></h3>
<div class="paragraph">
<p>With this type of messaging you send a message to a queue.
The message is then typically persisted to provide a guarantee of delivery, then some time later the messaging system delivers the message to a consumer.
The consumer then processes the message and when it is done, it acknowledges the message.
Once the message is acknowledged it disappears from the queue and is not available to be delivered again.
If the system crashes before the messaging server receives an acknowledgement from the consumer, then on recovery, the message will be available to be delivered to a consumer again.</p>
</div>
<div class="paragraph">
<p>With point-to-point messaging, there can be many consumers on the queue but a particular message will only ever be consumed by a maximum of one of them.
Senders (also known as <em>producers</em>) to the queue are completely decoupled from receivers (also known as <em>consumers</em>) of the queue - they do not know of each other&#8217;s existence.</p>
</div>
<div class="paragraph">
<p>A classic example of point to point messaging would be an order queue in a company&#8217;s book ordering system.
Each order is represented as a message which is sent to the order queue.
Let&#8217;s imagine there are many front end ordering systems which send orders to the order queue.
When a message arrives on the queue it is persisted - this ensures that if the server crashes the order is not lost.
Let&#8217;s also imagine there are many consumers on the order queue - each representing an instance of an order processing component - these can be on different physical machines but consuming from the same queue.
The messaging system delivers each message to one and only one of the ordering processing components.
Different messages can be processed by different order processors, but a single order is only processed by one order processor - this ensures orders aren&#8217;t processed twice.</p>
</div>
<div class="paragraph">
<p>As an order processor receives a message, it fulfills the order, sends order information to the warehouse system and then updates the order database with the order details.
Once it&#8217;s done that it acknowledges the message to tell the server that the order has been processed and can be forgotten about.
Often the send to the warehouse system, update in database and acknowledgement will be completed in a single transaction to ensure <a href="https://en.wikipedia.org/wiki/ACID">ACID</a> properties.</p>
</div>
</div>
<div class="sect2">
<h3 id="publish-subscribe"><a class="anchor" href="#publish-subscribe"></a><a class="link" href="#publish-subscribe">2.2. Publish-Subscribe</a></h3>
<div class="paragraph">
<p>With publish-subscribe messaging many senders can send messages to an entity on the server, often called a <em>topic</em> (e.g. in the JMS world).</p>
</div>
<div class="paragraph">
<p>There can be many <em>subscriptions</em> on a topic, a subscription is just another word for a consumer of a topic.
Each subscription receives a <em>copy</em> of <em>each</em> message sent to the topic.
This differs from the message queue pattern where each message is only consumed by a single consumer.</p>
</div>
<div class="paragraph">
<p>Subscriptions can optionally be <em>durable</em> which means they retain a copy of each message sent to the topic until the subscriber consumes them - even if the server crashes or is restarted in between.
Non-durable subscriptions only last a maximum of the lifetime of the connection that created them.</p>
</div>
<div class="paragraph">
<p>An example of publish-subscribe messaging would be a news feed.
As news articles are created by different editors around the world they are sent to a news feed topic.
There are many subscribers around the world who are interested in receiving news items - each one creates a subscription and the messaging system ensures that a copy of each news message is delivered to each subscription.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="delivery-guarantees"><a class="anchor" href="#delivery-guarantees"></a><a class="link" href="#delivery-guarantees">3. Delivery guarantees</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>A key feature of most messaging systems is <em>reliable messaging</em>.
With reliable messaging the server gives a guarantee that the message will be delivered once and only once to each consumer of a queue or each durable subscription of a topic, even in the event of system failure.
This is crucial for many businesses; e.g. you don&#8217;t want your orders fulfilled more than once or any of your orders to be lost.</p>
</div>
<div class="paragraph">
<p>In other cases you may not care about a once and only once delivery guarantee and are happy to cope with duplicate deliveries or lost messages - an example of this might be transient stock price updates - which are quickly superseded by the next update on the same stock.
The messaging system allows you to configure which delivery guarantees you require.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="transactions"><a class="anchor" href="#transactions"></a><a class="link" href="#transactions">4. Transactions</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Messaging systems typically support the sending and acknowledgement of multiple messages in a single local transaction.
Apache ActiveMQ Artemis also supports the sending and acknowledgement of message as part of a large global transaction - using the Java mapping of XA: JTA.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="durability"><a class="anchor" href="#durability"></a><a class="link" href="#durability">5. Durability</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Messages are either durable or non durable.
Durable messages will be persisted in permanent storage and will survive server failure or restart.
Non durable messages will not survive server failure or restart.
Examples of durable messages might be orders or trades, where they cannot be lost.
An example of a non durable message might be a stock price update which is transitory and doesn&#8217;t need to survive a restart.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="messaging-apis"><a class="anchor" href="#messaging-apis"></a><a class="link" href="#messaging-apis">6. Messaging APIs</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>How do client applications interact with messaging systems in order to send and consume messages?</p>
</div>
<div class="paragraph">
<p>Several messaging systems provide their own proprietary APIs with which the client communicates with the messaging system.</p>
</div>
<div class="paragraph">
<p>There are also some standard ways of operating with messaging systems and some emerging standards in this space. Let&#8217;s take a brief look at these.</p>
</div>
<div class="sect2">
<h3 id="jms-jakarta-messaging"><a class="anchor" href="#jms-jakarta-messaging"></a><a class="link" href="#jms-jakarta-messaging">6.1. JMS &amp; Jakarta Messaging</a></h3>
<div class="paragraph">
<p><a href="https://en.wikipedia.org/wiki/Java_Message_Service">JMS</a> was historically part of Oracle&#8217;s Java EE specification.
However, in 2017 control was transferred to the Eclipse Foundation and it is now known as <a href="https://jakarta.ee/specifications/messaging/">Jakarta Messaging</a> which is part of Jakarta EE.</p>
</div>
<div class="paragraph">
<p>It is a Java API that encapsulates both message queue and publish-subscribe messaging patterns.
It is a lowest common denominator specification - i.e. it was created to encapsulate common functionality of the already existing messaging systems that were available at the time of its creation.</p>
</div>
<div class="paragraph">
<p>It is a very popular API and is implemented by most messaging systems.
It is only available to clients running Java.</p>
</div>
<div class="paragraph">
<p>It does not define a standard wire format - it only defines a programmatic API so clients and servers from different vendors cannot directly interoperate since each will use the vendor&#8217;s own internal wire protocol.</p>
</div>
<div class="paragraph">
<p>Apache ActiveMQ Artemis provides client implementations which are a fully compliant with <a href="using-jms.html#using-jms-or-jakarta-messaging">JMS 1.1 &amp; 2.0 as well as Jakarta Messaging 2.0 &amp; 3.0</a>.</p>
</div>
</div>
<div class="sect2">
<h3 id="system-specific-apis"><a class="anchor" href="#system-specific-apis"></a><a class="link" href="#system-specific-apis">6.2. System specific APIs</a></h3>
<div class="paragraph">
<p>Many systems provide their own programmatic API for which to interact with the messaging system.
The advantage of this it allows the full set of system functionality to be exposed to the client application.
API&#8217;s like JMS are not normally rich enough to expose all the extra features that most messaging systems provide.</p>
</div>
<div class="paragraph">
<p>Apache ActiveMQ Artemis provides its own core client API for clients to use if they wish to have access to functionality over and above that accessible via the JMS API.</p>
</div>
<div class="paragraph">
<p>Please see <a href="core.html#core-api">Core</a> for using the Core API with Apache ActiveMQ Artemis.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="high-availability"><a class="anchor" href="#high-availability"></a><a class="link" href="#high-availability">7. High Availability</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>High Availability (HA) means that the system should remain operational after failure of one or more of the servers.
The degree of support for HA varies between various messaging systems.</p>
</div>
<div class="paragraph">
<p>Apache ActiveMQ Artemis provides automatic failover where your sessions are automatically reconnected to a backup server on event of a server failure.</p>
</div>
<div class="paragraph">
<p>For more information on HA, please see <a href="ha.html#high-availability-and-failover">High Availability and Failover</a>.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="clusters"><a class="anchor" href="#clusters"></a><a class="link" href="#clusters">8. Clusters</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Many messaging systems allow you to create groups of messaging servers called <em>clusters</em>.
Clusters allow the load of sending and consuming messages to be spread over many servers.
This allows your system to scale horizontally by adding new servers to the cluster.</p>
</div>
<div class="paragraph">
<p>Degrees of support for clusters varies between messaging systems, with some systems having fairly basic clusters with the cluster members being hardly aware of each other.</p>
</div>
<div class="paragraph">
<p>Apache ActiveMQ Artemis provides very configurable state-of-the-art clustering model where messages can be intelligently load balanced between the servers in the cluster, according to the number of consumers on each node, and whether they are ready for messages.</p>
</div>
<div class="paragraph">
<p>Apache ActiveMQ Artemis also has the ability to automatically redistribute messages between nodes of a cluster to prevent starvation on any particular node.</p>
</div>
<div class="paragraph">
<p>For full details on clustering, please see <a href="clusters.html#clusters">Clusters</a>.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="bridges-and-routing"><a class="anchor" href="#bridges-and-routing"></a><a class="link" href="#bridges-and-routing">9. Bridges and routing</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Some messaging systems allow isolated clusters or single nodes to be bridged together, typically over unreliable connections like a wide area network (WAN), or the internet.</p>
</div>
<div class="paragraph">
<p>A bridge normally consumes from a queue on one server and forwards messages to another queue on a different server.
Bridges cope with unreliable connections, automatically reconnecting when the connections becomes available again.</p>
</div>
<div class="paragraph">
<p>Apache ActiveMQ Artemis bridges can be configured with filter expressions to only forward certain messages, and transformation can also be hooked in.</p>
</div>
<div class="paragraph">
<p>Apache ActiveMQ Artemis also allows routing between queues to be configured in server side configuration.
This allows complex routing networks to be set up forwarding or copying messages from one destination to another, forming a global network of interconnected brokers.</p>
</div>
<div class="paragraph">
<p>For more information please see <a href="core-bridges.html#core-bridges">Core Bridges</a> and <a href="diverts.html#diverting-and-splitting-message-flows">Diverting and Splitting Message Flows</a>.</p>
</div>
</div>
</div>
</div>
</body>
</html>