docs/amqp-1.0.txt

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

AMQP 1.0 support for the qpid::messaging API
--------------------------------------------

* Connections, Session and Links

The protocol used is selected at runtime via the 'protocol' connection
property. The recognised values are 'amqp1.0' and 'amqp0-10'. AMQP
0-10 is still the default so in order to use AMQP 1.0 you must
explicitly specify this option.

The SASL negotiation is optional in AMQP 1.0. If no SASL layer is
desired, the sasl_mechanisms connection option can be set to NONE.

AMQP 1.0 can be used over SSL, however the messaging client does not
at this stage use an AMQP negotiated security layer for that
prupose. Peers must expect SSL on the port being used (either
exclusively or by being able to detect an SSL header).

The container id that the client advertises when establishing the
connection can be set through the container_id property on the
connection. If not set a UUID will be used.

Transactional sessions are not yet supported[1].

The creation of senders or receivers results in the attaching of a
link to the peer. The details of the attach, in particular the source
and/or target, are controlled through the address string.

* Addresses

The name specified in the address supplied when creating a sender or
receiver is used to set the 'address' field of the target or source
respectively.

The special address name '<null>' can be used to indicate that the
address hould be left null.

If the subject is specified for a sender it is used the default
subject for messages sent without an explicit subject set.

If the subject is specified for a receiver it is interpreted as a
filter on the set of messages of interest. If it includes a wildcard
(i.e. a '*' or a '#') it is sent as a legacy-amqp-topic-binding, if
not it is sent as a legacy-amqp-direct-binding.

An address of '#' indicates that the sender or receiver being created
should be to or from a dynamically created node. I.e. the dynamic flag
will be set on the source or target in the attach to request that the
broker create a node and return the name of it. The getAddress()
method on Sender and Receiver can then be used to set the reply-to on
any message (or to establish other links to or from that node).

For dynamically created nodes, the dynamic-node-properties can be
specified as a nested 'properties' map within the node
options. Additionally the durable and type properties in the node map
itself are sent if specified.

For receivers, where the node is an exchange, the binding can be
controlled through the filters for the link. These can be specified
through the filter property in the link properties specified in the
address. The value of this should be a list of maps, with each map
specifying a filter through key-value pairs for name, descriptor (can
be specified as numeric or symbolic) and a value.

The subject in the address and the selector property within the link
properties of the address provide shorter ways of specifying the most
common use cases.

The source/target capabilities sent by the client can be controlled
through a nested list within the node options. Note that capabilities
are simply strings (symbols in 1.0 to be precise), not name value
pairs.

E.g. topic subscriptions can be shared between receivers by requesting
the 'shared' capability and ensuring the receivers all specify the
same link name.

If durable is set in the node properties, then a capability of
'durable' will be requested, meaning the node will not lose messages
if its volatile memory is lost.

If the type is set, then that will also be passed as a requested
capability e.g. 'queue' meaning the node supports queue-like
characteristics (stores messages until consumers claim them and
allocates messages between competing consumers), 'topic' meaning the
node supports classic pub-sub characteristics.

* Messages

The message-id, correlation-id, user-id, subject, reply-to and
content-type fields in the properties section of a 1.0 message can all
be set or retreived via accessors of the same name on the Message
instance. Likewise for the durable, priority and ttl fields in the
header section.

An AMQP 1.0 message has a delivery-count field (within the header
section) for which there is no direct accessor yet on the Message
class. However if the value is greater than 0, then the
Message::getRedelivered() method will return true. If
Message::setRedelivered() is called with a value of true, then the
delivery count will be set to 1, else it will be set to 0. The
delivery count can also be retrieved or set through the
'x-amqp-delivery-count' pseudo-property.

The application-properties section of a 1.0 message is made available
via the properties map of the Message class. Likewise on sending a
message the properties map is used to poplulate the
application-properties section.

There are other fields defined in the AMQP 1.0 message format that as
yet do not have direct accessors on the Message class. These are made
available on received messages via special keys into the properties
map, and can be controlled for outgoing messages by setting the
properties with the same keys.

The format for the keys in general is x-amqp-<field-name>.  The keys
currently in use are: x-amqp-first-acquirer and x-amqp-delivery-count
for the header section, and x-amqp-to, x-amqp-absolute-expiry-time,
x-amqp-creation-time, x-amqp-group-id, x-amqp-qroup-sequence and
x-amqp-reply-to-group-id for the properties section.

The delivery- and message- annotations sections can be made available
via nested maps with key x-amqp-delivery-annotations and
x-amqp-message-annotations respectively by specifying the
'nest_annotations' connection option. Otherwise they will simply be
included in the properties.

AMQP 1.0 support in qpidd
-------------------------

To enable 1.0 support in qpidd, the amqp module must be loaded. This
allows the broker to recognise the 1.0 protocol header alongside the
0-10 one. If installed, the module directory should be configured such
that this happens automatically. If not, use the --load-module option
and point to the amqp.so module.

The broker can accept connections with or without and underlying SASL
security layer as defined by the 1.0 specification. However if
authentication is turned on -- i.e. auth=yes -- then a SASL security
layer MUST be used.

The broker allows links in both directions to be attached to queues or
exchanges. The address in the source or target is resolved by checking
whether it matches the name of a queue or an exchange. If there exists
a queue and an exchange of the same name, the queue is used but a
warning is logged.

The incoming and outgoing links attached to the broker can be viewed
via the qpid management framework (aka QMF), e.g. with the qpid-config
tool:

    qpid-config list incoming

or

    qpid-config list outgoing

If the node is an exchange, then an outgoing link (i.e. messages to
travel out from broker) will cause a temporary, link-scoped queue to
be created on the broker and bound to the exchange. [See section on
'Topics' below]. The name of the queue will be of the form
<container-id>_<link-name>.

If the target address of an attaching sender link is null, and the
dynamic flag is not set, then the broker will route all messages
received over this sender link using the 'to' field from the
properties. If that matches the name of a queue, the message will be
enqueued to that queue. If it doesn't match a queue but matches an
exchange the message will be routed through that exchange. If it
doesn't match either a queue or an exchange then it will be
dropped. This behaviour is known as 'ANONYMOUS-RELAY', a capability
which the broker advertises in the open frames it sends over every
connection. To use it the sending process needs to have permission to
access the ANONYMOUS-RELAY 'pseudo-exchange' (note: there isn't
actually an exchange of that name, but this is how the capability is
represented in the ACL permissions model).

Outgoing links may have a filter set on their source. The filters
currently supported by the broker are 'legacy-amqp-direct-binding',
'legacy-amqp-topic-binding', 'legacy-amqp-headers-binding',
'selector-filter' and 'xquery-filter' as defined in the registered
extensions.

The 'selector-filter' is supported for all nodes.

The 'legacy-amqp-direct-binding' and 'legacy-amqp-topic-binding'
filters are supported when the node is a topic, direct or xml exchange
- where it is used as the binding key when binding the subscription
queue to the exchange - and when the node is a queue, where the
subscription will then skip over messages that do not match that
subject. In the case where 'legacy-amqp-topic-binding' was requested
for a direct- or xml- exchange it will be interpreted as a direct
binding (and this will be indicated via the filters set on the attach
sent by the broker).

The 'legacy-amqp-headers-binding' is supported where the node is a
headers exchange and implemented as a binding to the exchange.

The 'xquery-filter' is supported where the node is an xml exchange and
provides the xquery to use in the binding. This can be used in
conjunction with a 'legacy-amqp-direct-binding' in order to control
the binding ley used for the binding.

If the dynamic flag is set on the source or target, then the
dynamic-node-properties are inspected to determine the characteristics
of the node to be created. The properties that are recognised are the
same as accepted via the create qmf method; the 0-10 defined options
durable, auto-delete, alternate-exchange, exchange-type and then any
qpidd specific options (e.g. qpid.max-count). The broker supports all
standard values for the standard lifetime-policy property also.

The supported-dist-modes property as defined by the 1.0 specification
is used to determine whether a queue or exchange is desired (where the
create method uses the 'type' property). If 'move' is specified a
queue will be created, if 'copy' is specified an exchange will be
created. If this property is not set, then a queue is assumed.

Messages sent over AMQP 0-10 will be converted by the broker for
sending over AMQP 1.0 and vice versa.

The message-id, correlation-id, userid, content-type and
content-encoding all map fairly simply in both directions between the
properties section in 1.0 and the message-properties in an 0-10
header. Note however that in 0-10 a message-id must be a UUID, and in
translating to 0-10 from 1.0 this field will be skipped unless it is a
valid UUID.

Likewise the priority directly between the field in the header section
of a 1.0 message and the corresponding field in the
delivery-properties of an 0-10 message. The durable header in a 1.0
message is equivalent to the delivery-mode in the delivery-properties
of an 0-10 message, with a value of true in the former being
equivalent to a value of 2 in the latter and a value of false in the
former equivalent to 1 in the latter.

When converting from 0-10 to 1.0, the reply-to will be simply the
routing-key if the exchange is not set, or if it is the reply-to
address for 1.0 will be composed of the exchange and if specified the
routing key (separated by a forward slash).

When converting from 1.0 to 0-10, if the address contains a forward
slash it is assumed to be of the form exchange/routing key. If not it
is assumed to be a simple node name. If that name matches an existing
queue, then the resulting 0-10 reply to will have the exchange empty
and the routing key populated with the queue name. If not, but the
name matches and exchange, then the reply to will have the exchange
populated with the node name and the routing key left empty. If the
node refers to neither a known queue nor exchange then the resulting
reply to will be empty.

When converting from 0-10 to 1.0, if the 0-10 message has a non-empty
destination, then the subject field in the properties of the 1.0
message is set to the value of the routing-key from the
message-properties of the 0-10 message. In the reverse direction, the
subject field of the properties section of the 1.0 message is used to
populate the routing-key in the message-properties of the 0-10
message.

The destination of a 0-10 message is used to populate the 'to' field
of the properties section when converting to 1.0, but the reverse
translation is not done (as the destination for messages sent out by
the broker is controlled by the subscription in 0-10).

The application-properties section of a 1.0 message is converted to
the application-headers field in the message-properties of an 0-10
message and vice versa.

Map and list content is converted between 1.0 and 0-10
formats. Specifically, a 1.0 message with a list- or map- typed
AmqpValue section will be converetd into an 0-10 message with the
content=type set to amqp/list or amqp/map respectively and the content
encoded in the 0-10 typesystem. Likewise an 0-10 message with
content-type set to amqp/list or amqp/map will be converted into a 1.0
message with an AmqpValue section containing a list or map
respectively, encoded in the 1.0 typesystem.

The broker recognises particular capabilities for source and
targets. It will only include a given capability in the attach it
sends if that has been 'requested' in the attach sent by the peer to
trigger the establishment of the link. This gives the peer some means
of ensuring their expectations will be met.

The 'shared' capability allows subscriptions from an exchange to be
shared by multiple receivers. Where this is specified the subscription
queue created takes the name of the link (and does not include the
container id).

The 'durable' capability will be added if the queue or exchange
refered to by the source or target is durable. The 'queue' capability
will be added if the source or target references a queue. The 'topic'
capability will be added if the source or target references an
exchange. If the source or target references a queue or a direct
exchange the 'legacy-amqp-direct-binding' will be added. If it
references a queue or a topic exchange, 'legacy-amqp-topic-binding'
will be added.

* Topics: a mechanism for controlling subscription queues

As there is no standard or obvious mechanism through which to
configure subscription queues in AMQP 1.0, a new broker entity of type
'topic' has been added.

A topic references an existing exchange and additionally specifies the
queue options to use when creating the subscription queue for any
receiver link attached to that topic. There can be topics with
different names all referencing the same exchange where different
policies should be applied to queues.

Topics can be created and deleted using the qpid-config tool, e.g.

  qpid-config add topic my-topic --argument exchange=amq.topic\
      --argument qpid.max_count=500 --argument qpid.policy_type=self-destruct

If a receiver is established for address 'my-topic/my-key' over 1.0
now, it will result in a subscription queue being created with a limit
of 500 messages, that deletes itself (thus ending the subscription) if
that limit is exceeded and is bound to 'amq.topic' with the key
'my-key'.

* 'Policies': a mechanism for broker configured, on-demand creation of
  nodes

As the core AMQP 1.0 protocol deliberately doesn't cover the creation
of nodes, a new mechanism has been added that allows nodes to be
created on-demand requiring only broker configuration. A client can
establish a link to or from a node, and if the desired target/source
matches a preconfigured policy, then the node will be created on
demand in accprdance with that policy.

There are currently two types of policy: QueuePolicy and
TopicPolicy. These allow for the on-demand creation of queues or
'topics' (in the JMS sense of the word), which are currently backed by
pre-1.0 AMQP style exchanges.

Policies of either type can be created using the qpid-config tool,
e.g.

  qpid-config add QueuePolicy queue_ --argument qpid.max_count=500

will create a policy such that any attempt to establish a link to or
from a node that begins with queue_ will result on a queue with the
required name being created if it doesn't exist. In this example the
queue would have a max depth of 500 messages.

* Broker Initiated Links

The existing 'federation' mechanism is quite tightly bound to the AMQP
0-10 implementation at present. The AMQP 1.0 support includes a
slightly different mechanism that offers the ability to establish
links to and from nodes in remote containers, and thus setup the
automatic transfer of messages.

To begin with, a 'domain' entity must be created describing how to
connect to the given external process (i.e. the remote
container). This can again be done with qpid-config, e.g.

  qpid-config add domain another-broker --argument url=host.acme.com

Incoming or outgoing links can then be established, e.g.

  qpid-config add incoming link_a --argument domain=another-broker \
  --argument src=some_remote_queue --argument dest=my_local_queue

will cause messages on a node named 'some_remote_queue' on the process
reachable through the default AMQP port on host.acme.com, to be
transferred to a queue, on the qpidd instance these command were
issued to, named my_local_queue.

Note that at present there is no automatic re-establishment of these
broker initiated AMQP 1.0 based links, nor is there any loop
prevention mechanism for messages transmitted over them in the event
that the links form circular paths.

[1] https://issues.apache.org/jira/browse/QPID-4710