| .Apache License |
| ____ |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| ____ |
| |
| = Qpid Interoperability Test Users Guide |
| |
| == 1. Introduction |
| qpid-interop-test is an AMQP client interoperability test suite. It tests |
| various aspects of the AMQP protocol and/or test client features against |
| each other to ensure that they can interoperate. |
| |
| The test suite consists of tests and shims. Each test has a set of test-cases |
| which make up the test. Each test case will pass or fail a specific feature |
| or piece of functionality under test. |
| |
| Each test has a set of shims, which are small and specific clients which |
| send and receive messages, and is written using one of the client libraries |
| under test. For example, the amqp_types test has shims for the following |
| clients: |
| * AmqpNetLite |
| * ProtonCpp |
| * ProtonPython2 |
| * ProtonPython3 |
| * RheaJS |
| |
| To obtain both self- and interoperability testing, each test program will |
| run each shim against every other shim in the role of both sender and |
| receiver. For the amqp-type-test example above, this will result in the |
| following combinations of shims being used: |
| [width="30%", cols="10%,45%,45%"] |
| |=== |
| | |*Sender shim* |*Receiver shim* |
| |
| |1 |AmqpNetLite | AmqpNetLite |
| |2 |AmqpNetLite | ProtonCpp |
| |3 |AmqpNetLite | ProtonPython2 |
| |4 |AmqpNetLite | ProtonPython3 |
| |5 |AmqpNetLite | RheaJS |
| |6 |ProtonCpp | AmqpNetLite |
| |7 |ProtonCpp | ProtonCpp |
| |8 |ProtonCpp | ProtonPython2 |
| |9 |ProtonCpp | ProtonPython3 |
| |10 |ProtonCpp | RheaJS |
| |11 |ProtonPython2 | AmqpNetLite |
| |12 |ProtonPython2 | ProtonCpp |
| |13 |ProtonPython2 | ProtonPython2 |
| |14 |ProtonPython2 | ProtonPython3 |
| |15 |ProtonPython2 | RheaJS |
| |16 |ProtonPython3 | AmqpNetLite |
| |17 |ProtonPython3 | ProtonCpp |
| |18 |ProtonPython3 | ProtonPython2 |
| |19 |ProtonPython3 | ProtonPython3 |
| |20 |ProtonPython3 | RheaJS |
| |21 |RheaJS | AmqpNetLite |
| |22 |RheaJS | ProtonCpp |
| |23 |RheaJS | ProtonPython2 |
| |24 |RheaJS | ProtonPython3 |
| |25 |RheaJS | RheaJS |
| |=== |
| |
| so that for each test case, 25 individual tests are run. The test program |
| will by default run all the available shims against each other in this way, |
| but it is possible to control which shims are used using the --include-shim |
| or --exclude-shim arguments (see below). |
| |
| == 2. Obtaining |
| qpid-interop-test is an Apache Qpid project. |
| |
| Web page: https://qpid.apache.org/components/interop-test/index.html |
| |
| Download soruce: https://qpid.apache.org/download.html |
| |
| Git: https://github.com/apache/qpid-interop-test.git |
| |
| == 3. Building |
| |
| === a. Install dependencies: |
| * Build tools: git, gcc-c++, cmake, maven, json-cpp |
| * Qpid Proton: qpid-proton-cpp-devel, python2-qpid-proton, python3-qpid-proton |
| |
| === b. Decide on local vs system install |
| ==== Local install: |
| Installs all of the Proton and QIT bits in a local directory. This is |
| useful for limited testing where you don't want to have these files |
| in your system directories. Also, if you don't have root privileges, then |
| this is the only way to install. The drawback is that you may need to |
| adjust some environment settings (`PATH`, `PYTHONPATH`, `LD_LIBRARY_PATH`) so |
| that the test will run. |
| |
| ==== System install: |
| Installs the files into traditional system locations. This type of install |
| requires root privileges. As the files are located in expected locations, no |
| environment settings need be made. |
| |
| === c. Build qpid-interop-test |
| *System install:* |
| ---- |
| $ cd qpid-interop-test |
| $ mkdir build |
| $ cd build |
| $ cmake .. |
| $ make |
| $ sudo make install |
| ---- |
| |
| *Local install:* |
| ---- |
| $ cd qpid-interop-test |
| $ mkdir build |
| $ cd build |
| $ cmake -DCMAKE_INSTALL_PREFIX=<path/to/local/install/dir> .. |
| $ make install |
| ---- |
| |
| == 4. Running |
| |
| The tests by default assume a broker is available and running. The assumed default |
| is at `localhost:5672`. For other broker location(s), use the `--sender` and `--receiver` |
| arguments to specify where the clients should interact, see below. |
| |
| The tests do not start or stop brokers. |
| |
| There are several tests in the test suite: |
| |
| * *amqp_types_test.py* - Tests all of the AMQP primitive types. This primarily tests |
| the encoding and decoding of AMQP types. |
| * *amqp_large_content_test.py* - Tests large messages of various types. Messages sizes |
| are 1MB, 10MB, 100MB. Compound types (lists, maps, etc) send elements of various |
| sizes so that the total payload is the target size. |
| * *jms_messages_test.py* - Tests JMS message types (as implemented by Qpid-jms over AMQP) |
| from all the Qpid clients (including non-jms clients) |
| * *jms_hdrs_props_test.py* - Tests various combinations of JMS headers and properties |
| are correctly sent and received by the various clients. |
| |
| Each test is executed directly. |
| |
| === Command-line arguments |
| .Common to all tests |
| [cols="20%,80%"] |
| |=== |
| |`--help` |Print help. This is useful for seeing the available argument options and |
| defaults for some arguments. |
| |`--sender` |Node to which test suite will send messages. |
| Format: `ip-address:port` |
| Default: `localhost:5672` |
| |`--receiver` |Node from which test suite will receive messages. |
| Format: `ip-address:port` |
| Default: `localhost:5672` |
| |`--no-skip` |Do not skip tests that are excluded by default for reasons of a known bug. |
| Warning: some tests may lock up of freeze rather than fail. |
| |`--broker-type` |Specify the broker manually, which eliminates the test connection made |
| to the broker to determine its identity through connection properties. If |
| "None" is specified, then Artemis broker will be assumed, but this will |
| change in the future when Artemis fixes the connection properties issue. |
| Format: For our current brokers: one of: "ActiveMQ", "qpid-cpp", |
| "qpid-dispatch-router". Artemis does not currently pass its name |
| in connection properties, and is equivalent to "None". |
| |`--timeout' |Timeout for the test in seconds (default: 10 seconds, may vary with test). If the |
| test does not complete within this time, it will be terminated and marked as a |
| failure. |
| |`--include-shim` |Name of shim to include. Cannot be used together with `--exclude-shim`. May |
| be used multiple times to include more than one shim. |
| |`--exclude-shim` |Name of shim to exclude. Cannot be used together with `--include-shim`. May |
| be used multiple times to exclude more that one shim. |
| |`--xunit-log` |Enable the generation of xUnit log files at the conclusion of the tests. | |
| |`--xunit-log-dir` |Path where xUnit log files are written. |
| |`--description` |Description of test used in xUnit log file. |
| |`--broker-topology` | Description of broker and broker topology (where needed) used in xUnit log file. |
| |=== |
| |
| .amqp-types-test |
| [cols="20%,80%"] |
| |=== |
| |`--include-type` |Name of AMQP type to include. Cannot be used together with `--exclude-type`. |
| May be used multiple times to include more than one type. |
| |`--exclude-type` |Name of AMQP type to exclude. Cannot be used together with `--include-type`. |
| May be used multiple times to exclude more than one type. |
| |=== |
| |
| .amqp-complex-types-test |
| [cols="20%,80%"] |
| |=== |
| |`--include-type` |Name of AMQP complex type to include. Cannot be used together with `--exclude-type`. |
| May be used multiple times to include more than one type. |
| |`--exclude-type` |Name of AMQP complex type to exclude. Cannot be used together with `--include-type`. |
| May be used multiple times to exclude more than one type. |
| |`--include-subtype` |Name of AMQP subtype to include. This is the AMQP type used within a primary complex |
| type (for example, the types used within an AMQP list). Cannot be used together with |
| `--exclude-subtype`. May be used multiple times to include more than one subtype. |
| NOTE: There is a special `*` wildcard that can be used for this, and will allow |
| all valid types to be used together in the same test. |
| |`--exclude-subtype` |Name of AMQP subtype to exclude. Cannot be used together with |
| `--include-subtype`. May be used multiple times to exclude more than one subtype. |
| |=== |
| |
| .amqp-large-content-test |
| |=== |
| |No other parameters. There is currently no way to select/limit the message size, but there |
| an issue open to address this limitation. |
| |=== |
| |
| .jms-messages-test |
| [cols="20%,80%"] |
| |=== |
| |`--include-type` |Name of JMS message type to include. Cannot be used together with |
| `--exclude-type`. May be used multiple times to include more than one type. |
| |`--exclude-type` |Name of JMS message type to exclude. Cannot be used together with |
| `--include-type`. May be used multiple times to exclude more than one type. |
| |=== |
| |
| .jms-hdrs-props-test |
| [cols="20%,80%"] |
| |=== |
| |`--include-type` |Name of Java property type to include. Cannot be used together with |
| `--exclude-type`. May be used multiple times to include more than one type. |
| |`--exclude-type` |Name of Java property type to exclude. Cannot be used together with |
| `--include-type`. May be used multiple times to exclude more than one type. |
| |=== |
| There is currently no way to control/limit the JMS header types in this test, but there is |
| an issue open to address this limitation. |
| |
| ==== Examples: |
| To limit amqp_types_test to boolean type only: |
| `$ amqp_types_test --include-type boolean` |
| |
| To limit amqp_types_test to Qpid-cpp and Proton-python shims only: |
| `$ amqp_types_test --include-shim ProtonCpp --include-shim ProtonPython` |
| |
| To test against a pair of Dispatch Routers and a broker running on the local machine |
| as follows (first set up the brokers and routers): |
| |
| ---- |
| +----------+ 9001 +----------+ 5672 +--------+ |
| | sender |--------->| dispatch |-------->| | |
| | shim | | router 1 | | | |
| +----------+ +----------+ | | |
| | broker | |
| +----------+ 9002 +----------+ 5672 | | |
| | receiver |<---------| dispatch |<--------| | |
| | shim | | router 2 | | | |
| +----------+ +----------+ +--------+ |
| ---- |
| |
| `$ amqp_types_test --sender localhost:9001 --receiver localhost:9002` |
| |
| == 5. Output |
| All the tests will list each test as it runs and whether it passes or fails. |
| If a test fails, then the details of the failure will be printed at the end of |
| the test. |
| |
| ---- |
| ====================================================================== |
| FAIL: test.B.MESSAGE.JMS_TYPE_HEADER:string+JMS_REPLYTO_HEADER:topic.QpidJms->ProtonCpp (__main__.PartB_JmsHeaderCombination_TestCase) |
| ---------------------------------------------------------------------- |
| Traceback (most recent call last): |
| File "./install/lib/python2.7/site-packages/qpid_interop_test/jms_hdrs_props_test.py", line 422, in inner_test_method |
| receive_shim) |
| File "./install/lib/python2.7/site-packages/qpid_interop_test/jms_hdrs_props_test.py", line 313, in run_test |
| self.fail(str(receive_obj)) |
| AssertionError: JmsReceiver error: Unexpected JMS message header: JMS_PRIORITY: Expected default priority (4), found priority 0 |
| ---- |
| |
| Currently, the tests do not produce text log files. However, by using the `--xunit-log` command-line option, xunit log files |
| will be generated at the conclusion of the test in the `xunit_logs` directory. The directory may be changed by |
| using the `--xunit-log-dir` command-line option. Log file management is the responsibility of the user, and if |
| xunit logging is used regularly, these files will accumulate, possibly exhausting the available disk space. NOTE |
| that if a test is interrupted (by using ctl+c, for example), then the log files will not be generated.) |
| |