| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| |
| QPID INTEROP TEST DEVELOPMENT OVERVIEW |
| ====================================== |
| |
| |
| Introduction |
| ============ |
| This document provides an overview for developers of new tests and shims |
| for qpid-interop-test. |
| |
| The purpose of qpid-interop-test is to test interoperability between |
| Qpid's AMQP clients. The tests use "Shims" to send messages through |
| any AMQP broker and to compare the received results with those sent. |
| |
| There are two layers in the qpid-interop-test architecture: |
| * Tests |
| * Shims |
| |
| |
| Tests |
| ===== |
| |
| The test is a high-level Python program which will coordinate the running |
| of individual test cases and provide the data for each test case. It will |
| also check for errors and failures. A test usually consists of a number |
| of test cases as described below under "Shims". |
| |
| |
| Shims |
| ===== |
| |
| For each test case, the Test will use a pair of Shims to send and receive |
| messages (or perform some other task). The Test will send and receive test |
| data to/from the shims using JSON. |
| |
| The shims are low-level AMQP clients which are designed to perform a |
| single test task - usually to send and receive messages using the data |
| supplied by the Test. Because this test suite has interoperability as one of |
| its primary goals, it is necessary for the test to be able to send |
| test data to several shims written using the API and language of the |
| target client. So, for example, the Test may organize a set of three |
| send and receive shims as follows to achieve interoperability testing: |
| |
| Test Case Sender Shim Receiver Shim |
| --------- ----------- ------------- |
| 1 Client_A_sender Client_A_receiver |
| 2 Client_A_sender Client_B_receiver |
| 3 Client_A_sender Client_C_receiver |
| 4 Client_B_sender Client_A_receiver |
| 5 Client_B_sender Client_B_receiver |
| 6 Client_B_sender Client_C_receiver |
| 7 Client_C_sender Client_A_receiver |
| 8 Client_C_sender Client_B_receiver |
| 9 Client_C_sender Client_C_receiver |
| |
| Each sender shim would be sent the same JSON test data, and each receiver |
| shim would return its test data to the Test for comparison and error |
| checking. The test would then determine which test cases passed or |
| failed. |
| |
| Although interoperability is the primary goal of Qpid Interop Test, it |
| is up to the Test to determine the test data, the order of the test |
| cases, which shims to use, and to have an understanding of what test |
| data will be received by the shim. |
| |
| |
| Relationship between Shims and Tests |
| ==================================== |
| |
| Each set of shims is written specifically for a test, and knows |
| how to interpret the data it is sent, and what to return (if needed). |
| |
| 1 +--------+ |
| +--<>| Sender | |
| +-----+ 1 * +------+ 1 * +------+ | +--------+ |
| | QIT |----<>| Test |----<>| Shim |----+ |
| +-----+ +------+ +------+ | 1 +----------+ |
| +--<>| Receiver | |
| +----------+ |
| |
| There may be quite a lot of duplication between shims. However, it |
| seems better to keep the shims as simple as possible than force |
| a more complex shim architecture in which shims perform a multitude |
| of tasks, but allow re-use. In addition, the patterns of re-use differ |
| across client languages, and it would be difficult to achieve any |
| consistency in re-use that works for all of them. |
| |
| The shims should be: |
| * As simple as possible (modeled off the simple sender/simple_receiver |
| examples) |
| * Perform only one type of test task |
| * Be easy to maintain and understand |
| |
| |
| Building and Installation |
| ------------------------- |
| |
| Cmake is the primary build and install mechanism. By running cmake on |
| the top level CMakeLists.txt file, and subsequently a make/make install |
| should build and install all components, regardless of language. |
| For some client shims (depending on language), the local cmake may call |
| external programs to perform compilation and installation of that shim. |
| |
| Interpreted languages (such as Python) don't compile, the source is |
| installed directly as a script. |
| |
| |
| Source Directory layout |
| ----------------------- |
| |
| qpid-interop-test |
| +-- CMakeLists.txt |
| +-- src |
| | +-- python |
| | +-- qpid_interop_test |
| | +-- <test_1>.py |
| | +-- <test_2>.py |
| | ... |
| +-- shims |
| +-- <client_1> |
| | +-- CMakeLists.txt |
| | +-- <test_1> |
| | | +-- Receiver.src |
| | | +-- Sender.src |
| | +-- <test_2> |
| | | +-- Receiver.src |
| | | +-- Sender.src |
| | ... |
| +-- <client_2> |
| | +-- CMakeLists.txt |
| | +-- <test_1> |
| | | +-- Receiver.src |
| | | +-- Sender.src |
| | +-- <test_2> |
| | | +-- Receiver.src |
| | | +-- Sender.src |
| ... |
| |
| |
| Install directory layout |
| ------------------------ |
| The tests (being Python) are installed into |
| ${CMAKE_INSTALL_PREFIX}/lib/python2.7/site-packages/qpid_interop_test/. |
| |
| The shim executable pairs (a Sender and Receiver) are installed into |
| ${CMAKE_INSTALL_PREFIX}/libexec/qpid_interop_test/shims/. |
| |
| The layout is as follows: |
| |
| <CMAKE_INSTALL_PREFIX> (default: /usr/local/) |
| +-- lib |
| | +-- python2.7 |
| | +-- site-packages |
| | +-- qpid_interop_test |
| | +-- <test_1.py> |
| | +-- <test_2.py> |
| | ... |
| +-- libexec |
| +-- qpid_interop_test |
| +-- shims |
| +-- <client_1> |
| | +-- <test_1> |
| | | +-- Receiver |
| | | +-- Sender |
| | +-- <test_2> |
| | | +-- Receiver |
| | | +-- Sender |
| | ... |
| +-- <client_2> |
| | +-- <test_1> |
| | | +-- Receiver |
| | | +-- Sender |
| | +-- <test_2> |
| | | +-- Receiver |
| | | +-- Sender |
| | ... |
| ... |