This is the cross everything integration test suite for Apache Thrift.
The test can be executed by:
make cross
This starts the test.py script which does the real cross test with different transports, protocols and languages.
Note that this skips any language that is not built locally. It also skips tests that are known to be failing. If you need more control over which tests to run, read following section.
Alternatively, you can invoke test.py directly. You need to runmake precross
once before executing it for the first time.
For example, if you changed something in nodejs
library and need to verify the patch, you can skip everything except nodejs
itself and some reference implementation (currently cpp
and java
are recommended) like this:
./configure --without-c_glib -without-csharp --without-erlang --without-lua ... make precross -j8 test/test.py --server cpp,java --client nodejs test/test.py --server nodejs --client cpp,java
The cross test cases are defined in tests.json. The root element is collection of test target definitions. Each test target definition looks like this:
{ "name": "somelib", "client": { "command": ["somelib_client_executable"], "workdir": "somelib/bin", "protocols": ["binary"], "transports": ["buffered"], "sockets": ["ip"], }, "server": { "command": ["somelib_server_executable"], "workdir": "somelib/bin", "protocols": ["binary"], "transports": ["buffered"], "sockets": ["ip", "ip-ssl"], } }
Either client or server definition or both should be present.
Parameters that are common to both client
and server
can be put to target definition root:
{ "name": "somelib", "workdir": "somelib/bin", "protocols": ["binary"], "transports": ["buffered"], "sockets": ["ip"], "client": { "command": ["somelib_client_executable"] }, "server": { "command": ["somelib_server_executable"], "sockets": ["ip-ssl"] } }
For the complete list of supported keys and their effect, see source code comment at the opt of crossrunner/collect.py.
Since many cross tests currently fail (mainly due to partial incompatibility around exception handling), the test script specifically report for “not known before” failures.
For this purpose, test cases known to (occasionally) fail are listed in known_failures_<platform>.json
where <platform>
matches with python platform.system()
string.
Currently, only Linux version is included.
FYI, the file is initially generated by
test/test.py --update-expected-failures=overwrite
after a full test run, then repeatedly
test/test.py --skip-known-failures test/test.py --update-expected-failures=merge
to update the known failures, run
make fail
Unit tests for languages are usually located under lib//test/ cross language tests according to ThriftTest.thrift shall be provided for every language including executables with the following command line interface:
Server command line interface:
$ ./cpp/TestServer -h Allowed options: -h [ --help ] produce help message --port arg (=9090) Port number to listen --domain-socket arg Unix Domain Socket (e.g. /tmp/ThriftTest.thrift) --named-pipe arg Windows Named Pipe (e.g. MyThriftPipe) --server-type arg (=simple) type of server, "simple", "thread-pool", "threaded", or "nonblocking" --transport arg (=buffered) transport: buffered, framed, http, anonpipe --protocol arg (=binary) protocol: binary, compact, json --ssl Encrypted Transport using SSL --processor-events processor-events -n [ --workers ] arg (=4) Number of thread pools workers. Only valid for thread-pool server type
Client command line interface:
$ ./cpp/TestClient -h Allowed options: -h [ --help ] produce help message --host arg (=localhost) Host to connect --port arg (=9090) Port number to connect --domain-socket arg Domain Socket (e.g. /tmp/ThriftTest.thrift), instead of host and port --named-pipe arg Windows Named Pipe (e.g. MyThriftPipe) --anon-pipes hRead hWrite Windows Anonymous Pipes pair (handles) --transport arg (=buffered) Transport: buffered, framed, http, evhttp --protocol arg (=binary) Protocol: binary, compact, json --ssl Encrypted Transport using SSL -n [ --testloops ] arg (=1) Number of Tests -t [ --threads ] arg (=1) Number of Test threads
If you have executed the make check or make cross then you will be able to browse gen-html/ThriftTest.html with the test documentation.
The return code (exit code) shall be 0 on success, or an integer in the range 1 - 255 on errors. In order to signal failed tests, the return code shall be composed from these bits to indicate failing tests:
#define TEST_BASETYPES 1 // 0000 0001 #define TEST_STRUCTS 2 // 0000 0010 #define TEST_CONTAINERS 4 // 0000 0100 #define TEST_EXCEPTIONS 8 // 0000 1000 #define TEST_NOTUSED 240 // 1111 0000 (reserved bits)
Tests that have not been executed at all count as errors.
Example:
During tests, the test client notices that some of the Struct tests fail. Furthermore, due to some other problem none of the Exception tests is executed. Therefore, the test client returns the code 10 = 2 | 8
, indicating the failure of both test 2 (TEST_STRUCTS) and test 8 (TEST_EXCEPTIONS).
Test Keys and Certificates are provided in multiple formats under the following directory test/keys