As the entrypoint of an Apache Traffic Control CDN, Traffic Router is the most exposed, most critical component, and it must be able to route traffic quickly and at a very high rate. Although Traffic Router has met this requirement in the past, if Traffic Router undergoes no performance tests as it evolves, there is no guarantee that its performance will not decline with time.
Important times to run performance tests:
A load test for Delivery Services exists in the project at /test/router, but
The Traffic Router Ultimate Test Harness will include an end-to-end performance test suite verify that the features of Traffic Router meet expected performance thresholds, as well as additional end-to-end tests of other Traffic Router features.
The TR Ultimate Test Harness may extend /test/router where possible, but it should not limit itself for that secondary goal.
No Traffic Portal impact is anticipated.
No Traffic Ops impact is anticipated.
No Traffic Ops REST API impact is anticipated.
Clients importing the github.com/apache/trafficcontrol/lib/go-tc package will optionally be able to import a constant for X-MM-Client-IP, a request header Traffic Router to specify to Traffic Router the IP address to use to geolocate that client:
https://github.com/apache/trafficcontrol/blob/1ed2964d16618aeebef142b01a538336a44d07dd/traffic_router/core/src/main/java/org/apache/traffic_control/traffic_router/core/request/HTTPRequest.java#L29
Additionally, a struct used to unmarshall a Coverage Zone File could be placed in lib/go-tc.
No Data Model impact is anticipated.
No Cache Config impact is anticipated.
No Traffic Monitor impact is anticipated.
The addition of the TR Ultimate Test Harness themselves will not change Traffic Router functionality in any way. For visibility, however, the TR Ultimate Test Harness should reside in a directory within the traffic_router directory. This will be the first time since 545929f7cc that Golang sources will exist in the traffic_router directory, so any assumption that all sources within the traffic_router directory directly impact Traffic Router's ability to compile should be abandoned.
The TR Ultimate Test Harness should not be included in the Traffic Router RPM, as it is meant to be run on a host separate from Traffic Routers.
No Traffic Stats impact is anticipated.
No Traffic Vault impact is anticipated.
Instructions for using the Traffic Router Ultimate Test Harness should be added to the documentation. This should include:
The Router Ultimate Test Harness should include a load test for HTTP-routed Delivery Services and for DNS-routed Delivery Services
| Option | Description | Delivery Service Type | Default |
|---|---|---|---|
| IPv4 TR addresses only | Test IPv4 Traffic Router addresses only | HTTP, DNS | False |
| IPv6 TR addresses only | Test IPv6 Traffic Router addresses only | HTTP, DNS | False |
| CDN name | The name of a CDN to search for Delivery Services | HTTP, DNS | all |
| Delivery Service name | The name (XMLID) of a Delivery Service to use for tests | HTTP, DNS | None |
| Traffic Router name | Instead of iterating through Traffic Routers, test only a specific Traffic Router, identified by hostname. | HTTP, DNS | all |
| Client IP address | If provided, Traffic Router will use the value of the X-MM-Client-IP request header as the IP address that Traffic Router's geolocation considers. This option should you specify such an IP address. | HTTP, DNS | None |
| Use coverage zone map | Whether to use an IP address from the Traffic Router's Coverage Zone File | HTTP, DNS | False |
| Coverage zone location | The coverage zone location to use (implies Use coverage zone map) | HTTP, DNS | None |
| Requests per second threshold | The minimum number of requests per second a Traffic Router must successfully respond to | HTTP, DNS | 8000 for HTTP, 7200 for DNS |
| Benchmark time | The duration of each load test, in seconds | HTTP, DNS | 300 |
| Thread count | The number of threads to spawn for each test | HTTP, DNS | 12 |
| Path count | The number of paths to generate for use in requests to Delivery Services | HTTP | 10000 |
| Maximum path length | The maximum string length for each generated path | HTTP | 100 |
| Use location header | Whether the HTTP HTTP Delivery service should redirect the user or server the routing information as a JSON response. | HTTP | True |
These options will be structured in a config file:
{ "all": { "cdn_name": "Kabletown" }, "http": { "ipv4_only": true, /* more options */ "path_count": 5000 }, "dns": { "delivery_service_name": "static" } }
Options that should apply to a specific type of test should go under a key named that test type, while options applying to all tests can go under the "all" key.
Additionally, the TR Ultimate Test Harness should provide the ability to verify that a DNS-routed Delivery Service assigned to a Federation resolves to that Federation‘s CNAME, rather than a Cache’s IP address, depending on the IP address of the client querying Traffic Router.
A GitHub Action to run the tests that the TR Ultimate Test Harness should be added, but only if it consistently meets a constant requests per second threshold. Traffic Router will perform better on some GitHub Actions runners than others, so this should be tested after writing the GitHub Action.
If a meaningful requests per second threshold cannot be found for GitHub Actions runners, we may consider trying again in the future, in case consistency of Traffic Router performance on GitHub Actions runners improves.
By increasing our attention to Traffic Router‘s performance, Traffic Router’s performance should not decrease, and its performance may increase.
If Traffic Router is vulnerable to denial-of-service attacks relating to HTTP requests or DNS queries, there is potential for the TR Ultimate Test Harness to uncover such vulnerabilities.
An Apache Traffic Control administrator may choose to use the TR Ultimate Test Harness to verify that they can get as good of Traffic Router performance in a new Apache Traffic Control version as they can in the version they are upgrading from. If, according to their testing, performance decreases in the newer Traffic Router, the administrator may choose to delay upgrading until they are able to attain the same level of performance, either by changing Traffic Router configuration or waiting for an even newer Traffic Router version to be released.
Needless to say, the Traffic Router Ultimate Test Harness should not be run against a Traffic Router that is simultaneously being used to route production traffic. In order to avoid this. the Traffic Router Ultimate Test Harness should only be run in non-production environments.
When developing Traffic Router features, especially ones found or anticipated to affect Traffic Router performance, a developer may choose to test the result of the new feature on Traffic Router's performance using the Traffic Router Ultimate Test Harness.
wrk is an alternative for HTTP load testing.
flamethrower is an alternative for DNS load testing.
No additional dependencies are anticipated. If additional Go dependencies are required, those dependencies will be added to the Apache Traffic Control go.mod file.
wrk and flamethrower: https://the-asf.slack.com/archives/C023PVBLWE4/p1631639255008200