After updating the features of e2e, you can use the files in the test/e2e/ directory to perform testing for both new and old features of e2e. You can perform testing locally. And when you submit a pull request (PR), GitHub Actions will automatically run the tests.
|- test |- e2e |- concurrency |- fail-fast (concurrency & fail-fast mode) |- internal |- expected.yaml |- verify.yaml (configuration file) |- expected.yaml |- non-fail-fast (concurrency & non-fail-fast mode) |- internal |- expected.yaml |- verify.yaml (configuration file) |- expected.yaml |- non-concurrency |- fail-fast (non-concurrency & fail-fast mode) |- internal |- expected.yaml |- verify.yaml (configuration file ) |- expected.yaml |- non-fail-fast (non-concurrency & non-fail-fast mode) |- internal |- expected.yaml |- verify.yaml (configuration file) |- expected.yaml |- docker-compose.yaml (run a httpbin container, which can return YAML data) |- e2e.yaml (configuration file for the outer infra E2E)
We use the E2E(released version) to test E2E(dev version) of each mode. The external E2E(released version) will compare the summary of the internal E2E(dev version) of each mode with the expected file. If the summary is as expected, the test of the E2E(dev version) of that mode is passed.
From the view of the external E2E(released version), the internal E2E(dev verison) is the SUT. Also, from the view of each internal E2E(dev verison), the ‘httpbin’ is the SUT. We use the docker container of ‘httpbin’ as the SUT, which can receive the ‘query’ of E2E and return YAML data to E2E.After receiving the YAML data from ‘httpbin’, the E2E will compare the YAML data with the expected YAML file. At last, the E2E will generate a summary of the result.
cases: - name: case-1 query: 'curl -s 127.0.0.1:8080/get?case=success -H "accept: application/json"' expected: ./expected.yaml - name: case-2 query: 'curl -s 127.0.0.1:8080/get?case=success -H "accept: application/json"' expected: ./expected.yaml - name: case-3 query: 'curl -s 127.0.0.1:8080/get?case=success -H "accept: application/json"' expected: ./expected.yaml # Add a new case named 'case-4' - name: case-4 query: 'curl -s 127.0.0.1:8080/get?case=success -H "accept: application/json"' expected: ./expected.yaml
the ‘case-4’ will be the passed case, because the parameter is ‘success’. In the ‘concurrency&fail-fast’ mode, the name of the cases should begin with ‘passed’ or ‘failed’.
passed: - case-1 - case-2 - case-3 - case-4 - case-5 - case-7 failed: - case-6 - case-8 - case-9 skipped: [] passedCount: 6 failedCount: 3 skippedCount: 0
add the name of the cases to ‘passed’ or ‘failed’. And add the number of cases on ‘passedCount’ and ‘failedCount’.
passed: - case-1 - case-2 - case-3 - case-4 - case-5 failed: - case-6 skipped: - case-7 - case-8 - case-9 passedCount: 5 failedCount: 1 skippedCount: 3
add the name of the cases to ‘passed’,‘failed’ or ‘skipped’. And add the number of cases on ‘passedCount’,‘failedCount’ and ‘skippedCount’.
{{range .passed}} - {{ if hasPrefix . "passed" }}{{.}}{{ end }} {{end}} failed: {{range .failed}} - {{ if hasPrefix . "failed" }}{{.}}{{ end }} {{end}} skipped: {{range .skipped}} - {{.}} {{end}} passedCount: {{le .passedCount x}} failedCount: {{le .failedCount y}} skippedCount: {{subtractor z .passedCount .failedCount}}
change the number of cases on ‘x’ of ‘passedCount’, ‘y’ of ‘failedCount’ and ‘z’ of ‘skippedCount’.
{{- contains .passed}} - passed-case-1 - passed-case-2 - passed-case-4 - passed-case-5 - passed-case-7 - passed-case-8 {{- end}} failed: {{- contains .failed }} - failed-case-3 - failed-case-6 - failed-case-9 {{- end }} skipped: [] passedCount: 6 failedCount: 3 skippedCount: 0
add the name of the cases to ‘passed’,‘failed’ or ‘skipped’. And add the number of cases on ‘passedCount’,‘failedCount’ and ‘skippedCount’.
make e2e-test
to run the test locally# /internal/expected.yaml args: case: success headers: Accept: application/json Host: 127.0.0.1:8080 # The 'curl' version in the testing environment may vary. User-Agent: curl/7.81.0 # The 'origin' in the testing environment may vary origin: 172.18.0.1 url: http://127.0.0.1:8080/get?case=success
If the above issues prevent the e2e tests from being passed, you can follow these steps to address the problem: