commit | 054b6a80439532b6c74be6b8c27f494a9f8330e2 | [log] [tgz] |
---|---|---|
author | JosiahWI <41302989+JosiahWI@users.noreply.github.com> | Tue Jun 11 15:16:08 2024 -0500 |
committer | GitHub <noreply@github.com> | Tue Jun 11 15:16:08 2024 -0500 |
tree | f532541be07a5f8ae09d4e649bb01486fb590bb3 | |
parent | 348aedf77c2c19a3a2269c13aceced4d4f53e512 [diff] |
Fix race condition causing interleaved logs (#11432) * Remove unused argument from mime_hdr_print This removes the unused heap argument and updates calls across the whole codebase accordingly. * Constify some arguments for MIME printing This constifies some arguments to mime_field_print and mime_hdr_print and updates usage throughout the codebase accordingly. * Remove unused argument from http_hdr_print This removes the heap argument from the method and updates the usage throughout the codebase. * Constify argument to http_hdr_status_get * Constify argument to http_hdr_print This marks the hdr argument to http_hdr_print as const. * Constify the HTTPHdr::print method * Fix race condition causing interleaved logs The output of the `DUMP_HEADER` macro was being written in multiple calls to fprintf() that raced with other logging, resulting in interleaved output that intermittently failed the ja3_fingerprint AuTest on CI. Whether other AuTest failures were related to the same issue is unknown at this point. This fixes the race condition by batching the calls to a single fprintf() call. This also refactors the macro to two inline functions. In a benchmark using an expanded version of the headers in the HttpTransact tests, a 60% speedup was measured over the macro if all the header data fit in the buffer. In the worst case that the buffer could only hold a byte at a time, the speedup was 80%. The benchmark was compiled with -O3 and stderr was piped to /dev/null. These were the headers used for the benchmark: ``` header input1[] = { {"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", "111111111111111111111111"}, {"BBB", "222" }, {"CCC", "333" }, {"DDD", "333" }, {"EEE", "333" }, {"FFF", "333" }, {"GGG", "333" }, {"HHH", "333" }, {"III", "333" }, {"JJJ", "333" }, {"KKK", "333" }, }; ``` This fixes #11431. * Implement changes requested by Brian Neradt This makes the following changes: * The length parameter to string::append is used instead of adding a null terminator. * Some erroneous logic left over from testing is fixed. * Fix formatting * Add log output to dump_header for invalid hdrs * Remove bufp parameter from TSMimeHdrPrint This is an API change. The parameter is no longer used, so this commit removes it and updates the developer documentation and core plugins accordingly. * Fix the example plugins Since the API changed, the example plugins now need updating as well.
Traffic Server is a high-performance building block for cloud services. It's more than just a caching proxy server; it also has support for plugins to build large scale web applications.
AS of version 10, ATS has transitioned to cmake as its build system. Below is a quick-start guide to building ATS with cmake:
With cmake, you definitely want to create an out-of-source build. You will give that directory to every cmake command. For these examples, it will just be build
.
$ cmake -B build
This will configure the project with defaults.
If you want to customize the build, you can pass values for variables on the command line. Or, you can interactively change them using the ccmake
program.
$ cmake -B build -DCMAKE_INSTALL_PREFIX=/tmp/ats -DBUILD_EXPERIMENTAL_PLUGINS=ON
-- or --
$ ccmake build
To specify the location of a dependency (like --with-*
for autotools builds), you generally set a variable with the ROOT
. The big exception to this is for openssl. This variable is called OPENSSL_ROOT_DIR
$ cmake -B build -Djemalloc_ROOT=/opt/jemalloc -DPCRE_ROOT=/opt/edge -DOPENSSL_ROOT_DIR=/opt/boringssl
cmake has a feature for grouping configurations together to make configuration and reproduction easier. The file CMakePresets.json
declares presets that you can use from the command line. You can provide your own CMakeUserPresets.json
and further refine those via inheritance:
$ cmake --preset dev
You can start out your user presets by just copying CMakePresets.json
and removing everything in configurePresets
Here is an example user preset:
{ "name": "clang", "hidden": true, "environment": { "LDFLAGS": "-L/opt/homebrew/opt/llvm/lib -L/opt/homebrew/opt/llvm/lib/c++ -Wl,-rpath,/opt/homebrew/opt/llvm/lib/c++ -fuse-ld=/opt/homebrew/opt/llvm/bin/ld64.lld", "CPPFLAGS": "-I/opt/homebrew/opt/llvm/include", "CXXFLAGS": "-stdlib=libc++", "CC": "/opt/homebrew/opt/llvm/bin/clang", "CXX": "/opt/homebrew/opt/llvm/bin/clang++" } }, { "name": "mydev", "displayName": "my development", "description": "My Development Presets", "binaryDir": "${sourceDir}/build-dev-clang", "inherits": ["clang", "dev"], "cacheVariables": { "CMAKE_INSTALL_PREFIX": "/opt/ats-cmake", "jemalloc_ROOT": "/opt/homebrew", "ENABLE_LUAJIT": false, "ENABLE_JEMALLOC": true, "ENABLE_MIMALLOC": false, "ENABLE_MALLOC_ALLOCATOR": true, "BUILD_EXPERIMENTAL_PLUGINS": true, "BUILD_REGRESSION_TESTING": true } },
And then use it like:
cmake --preset mydev
$ cmake --build build
$ cmake --build build -t traffic_server
$ cd build $ ctest
$ cmake --install build
trafficserver ............. Top src dir ├── ci .................... Quality assurance and other CI tools and configs ├── configs ............... Configurations ├── contrib ............... Various contributed auxiliary pieces ├── doc ................... Documentation for Traffic Server │ ├── admin-guide ....... Admin guide documentations │ ├── appendices ........ Appendices of Traffic Server │ ├── developer-guide ... Documentation for developers │ ├── dot ............... Graphviz source files for docs pictures │ ├── static ............ Static resources │ └── uml ............... Documentation in UML ├── example ............... Example plugins ├── iocore ................ │ ├── aio ............... Asynchronous I/O core │ ├── cache ............. Disk and RAM cache │ ├── dns ............... DNS (asynchronous) │ ├── eventsystem ....... Event Driven Engine │ ├── hostdb ............ Internal DNS cache │ ├── net ............... Network │ │ └── quic .......... QUIC implementation │ └── utils ............. Utilities ├── lib ................... │ ├── records ........... Library for config files │ └── yamlcpp ........... Library for YAML of C++ ├── m4 .................... Custom macros for configure.ac ├── mk .................... Includes for Makefiles ├── mgmt .................. JSONRPC server/management and tools ├── plugins ............... Stable core plugins │ └── experimental ...... Experimental core plugins ├── proxy ................. HTTP proxy logic │ ├── hdrs .............. Headers parsing and management │ ├── http .............. The actual HTTP protocol implementation │ ├── http2 ............. HTTP/2 implementation │ ├── http3 ............. HTTP/3 implementation │ ├── logging ........... Flexible logging │ └── shared ............ Shared files ├── rc .................... Installation programs and scripts ├── src ................... Source for all the main binaries / applications │ ├── traffic_cache_tool Tool to interact with the Traffic Server cache │ ├── traffic_crashlog .. Helper process that catches Traffic Server crashes │ ├── traffic_ctl ....... Command line management tool │ ├── traffic_layout .... Display information on the build and runtime directory structure │ ├── traffic_logcat .... Convert binary log file to plain text │ ├── traffic_logstats .. Log parsing and metrics calculation utility │ ├── traffic_server .... Main proxy server │ ├── traffic_top ....... Top like tool for viewing Traffic Server statistics │ ├── traffic_via ....... Tool for decoding the Traffic Server Via header codes │ ├── tscore ............ Base / core library │ ├── tscpp ............. C++ api wrapper for plugin developers ├── tests ................. Different tests for Traffic Server ├── tools ................. Directory of various tools ├── INSTALL ............... Build and installation guide ├── LAYOUT ................ Traffic Server default layout ├── LICENSE ............... Full license text ├── NOTICE ................ Copyright notices ├── README ................ Intro, links, build info ├── README-EC2 ............ Info on EC2 support ├── REVIEWERS ............. (Incomplete) list of areas with committer interest └── STATUS ................ Release history and information
This section outlines build requirements for different OS distributions. This may be out of date compared to the on-line requirements at
https://cwiki.apache.org/confluence/display/TS/Building.
As of ATS v9.0.0 and later, gcc 7 or later is required, since we now use and require the C++17 standard.
cmake ninja pkgconfig gcc/g++ or clang/clang++ openssl-devel pcre-devel ncurses-devel and libcurl-devel(optional, needed for traffic_top) libcap-devel (optional, highly recommended) hwloc-devel (optional, highly recommended)
cmake ninja pkg-config gcc/g++ or clang/clang++ zlib1g-dev libssl-dev libpcre3-dev libcap-dev (optional, highly recommended) libhwloc-dev (optional, highly recommended) libncurses5-dev (optional, required for e.g.: traffic_top) libcurl4-openssl-dev (optional, required for e.g.: traffic_top)
build-base libexecinfo-dev pcre-dev libressl-dev cmake ninja linux-headers
cmake ninja pkg-config openssl pcre
cmake ninja devel/gmake devel/pkgconf security/openssl devel/pcre textproc/flex (optional, install newer version from ports, fix PATH) devel/hwloc (optional, highly recommended)
You can download the latest source code from the official Apache Traffic Server site:
https://trafficserver.apache.org/downloads
(or via the URL shortener: http://s.apache.org/uG). Once downloaded, follow the instructions:
tar jxvf trafficserver-9.1.3.tar.bz2 cd trafficserver-9.1.3 cmake -B build cmake --build build
This will build with a destination prefix of /usr/local. You can finish the installation with
sudo cmake --install build
git clone https://github.com/apache/trafficserver.git # get the source code from ASF Git repository cd trafficserver # enter the checkout directory cmake --preset default # configure the build cmake --build build-default # execute the compile cmake --build build-default -t test # run tests (optional) cmake --install build-default # install
NOTE: Alternately you may use the scripts under ‘contrib’ which will automate the install for trafficserver under EC2 which is HIGHLY RECOMMENDED. See ‘README-EC2’ for further details.
mkdir -p /mnt #EC2 Storage Mount, where storage is located cd /mnt git clone ... # get the source code from ASF Git repo cd trafficserver # enter the checkout dir cmake --preset default # configure the build cmake --build build-default # execute the compile cmake --build build-default -t test cmake --install build-default
mkdir -p /mnt #EC2 Storage Mount, where storage is located cd /mnt git clone ... # get the source code from ASF Git repo cd trafficserver # enter the checkout dir cmake --preset default # configure the build cmake --build build-default # execute the compile cmake --build build-default -t test # run tests (optional) cmake --install build-default # install
/usr/local ├── /var/log/trafficserver log files created at runtime ├── /var/trafficserver runtime files ├── /etc/trafficserver configuration files ├── /bin executable binaries └── /libexec/trafficserver plugins
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.
The following provides more details on the included cryptographic software:
The functionality of OpenSSL http://www.openssl.org/ is utilized in parts of the software.
export CC=clang export CXX=clang++ export CFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link" export CXXFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link" export LIB_FUZZING_ENGINE=-fsanitize=fuzzer
cmake -B build -DENABLE_POSIX_CAP=OFF -DENABLE_FUZZING=ON -DYAML_BUILD_SHARED_LIBS=OFF cmake --build build