hrw4u: Add AST for static analysis and codegen (#13126)

* Add AST node dataclasses for hrw4u linter

Frozen dataclasses representing the semantic AST that a
visitor produces from the ANTLR parse tree. Includes Target
decomposition (namespace/field/modifier), all statement nodes
(Assignment, FunctionCall, BreakStatement, StandaloneOperator),
condition expression nodes (Comparison, LogicalOp, Negation),
control flow (IfBlock, ElifBranch), and top-level constructs
(VarDecl, UseDecl, ProcedureDecl, Section).

Type aliases ConditionExpr, BodyNode, and TopLevelNode provide
convenience unions. Tests cover Target.from_dotted parsing, node
construction, and immutability.

* Add AST visitor to build AST from ANTLR parse tree

ASTVisitor walks the ANTLR parse tree and produces HRW4UAST.
Handles named sections, assignments (= and +=), function calls,
break statements, standalone operators, condition expressions
(comparisons, logical operators, negation, set membership, IP
ranges, WITH modifiers), if/elif/else blocks with arbitrary
nesting, and top-level var/use/procedure declarations.

Only visitProgram is overridden from the ANTLR visitor base
class; all other dispatch is internal, keeping the public API
surface minimal. Raises ValueError for unhandled grammar
alternatives to surface visitor-grammar drift early.

Makefile updated to include ast_nodes.py and ast_visitor.py
in the build copy step.

* Add AST visitor tests

Integration tests covering the full visitor pipeline from
source text to AST nodes. Tests are organized by concern:
sections and simple statements, condition expressions (all
operators, logical combinators, negation, parenthesized
grouping), if/elif/else blocks with nesting, real config
patterns (nested conditionals, boolean coercion, IP ranges,
set membership with modifiers, exact match patterns), line
number tracking across all 17 node types, and error handling
for unhandled grammar alternatives.

* Add tagged Value type to AST for codegen support

Introduce ValueKind enum and Value dataclass to preserve
semantic distinction between string literals, identifiers,
param refs, IPs, and regexes in the AST. Without this, the
codegen visitor cannot re-emit values correctly since
_extract_value was collapsing all string-like values into
bare Python str.

* Handle commentLine explicitly in visitProgram

Skip comments intentionally and raise on unrecognized
programItem alternatives to catch visitor/grammar drift.

* Replace tagged Value type with concrete value types

Split Value/ValueKind into LiteralStringValue, IdentValue,
IPValue, ParamRef, and RegexValue so the type system encodes
what kind of value each node carries. This makes the AST more
precise — e.g. iprange is now tuple[IPValue, ...] instead of
tuple[Value, ...], and RegexValue only appears in
Comparison.right where the grammar allows it.

* Raise on unhandled grammar alternatives in AST visitor

Add explicit commentLine handling and ValueError raises
in _visit_var_section, _visit_body, and _extract_value
so new grammar alternatives fail fast instead of being
silently dropped or misclassified.

* Format code

* Add tests for comment skipping and modifier casing

Cover comment handling in section bodies, blocks, and
var sections. Add test confirming modifiers preserve
source casing rather than normalizing.

* Add __all__ to ast_nodes.py for safe wildcard imports

Defines the public API surface explicitly, enabling shorter
`from hrw4u.ast_nodes import *` in downstream modules.

* Use wildcard imports from ast_nodes

Now that __all__ is defined, switch ast_visitor.py and
test_ast_visitor.py to `from hrw4u.ast_nodes import *`.

* Add return type annotations to ASTVisitor methods

Matches the convention used in visitor.py where all visit/helper
methods declare their return types.

* Collapse short constructor calls to single lines

Fits within the project's 132-char column limit per .style.yapf.
5 files changed
tree: a1ff7dd9781fd9d735d8d5a93cb42123a66cce48
  1. .claude/
  2. .cursor/
  3. .github/
  4. .vscode/
  5. ci/
  6. cmake/
  7. configs/
  8. contrib/
  9. doc/
  10. example/
  11. ext/
  12. include/
  13. lib/
  14. plugins/
  15. rc/
  16. src/
  17. tests/
  18. tools/
  19. .asf.yaml
  20. .clang-analyzer
  21. .clang-format
  22. .clang-tidy
  23. .cmake-format.yaml
  24. .editorconfig
  25. .git-blame-ignore-revs
  26. .gitignore
  27. .perltidyrc
  28. .ripgreprc
  29. .style.yapf
  30. .tsan_suppressions
  31. .vimrc
  32. .yapfignore
  33. AGENTS.md
  34. CMakeLists.txt
  35. CMakePresets.json
  36. CONTRIBUTING.md
  37. emacs-style
  38. Findtsapi.cmake.in
  39. INSTALL
  40. LICENSE
  41. NOTICE
  42. README-EC2
  43. README.md
  44. SECURITY.md
  45. setup.cfg
  46. STATUS
  47. ts.pc.in
README.md

Apache Traffic Server

Jenkins Jenkins Jenkins Jenkins Jenkins Jenkins Jenkins Jenkins Jenkins Jenkins Jenkins

Traffic Server is a high-performance building block for cloud services. It's more than just a caching proxy server; it also has support for plugins to build large scale web applications.

Important notice to ATS developers

AS of version 10, ATS has transitioned to cmake as its build system. Below is a quick-start guide to building ATS with cmake:

Step 1: Configuration

With cmake, you definitely want to create an out-of-source build. You will give that directory to every cmake command. For these examples, it will just be build.

$ cmake -B build

This will configure the project with defaults.

If you want to customize the build, you can pass values for variables on the command line. Or, you can interactively change them using the ccmake program.

$ cmake -B build -DCMAKE_INSTALL_PREFIX=/tmp/ats -DBUILD_EXPERIMENTAL_PLUGINS=ON

-- or --

$ ccmake build

Specifying locations of dependencies

To specify the location of a dependency (like --with-* for autotools builds), you generally set a variable with the ROOT. The big exception to this is for openssl. This variable is called OPENSSL_ROOT_DIR

$ cmake -B build -Djemalloc_ROOT=/opt/jemalloc -DOPENSSL_ROOT_DIR=/opt/boringssl

Using presets to configure the build

cmake has a feature for grouping configurations together to make configuration and reproduction easier. The file CMakePresets.json declares presets that you can use from the command line. You can provide your own CMakeUserPresets.json and further refine those via inheritance:

$ cmake --preset dev

You can start out your user presets by just copying CMakePresets.json and removing everything in configurePresets

Here is an example user preset:


{ "name": "clang", "hidden": true, "environment": { "LDFLAGS": "-L/opt/homebrew/opt/llvm/lib -L/opt/homebrew/opt/llvm/lib/c++ -Wl,-rpath,/opt/homebrew/opt/llvm/lib/c++ -fuse-ld=/opt/homebrew/opt/llvm/bin/ld64.lld", "CPPFLAGS": "-I/opt/homebrew/opt/llvm/include", "CXXFLAGS": "-stdlib=libc++", "CC": "/opt/homebrew/opt/llvm/bin/clang", "CXX": "/opt/homebrew/opt/llvm/bin/clang++" } }, { "name": "mydev", "displayName": "my development", "description": "My Development Presets", "binaryDir": "${sourceDir}/build-dev-clang", "inherits": ["clang", "dev"], "cacheVariables": { "CMAKE_INSTALL_PREFIX": "/opt/ats-cmake", "jemalloc_ROOT": "/opt/homebrew", "ENABLE_LUAJIT": false, "ENABLE_JEMALLOC": true, "ENABLE_MIMALLOC": false, "ENABLE_MALLOC_ALLOCATOR": true, "BUILD_EXPERIMENTAL_PLUGINS": true, "BUILD_REGRESSION_TESTING": true } },

And then use it like:

cmake --preset mydev

Building the project

$ cmake --build build
$ cmake --build build -t traffic_server

running tests

$ cd build
$ ctest

installing

$ cmake --install build

DIRECTORY STRUCTURE

trafficserver ............. Top src dir
├── ci .................... Quality assurance and other CI tools and configs
├── cmake ................. CMake build system files
├── configs ............... Configurations
├── contrib ............... Various contributed auxiliary pieces
├── doc ................... Documentation for Traffic Server
│   ├── admin-guide ....... Admin guide documentations
│   ├── appendices ........ Appendices of Traffic Server
│   ├── developer-guide ... Documentation for developers
│   ├── dot ............... Graphviz source files for docs pictures
│   ├── static ............ Static resources
│   └── uml ............... Documentation in UML
├── example ............... Example plugins
├── ext ................... External dependencies and extensions
├── include ............... Header files
├── lib ................... Third-party libraries
│   ├── Catch2 ............ Unit testing framework
│   ├── fastlz ............ Fast compression library
│   ├── ls-hpack .......... HPACK compression for HTTP/2
│   ├── swoc .............. Solid Wall of Code utility library
│   ├── systemtap ......... SystemTap integration
│   └── yamlcpp ........... YAML parser library
├── memory-bank ........... AI assistant context documentation
├── plugins ............... Stable core plugins
│   └── experimental ...... Experimental core plugins
├── rc .................... Installation programs and scripts
├── src ................... Source for all the main binaries / applications
│   ├── api ............... C Plugin API implementation
│   ├── cripts ............ Cripts scripting framework
│   ├── iocore ............ I/O core subsystem
│   │   ├── aio ........... Asynchronous I/O core
│   │   ├── cache ......... Disk and RAM cache
│   │   ├── dns ........... DNS (asynchronous)
│   │   ├── eventsystem ... Event Driven Engine
│   │   ├── hostdb ........ Internal DNS cache
│   │   ├── io_uring ...... io_uring support (Linux)
│   │   ├── net ........... Network I/O
│   │   │   └── quic ...... QUIC implementation
│   │   └── utils ......... I/O utilities
│   ├── mgmt .............. JSONRPC server/management and tools
│   ├── proxy ............. HTTP proxy logic
│   │   ├── hdrs .......... Headers parsing and management
│   │   ├── http .......... HTTP/1.x implementation
│   │   ├── http2 ......... HTTP/2 implementation
│   │   ├── http3 ......... HTTP/3 implementation
│   │   ├── logging ....... Flexible logging
│   │   └── shared ........ Shared proxy files
│   ├── records ........... Configuration records library
│   ├── shared ............ Shared utilities and RPC
│   ├── traffic_cache_tool  Tool to interact with the Traffic Server cache
│   ├── traffic_crashlog .. Helper process that catches Traffic Server crashes
│   ├── traffic_ctl ....... Command line management tool
│   ├── traffic_layout .... Display information on the build and runtime directory structure
│   ├── traffic_logcat .... Convert binary log file to plain text
│   ├── traffic_logstats .. Log parsing and metrics calculation utility
│   ├── traffic_quic ...... QUIC client tool
│   ├── traffic_server .... Main proxy server
│   ├── traffic_top ....... Top like tool for viewing Traffic Server statistics
│   ├── traffic_via ....... Tool for decoding the Traffic Server Via header codes
│   ├── tscore ............ Base / core library
│   ├── tscpp ............. C++ api wrapper for plugin developers
│   └── tsutil ............ Core utilities (metrics, debugging, regex, etc.)
├── tests ................. Different tests for Traffic Server
├── tools ................. Directory of various tools
├── CONTRIBUTING.md ....... How to contribute to the project
├── INSTALL ............... Build and installation guide
├── LICENSE ............... Full license text
├── NOTICE ................ Copyright notices
├── README.md ............. This file: intro, links, build info
├── README-EC2 ............ Info on EC2 support
├── SECURITY.md ........... Security policy and vulnerability reporting
└── STATUS ................ Release history and information

REQUIREMENTS

This section outlines build requirements for different OS distributions. This may be out of date compared to the on-line requirements at

https://cwiki.apache.org/confluence/display/TS/Building.

As of ATS v9.0.0 and later, gcc 7 or later is required, since we now use and require the C++17 standard.

Fedora / CentOS / RHEL:

cmake
ninja
pkgconfig
gcc/g++ or clang/clang++
openssl-devel
pcre2-devel
ncurses-devel and libcurl-devel(optional, needed for traffic_top)
libcap-devel (optional, highly recommended)
hwloc-devel (optional, highly recommended)

Ubuntu / Debian

cmake
ninja
pkg-config
gcc/g++ or clang/clang++
zlib1g-dev
libssl-dev
libpcre2-dev
libcap-dev (optional, highly recommended)
libhwloc-dev (optional, highly recommended)
libncurses5-dev (optional, required for e.g.: traffic_top)
libcurl4-openssl-dev (optional, required for e.g.: traffic_top)

Alpine Linux

build-base
libexecinfo-dev
pcre2-dev
libressl-dev
cmake
ninja
linux-headers

macOS (we recommend HomeBrew):

cmake
ninja
pkg-config
openssl
pcre2

FreeBSD

cmake
ninja
devel/gmake
devel/pkgconf
security/openssl
devel/pcre2
textproc/flex (optional, install newer version from ports, fix PATH)
devel/hwloc (optional, highly recommended)

Building from distribution

You can download the latest source code from the official Apache Traffic Server site:

https://trafficserver.apache.org/downloads

(or via the URL shortener: http://s.apache.org/uG). Once downloaded, follow the instructions:

tar jxvf trafficserver-9.1.3.tar.bz2
cd trafficserver-9.1.3
cmake -B build
cmake --build build

This will build with a destination prefix of /usr/local. You can finish the installation with

sudo cmake --install build

BUILDING FROM GIT REPO

git clone https://github.com/apache/trafficserver.git   # get the source code from ASF Git repository
cd trafficserver                                        # enter the checkout directory
cmake --preset default                                  # configure the build
cmake --build build-default                             # execute the compile
cmake --build build-default -t test                     # run tests (optional)
cmake --install build-default                           # install

Instructions for building on EC2

NOTE: Alternately you may use the scripts under ‘contrib’ which will automate the install for trafficserver under EC2 which is HIGHLY RECOMMENDED. See ‘README-EC2’ for further details.

As root do the following when using Ubuntu

mkdir -p /mnt          #EC2 Storage Mount, where storage is located
cd /mnt
git clone ...          # get the source code from ASF Git repo
cd trafficserver       # enter the checkout dir
cmake --preset default                                  # configure the build
cmake --build build-default                                   # execute the compile
cmake --build build-default -t test
cmake --install build-default

As root do the following when using Fedora Core 8 kernel

mkdir -p /mnt                             #EC2 Storage Mount, where storage is located
cd /mnt
git clone ...                             # get the source code from ASF Git repo
cd trafficserver                          # enter the checkout dir
cmake --preset default                    # configure the build
cmake --build build-default               # execute the compile
cmake --build build-default -t test       # run tests (optional)
cmake --install build-default             # install

INSTALLATION

/usr/local
├── /var/log/trafficserver   log files created at runtime
├── /var/trafficserver       runtime files
├── /etc/trafficserver       configuration files
├── /bin                     executable binaries
└── /libexec/trafficserver   plugins

CRYPTO NOTICE

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

The following provides more details on the included cryptographic software:

The functionality of OpenSSL http://www.openssl.org/ is utilized in parts of the software.

Fuzzing

FLAGS

export CC=clang
export CXX=clang++
export CFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link"
export CXXFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link"
export LIB_FUZZING_ENGINE=-fsanitize=fuzzer

Compile

cmake -B build -DENABLE_POSIX_CAP=OFF -DENABLE_FUZZING=ON -DYAML_BUILD_SHARED_LIBS=OFF
cmake --build build

ADDITIONAL INFO