All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog.
PUT a requested configuration change for a full configuration or per host and an endpoint to approve or deny the request.GET capacity and telemetry data for CDNi integration.status field logging with the addition of the filed to publish/CrStatesGET /deliveryservices/{id}/servers/ and /deliveryservices/{id}/servers/eligible.GET /servers/, POST /servers/, PUT /servers/{id} and DELETE /servers/{id}.tm_query_status_override -- to override which status of Traffic Monitors to query (default: ONLINE).ats #6879.distributed_polling which enables the ability for Traffic Monitor to poll a subset of the CDN and divide into “local peer groups” and “distributed peer groups”. Traffic Monitors in the same group are local peers, while Traffic Monitors in other groups are distibuted peers. Each TM group polls the same set of cachegroups and gets availability data for the other cachegroups from other TM groups. This allows each TM to be responsible for polling a subset of the CDN while still having a full view of CDN availability. In order to use this, stat_polling must be disabled.access.log to TM.cdn.conf option -- server_update_status_cache_refresh_interval_sec -- which enables an in-memory server update status cache to improve performance. Default: 0 (disabled).cdn.conf option -- user_cache_refresh_interval_sec -- which enables an in-memory users cache to improve performance. Default: 0 (disabled).file-protocol URLs for the geolocation.polling.url for the Geolocation database.status and lastPoll fields to the publish/CrStates endpoint of Traffic Monitor (TM) #6448.__HOSTNAME__ in “unknown” files (others than the defaults ones), was being replaced by the full FQDN instead of the shot hostname.GET /deliveryservicesserver causing error when an IMS request is made with the cdn and maxRevalDurationDays parameters set.PUT /servers/:id/status to only queue updates on the same CDN as the updated serverRHEL_VERSION=7operations and admin roles should have the DELIVERY-SERVICE:UPDATE permission.npm audit issues.api/{{version}/deliveryservices/{id}/health returns no info if the delivery service uses a topology./acme_accounts/acme_accounts endpoint to validate email and URL fieldsPOST and response code for PUT to /acme_accounts endpointcreate_tables.sql more than once./server/details associated with query parameters.GET for /servers to display all profiles irrespective of the index position. Also, replaced query param profileId with profileName.POST api/cachegroups/id/queue_updates endpoint so that it doesn't give an internal server error anymore.dequeueing server updates should not require checking for cdn locks.client.steering.forced.diversity feature flag(profile parameter) from Traffic Router (TR). Client steering responses now have cache diversity by default.npm audit issues, specifically grunt-concurrent, grunt-contrib-concat, grunt-contrib-cssmin, grunt-contrib-jsmin, grunt-contrib-uglify, grunt-contrib-htmlmin, grunt-newer, and grunt-wiredeppeer_polling_protocol option. Traffic Monitor now just uses hostnames to request peer states, which can be handled via IPv4 or IPv6 depending on the underlying IP version in use.forever with pm2 for process management of the traffic portal node server to remediate security issues./servers/details endpoint of the Traffic Ops API has been dropped in version 4.0, and marked deprecated in earlier versions.deliveryservice_tmuser table from Traffic Ops databaseTRAFFIC_ROUTER-type Profiles no longer need to have names that match any kind of pattern (e.g. CCR_.*)cdn and maxRevalDurationDays to the GET /api/x/jobs Traffic Ops API to filter by CDN name and within the start_time window defined by the maxRevalDurationDays GLOBAL profile parameter, respectively.disable_auto_cert_deletion -- in order to optionally prevent the automatic deletion of certificates for delivery services that no longer exist whenever a CDN snapshot is taken.cdn to the GET /api/x/deliveryserviceserver Traffic Ops API to filter by CDN nameshort_hostname_override -- to traffic_monitor.cfg to allow overriding the system hostname that Traffic Monitor uses.stat_polling (default: true) -- to traffic_monitor.cfg to disable stat polling.-c to specify a configuration file, and the ability to set log: null to log to stdout (consult documentation for details).heartbeat.polling.interval for CDN Traffic Monitor config in API documentation.pkg script options, -h, -s, -S, and -L.Invalidation Type (REFRESH or REFETCH) for invalidating content to Traffic Portal.__CACHEGROUP__ preprocess directive, to allow injecting the local server's cachegroup name into any config fileGET /cdns/routing Traffic Ops APIuser/current to work with v4 User Roles and Permissionslog4j.properties with Log4j 1.2t3c to request less unnecessary deliveryservice-server assignment and invalidation jobs data via new query params supported by Traffic OpsToDnssecRefresh binary and make the trafops_dnssec_refresh cron job use itruby compass to compile sass and now uses dart-sass.maxConnections value on Traffic Router, to prevent the thundering herd problem (TR).admin Role is now always guaranteed to exist, and can't be deleted or modified.t3c-apply to reduce mutable state in TrafficOpsReq struct.jackson-databind and jackson-annotations Traffic Router dependencies to version 2.13.1/api_capability and /capabilities.logs property of the Traffic Stats configuration file (refer to documentation for details).user_role table.traffic_ops.sh shell profile no longer sets GOPATH or adds its bin folder to the PATH/capabilities removed from Traffic Ops API version 4.log4j module in Traffic Router from version 1.2.17 to 2.17.0/cdns/{name}/federations?id=# to search for CDN./deliveryservices/:id/routing makes requests to all TRs instead of by CDN.GET /user/current, GET /users, GET /user?id=) response payloadtodb-tests GitHub action now runs the Traffic Ops DB testsinstall/bin/_postinstall.pl and has been deprecated, pending removal in a future release.traffic_vault_backend and traffic_vault_configGET request method for /deliveryservices/{{ID}}/assignGET request method for /deliveryservices/{{ID}}/statusGetServersByDeliveryService method to the TO Go clientDELETE request method for deliveryservices/xmlId/{name}/urlkeys and deliveryservices/{id}/urlkeys.t3c apply ..., moved to cache-config and RPM changed to trafficcontrol-cache-config. See cache-config README.md.svc="..." field to request logging output.traffic_ops/app/db/traffic_vault_migrate to help with migrating Traffic Ops Traffic Vault backends/traffic_ops/app/db/reencrypt to re-encrypt the data in the Postgres Traffic Vault with a new key.tlsVersions - that explicitly lists the TLS versions that may be used to retrieve their content from Cache Servers.log_location_info instead of log_location_warning403 Forbidden when the user tries to get servers of a non-existent DS using GET /servers?dsId={{nonexistent DS ID}}monitor to true./deliveryservices/{{ID}}/safe returns incorrect response for the requested API version/servers/{{ID}}/deliveryservices returns incorrect response for the requested API versiondb/admin tool to use Migrate instead of Goose and converted the migrations to Migrate format (split up/down for each migration into separate files)serve_write_timeout_ms to 20000 by default because 10 seconds can be too short for relatively large CDNs./cdns/dnsseckeys/refresh). As of TO API v4, its method is PUT instead of GET, its response format was changed to return an alert instead of a string-based response, it returns a 202 instead of a 200, and it now works with the async_status API in order for the client to check the status of the async job: #3054goose provider to the maintained fork github.com/kevinburke/goose/servers/{{host name}}/update_status Traffic Ops API endpoint has been changed to use a top-level response property, in keeping with (most of) the rest of the API.t3c for cache configuration./deliveryservices/request Traffic Ops API endpoint can no longer contain characters besides alphanumerics, @, !, #, $, %, ^, &, *, (, ), [, ], ‘.’, ' ', and ‘-’. This fixes a vulnerability that allowed email content injection.riak.conf config file and its corresponding --riakcfg option in traffic_ops_golang have been deprecated. Please use "traffic_vault_backend": "riak" and "traffic_vault_config" (with the existing contents of riak.conf) instead.GET /api/{version}/vault/bucket/{bucket}/key/{key}/values has been deprecated and will no longer be available as of Traffic Ops API v4POST /api/{version}/deliveryservices/request has been deprecated and will no longer be available as of Traffic Ops API v4GET /api/{version}/cachegroupparameters, POST /api/{version}/cachegroupparameters, GET /api/{version}/cachegroups/{id}/parameters, and DELETE /api/{version}/cachegroupparameters/{cachegroupID}/{parameterId} have been deprecated and will no longer be available as of Traffic Ops API v4riak_port option in cdn.conf is now deprecated. Please use the "port" field in traffic_vault_config instead.traffic_ops_ort.pl tool has been deprecated in favor of t3c, and will be removed in the next major version.backend_max_connections option from cdn.conf.http_poll_no_sleep, max_stat_history, max_health_history, cache_health_polling_interval_ms, cache_stat_polling_interval_ms, and peer_polling_interval_ms Traffic Monitor config options.Long Description 2 and Long Description 3 fields of DeliveryService from the UI, and changed the backend so that routes corresponding API 4.0 and above no longer accept or return these fields.app/public directory, as the static webserver has been removed along with the Perl Traffic Ops implementation. Traffic Ops also no longer attempts to download MaxMind GeoIP City databases when running the Traffic Ops Postinstall script.compare tool stack has been removed, as it no longer serves a purpose.cdn.conf option geniso.iso_root_pathcdns/routing to avoid incorrect success response.cache_statsONLINE caches as available.TM UI and /api/cache-statuses to report aggregate bandwidth_kbps correctly.maxRequestHeaderBytes on a per delivery service basisweight parameter of parent.config is a floatparent_reval_pending for servers in a Flexible Topology CDN-specific on GET /servers/{name}/update_status200 OK from all delivery servicesGET /api/3.0/topologies, to create, read, update and delete flexible topologies.POST /api/3.0/topologies/{name}/queue_update, to queue or dequeue updates for all servers assigned to the Cachegroups in a given Topology.topology field to the /api/3.0/deliveryservices APIstopology query parameter to GET /api/3.0/cachegroups to return all cachegroups used in the given topology.topology query parameter to GET /api/3.0/deliveryservices to return all delivery services that employ a given topology.dsId query parameter for GET /api/3.0/servers for topology-based delivery services.GET /api/3.0/servers?dsId=# for Topology-based Delivery Services unless the ORG server is assigned to that Delivery Service.topology query parameter for GET /api/3.0/servers to return all servers whose cachegroups are in a given topology.firstHeaderRewrite, innerHeaderRewrite, lastHeaderRewrite/api/3.0/topologies endpoint and the /api/3.0/servers/{{ID}} endpoint.demo1-top, and make the demo1 delivery service topology-basedAccept: text/csv and return a csv formatted stats listcachegroupName query parameter for GET /api/3.0/servers in Traffic Ops/traffic_router/tests/traffic_router/tests run in Alpine LinuxStatus Last Updated field to servers, and the UI, so that we can see when the last status change took place for a server.system_stats.so when using stats over http in order to keep all the same functionality (and included stats) that astats_over_http provides.stats_over_http health.polling.format value that allows monitoring of multiple interfaces will first require that all Traffic Monitors monitoring the affected cache server be upgraded.--traffic_ops_insecure=<0|1> optional option to traffic_ops_ort.plcrs/stats/ip/{ip} endpoint in the Traffic Router APIactive flagGET /api/x/cdns/capacity gives back 500, with the message capacity was zeroGET /api/x/jobs and GET /api/x/jobs/:id Traffic Ops API routes to allow falling back to Perl via the routing blacklistPOST /api/x/serverchecks Traffic Ops API endpoint in order to reduce audit log spam/jobs APIs to bring them back to the same level of information provided by TO-PerlmaxRevalDurationDays validation for POST /api/1.x/user/current/jobs and added that validation to the /api/x/jobs endpointsPOST /api/x/steering and PUT /api/x/steering so that a steering target with an invalid type is no longer accepted. Related github issuecachegroups READ endpoint, so that if a request is made with the type specified as a non integer value, you get back a 400 with error details, instead of a 500. Related github issue__RANGE_DIRECTIVE__ directive to allow inserting the Range Directive after the Raw Remap text. This allows Raw Remaps which manipulate the Range.coordinateRange in the RGB configuration file, so that in case a client doesn't have a postal code, we can still determine if it should be allowed or not, based on whether or not the latitude/ longitude of the client falls within the supplied ranges. Related github issuecreate_tables.sql be run concurrently without issuePOST /api/x/profileparameters route/opt/traffic_router/db/cr-config.jsonDeliveryServiceNullable inputs and outputs.tmconfig.backup to store the result of GET /api/2.0/cdns/{{name}}/configs/monitoring instead of a transformed map/servers and /servers/{{ID}}} TO API endpoints have been updated to use and reflect multi-interface servers./cdns/{{name}}/configs/monitoring TO API endpoint to return multi-interface data.Cache States tab of the Traffic Monitor UI to properly handle multiple interfaces./publish/CacheStats in Traffic Monitor to support multiple interfaces.RHEL_VERSION Docker build arg so CDN in a Box can be built for CentOS 7, if desiredDeliveryService Go struct and other structs that use it. DeliveryServiceNullable structs should be used instead.insecure option in traffic_ops_golang in favor of "tls_config": { "InsecureSkipVerify": <bool> }github.com/apache/trafficcontrol/traffic_ops/client is now deprecated in favor of versioned import paths e.g. github.com/apache/trafficcontrol/traffic_ops/v3-client.chi field and the resolver address is in the rhi field.(DELETE)(POST)(GET, POST)POSTDELETEPOSTGETGETGETGET(DELETE)(GET, POST)POST(GET)PUTtotalBytes property of responses to GET requests to /deliveryservice_stats to the more appropriate totalKiloBytes in API 2.xdb/admin.pl script has now been removed. Please use the db/admin binary instead.Traffic Router: TR now generates a self-signed certificate at startup and uses it as the default TLS cert. The default certificate is used whenever a client attempts an SSL handshake for an SNI host which does not match any of the other certificates.
Client Steering Forced Diversity: force Traffic Router to return more unique edge caches in CLIENT_STEERING results instead of the default behavior which can sometimes return a result of multiple targets using the same edge cache. In the case of edge cache failures, this feature will give clients a chance to retry a different edge cache. This can be enabled with the new “client.steering.forced.diversity” Traffic Router profile parameter.
Traffic Ops Golang Endpoints
(GET,POST,PUT)(GET,POST,PUT)GETGETPOSTGET(GET,POST,DELETE)GETGETGETGETGETPOSTGETPOSTGETGETGETPUTGETPOSTGETGETGETPUT(GET, POST)Traffic Router: Added a tunable bounded queue to support DNS request processing.
Traffic Ops API Routing Blacklist: via the routing_blacklist field in cdn.conf, enable certain whitelisted Go routes to be handled by Perl instead (via the perl_routes list) in case a regression is found in the Go handler, and explicitly disable any routes via the disabled_routes list. Requests to disabled routes are immediately given a 503 response. Both fields are lists of Route IDs, and route information (ID, version, method, path, and whether or not it can bypass to Perl) can be found by running ./traffic_ops_golang --api-routes. To disable a route or have it bypassed to Perl, find its Route ID using the previous command and put it in the disabled_routes or perl_routes list, respectively.
To support reusing a single riak cluster connection, an optional parameter is added to riak.conf: “HealthCheckInterval”. This options takes a ‘Duration’ value (ie: 10s, 5m) which affects how often the riak cluster is health checked. Default is currently set to: “HealthCheckInterval”: “5s”.
Added a new Go db/admin binary to replace the Perl db/admin.pl script which is now deprecated and will be removed in a future release. The new db/admin binary is essentially a drop-in replacement for db/admin.pl since it supports all of the same commands and options; therefore, it should be used in place of db/admin.pl for all the same tasks.
Added an API 1.4 endpoint, /api/1.4/cdns/dnsseckeys/refresh, to perform necessary behavior previously served outside the API under /internal.
Added the DS Record text to the cdn dnsseckeys endpoint in 1.4.
Added monitoring.json snapshotting. This stores the monitoring json in the same table as the crconfig snapshot. Snapshotting is now required in order to push out monitoring changes.
To traffic_ops_ort.pl added the ability to handle ##OVERRIDE## delivery service ANY_MAP raw remap text to replace and comment out a base delivery service remap rules. THIS IS A TEMPORARY HACK until versioned delivery services are implemented.
Snapshotting the CRConfig now deletes HTTPS certificates in Riak for delivery services which have been deleted in Traffic Ops.
Added a context menu in place of the “Actions” column from the following tables in Traffic Portal: cache group tables, CDN tables, delivery service tables, parameter tables, profile tables, server tables.
Traffic Portal standalone Dockerfile
In Traffic Portal, removes the need to specify line breaks using __RETURN__ in delivery service edge/mid header rewrite rules, regex remap expressions, raw remap text and traffic router additional request/response headers.
In Traffic Portal, provides the ability to clone delivery service assignments from one cache to another cache of the same type. Issue #2963.
Added an API 1.4 endpoint, /api/1.4/server_capabilities, to create, read, and delete server capabilities.
Traffic Ops now allows each delivery service to have a set of query parameter keys to be retained for consistent hash generation by Traffic Router.
In Traffic Portal, delivery service table columns can now be rearranged and their visibility toggled on/off as desired by the user. Hidden table columns are excluded from the table search. These settings are persisted in the browser.
Added an API 1.4 endpoint, /api/1.4/user/login/oauth to handle SSO login using OAuth.
Added /#!/sso page to Traffic Portal to catch redirects back from OAuth provider and POST token into the API.
In Traffic Portal, server table columns can now be rearranged and their visibility toggled on/off as desired by the user. Hidden table columns are excluded from the table search. These settings are persisted in the browser.
Added pagination support to some Traffic Ops endpoints via three new query parameters, limit and offset/page
Traffic Ops now supports a “sortOrder” query parameter on some endpoints to return API responses in descending order
Traffic Ops now uses a consistent format for audit logs across all Go endpoints
Added cache-side config generator, atstccfg, installed with ORT. Includes all configs. Includes a plugin system.
Fixed ATS config generation to omit regex remap, header rewrite, URL Sig, and URI Signing files for delivery services not assigned to that server.
In Traffic Portal, all tables now include a ‘CSV’ link to enable the export of table data in CSV format.
Pylint configuration now enforced (present in a file in the Python client directory)
Added an optional SMTP server configuration to the TO configuration file, api now has unused abilitiy to send emails
Traffic Monitor now has “gbps” calculated stat, allowing operators to monitor bandwidth in Gbps.
Added an API 1.4 endpoint, /api/1.4/deliveryservices_required_capabilities, to create, read, and delete associations between a delivery service and a required capability.
Added ATS config generation omitting parents without Delivery Service Required Capabilities.
In Traffic Portal, added the ability to create, view and delete server capabilities and associate those server capabilities with servers and delivery services. See blueprint
Added validation to prevent assigning servers to delivery services without required capabilities.
Added deep coverage zone routing percentage to the Traffic Portal dashboard.
Added a traffic_ops/app/bin/osversions-convert.pl script to convert the osversions.cfg file from Perl to JSON as part of the /osversions endpoint rewrite.
Added [Experimental] - Emulated Vault suppling a HTTP server mimicking RIAK behavior for usage as traffic-control vault.
Added Traffic Ops Client function that returns a Delivery Service Nullable Response when requesting for a Delivery Service by XMLID
Expires cookie header was not being set properly in responses. Also, added the Max-Age cookie header in responses.null value for that fieldREFUSED DNS answers if the zone priming execution did not complete within the configured zonemanager.init.timeout period.rhi. This defaults to ‘-’ and is only used when EDNS0 client subnet extensions are enabled and a client subnet is present in the request. When enabled and a subnet is present, the subnet appears in the chi field and the resolver address is in the rhi field.POST(GET,POST,PUT)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)GETGETPOSTGETorg_server_fqdn column in the deliveryservice table has been removed. The org_server_fqdn data is now computed from the Delivery Service's primary origin (note: the name of the primary origin is the xml_id of its delivery service).latitude and longitude columns in the cachegroup table have been replaced with coordinate (a foreign key to Coordinate). Coordinates created from Cachegroups are given the name from_cachegroup_\<cachegroup name\>.CLIENT_STEERING delivery services: STEERING_GEO_ORDER and STEERING_GEO_WEIGHT. When targets of these types have an Origin with a Coordinate, Traffic Router will order and prioritize them based upon the shortest total distance from client -> edge -> origin. Co-located targets are grouped together and can be weighted or ordered within the same location using STEERING_GEO_WEIGHT or STEERING_GEO_ORDER, respectively.use_tenancy global parameter and will allow for code to be simplified as a result. If all user and delivery services reference the root tenant, then there will be no difference from having use_tenancy set to 0.localizationMethods field which specifies the localization methods allowed for that cachegroup (currently ‘DEEP_CZ’, ‘CZ’, and ‘GEO’). By default if this field is null/empty, all localization methods are enabled. After Traffic Router has localized a client, it will only route that client to cachegroups that have enabled the localization method used. For example, this can be used to prevent GEO-localized traffic (i.e. most likely from off-net/internet clients) to cachegroups that aren't optimal for internet traffic.traffic_monitor_javainfrastructure/cdn-in-a-box for Apachecon 2018 demonstration(GET)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)(GET)(GET)(GET)(GET)(GET)(GET)(GET)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)(GET)(POST)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)(GET)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)(GET,POST,PUT,DELETE)(GET)(GET)(GET)(GET)(GET,POST,PUT,DELETE)(GET)(GET,POST,PUT,DELETE)