| .. Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| |
| .. include:: ../common.defs |
| .. default-domain:: c |
| .. _layer-4-routing: |
| |
| Layer 4 Routing |
| ************************** |
| |
| |TS| supports a limited set of layer 4 routing options. In such use |TS| acts effectively as a |
| router, moving network data between two endpoints without modifying the data. The routing is |
| accomplished by examining the initial data from the inbound connection to decide the outbound |
| destination. The initial data is then sent to the destination and subsequently |TS| forwards all |
| data read on one connection to the other and vice versa. |
| |
| .. figure:: ../uml/images/l4-basic-sequence.svg |
| :align: center |
| |
| In this way it acts similarly to `nc <https://linux.die.net/man/1/nc>`__. |
| |
| The primary differences between different types of layer 4 routing is the mechanism by which |TS| |
| creates the outbound connection. This is described in detail in the type specific documentation. |
| |
| Transparency |
| ============ |
| |
| Transparency is in some sense layer 4 routing because the outbound connection is determined by |
| examining the destination address in the client TCP packets. This is discussed in detail |
| :ref:`elsewhere <transparent-proxy>`. |
| |
| .. _sni-routing: |
| |
| SNI Routing |
| =========== |
| |
| Currently the only directly supported layer 4 routing (as of version 8.0) is SNI based routing. This |
| imposes the requirement on the traffic that the inbound connection must be TLS. |
| |
| SNI routing is configured by :file:`sni.yaml`. |
| |
| If SNI Routing is enabled the initial "`CLIENT HELLO |
| <https://tools.ietf.org/html/rfc5246#section-7.4.1.2>`__" data of an inbound TLS connection is |
| examined to extract the "`SNI <https://tools.ietf.org/html/rfc3546#section-3.1>`__" value. This is |
| matched against the configuration data to select an action for the inbound connection. In this case |
| the option of interest is ``tunnel_route``. If this is set then |TS| synthesizes an HTTP ``CONNECT`` |
| request to itself with the ``tunnel_route`` host and port as the upstream. That is, the inbound |
| connection is treated as if the user agent had sent a |
| ``CONNECT`` to the upstream and forwards the "`CLIENT HELLO |
| <https://tools.ietf.org/html/rfc5246#section-7.4.1.2>`__" to it. In addition to the method appearing |
| as ``CONNECT``, be aware that logs printing the URL via the ``<%cquc>`` field format will show the |
| scheme in the URL as ``tunnel``. The scheme as printed via ``<%cqus>``, however, will show the |
| scheme used in the original client request. |
| |
| Example |
| ------- |
| |
| Consider a Content Delivery Network (CDN) that has an edge layer of externally facing |TS| |
| instances. The goal is to enable external clients to connect to internal services and do their own |
| client certificate verification, possibly because distribution of private keys to the edge |TS| |
| instances is too difficult or too risky. To achieve this, the edge |TS| instances can be configured |
| to route inbound TLS connections with specific SNI values directly to the internal services without |
| TLS termination on the edge. This enables the edge to provide controlled external access to the |
| internal services without each internal service having to stand up its own edge. Note the services |
| do not require global routable addresses as long as the edge |TS| instances can route to the |
| services. |
| |
| The basic set up is therefore |
| |
| .. figure:: ../uml/images/l4-example-cdn-layout.svg |
| :align: center |
| |
| A Client connects to an edge |TS| which forwards the connection to the internal Service. |
| The Client then negotiates TLS with the Service. |
| |
| |
| For the example, let us define two services inside the corporate network of Example, Inc. |
| ``service-1`` and ``service-2``. ``service-1`` is on port 443 on host ``app-server-29`` while |
| ``service-2`` is on port 4443 on host ``app-server-56``. The SNI routing set up for this would be |
| |
| ========================== ===================================== |
| SNI value Destination |
| ========================== ===================================== |
| service-1.example.com app-server-29:443 |
| service-2.example.com app-server-56:4443 |
| ========================== ===================================== |
| |
| The :file:`sni.yaml` contents would be |
| |
| .. code:: yaml |
| |
| |
| sni: |
| - tunnel_route: app-server-29:443 |
| fqdn: service-1.example.com |
| |
| - tunnel_route: app-server-56:4443 |
| fqdn: service-2.example.com |
| |
| In addition to this, in the :file:`records.config` file, edit ``connect_ports`` like so: |
| |
| - :ts:cv:`proxy.config.http.connect_ports`: ``443 4443`` to allow |TS| to connect |
| to the destination port |
| |
| The sequence of network activity for a Client connecting to ``service-2`` is |
| |
| .. figure:: ../uml/images/l4-sni-routing-seq.svg |
| :align: center |
| |
| Note the destination for the outbound TCP connection and the HTTP ``CONNECT`` is the same. If this |
| is a problem (which it will be in general) a plugin is needed to change the URL in the ``CONNECT``. |
| In this case the proxy request is available in the :c:macro:`TS_HTTP_TXN_START_HOOK` hook. This |
| cannot be done using remap because for a ``CONNECT`` there is no remap phase. Note that for a |
| tunneled connection like this, the only transaction hooks that will be triggered are |
| :c:macro:`TS_HTTP_TXN_START_HOOK` and :c:macro:`TS_HTTP_TXN_CLOSE_HOOK`. In addition, because |TS| |
| does not terminate (and therefore does not decrypt) the connection, it cannot be cached or served from |
| cache. |
| |
| .. _pre-warming-tls-tunnel: |
| |
| Pre-warming TLS Tunnel |
| ====================== |
| |
| Pre-warming TLS Tunnel reduces the latency of TLS connections (``forward_route`` and ``partial_blind_route`` type SNI |
| Routing). When this feature is enabled, each ET_NET thread makes TLS connections pool per routing type, SNI, and ALPN. |
| |
| .. figure:: ../uml/images/l4-pre-warming-overview.svg |
| :align: center |
| |
| Stats for connection pools are registered dynamically on start up. Details in :ref:`pre-warming-tls-tunnel-stats`. |
| |
| Examples |
| -------- |
| |
| .. code:: yaml |
| |
| sni: |
| - fqdn: foo.com |
| http2: off |
| partial_blind_route: bar.com |
| client_sni_policy: server_name |
| tunnel_prewarm: true |
| tunnel_prewarm_connect_timeout: 10 |
| tunnel_prewarm_inactive_timeout: 150 |
| tunnel_prewarm_max: 100 |
| tunnel_prewarm_min: 10 |
| tunnel_alpn: |
| - h2 |
| |
| .. code:: |
| |
| proxy.process.tunnel.prewarm.bar.com:443.tls.current_init 0 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.current_open 10 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.total_hit 0 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.total_miss 0 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.total_handshake_time 1106250000 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.total_handshake_count 10 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.total_retry 0 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.http2.current_init 0 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.http2.current_open 10 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.http2.total_hit 0 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.http2.total_miss 0 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.http2.total_handshake_time 1142368000 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.http2.total_handshake_count 10 |
| proxy.process.tunnel.prewarm.bar.com:443.tls.http2.total_retry 0 |