| .. _http-transformation-plugin: |
| |
| HTTP Transformation Plugins |
| *************************** |
| |
| .. Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| |
| Transform plugins examine or transform HTTP message body content. For |
| example, transform plugins can: |
| |
| - Append text to HTML documents |
| |
| - Compress images |
| |
| - Do virus checking (on client ``POST`` data or server response data) |
| |
| - Do content-based filtering (filter out HTML documents that contain |
| certain terms or expressions) |
| |
| This chapter explains how to write transform plugins. The following |
| examples are discussed in detail: |
| |
| .. toctree:: |
| :maxdepth: 2 |
| |
| http-transformation-plugin/sample-null-transformation-plugin.en |
| http-transformation-plugin/append-transform-plugin.en |
| http-transformation-plugin/sample-buffered-null-transformation-plugin.en |
| |
| .. _WritingContentTransformPlugin: |
| |
| Writing Content Transform Plugins |
| --------------------------------- |
| |
| Content transformation plugins transform HTTP response content (such as |
| images or HTML documents) and HTTP request content (such as client |
| ``POST`` data). Because the data stream to be transformed is of variable |
| length, these plugins must use a mechanism that passes data from buffer |
| to buffer *and* checks to see if the end of the data stream is reached. |
| This mechanism is provided by virtual connections (``VConnection``\ s) |
| and virtual IO descriptors (``VIO``\ s). |
| |
| A ``VConnection`` is an abstraction for a data pipe that allows its |
| users to perform asynchronous reads and writes without knowing the |
| underlying implementation. A transformation is a specific type of |
| ``VConnection``. A **transformation** connects an input data source and |
| an output data sink; this feature enables it to view and modify all the |
| data passing through it. |
| |
| Transformations can be chained together, one after the other, so that |
| multiple transformations can be performed on the same content. The |
| ``VConnection`` type, ``TSVConn``, is actually a subclass of ``TSCont``, |
| which means that ``VConnection``\ s (and transformations) are |
| continuations. ``VConnection``\ s and transformations can thus exchange |
| events, informing one another that data is available for reading or |
| writing, or that the end of a data stream is reached. |
| |
| A ``VIO`` is a description of an IO operation that is in progress. |
| Every ``VConnection`` has an associated *input VIO* and an associated |
| *output VIO*. When ``VConnection``\ s are transferring data to one |
| another, one ``VConnection``'s input ``VIO`` is another |
| ``VConnection``'s output ``VIO``. A ``VConnection``'s input ``VIO`` is |
| also called its **write ``VIO``** because the input ``VIO`` refers to a |
| write operation performed on the ``VConnection`` itself. Similarly, the |
| outpt ``VIO`` is also called the **read ``VIO``**. For transformations, |
| which are designed to pass data in one direction, you can picture the |
| relationship between the transformation ``VConnection`` and its |
| ``VIO``\ s as follows: |
| |
| **Transformation and its ``VIO``\ s** {#TransformationAndVIOs} |
| |
| .. figure:: /static/images/sdk/vconnection.jpg |
| :alt: A Transformation and its VIOs |
| |
| A Transformation and its VIOs |
| |
| Because the Traffic Server API places transformations directly in the |
| response or request data stream, the transformation ``VConnection`` is |
| responsible only for reading the data from the input buffer, |
| transforming it, and then writing it to the output buffer. The upstream |
| ``VConnection`` writes the incoming data to the transformation's input |
| buffer. In the figure above, `A Transformation and its |
| ``VIO``\ s <#TransformationAndVIOs>`__, the input ``VIO`` describes the |
| progress of the upstream ``VConnection``'s write operation on the |
| transformation, while the output ``VIO`` describes the progress of the |
| transformation's write operation on the output (downstream) |
| ``VConnection``. The **nbytes** value in the ``VIO`` is the total number |
| of bytes to be written. The **ndone** value is the current progress, or |
| the number of bytes that have been written at a specific point in time. |
| |
| When writing a transformation plugin, you must understand implementation |
| as well as the use of ``VConnection``\ s. The *implementor's side* |
| refers to how to implement a ``VConnection`` that others can use. At |
| minimum, a transform plugin creates a transformation that sits in the |
| data stream and must be able to handle the events that the upstream and |
| downstream ``VConnection``\ s send to it. The *user's side* refers to |
| how to use a ``VConnection`` to read or write data. At the very least, |
| transformations output (write) data. |
| |
| .. _transformations: |
| |
| Transformations |
| ~~~~~~~~~~~~~~~ |
| |
| VIOs |
| ~~~~ |
| |
| A ``VIO``*or virtual IO is a description of an in progress IO |
| operation. The ``VIO`` data structure is used by ``VConnection`` users |
| to determine how much progress has been made on a particular IO |
| operation, and to reenable an IO operation when it stalls due to buffer |
| space. ``VConnection`` implementors use ``VIO``\ s to determine the |
| buffer for an IO operation, how much work to do on the IO operation, and |
| which continuation to call back when progress on the IO operation is |
| made. |
| |
| The ``TSVIO`` data structure itself is opaque, but it might have been |
| defined as follows: |
| |
| .. code-block:: c |
| |
| typedef struct { |
| TSCont continuation; |
| TSVConn vconnection; |
| TSIOBufferReader reader; |
| TSMutex mutex; |
| int nbytes; |
| int ndone; |
| } *TSVIO; |
| |
| IO Buffers |
| ~~~~~~~~~~ |
| |
| The **IO buffer** data structure is the building block of the |
| ``VConnection`` abstraction. An IO buffer is composed of a list of |
| buffer blocks which, in turn, point to buffer data. Both the *buffer |
| block* (``TSIOBufferBlock``) and *buffer data* (``TSIOBufferData``) data |
| structures are reference counted so they can reside in multiple buffers |
| at the same time. This makes it extremely efficient to copy data from |
| one IO buffer to another using ``TSIOBufferCopy``, since Traffic Server |
| only needs to copy pointers and adjust reference counts appropriately |
| (instead of actually copying any data). |
| |
| The IO buffer abstraction provides for a single writer and multiple |
| readers. In order for the readers to have no knowledge of each other, |
| they manipulate IO buffers through the\ ``TSIOBufferReader`` data |
| structure. Since only a single writer is allowed, there is no |
| corresponding ``TSIOBufferWriter`` data structure. The writer simply |
| modifies the IO buffer directly. |
| |