| <html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><title>Rivet Internals</title><link rel="stylesheet" type="text/css" href="rivet.css"><meta name="generator" content="DocBook XSL Stylesheets V1.79.1"><link rel="home" href="index.html" title="Apache Rivet 3.0"><link rel="up" href="index.html" title="Apache Rivet 3.0"><link rel="prev" href="help.html" title="Resources - How to Get Help"><link rel="next" href="lazybridge.html" title="Example: the “Lazy” bridge"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Rivet Internals</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="help.html"><img src="images/prev.png" alt="Prev"></a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="lazybridge.html"><img src="images/next.png" alt="Next"></a></td></tr></table></div><div class="section"><div class="titlepage"><div><div><hr><h2 class="title" style="clear: both"><a name="internals"></a>Rivet Internals</h2></div></div></div><p style="width:90%"> |
| This section easily falls out of date, as new code is added, old |
| code is removed, and changes are made. The best place to look |
| is the source code itself. If you are interested in the changes |
| themselves, the Subversion revision control system |
| (<span style="font-family:monospace"><span class="command"><strong>svn</strong></span></span>) can provide you with information about |
| what has been happening with the code. |
| </p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4363"></a>Rivet approach to Apache Multiprocessing Models</h3></div></div></div><p style="width:90%"> |
| The Apache HTTP web server has an extremely modular architecture |
| that made it very popular among web developers. Most of the server |
| features can be implemented in external modules, including some of |
| the way the server interfaces to the operative system. The multiprocessing |
| modules are meant to provide different models for distributing the |
| server workload but also to cope with different operative systems |
| having their specific architectures and services. |
| </p><p style="width:90%"> |
| From the very beginning mod_rivet was designed to work with |
| the <a class="ulink" href="https://httpd.apache.org/docs/2.4/mod/prefork.html" target="_top">prefork MPM</a> |
| MPM (Multi Processing Module) which assumes the OS to have 'fork' capabilities. |
| This prerequisite basically restricted mod_rivet to work only with |
| Unix-like operative systems. Starting with version 3.0 we reorganized |
| mod_rivet to offer a design that could work together with more MPM and |
| hopefully pave the way to support different OS that have no 'fork' |
| call. At the same time we tried to preserve some of the basic |
| features of mod_rivet when working with the prefork MPM, chiefly the feature of |
| the Unix fork system call of 'cloning' a parent process |
| memory into its child, thus allowing fast initialization of interpreters. |
| </p><p style="width:90%"> |
| The central design of mod_rivet now relies on the idea of <span class="quote">“<span class="quote">MPM bridges</span>”</span>, |
| loadable modules that are responsible to adapt the module procedural design to |
| a given class of Apache MPMs. This design is open to the development of more |
| MPM bridges coping with different multi-processing models but also to the development of |
| different approaches to resource consumption and workload balance. By now we have 3 bridges: |
| </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">rivet_prefork_mpm.c: a bridge for the prefork MPM</li><li class="listitem">rivet_worker_mpm.c: a threaded bridge creating a pool of threads |
| each running Tcl interpreters and communicating with the worker MPM threads |
| through a thread safe queue. This bridge is needed by the worker MPM.</li><li class="listitem">rivet_lazy_mpm.c: a threaded bridge where Tcl threads are |
| started <span class="quote">“<span class="quote">on demand</span>”</span>. The bridge creates no threads and Tcl interpreters |
| at start up and only when requests come in Tcl execution threads are created. |
| This bridge is explained in detail in the <a class="xref" href="lazybridge.html" title="Example: the “Lazy” bridge">the section called “Example: the <span class="quote">“<span class="quote">Lazy</span>”</span> bridge”</a>. |
| Since the resource demand at startup is minimal this bridge should suite |
| development machines that go through frequent web server restarts.</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4376"></a>mod_rivet MPM Bridge callbacks</h3></div></div></div><p style="width:90%"> |
| A bridge is a loadable library implementing different ways to handle |
| specific features needed to mod_rivet. It was originally meant as a way |
| to handle the prefork/worker/event MPM specificities that prevented mod_rivet |
| from supporting each of them, at the same time avoiding the need to stuff the |
| code with conditional statements that would have implied useless complexity (an |
| instance of the Apache web server can run only an MPM at a time), |
| error prone programming and performance costs. |
| New bridges could be imagined also to implement different models of workload |
| and resource management (like the resources demanded by the Tcl interpreters). |
| We designed an interface between the core of mod_rivet and its MPM bridges |
| based on a set of functions defined in the rivet_bridge_table structure. |
| </p><pre class="programlisting">typedef struct _mpm_bridge_table { |
| RivetBridge_ServerInit *mpm_server_init; |
| RivetBridge_ChildInit *mpm_child_init; |
| RivetBridge_Request *mpm_request; |
| RivetBridge_Finalize *mpm_finalize; |
| RivetBridge_Exit_Handler *mpm_exit_handler; |
| RivetBridge_Thread_Interp *mpm_thread_interp; |
| } rivet_bridge_table;</pre><p style="width:90%"> |
| </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="emphasis"><em>mpm_server_init</em></span>: pointer to any |
| specific server inititalization function. This field can be NULL |
| if no bridge specific initialization has to be done. The core of |
| mod_rivet runs the <span style="font-family:monospace"><span class="command"><strong>ServerInitScript</strong></span></span> before |
| calling this function.</li><li class="listitem"><span class="emphasis"><em>mpm_child_init</em></span>: Bridge specific |
| child process initialization. If the pointer is assigned with |
| a non-NULL value the function is called by Rivet_ChildInit. |
| </li><li class="listitem"><span class="emphasis"><em>mpm_request</em></span>: This pointer must |
| be assigned with a valid function pointer to the content generator |
| implemented by the bridge. If the pointer is not defined the Apache |
| web server will stop during start up. This condition is motivated by |
| the need of avoiding useless testing of the pointer. The fundamental |
| purpose of a content generator module (like mod_rivet) is to respond |
| to requests creating content, thus whatever it is |
| a content generating function must exist (during the early stages of |
| development you can create a simple test function for that). In a |
| threaded MPM this function typically prepares the request processing |
| stuffing somewhere the pointer to the request_rec structure |
| passed by the web server and then it calls some method to communicate |
| these data to the Tcl execution thread waiting for result to be |
| returned. The <span class="quote">“<span class="quote">prefork</span>”</span> bridge is an exception since there |
| are no multiple threads and the bridge calls directly Rivet_SendContent |
| </li><li class="listitem"><span class="emphasis"><em>mpm_finalize</em></span>: pointer to a finalization |
| function called during a child process exit. This function is registered |
| as child process memory pool cleanup function. If the pointer is NULL |
| the pool is given a default cleanup function (apr_pool_cleanup_null) |
| defined in src/mod_rivet/mod_rivet.c. For instance the finalize function |
| in the <span class="emphasis"><em>worker</em></span> MPM bridge notifies |
| a supervisor thread demanding the whole pool of threads running Tcl |
| interpreters to orderly exit. This pointer can be NULL if the bridge |
| has no special need when a child process must exit (unlikely if you have |
| multiple threads running) |
| </li><li class="listitem"><span class="emphasis"><em>mpm_exit_handler</em></span>: mod_rivet replaces |
| the core <span style="font-family:monospace"><span class="command"><strong>exit</strong></span></span> command with a new one |
| (<span style="font-family:monospace"><span class="command"><strong>::rivet::exit</strong></span></span>). This command must handle |
| the process exit in the best possible way for the bridge and the |
| threading model it implements (for the 2 current threaded bridges this implies |
| signaling the threads to exit). The <span style="font-family:monospace"><span class="command"><strong>::rivet::exit</strong></span></span> |
| actually doesn't terminate the process, but interrupts execution |
| returning a specific error code commands <span style="font-family:monospace"><span class="command"><strong>::rivet::catch</strong></span></span> |
| and <span style="font-family:monospace"><span class="command"><strong>::rivet::try</strong></span></span> can detect. Before the process is terminated |
| the <span style="font-family:monospace"><span class="command"><strong>AbortScript</strong></span></span> script is fired and <span style="font-family:monospace"><span class="command"><strong>::rivet::abort_code</strong></span></span> |
| returns a message describing the exit condition. For instance |
| the <span class="emphasis"><em>worker</em></span> MPM bridge the finalize function |
| is called after the current thread itself is set up for termination. |
| See function Rivet_ExitCmd in |
| <a class="ulink" href="https://svn.apache.org/repos/asf/tcl/rivet/trunk/src/mod_rivet_ng/rivetCore.c" target="_top">rivetCore.c</a> |
| to have details on how and at what stage this callback is invoked. |
| </li><li class="listitem"><span class="emphasis"><em>mpm_thread_interp</em></span> must be a function returning |
| the interpreter object (a pointer to record of type |
| <span style="font-family:monospace"><span class="command"><strong>rivet_thread_interp</strong></span></span>) associated |
| to a given configuration as stored in a <span style="font-family:monospace"><span class="command"><strong>rivet_server_conf*</strong></span></span> |
| object. This element was temporarily introduced in the |
| <span style="font-family:monospace"><span class="command"><strong>mpm_bridge_table</strong></span></span> table and should be accessed |
| through the macro RIVET_PEEK_INTERP. |
| <pre class="programlisting">interp_obj = RIVET_PEEK_INTERP(private,private->conf);</pre> |
| Every bridge implementation should have its own way to store interpreter data and manage their |
| status. So this macro (and associated function) should hide from the module core function |
| the specific approach followed in a particular bridge |
| </li></ul></div><p style="width:90%"> |
| </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4410"></a>Server Initialization and MPM Bridge</h3></div></div></div><p style="width:90%"> |
| </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4413"></a>RivetChan</h3></div></div></div><p style="width:90%"> |
| The <span class="structname">RivetChan</span> system was created in |
| order to have an actual Tcl channel that we could redirect |
| standard output to. This enables us use, for instance, the |
| regular <span style="font-family:monospace"><span class="command"><strong>puts</strong></span></span> command in .rvt pages. It |
| works by creating a channel that buffers output, and, at |
| predetermined times, passes it on to Apache's I/O system. |
| Tcl's regular standard output is replaced with an instance of |
| this channel type, so that, by default, output will go to the |
| web page. |
| </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4418"></a>The <span style="font-family:monospace"><span class="command"><strong>global</strong></span></span> Command</h3></div></div></div><p style="width:90%"> |
| Rivet aims to run standard Tcl code with as few surprises as |
| possible. At times this involves some compromises - in this |
| case regarding the <span style="font-family:monospace"><span class="command"><strong>global</strong></span></span> command. The |
| problem is that the command will create truly global |
| variables. If the user is just cut'n'pasting some Tcl code |
| into Rivet, they most likely just want to be able to share the |
| variable in question with other procs, and don't really care |
| if the variable is actually persistant between pages. The |
| solution we have created is to create a proc |
| <span style="font-family:monospace"><span class="command"><strong>::request::global</strong></span></span> that takes the place of |
| the <span style="font-family:monospace"><span class="command"><strong>global</strong></span></span> command in Rivet templates. If |
| you really need a true global variable, use either |
| <span style="font-family:monospace"><span class="command"><strong>::global</strong></span></span> or add the :: namespace qualifier |
| to variables you wish to make global. |
| </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4426"></a>Page Parsing, Execution and Caching</h3></div></div></div><p style="width:90%"> |
| When a Rivet page is requested, it is transformed into an |
| ordinary Tcl script by parsing the file for the <? ?> |
| processing instruction tags. Everything outside these tags |
| becomes a large <span style="font-family:monospace"><span class="command"><strong>puts</strong></span></span> statement, and |
| everything inside them remains Tcl code. |
| </p><p style="width:90%"> |
| Each .rvt file is evaluated in its own |
| <code class="constant">::request</code> namespace, so that it is not |
| necessary to create and tear down interpreters after each |
| page. By running in its own namespace, though, each page will |
| not run afoul of local variables created by other scripts, |
| because they will be deleted automatically when the namespace |
| goes away after Apache finishes handling the request. |
| </p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"> |
| One current problem with this system is that while |
| variables are garbage collected, file handles are not, so |
| that it is very important that Rivet script authors make |
| sure to close all the files they open. |
| </td></tr></table></div><p style="width:90%"> |
| </p><p style="width:90%"> |
| After a script has been loaded and parsed into it's "pure Tcl" |
| form, it is also cached, so that it may be used in the future |
| without having to reload it (and re-parse it) from the disk. |
| The number of scripts stored in memory is configurable. This |
| feature can significantly improve performance. |
| </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4434"></a>Extending Rivet by developing C code procedures</h3></div></div></div><p style="width:90%"> |
| Rivet endows the Tcl interpreter with new commands |
| serving as interface between the application layer and the |
| Apache web server. Many of these commands |
| are meaningful only when a HTTP request is under way and |
| therefore a request_rec object allocated by the framework |
| is existing and was passed to mod_rivet as argument of a callback. |
| In case commands have to gain access to a valid request_rec |
| object the C procedure must check if such |
| a pointer exists and it's initialized |
| with valid data. For this purpose the procedure handling requests |
| (Rivet_SendContent) makes a copy of such pointer and keeps it |
| in an internal structure. The copy is set to NULL just before |
| returning to the framework, right after mod_rivet's has |
| carried out its request processing. When the pointer copy is NULL |
| the module is outside any request processing and this |
| condition invalidates the execution of |
| many of the Rivet commands. In case they are called |
| (for example in a ChildInitScript, GlobalInitScript, |
| ServerInitScript or ChildExitScript) they fail with a Tcl error |
| you can handle with a <span style="font-family:monospace"><span class="command"><strong>catch</strong></span></span> command. |
| </p><p style="width:90%"> |
| For this purpose in src/rivet.h the macro |
| CHECK_REQUEST_REC was defined accepting two arguments: the thread |
| private data object and the command name. If the pointer is NULL |
| the macro calls Tcl_NoRequestRec and returns TCL_ERROR |
| causing the command to fail. These are the steps to follow |
| in order to write a new C language command for mod_rivet |
| </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"> |
| Define the command and associated C language procedure |
| in src/mod_rivet_ng/rivetCore.c using the macro |
| RIVET_OBJ_CMD<pre class="programlisting">RIVET_OBJ_CMD("mycmd",Rivet_MyCmd,private)</pre> |
| This macro ensures the command is defined as <span style="font-family:monospace"><span class="command"><strong>::rivet::mycmd</strong></span></span> |
| and its ClientData pointer is defined with the thread private data |
| </li><li class="listitem"> |
| Add the code of Rivet_MyCmd to src/mod_rivet_ng/rivetCore.c (in case |
| the code resides in a different file also src/Makefile.am should be |
| changed to tell the build system how to compile the code and |
| link it into mod_rivet.so) |
| </li><li class="listitem"> |
| If the code must have access to the request record in <span style="font-family:monospace"><span class="command"><strong>private->r</strong></span></span> |
| use the macro THREAD_PRIVATE_DATA in order to claim the thread private data, then |
| check for the validity of the pointer using the macro |
| CHECK_REQUEST_REC(private,"::rivet::<cmd_name>") |
| |
| <pre class="programlisting">TCL_CMD_HEADER(Rivet_MyCmd) |
| { |
| /* we have to get the thread private data */ |
| |
| THREAD_PRIVATE_DATA(private) |
| |
| /* if ::rivet::mycmd works within a request processing we have |
| * to check if 'private' is bringing a non null request_rec pointer |
| */ |
| |
| CHECK_REQUEST_REC(private,"::rivet::mycmd"); |
| .... |
| |
| return TCL_OK; |
| }</pre></li><li class="listitem"> |
| Add a test for this command in tests/checkfails.tcl. For |
| instance |
| <pre class="programlisting">... |
| check_fail no_body |
| check_fail virtual_filename unkn |
| check_fail my_cmd <arg1> <arg2> |
| ....</pre> |
| Where <arg1> <arg2> are optional |
| arguments in case the command has different forms depending on |
| the arguments. Then, if <span style="font-family:monospace"><span class="command"><strong>::rivet::mycmd</strong></span></span> must fail also |
| tests/failtest.tcl should modified as |
| <pre class="programlisting">virtual_filename->1 |
| mycmd->1</pre> |
| The value associated to the test must be 0 in case the |
| command doesn't need to test the <span style="font-family:monospace"><span class="command"><strong>private->r</strong></span></span> pointer. |
| </li></ul></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4458"></a>Debugging Rivet and Apache</h3></div></div></div><p style="width:90%"> |
| If you are interested in hacking on Rivet, you're welcome to |
| contribute! Invariably, when working with code, things go |
| wrong, and it's necessary to do some debugging. In a server |
| environment like Apache, it can be a bit more difficult to |
| find the right way to do this. Here are some techniques to |
| try. |
| </p><p style="width:90%"> |
| The first thing you should know is that Apache can be launched |
| as a <span class="emphasis"><em>single process</em></span> with the |
| -X argument: |
| </p><pre class="programlisting">httpd -X</pre>. |
| |
| <p style="width:90%"> |
| On Linux, one of the first things to try is the system call |
| tracer, <span style="font-family:monospace"><span class="command"><strong>strace</strong></span></span>. You don't even have to |
| recompile Rivet or Apache for this to work. |
| </p><pre class="programlisting">strace -o /tmp/outputfile -S 1000 httpd -X</pre><p style="width:90%"> |
| This command will run httpd in the system call tracer, |
| which leaves its output (there is potentially a lot of it) in |
| <code class="filename">/tmp/outputfile</code>. The -S |
| option tells <span style="font-family:monospace"><span class="command"><strong></strong></span></span>strace to only record the |
| first 1000 bytes of a syscall. Some calls such as |
| <code class="function">write</code> can potentially be much longer than |
| this, so you may want to increase this number. The results |
| are a list of all the system calls made by the program. You |
| want to look at the end, where the failure presumably occured, |
| to see if you can find anything that looks like an error. If |
| you're not sure what to make of the results, you can always |
| ask on the Rivet development mailing list. |
| </p><p style="width:90%"> |
| If <span style="font-family:monospace"><span class="command"><strong>strace</strong></span></span> (or its equivalent on your |
| operating system) doesn't answer your question, it may be time |
| to debug Apache and Rivet. To do this, you will need to rebuild mod_rivet. |
| First of all you have to configure the build by running the |
| <span style="font-family:monospace"><span class="command"><strong>./configure</strong></span></span> script with the |
| -enable-symbols option and after you have |
| set the CFLAGS and LDFLAGS environment variables |
| </p><pre class="programlisting">export CFLAGS="-g -O0" |
| export LDFLAGS="-g" |
| ./configure --enable-symbols ...... |
| make |
| make install</pre><p style="width:90%"> |
| Arguments to <span style="font-family:monospace"><span class="command"><strong>./configure</strong></span></span> must fit your Apache HTTP |
| web server installation. See the output produced by |
| </p><pre class="programlisting">./configure --help</pre><p style="width:90%"> |
| And check the <a class="xref" href="installation.html" title="Apache Rivet 3.0 Installation">the section called “Apache Rivet 3.0 Installation”</a> page to |
| have further information. |
| Since it's easier to debug a single process, we'll still run |
| Apache in single process mode with -X: |
| </p><pre class="programlisting">@ashland [~] $ gdb /usr/sbin/apache.dbg |
| GNU gdb 5.3-debian |
| Copyright 2002 Free Software Foundation, Inc. |
| GDB is free software, covered by the GNU General Public License, and you are |
| welcome to change it and/or distribute copies of it under certain conditions. |
| Type "show copying" to see the conditions. |
| There is absolutely no warranty for GDB. Type "show warranty" for details. |
| This GDB was configured as "powerpc-linux"... |
| (gdb) run -X |
| Starting program: /usr/sbin/apache.dbg -X |
| [New Thread 16384 (LWP 13598)] |
| . |
| . |
| .</pre><p style="width:90%"> |
| When your apache session is up and running, you can request a |
| web page with the browser, and see where things go wrong (if |
| you are dealing with a crash, for instance). |
| </p></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="help.html"><img src="images/prev.png" alt="Prev"></a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> <a accesskey="n" href="lazybridge.html"><img src="images/next.png" alt="Next"></a></td></tr><tr><td width="40%" align="left" valign="top">Resources - How to Get Help </td><td width="20%" align="center"><a accesskey="h" href="index.html"><img src="images/home.png" alt="Home"></a></td><td width="40%" align="right" valign="top"> Example: the <span class="quote">“<span class="quote">Lazy</span>”</span> bridge</td></tr></table></div></body></html> |
