| <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> |
| <HTML> |
| <HEAD> |
| <TITLE>Descriptors and Apache</TITLE> |
| </HEAD> |
| |
| <!-- Background white, links blue (unvisited), navy (visited), red (active) --> |
| <BODY |
| BGCOLOR="#FFFFFF" |
| TEXT="#000000" |
| LINK="#0000FF" |
| VLINK="#000080" |
| ALINK="#FF0000" |
| > |
| <!--#include virtual="header.html" --> |
| <H1 ALIGN="CENTER">Descriptors and Apache</H1> |
| |
| <P>A <EM>descriptor</EM>, also commonly called a <EM>file handle</EM> is |
| an object that a program uses to read or write an open file, or open |
| network socket, or a variety of other devices. It is represented |
| by an integer, and you may be familiar with <CODE>stdin</CODE>, |
| <CODE>stdout</CODE>, and <CODE>stderr</CODE> which are descriptors 0, |
| 1, and 2 respectively. |
| Apache needs a descriptor for each log file, plus one for each |
| network socket that it listens on, plus a handful of others. Libraries |
| that Apache uses may also require descriptors. Normal programs don't |
| open up many descriptors at all, and so there are some latent problems |
| that you may experience should you start running Apache with many |
| descriptors (<EM>i.e.</EM>, with many virtual hosts). |
| |
| <P>The operating system enforces a limit on the number of descriptors |
| that a program can have open at a time. There are typically three limits |
| involved here. One is a kernel limitation, depending on your operating |
| system you will either be able to tune the number of descriptors available |
| to higher numbers (this is frequently called <EM>FD_SETSIZE</EM>). Or you |
| may be stuck with a (relatively) low amount. The second limit is called |
| the <EM>hard resource</EM> limit, and it is sometimes set by root in an |
| obscure operating system file, but frequently is the same as the kernel |
| limit. The third limit is called the <EM>soft |
| resource</EM> limit. The soft limit is always less than or equal to |
| the hard limit. For example, the hard limit may be 1024, but the soft |
| limit only 64. Any user can raise their soft limit up to the hard limit. |
| Root can raise the hard limit up to the system maximum limit. The soft |
| limit is the actual limit that is used when enforcing the maximum number |
| of files a process can have open. |
| |
| <P>To summarize: |
| |
| <CENTER><PRE> |
| #open files <= soft limit <= hard limit <= kernel limit |
| </PRE></CENTER> |
| |
| <P>You control the hard and soft limits using the <CODE>limit</CODE> (csh) |
| or <CODE>ulimit</CODE> (sh) directives. See the respective man pages |
| for more information. For example you can probably use |
| <CODE>ulimit -n unlimited</CODE> to raise your soft limit up to the |
| hard limit. You should include this command in a shell script which |
| starts your webserver. |
| |
| <P>Unfortunately, it's not always this simple. As mentioned above, |
| you will probably run into some system limitations that will need to be |
| worked around somehow. Work was done in version 1.2.1 to improve the |
| situation somewhat. Here is a partial list of systems and workarounds |
| (assuming you are using 1.2.1 or later): |
| |
| <DL> |
| |
| <DT><STRONG>BSDI 2.0</STRONG> |
| <DD>Under BSDI 2.0 you can build Apache to support more descriptors |
| by adding <CODE>-DFD_SETSIZE=nnn</CODE> to |
| <CODE>EXTRA_CFLAGS</CODE> (where nnn is the number of descriptors |
| you wish to support, keep it less than the hard limit). But it |
| will run into trouble if more than approximately 240 Listen |
| directives are used. This may be cured by rebuilding your kernel |
| with a higher FD_SETSIZE. |
| <P> |
| |
| <DT><STRONG>FreeBSD 2.2, BSDI 2.1+</STRONG> |
| <DD>Similar to the BSDI 2.0 case, you should define |
| <CODE>FD_SETSIZE</CODE> and rebuild. But the extra |
| Listen limitation doesn't exist. |
| <P> |
| |
| <DT><STRONG>Linux</STRONG> |
| <DD>By default Linux has a kernel maximum of 256 open descriptors |
| per process. There are several patches available for the |
| 2.0.x series which raise this to 1024 and beyond, and you |
| can find them in the "unofficial patches" section of <A |
| HREF="http://www.linuxhq.com/">the Linux Information HQ</A>. |
| None of these patches are perfect, and an entirely different |
| approach is likely to be taken during the 2.1.x development. |
| Applying these patches will raise the FD_SETSIZE used to compile |
| all programs, and unless you rebuild all your libraries you should |
| avoid running any other program with a soft descriptor limit above |
| 256. As of this writing the patches available for increasing |
| the number of descriptors do not take this into account. On a |
| dedicated webserver you probably won't run into trouble. |
| <P> |
| |
| <DT><STRONG>Solaris through 2.5.1</STRONG> |
| <DD>Solaris has a kernel hard limit of 1024 (may be lower in earlier |
| versions). But it has a limitation that files using |
| the stdio library cannot have a descriptor above 255. |
| Apache uses the stdio library for the ErrorLog directive. |
| When you have more than approximately 110 virtual hosts |
| (with an error log and an access log each) you will need to |
| build Apache with <CODE>-DHIGH_SLACK_LINE=256</CODE> added to |
| <CODE>EXTRA_CFLAGS</CODE>. You will be limited to approximately |
| 240 error logs if you do this. |
| <P> |
| |
| <DT><STRONG>AIX</STRONG> |
| <DD>AIX version 3.2?? appears to have a hard limit of 128 descriptors. |
| End of story. Version 4.1.5 has a hard limit of 2000. |
| <P> |
| |
| <DT><STRONG>Others</STRONG> |
| <DD>If you have details on another operating system, please submit |
| it through our <A HREF="http://www.apache.org/bug_report.html">Bug |
| Report Page</A>. |
| <P> |
| |
| </DL> |
| |
| <P>In addition to the problems described above there are problems with |
| many libraries that Apache uses. The most common example is the bind |
| DNS resolver library that is used by pretty much every unix, which |
| fails if it ends up with a descriptor above 256. We suspect there |
| are other libraries that similar limitations. So the code as of 1.2.1 |
| takes a defensive stance and tries to save descriptors less than 16 |
| for use while processing each request. This is called the <EM>low |
| slack line</EM>. |
| |
| <P>Note that this shouldn't waste descriptors. If you really are pushing |
| the limits and Apache can't get a descriptor above 16 when it wants |
| it, it will settle for one below 16. |
| |
| <P>In extreme situations you may want to lower the low slack line, |
| but you shouldn't ever need to. For example, lowering it can |
| increase the limits 240 described above under Solaris and BSDI 2.0. |
| But you'll play a delicate balancing game with the descriptors needed |
| to serve a request. Should you want to play this game, the compile |
| time parameter is <CODE>LOW_SLACK_LINE</CODE> and there's a tiny |
| bit of documentation in the header file <CODE>httpd.h</CODE>. |
| |
| <P>Finally, if you suspect that all this slack stuff is causing you |
| problems, you can disable it. Add <CODE>-DNO_SLACK</CODE> to |
| <CODE>EXTRA_CFLAGS</CODE> and rebuild. But please report it to |
| our <A HREF="http://www.apache.org/bug_report.html">Bug |
| Report Page</A> so that |
| we can investigate. |
| |
| <!--#include virtual="footer.html" --> |
| </BODY> |
| </HTML> |