docs/manual/misc/descriptors.html - httpd - Git at Google

 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
 <HTML>
 <HEAD>
 <TITLE>Descriptors and Apache</TITLE>
 </HEAD>

 <!-- Background white, links blue (unvisited), navy (visited), red (active) -->
 <BODY
  BGCOLOR="#FFFFFF"
  TEXT="#000000"
  LINK="#0000FF"
  VLINK="#000080"
  ALINK="#FF0000"
 >
 <!--#include virtual="header.html" -->
 <H1 ALIGN="CENTER">Descriptors and Apache</H1>

 <P>A <EM>descriptor</EM>, also commonly called a <EM>file handle</EM> is
 an object that a program uses to read or write an open file, or open
 network socket, or a variety of other devices.  It is represented
 by an integer, and you may be familiar with <CODE>stdin</CODE>,
 <CODE>stdout</CODE>, and <CODE>stderr</CODE> which are descriptors 0,
 1, and 2 respectively.
 Apache needs a descriptor for each log file, plus one for each
 network socket that it listens on, plus a handful of others.  Libraries
 that Apache uses may also require descriptors.  Normal programs don't
 open up many descriptors at all, and so there are some latent problems
 that you may experience should you start running Apache with many
 descriptors (<EM>i.e.</EM>, with many virtual hosts).

 <P>The operating system enforces a limit on the number of descriptors
 that a program can have open at a time.  There are typically three limits
 involved here.  One is a kernel limitation, depending on your operating
 system you will either be able to tune the number of descriptors available
 to higher numbers (this is frequently called <EM>FD_SETSIZE</EM>).  Or you
 may be stuck with a (relatively) low amount.  The second limit is called
 the <EM>hard resource</EM> limit, and it is sometimes set by root in an
 obscure operating system file, but frequently is the same as the kernel
 limit.  The third limit is called the <EM>soft
 resource</EM> limit.  The soft limit is always less than or equal to
 the hard limit.  For example, the hard limit may be 1024, but the soft
 limit only 64.  Any user can raise their soft limit up to the hard limit.
 Root can raise the hard limit up to the system maximum limit.  The soft
 limit is the actual limit that is used when enforcing the maximum number
 of files a process can have open.

 <P>To summarize:

 <CENTER><PRE>
   #open files  &lt;=  soft limit  &lt;=  hard limit  &lt;=  kernel limit
 </PRE></CENTER>

 <P>You control the hard and soft limits using the <CODE>limit</CODE> (csh)
 or <CODE>ulimit</CODE> (sh) directives.  See the respective man pages
 for more information.  For example you can probably use
 <CODE>ulimit -n unlimited</CODE> to raise your soft limit up to the
 hard limit.  You should include this command in a shell script which
 starts your webserver.

 <P>Unfortunately, it's not always this simple.  As mentioned above,
 you will probably run into some system limitations that will need to be
 worked around somehow.  Work was done in version 1.2.1 to improve the
 situation somewhat.  Here is a partial list of systems and workarounds
 (assuming you are using 1.2.1 or later):

 <DL>

     <DT><STRONG>BSDI 2.0</STRONG>
     <DD>Under BSDI 2.0 you can build Apache to support more descriptors
         by adding <CODE>-DFD_SETSIZE=nnn</CODE> to
         <CODE>EXTRA_CFLAGS</CODE> (where nnn is the number of descriptors
         you wish to support, keep it less than the hard limit).  But it
         will run into trouble if more than approximately 240 Listen
         directives are used.  This may be cured by rebuilding your kernel
         with a higher FD_SETSIZE.
     <P>

     <DT><STRONG>FreeBSD 2.2, BSDI 2.1+</STRONG>
     <DD>Similar to the BSDI 2.0 case, you should define
         <CODE>FD_SETSIZE</CODE> and rebuild.  But the extra
         Listen limitation doesn't exist.
     <P>

     <DT><STRONG>Linux</STRONG>
     <DD>By default Linux has a kernel maximum of 256 open descriptors
         per process.  There are several patches available for the
         2.0.x series which raise this to 1024 and beyond, and you
         can find them in the "unofficial patches" section of <A
         HREF="http://www.linuxhq.com/">the Linux Information HQ</A>.
         None of these patches are perfect, and an entirely different
         approach is likely to be taken during the 2.1.x development.
         Applying these patches will raise the FD_SETSIZE used to compile
         all programs, and unless you rebuild all your libraries you should
         avoid running any other program with a soft descriptor limit above
         256.  As of this writing the patches available for increasing
         the number of descriptors do not take this into account.  On a
         dedicated webserver you probably won't run into trouble.
     <P>

     <DT><STRONG>Solaris through 2.5.1</STRONG>
     <DD>Solaris has a kernel hard limit of 1024 (may be lower in earlier
         versions).  But it has a limitation that files using
         the stdio library cannot have a descriptor above 255.
         Apache uses the stdio library for the ErrorLog directive.
         When you have more than approximately 110 virtual hosts
         (with an error log and an access log each) you will need to
         build Apache with <CODE>-DHIGH_SLACK_LINE=256</CODE> added to
         <CODE>EXTRA_CFLAGS</CODE>.  You will be limited to approximately
         240 error logs if you do this.
     <P>

     <DT><STRONG>AIX</STRONG>
     <DD>AIX version 3.2?? appears to have a hard limit of 128 descriptors.
 	End of story.  Version 4.1.5 has a hard limit of 2000.
     <P>

     <DT><STRONG>Others</STRONG>
     <DD>If you have details on another operating system, please submit
         it through our <A HREF="http://www.apache.org/bug_report.html">Bug
         Report Page</A>.
     <P>

 </DL>

 <P>In addition to the problems described above there are problems with
 many libraries that Apache uses.  The most common example is the bind
 DNS resolver library that is used by pretty much every unix, which
 fails if it ends up with a descriptor above 256.  We suspect there
 are other libraries that similar limitations.  So the code as of 1.2.1
 takes a defensive stance and tries to save descriptors less than 16
 for use while processing each request.  This is called the <EM>low
 slack line</EM>.

 <P>Note that this shouldn't waste descriptors.  If you really are pushing
 the limits and Apache can't get a descriptor above 16 when it wants
 it, it will settle for one below 16.

 <P>In extreme situations you may want to lower the low slack line,
 but you shouldn't ever need to.  For example, lowering it can
 increase the limits 240 described above under Solaris and BSDI 2.0.
 But you'll play a delicate balancing game with the descriptors needed
 to serve a request.  Should you want to play this game, the compile
 time parameter is <CODE>LOW_SLACK_LINE</CODE> and there's a tiny
 bit of documentation in the header file <CODE>httpd.h</CODE>.

 <P>Finally, if you suspect that all this slack stuff is causing you
 problems, you can disable it.  Add <CODE>-DNO_SLACK</CODE> to
 <CODE>EXTRA_CFLAGS</CODE> and rebuild.  But please report it to
 our <A HREF="http://www.apache.org/bug_report.html">Bug
 Report Page</A> so that
 we can investigate.

 <!--#include virtual="footer.html" -->
 </BODY>
 </HTML>
	<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
	<HTML>
	<HEAD>
	<TITLE>Descriptors and Apache</TITLE>
	</HEAD>

	<!-- Background white, links blue (unvisited), navy (visited), red (active) -->
	<BODY
	BGCOLOR="#FFFFFF"
	TEXT="#000000"
	LINK="#0000FF"
	VLINK="#000080"
	ALINK="#FF0000"
	>
	<!--#include virtual="header.html" -->
	<H1 ALIGN="CENTER">Descriptors and Apache</H1>

	<P>A <EM>descriptor</EM>, also commonly called a <EM>file handle</EM> is
	an object that a program uses to read or write an open file, or open
	network socket, or a variety of other devices. It is represented
	by an integer, and you may be familiar with <CODE>stdin</CODE>,
	<CODE>stdout</CODE>, and <CODE>stderr</CODE> which are descriptors 0,
	1, and 2 respectively.
	Apache needs a descriptor for each log file, plus one for each
	network socket that it listens on, plus a handful of others. Libraries
	that Apache uses may also require descriptors. Normal programs don't
	open up many descriptors at all, and so there are some latent problems
	that you may experience should you start running Apache with many
	descriptors (<EM>i.e.</EM>, with many virtual hosts).

	<P>The operating system enforces a limit on the number of descriptors
	that a program can have open at a time. There are typically three limits
	involved here. One is a kernel limitation, depending on your operating
	system you will either be able to tune the number of descriptors available
	to higher numbers (this is frequently called <EM>FD_SETSIZE</EM>). Or you
	may be stuck with a (relatively) low amount. The second limit is called
	the <EM>hard resource</EM> limit, and it is sometimes set by root in an
	obscure operating system file, but frequently is the same as the kernel
	limit. The third limit is called the <EM>soft
	resource</EM> limit. The soft limit is always less than or equal to
	the hard limit. For example, the hard limit may be 1024, but the soft
	limit only 64. Any user can raise their soft limit up to the hard limit.
	Root can raise the hard limit up to the system maximum limit. The soft
	limit is the actual limit that is used when enforcing the maximum number
	of files a process can have open.

	<P>To summarize:

	<CENTER><PRE>
	#open files <= soft limit <= hard limit <= kernel limit
	</PRE></CENTER>

	<P>You control the hard and soft limits using the <CODE>limit</CODE> (csh)
	or <CODE>ulimit</CODE> (sh) directives. See the respective man pages
	for more information. For example you can probably use
	<CODE>ulimit -n unlimited</CODE> to raise your soft limit up to the
	hard limit. You should include this command in a shell script which
	starts your webserver.

	<P>Unfortunately, it's not always this simple. As mentioned above,
	you will probably run into some system limitations that will need to be
	worked around somehow. Work was done in version 1.2.1 to improve the
	situation somewhat. Here is a partial list of systems and workarounds
	(assuming you are using 1.2.1 or later):

	<DL>

	<DT><STRONG>BSDI 2.0</STRONG>
	<DD>Under BSDI 2.0 you can build Apache to support more descriptors
	by adding <CODE>-DFD_SETSIZE=nnn</CODE> to
	<CODE>EXTRA_CFLAGS</CODE> (where nnn is the number of descriptors
	you wish to support, keep it less than the hard limit). But it
	will run into trouble if more than approximately 240 Listen
	directives are used. This may be cured by rebuilding your kernel
	with a higher FD_SETSIZE.
	<P>

	<DT><STRONG>FreeBSD 2.2, BSDI 2.1+</STRONG>
	<DD>Similar to the BSDI 2.0 case, you should define
	<CODE>FD_SETSIZE</CODE> and rebuild. But the extra
	Listen limitation doesn't exist.
	<P>

	<DT><STRONG>Linux</STRONG>
	<DD>By default Linux has a kernel maximum of 256 open descriptors
	per process. There are several patches available for the
	2.0.x series which raise this to 1024 and beyond, and you
	can find them in the "unofficial patches" section of <A
	HREF="http://www.linuxhq.com/">the Linux Information HQ</A>.
	None of these patches are perfect, and an entirely different
	approach is likely to be taken during the 2.1.x development.
	Applying these patches will raise the FD_SETSIZE used to compile
	all programs, and unless you rebuild all your libraries you should
	avoid running any other program with a soft descriptor limit above
	256. As of this writing the patches available for increasing
	the number of descriptors do not take this into account. On a
	dedicated webserver you probably won't run into trouble.
	<P>

	<DT><STRONG>Solaris through 2.5.1</STRONG>
	<DD>Solaris has a kernel hard limit of 1024 (may be lower in earlier
	versions). But it has a limitation that files using
	the stdio library cannot have a descriptor above 255.
	Apache uses the stdio library for the ErrorLog directive.
	When you have more than approximately 110 virtual hosts
	(with an error log and an access log each) you will need to
	build Apache with <CODE>-DHIGH_SLACK_LINE=256</CODE> added to
	<CODE>EXTRA_CFLAGS</CODE>. You will be limited to approximately
	240 error logs if you do this.
	<P>

	<DT><STRONG>AIX</STRONG>
	<DD>AIX version 3.2?? appears to have a hard limit of 128 descriptors.
	End of story. Version 4.1.5 has a hard limit of 2000.
	<P>

	<DT><STRONG>Others</STRONG>
	<DD>If you have details on another operating system, please submit
	it through our <A HREF="http://www.apache.org/bug_report.html">Bug
	Report Page</A>.
	<P>

	</DL>

	<P>In addition to the problems described above there are problems with
	many libraries that Apache uses. The most common example is the bind
	DNS resolver library that is used by pretty much every unix, which
	fails if it ends up with a descriptor above 256. We suspect there
	are other libraries that similar limitations. So the code as of 1.2.1
	takes a defensive stance and tries to save descriptors less than 16
	for use while processing each request. This is called the <EM>low
	slack line</EM>.

	<P>Note that this shouldn't waste descriptors. If you really are pushing
	the limits and Apache can't get a descriptor above 16 when it wants
	it, it will settle for one below 16.

	<P>In extreme situations you may want to lower the low slack line,
	but you shouldn't ever need to. For example, lowering it can
	increase the limits 240 described above under Solaris and BSDI 2.0.
	But you'll play a delicate balancing game with the descriptors needed
	to serve a request. Should you want to play this game, the compile
	time parameter is <CODE>LOW_SLACK_LINE</CODE> and there's a tiny
	bit of documentation in the header file <CODE>httpd.h</CODE>.

	<P>Finally, if you suspect that all this slack stuff is causing you
	problems, you can disable it. Add <CODE>-DNO_SLACK</CODE> to
	<CODE>EXTRA_CFLAGS</CODE> and rebuild. But please report it to
	our <A HREF="http://www.apache.org/bug_report.html">Bug
	Report Page</A> so that
	we can investigate.

	<!--#include virtual="footer.html" -->
	</BODY>
	</HTML>