src/tools/backend/index.html - hawq - Git at Google

 <!-- src/tools/backend/index.html -->
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <meta name="generator"
 content="HTML Tidy for BSD/OS (vers 1st July 2002), see www.w3.org" />
 <title>How PostgreSQL Processes a Query</title>
 </head>
 <body bgcolor="#FFFFFF" text="#000000" link="#FF0000"
 vlink="#A00000" alink="#0000FF">
 <h1>How PostgreSQL Processes a Query</h1>

 <h2>by Bruce Momjian</h2>

 <center>
 <h3><i>Click on an item</i> to see more detail or look at the full
 <a href="backend_dirs.html">index.</a></h3>

 <p><img src="flow.gif" usemap="#flowmap" alt="flowchart" />

 <map name="flowmap" id="flowmap">
 <area coords="45, 0, 175, 30" href="backend_dirs.html#main" alt="main" />
 <area coords="255, 35, 380, 65" href="backend_dirs.html#libpq" alt="libpq" />
 <area coords="45,  65,  175,  95" href="backend_dirs.html#postmaster" alt="postmaster" />
 <area coords="45, 130, 175, 160" href="backend_dirs.html#tcop" alt="tcop" />
 <area coords="250, 130, 380, 160" href="backend_dirs.html#tcop" alt="tcop" />
 <area coords="45, 205, 175, 240" href="backend_dirs.html#parser" alt="parser" />
 <area coords="45, 270, 175, 300" href="backend_dirs.html#tcop" alt="tcop" />
 <area coords="255, 270, 380, 300" href="backend_dirs.html#commands" alt="commands" />
 <area coords="45, 335, 175, 365" href="backend_dirs.html#rewrite" alt="rewrite" />
 <area coords="45, 400, 175, 430" href="backend_dirs.html#optimizer_path" alt="path" />
 <area coords="45, 460, 175, 490" href="backend_dirs.html#optimizer_plan" alt="plan" />
 <area coords="45, 525, 175, 555" href="backend_dirs.html#executor" alt="executor" />
 <area coords="0, 640, 130, 675" href="backend_dirs.html#utils" alt="utils" />
 <area coords="175, 640, 300, 675" href="backend_dirs.html#catalog" alt="catalog" />
 <area coords="330, 640, 475, 675" href="backend_dirs.html#storage" alt="storage" />
 <area coords="75, 700, 210, 735" href="backend_dirs.html#access" alt="access" />
 <area coords="255, 705, 380, 735" href="backend_dirs.html#nodes" alt="nodes" />
 </map>
 </center>

 <br />

 <p>A query comes to the backend via data packets arriving through
 TCP/IP or Unix Domain sockets. It is loaded into a string, and
 passed to the <a href="../../backend/parser">parser,</a> where the
 lexical scanner, <a href="../../backend/parser/scan.l">scan.l,</a>
 breaks the query up into tokens(words). The parser uses <a
 href="../../backend/parser/gram.y">gram.y</a> and the tokens to
 identify the query type, and load the proper query-specific
 structure, like <a
 href="../../include/nodes/parsenodes.h">CreateStmt</a> or <a
 href="../../include/nodes/parsenodes.h">SelectStmt.</a></p>

 <p>The statement is then identified as complex (<i>SELECT / INSERT /
 UPDATE / DELETE</i>) or a simple, e.g <i> CREATE USER, ANALYZE, </i>,
 etc.  Simple utility commands are processed by statement-specific
 functions in <a href="../../backend/commands">backend/commands.</a>
 Complex statements require more handling.</p>

 <p>The parser takes a complex query, and creates a <a
 href="../../include/nodes/parsenodes.h">Query</a> structure that
 contains all the elements used by complex queries. Query.qual holds
 the <i>WHERE</i> clause qualification, which is filled in by <a
 href="../../backend/parser/parse_clause.c">transformWhereClause().</a>
 Each table referenced in the query is represented by a <a
 href="../../include/nodes/parsenodes.h">RangeTableEntry,</a> and
 they are linked together to form the <i>range table</i> of the
 query, which is generated by <a
 href="../../backend/parser/parse_clause.c">transformFromClause().</a>
 Query.rtable holds the query's range table.</p>

 <p>Certain queries, like <i>SELECT,</i> return columns of data.
 Other queries, like <i>INSERT</i> and <i>UPDATE,</i> specify the
 columns modified by the query. These column references are
 converted to <a
 href="../../include/nodes/primnodes.h">TargetEntry</a> entries,
 which are linked together to make up the <i>target list</i> of the
 query. The target list is stored in Query.targetList, which is
 generated by <a
 href="../../backend/parser/parse_target.c">transformTargetList().</a></p>

 <p>Other query elements, like aggregates(<i>SUM()</i>), <i>GROUP
 BY,</i> and <i>ORDER BY</i> are also stored in their own Query
 fields.</p>

 <p>The next step is for the Query to be modified by any
 <i>VIEWS</i> or <i>RULES</i> that may apply to the query. This is
 performed by the <a href="../../backend/rewrite">rewrite</a>
 system.</p>

 <p>The <a href="../../backend/optimizer">optimizer</a> takes the
 Query structure and generates an optimal <a
 href="../../include/nodes/plannodes.h">Plan,</a> which contains the
 operations to be performed to execute the query. The <a
 href="../../backend/optimizer/path">path</a> module determines the
 best table join order and join type of each table in the
 RangeTable, using Query.qual(<i>WHERE</i> clause) to consider
 optimal index usage.</p>

 <p>The Plan is then passed to the <a
 href="../../backend/executor">executor</a> for execution, and the
 result returned to the client. The Plan is actually as set of nodes,
 arranged in a tree structure with a top-level node, and various
 sub-nodes as children.</p>

 <p>There are many other modules that support this basic
 functionality. They can be accessed by clicking on the
 flowchart.</p>

 <hr />
 <p>Another area of interest is the shared memory area, which
 contains data accessable to all backends. It has recently used
 data/index blocks, locks, backend process information, and lookup
 tables for these structures:</p>

 <ul>
 <li>ShmemIndex - lookup shared memory addresses using structure
 names</li>

 <li><a href="../../include/storage/buf_internals.h">Buffer
 Descriptor</a> - control header for buffer cache block</li>

 <li><a href="../../include/storage/buf_internals.h">Buffer
 Block</a> - data/index buffer cache block</li>

 <li>Shared Buffer Lookup Table - lookup of buffer cache block
 addresses using table name and block number( <a
 href="../../include/storage/buf_internals.h">BufferTag</a>)</li>

 <li>Lock Manager Tables (lock hash) - the <a
 href="../../include/storage/lock.h">LOCK</a> structure, looked up
 using a <a href="../../include/storage/lock.h">LOCKTAG</a>.
 A LOCK structure exists for each lockable object that is currently
 locked by any backend.  Also, there is a subsidiary <a
 href="../../include/storage/lock.h">PROCLOCK</a> structure for each
 backend currently interested in a given LOCK</li>

 <li><a href="../../include/storage/proc.h">PGPROC Structures</a> -
 information about each backend, including locks held/waiting</li>
 </ul>

 <p>Each data structure is created by calling <a
 href="../../backend/storage/ipc/shmem.c">ShmemInitStruct(),</a> and
 the lookups are created by <a
 href="../../backend/storage/ipc/shmem.c">ShmemInitHash().</a></p>

 <hr />
 <small>Maintainer: Bruce Momjian (<a
 href="mailto:pgman@candle.pha.pa.us">pgman@candle.pha.pa.us</a>)<br />

 Last updated: Fri May  6 14:22:27 EDT 2005</small>
 </body>
 </html>
	<!-- src/tools/backend/index.html -->
	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
	<html xmlns="http://www.w3.org/1999/xhtml">
	<head>
	<meta name="generator"
	content="HTML Tidy for BSD/OS (vers 1st July 2002), see www.w3.org" />
	<title>How PostgreSQL Processes a Query</title>
	</head>
	<body bgcolor="#FFFFFF" text="#000000" link="#FF0000"
	vlink="#A00000" alink="#0000FF">
	<h1>How PostgreSQL Processes a Query</h1>

	<h2>by Bruce Momjian</h2>

	<center>
	<h3><i>Click on an item</i> to see more detail or look at the full
	<a href="backend_dirs.html">index.</a></h3>

	<p><img src="flow.gif" usemap="#flowmap" alt="flowchart" />

	<map name="flowmap" id="flowmap">
	<area coords="45, 0, 175, 30" href="backend_dirs.html#main" alt="main" />
	<area coords="255, 35, 380, 65" href="backend_dirs.html#libpq" alt="libpq" />
	<area coords="45, 65, 175, 95" href="backend_dirs.html#postmaster" alt="postmaster" />
	<area coords="45, 130, 175, 160" href="backend_dirs.html#tcop" alt="tcop" />
	<area coords="250, 130, 380, 160" href="backend_dirs.html#tcop" alt="tcop" />
	<area coords="45, 205, 175, 240" href="backend_dirs.html#parser" alt="parser" />
	<area coords="45, 270, 175, 300" href="backend_dirs.html#tcop" alt="tcop" />
	<area coords="255, 270, 380, 300" href="backend_dirs.html#commands" alt="commands" />
	<area coords="45, 335, 175, 365" href="backend_dirs.html#rewrite" alt="rewrite" />
	<area coords="45, 400, 175, 430" href="backend_dirs.html#optimizer_path" alt="path" />
	<area coords="45, 460, 175, 490" href="backend_dirs.html#optimizer_plan" alt="plan" />
	<area coords="45, 525, 175, 555" href="backend_dirs.html#executor" alt="executor" />
	<area coords="0, 640, 130, 675" href="backend_dirs.html#utils" alt="utils" />
	<area coords="175, 640, 300, 675" href="backend_dirs.html#catalog" alt="catalog" />
	<area coords="330, 640, 475, 675" href="backend_dirs.html#storage" alt="storage" />
	<area coords="75, 700, 210, 735" href="backend_dirs.html#access" alt="access" />
	<area coords="255, 705, 380, 735" href="backend_dirs.html#nodes" alt="nodes" />
	</map>
	</center>

	<br />

	<p>A query comes to the backend via data packets arriving through
	TCP/IP or Unix Domain sockets. It is loaded into a string, and
	passed to the <a href="../../backend/parser">parser,</a> where the
	lexical scanner, <a href="../../backend/parser/scan.l">scan.l,</a>
	breaks the query up into tokens(words). The parser uses <a
	href="../../backend/parser/gram.y">gram.y</a> and the tokens to
	identify the query type, and load the proper query-specific
	structure, like <a
	href="../../include/nodes/parsenodes.h">CreateStmt</a> or <a
	href="../../include/nodes/parsenodes.h">SelectStmt.</a></p>

	<p>The statement is then identified as complex (<i>SELECT / INSERT /
	UPDATE / DELETE</i>) or a simple, e.g <i> CREATE USER, ANALYZE, </i>,
	etc. Simple utility commands are processed by statement-specific
	functions in <a href="../../backend/commands">backend/commands.</a>
	Complex statements require more handling.</p>

	<p>The parser takes a complex query, and creates a <a
	href="../../include/nodes/parsenodes.h">Query</a> structure that
	contains all the elements used by complex queries. Query.qual holds
	the <i>WHERE</i> clause qualification, which is filled in by <a
	href="../../backend/parser/parse_clause.c">transformWhereClause().</a>
	Each table referenced in the query is represented by a <a
	href="../../include/nodes/parsenodes.h">RangeTableEntry,</a> and
	they are linked together to form the <i>range table</i> of the
	query, which is generated by <a
	href="../../backend/parser/parse_clause.c">transformFromClause().</a>
	Query.rtable holds the query's range table.</p>

	<p>Certain queries, like <i>SELECT,</i> return columns of data.
	Other queries, like <i>INSERT</i> and <i>UPDATE,</i> specify the
	columns modified by the query. These column references are
	converted to <a
	href="../../include/nodes/primnodes.h">TargetEntry</a> entries,
	which are linked together to make up the <i>target list</i> of the
	query. The target list is stored in Query.targetList, which is
	generated by <a
	href="../../backend/parser/parse_target.c">transformTargetList().</a></p>

	<p>Other query elements, like aggregates(<i>SUM()</i>), <i>GROUP
	BY,</i> and <i>ORDER BY</i> are also stored in their own Query
	fields.</p>

	<p>The next step is for the Query to be modified by any
	<i>VIEWS</i> or <i>RULES</i> that may apply to the query. This is
	performed by the <a href="../../backend/rewrite">rewrite</a>
	system.</p>

	<p>The <a href="../../backend/optimizer">optimizer</a> takes the
	Query structure and generates an optimal <a
	href="../../include/nodes/plannodes.h">Plan,</a> which contains the
	operations to be performed to execute the query. The <a
	href="../../backend/optimizer/path">path</a> module determines the
	best table join order and join type of each table in the
	RangeTable, using Query.qual(<i>WHERE</i> clause) to consider
	optimal index usage.</p>

	<p>The Plan is then passed to the <a
	href="../../backend/executor">executor</a> for execution, and the
	result returned to the client. The Plan is actually as set of nodes,
	arranged in a tree structure with a top-level node, and various
	sub-nodes as children.</p>

	<p>There are many other modules that support this basic
	functionality. They can be accessed by clicking on the
	flowchart.</p>

	<hr />
	<p>Another area of interest is the shared memory area, which
	contains data accessable to all backends. It has recently used
	data/index blocks, locks, backend process information, and lookup
	tables for these structures:</p>

	<ul>
	<li>ShmemIndex - lookup shared memory addresses using structure
	names</li>

	<li><a href="../../include/storage/buf_internals.h">Buffer
	Descriptor</a> - control header for buffer cache block</li>

	<li><a href="../../include/storage/buf_internals.h">Buffer
	Block</a> - data/index buffer cache block</li>

	<li>Shared Buffer Lookup Table - lookup of buffer cache block
	addresses using table name and block number( <a
	href="../../include/storage/buf_internals.h">BufferTag</a>)</li>

	<li>Lock Manager Tables (lock hash) - the <a
	href="../../include/storage/lock.h">LOCK</a> structure, looked up
	using a <a href="../../include/storage/lock.h">LOCKTAG</a>.
	A LOCK structure exists for each lockable object that is currently
	locked by any backend. Also, there is a subsidiary <a
	href="../../include/storage/lock.h">PROCLOCK</a> structure for each
	backend currently interested in a given LOCK</li>

	<li><a href="../../include/storage/proc.h">PGPROC Structures</a> -
	information about each backend, including locks held/waiting</li>
	</ul>

	<p>Each data structure is created by calling <a
	href="../../backend/storage/ipc/shmem.c">ShmemInitStruct(),</a> and
	the lookups are created by <a
	href="../../backend/storage/ipc/shmem.c">ShmemInitHash().</a></p>

	<hr />
	<small>Maintainer: Bruce Momjian (<a
	href="mailto:pgman@candle.pha.pa.us">pgman@candle.pha.pa.us</a>)<br />

	Last updated: Fri May 6 14:22:27 EDT 2005</small>
	</body>
	</html>