{{{ | |
#!rst | |
============== | |
a power-magnet | |
============== | |
------------------ | |
Module: mod_magnet | |
------------------ | |
.. contents:: Table of Contents | |
Requirements | |
============ | |
:Version: lighttpd 1.4.12 or higher | |
:Packages: lua >= 5.1 | |
Overview | |
======== | |
mod_magnet is a module to control the request handling in lighty. | |
.. note:: | |
Keep in mind that the magnet is executed in the core of lighty. EVERY long-running operation is blocking | |
ALL connections in the server. You are warned. For time-consuming or blocking scripts use mod_fastcgi and friends. | |
For performance reasons mod_magnet caches the compiled script. For each script-run the script itself is checked for | |
freshness and recompile if neccesary. | |
Installation | |
============ | |
mod_magnet needs a lighty which is compiled with the lua-support ( --with-lua). Lua 5.1 or higher are required by | |
the module. Use "--with-lua=lua5.1" to install on Debian and friends. :: | |
server.modules = ( ..., "mod_magnet", ... ) | |
Options | |
======= | |
mod_magnet can attract a request in several stages in the request-handling. | |
* either at the same level as mod_rewrite, before any parsing of the URL is done | |
* or at a later stage, when the doc-root is known and the physical-path is already setup | |
It depends on the purpose of the script which stage you want to intercept. Usually you want to use | |
the 2nd stage where the physical-path which relates to your request is known. At this level you | |
can run checks against lighty.env["physical.path"]. | |
:: | |
magnet.attract-raw-url-to = ( ... ) | |
magnet.attract-physical-path-to = ( ... ) | |
You can define multiple scripts when separated by a semicolon. The scripts are executed in the specified | |
order. If one of them a returning a status-code, the following scripts will not be executed. | |
Tables | |
====== | |
Most of the interaction between between mod_magnet and lighty is done through tables. Tables in lua are hashes (Perl), dictionaries (Java), arrays (PHP), ... | |
Request-Environment | |
------------------- | |
Lighttpd has its internal variables which are exported as read/write to the magnet. | |
If "http://example.org/search.php?q=lighty" is requested this results in a request like :: | |
GET /search.php?q=lighty HTTP/1.1 | |
Host: example.org | |
When you are using ``attract-raw-url-to`` you can access the following variables: | |
* parts of the request-line | |
* lighty.env["request.uri"] = "/search.php?q=lighty" | |
* HTTP request-headers | |
* lighty.request["Host"] = "example.org" | |
Later in the request-handling, the URL is splitted, cleaned up and turned into a physical path name: | |
* parts of the URI | |
* lighty.env["uri.path"] = "/search.php" | |
* lighty.env["uri.path-raw"] = "/search.php" | |
* lighty.env["uri.scheme"] = "http" | |
* lighty.env["uri.authority"] = "example.org" | |
* lighty.env["uri.query"] = "q=lighty" | |
* filenames, pathnames | |
* lighty.env["physical.path"] = "/my-docroot/search.php" | |
* lighty.env["physical.rel-path"] = "/search.php" | |
* lighty.env["physical.doc-root"] = "/my-docroot" | |
All of them are readable, not all of the are writable (or don't have an effect if you write to them). | |
As a start, you might want to use those variables for writing: :: | |
-- 1. simple rewriting is done via the request.uri | |
lighty.env["request.uri"] = ... | |
return lighty.RESTART_REQUEST | |
-- 2. changing the physical-path | |
lighty.env["physical.path"] = ... | |
-- 3. changing the query-string | |
lighty.env["uri.query"] = ... | |
Response Headers | |
---------------- | |
If you want to set a response header for your request, you can add a field to the lighty.header[] table: :: | |
lighty.header["Content-Type"] = "text/html" | |
Sending Content | |
=============== | |
You can generate your own content and send it out to the clients. :: | |
lighty.content = { "<pre>", { filename = "/etc/passwd" }, "</pre>" } | |
lighty.header["Content-Type"] = "text/html" | |
return 200 | |
The lighty.content[] table is executed when the script is finished. The elements of the array are processed left to right and the elements can either be a string or a table. Strings are included AS IS into the output of the request. | |
* Strings | |
* are included as is | |
* Tables | |
* filename = "<absolute-path>" is required | |
* offset = <number> [default: 0] | |
* length = <number> [default: size of the file - offset] | |
Internally lighty will use the sendfile() call to send out the static files at full speed. | |
Status Codes | |
============ | |
You might have seen it already in other examples: In case you are handling the request completly in the magnet you | |
can return your own status-codes. Examples are: Redirected, Input Validation, ... :: | |
if (lighty.env["uri.scheme"] == "http") then | |
lighty.header["Location"] = "https://" .. lighty.env["uri.authority"] .. lighty.env["request.uri"] | |
return 302 | |
end | |
You every number above and equal to 100 is taken as final status code and finishes the request. No other modules are | |
executed after this return. | |
A special return-code is lighty.RESTART_REQUEST (currently equal to 99) which is usually used in combination with | |
changing the request.uri in a rewrite. It restarts the splitting of the request-uri again. | |
If you return nothing (or nil) the request-handling just continues. | |
Debugging | |
========= | |
To easy debugging we overloaded the print()-function in lua and redirect the output of print() to the error-log. :: | |
print("Host: " .. lighty.request["Host"]) | |
print("Request-URI: " .. lighty.env["request.uri"]) | |
Examples | |
======== | |
Sending text-files as HTML | |
-------------------------- | |
This is a bit simplistic, but it illustrates the idea: Take a text-file and cover it in a <pre> tag. | |
Config-file :: | |
magnet.attract-physical-path-to = server.docroot + "/readme.lua" | |
readme.lua :: | |
lighty.content = { "<pre>", { filename = "/README" }, "</pre>" } | |
lighty.header["Content-Type"] = "text/html" | |
return 200 | |
Maintainance pages | |
------------------ | |
Your side might be on maintainance from time to time. Instead of shutting down the server confusing all | |
users, you can just send a maintainance page. | |
Config-file :: | |
magnet.attract-physical-path-to = server.docroot + "/maintainance.lua" | |
maintainance.lua :: | |
require "lfs" | |
if (nil == lfs.attributes(lighty.env["physical.doc-root"] .. "/maintainance.html")) then | |
lighty.content = ( lighty.env["physical.doc-root"] .. "/maintainance.html" ) | |
lighty.header["Content-Type"] = "text/html" | |
return 200 | |
end | |
mod_flv_streaming | |
----------------- | |
Config-file :: | |
magnet.attract-physical-path-to = server.docroot + "/flv-streaming.lua" | |
flv-streaming.lua:: | |
if (lighty.env["uri.query"]) then | |
-- split the query-string | |
get = {} | |
for k, v in string.gmatch(lighty.env["uri.query"], "(%w+)=(%w+)") do | |
get[k] = v | |
end | |
if (get["start"]) then | |
-- missing: check if start is numeric and positive | |
-- send te FLV header + a seek into the file | |
lighty.content = { "FLV\x1\x1\0\0\0\x9\0\0\0\x9", | |
{ filename = lighty.env["physical.path"], offset = get["start"] } } | |
lighty.header["Content-Type"] = "video/x-flv" | |
return 200 | |
end | |
end | |
selecting a random file from a directory | |
---------------------------------------- | |
Say, you want to send a random file (ad-content) from a directory. | |
To simplify the code and to improve the performance we define: | |
* all images have the same format (e.g. image/png) | |
* all images use increasing numbers starting from 1 | |
* a special index-file names the highest number | |
Config :: | |
server.modules += ( "mod_magnet" ) | |
magnet.attract-physical-path-to = "random.lua" | |
random.lua :: | |
dir = lighty.env["physical.path"] | |
f = assert(io.open(dir .. "/index", "r")) | |
maxndx = f:read("*all") | |
f:close() | |
ndx = math.random(maxndx) | |
lighty.content = { { filename = dir .. "/" .. ndx }} | |
lighty.header["Content-Type"] = "image/png" | |
return 200 | |
denying illegal character sequences in the URL | |
---------------------------------------------- | |
Instead of implementing mod_security, you might just want to apply filters on the content | |
and deny special sequences that look like SQL injection. | |
A common injection is using UNION to extend a query with another SELECT query. | |
:: | |
if (string.find(lighty.env["request.uri"], "UNION%s")) then | |
return 400 | |
end | |
Traffic Quotas | |
-------------- | |
If you only allow your virtual hosts a certain amount for traffic each month and want to | |
disable them if the traffic is reached, perhaps this helps: :: | |
host_blacklist = { ["www.example.org"] = 0 } | |
if (host_blacklist[lighty.request["Host"]]) then | |
return 404 | |
end | |
Just add the hosts you want to blacklist into the blacklist table in the shown way. | |
Complex rewrites | |
---------------- | |
If you want to implement caching on your document-root and only want to regenerate | |
content if the requested file doesn't exist, you can attract the physical.path: :: | |
magnet.attract-physical-path-to = ( server.document-root + "/rewrite.lua" ) | |
rewrite.lua :: | |
require "lfs" | |
attr = lfs.attributes(lighty.env["physical.path"]) | |
if (not attr) then | |
-- we couldn't stat() the file for some reason | |
-- let the backend generate it | |
lighty.env["uri.path"] = "/dispatch.fcgi" | |
lighty.env["physical.rel-path"] = lighty.env["uri.path"] | |
lighty.env["physical.path"] = lighty.env["physical.doc-root"] .. lighty.env["physical.rel-path"] | |
fi | |
luafilesystem | |
+++++++++++++ | |
We are requiring the lua-module 'lfs' (http://www.keplerproject.org/luafilesystem/). | |
I had to compile lfs myself for lua-5.1 which required a minor patch as compat-5.1 is not needed:: | |
$ wget http://luaforge.net/frs/download.php/1487/luafilesystem-1.2.tar.gz | |
$ wget http://www.lighttpd.net/download/luafilesystem-1.2-lua51.diff | |
$ gzip -cd luafilesystem-1.2.tar.gz | tar xf - | |
$ cd luafilesystem-1.2 | |
$ patch -ls -p1 < ../luafilesystem-1.2-lua51.diff | |
$ make install | |
It will install lfs.so into /usr/lib/lua/5.1/ which is where lua expects the extensions on my system. | |
SuSE and Gentoo are known to have their own lfs packages and don't require a compile. | |
Usertracking | |
------------ | |
... or how to store data globally in the script-context: | |
Each script has its own script-context. When the script is started it only contains the lua-functions | |
and the special lighty.* name-space. If you want to save data between script runs, you can use the global-script | |
context: | |
:: | |
if (nil == _G["usertrack"]) then | |
_G["usertrack"] = {} | |
end | |
if (nil == _G["usertrack"][lighty.request["Cookie"]]) then | |
_G["usertrack"][lighty.request["Cookie"]] | |
else | |
_G["usertrack"][lighty.request["Cookie"]] = _G["usertrack"][lighty.request["Cookie"]] + 1 | |
end | |
print _G["usertrack"][lighty.request["Cookie"]] | |
The global-context is per script. If you update the script without restarting the server, the context will still be maintained. | |
Counters | |
-------- | |
mod_status support a global statistics page and mod_magnet allows to add and update values in the status page: | |
Config :: | |
status.statistics-url = "/server-counters" | |
magnet.attract-raw-url-to = server.docroot + "/counter.lua" | |
counter.lua :: | |
lighty.status["core.connections"] = lighty.status["core.connections"] + 1 | |
Result:: | |
core.connections: 7 | |
fastcgi.backend.php-foo.0.connected: 0 | |
fastcgi.backend.php-foo.0.died: 0 | |
fastcgi.backend.php-foo.0.disabled: 0 | |
fastcgi.backend.php-foo.0.load: 0 | |
fastcgi.backend.php-foo.0.overloaded: 0 | |
fastcgi.backend.php-foo.1.connected: 0 | |
fastcgi.backend.php-foo.1.died: 0 | |
fastcgi.backend.php-foo.1.disabled: 0 | |
fastcgi.backend.php-foo.1.load: 0 | |
fastcgi.backend.php-foo.1.overloaded: 0 | |
fastcgi.backend.php-foo.load: 0 | |
Porting mod_cml scripts | |
----------------------- | |
mod_cml got replaced by mod_magnet. | |
A CACHE_HIT in mod_cml:: | |
output_include = { "file1", "file2" } | |
return CACHE_HIT | |
becomes:: | |
content = { { filename = "/path/to/file1" }, { filename = "/path/to/file2"} } | |
return 200 | |
while a CACHE_MISS like (CML) :: | |
trigger_handler = "/index.php" | |
return CACHE_MISS | |
becomes (magnet) :: | |
lighty.env["request.uri"] = "/index.php" | |
return lighty.RESTART_REQUEST | |
}}} |