Writing dynamic web pages using Lua

Posted by Nick Gammon on Fri 28 Apr 2006 03:26 AM — 25 posts, 191,173 views.

Nick Gammon Australia Forum Administrator Fri 28 Apr 2006 03:26 AM #0

So far my dynamic web pages have been written using PHP, see:



http://www.php.net/

However recently I have become interested in achieving much the same results using Lua.

There is already a Lua add-on called luacgi, however I found it a bit confusing and poorly documented, so this posting is an attempt to get you started at doing web pages in Lua from first principles.

Install Lua at your web host site

The first step would be to install Lua at your web hosting site, if it is not already installed. If necessary, obtain a copy of the source from:



http://www.lua.org/download.html

I used Lua 5.0.2, however Lua version 5.1 has now been released.

In my case, after compiling and installing, my copy of Lua ended up in:



/usr/local/bin/lua

This is important to know, as the path to the Lua executable has to be mentioned in the first line of your web script.

If it is already installed you could find its path by typing:



whereis lua

The basic components of a web page

Web pages sent from a web server to a web browser consist of two major parts, separated by a blank line:

The HTTP header
The body of the page

The header contains things like the type of body (text, html, etc.), cookies, date, server type and so on. This is an example header:


HTTP/1.1 200 OK
Date: Fri, 28 Apr 2006 03:14:58 GMT
Server: Apache/2.0.46 (Unix) PHP/4.3.2
Set-Cookie: foo=bar
Set-Cookie: food=apples
Connection: close
Content-Type: text/html; charset=iso-8859-1

The body of the page is the actual data the user sees (generally, the HTML code).

First example, straight text

Let's start off with a simple example dynamic page. The nice thing about using Apache is that it appears to supply most of the headers itself, so for a simple page the only header we need is:



Content-Type: text/plain

This tells the brower to interpret the body of the web page as pure text (not HTML). Thus, a minimal dynamic web page written in Lua could be this:


#! /usr/local/bin/lua

print [[
Content-Type: text/plain

Hello, world
]]

Here we are printing (sending to the browser) 3 lines:

The header (content-type)
The blank line which separates the header from the body)
The body of the page

The first line of the file indicates the path to the Lua executable.

I saved this file into my "cgi-bin" directory, specifically to:



/usr/local/httpd/cgi-bin/test1.lua

An important step is to make the file executable, like this:



chmod a+x test1.lua

Having done that, I can now enter this URL into my web browser:



http://10.0.0.2/cgi-bin/test1.lua

My internal web server is at the local address of 10.0.0.2 - you would replace that with the address of wherever your server is located.

This may not look very exciting, but a small modification, and we can do some maths on our page. Let's make a 2 times table:


#! /usr/local/bin/lua

-- HTTP header
print [[
Content-Type: text/plain

]]

-- body of page

for i = 1, 10 do
  print (i, i * 2)
end -- for

If I run this I see a page appear in my browser with this in it:

Nick Gammon Australia Forum Administrator Fri 28 Apr 2006 04:06 AM #1

Getting data from a web page

The simple example above will always do the same thing, however often we want to let the user specify further information. For example in the forum here, we specify which forum posting we want to see, or when we make a new posting, the new message has to be sent to the web server.

There are four ways we can send data to and from web servers (apart from the things in the header like the date):

The URL path itself
The query string
Cookies
Form data

Let's examine each of these to see how they are used and how they work. To do that we'll make a new Lua dynamic page (test2.lua), that will show the contents of all the environment variables:


#! /usr/local/bin/lua

-- HTTP header
print [[
Content-Type: text/plain
Set-Cookie: foo=bar
Set-Cookie: wonder=always

]]

-- body of page

-- find all environment variables using bash and a temporary file

fname = os.tmpname ()
os.execute ("/bin/bash -c set > " .. fname)
f = io.open (fname, "r")  -- open it
s = f:read ("*a")  -- read all of it
print (s)
f:close ()  -- close it
os.remove (fname)

This example also uses "Set-Cookie" twice to set two cookies. We connect to this page twice (once to set the cookie, once to get the value returned). I used this URL in my web browser:



http://10.0.0.2/cgi-bin/test2.lua/sausage/machine?type=meat&date=Friday

(Note the trailing parts of the path after test2.lua).

I see this in the browser:


BASH=/bin/bash
BASH_VERSINFO=([0]="2" [1]="05b" [2]="0" [3]="1" [4]="release" [5]="i386-redhat-linux-gnu")
BASH_VERSION='2.05b.0(1)-release'
DIRSTACK=()
DOCUMENT_ROOT=/usr/local/httpd/htdocs
EUID=99
GATEWAY_INTERFACE=CGI/1.1
GROUPS=()
HOSTNAME=bacall
HOSTTYPE=i386
HTTP_ACCEPT='text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'
HTTP_ACCEPT_CHARSET='ISO-8859-1,utf-8;q=0.7,*;q=0.7'
HTTP_ACCEPT_ENCODING=gzip,deflate
HTTP_ACCEPT_LANGUAGE='en-gb,en;q=0.5'
HTTP_CONNECTION=keep-alive
HTTP_COOKIE='foo=bar; wonder=always'
HTTP_HOST=10.0.0.2
HTTP_KEEP_ALIVE=300
HTTP_USER_AGENT='Mozilla/5.0 (Windows; U; WinNT4.0; en-GB; rv:1.7.12) Gecko/20050919 Firefox/1.0.7'
IFS=$' \t\n'
MACHTYPE=i386-redhat-linux-gnu
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/X11R6/bin
PATH_INFO=/sausage/machine
PATH_TRANSLATED=/usr/local/httpd/htdocs/sausage/machine
PPID=2192
PS4='+ '
PWD=/usr/local/httpd/cgi-bin
QUERY_STRING='type=meat&date=Friday'
REMOTE_ADDR=10.0.0.3
REMOTE_PORT=2433
REQUEST_METHOD=GET
REQUEST_URI='/cgi-bin/test2.lua/sausage/machine?type=meat&date=Friday'
SCRIPT_FILENAME=/usr/local/httpd/cgi-bin/test2.lua
SCRIPT_NAME=/cgi-bin/test2.lua
SERVER_ADDR=10.0.0.2
SERVER_ADMIN=you@your.address
SERVER_NAME=10.0.0.2
SERVER_PORT=80
SERVER_PROTOCOL=HTTP/1.1
SERVER_SIGNATURE=$'<address>Apache/2.0.46 (Unix) PHP/4.3.2 Server at 10.0.0.2 Port 80</address>\n'
SERVER_SOFTWARE='Apache/2.0.46 (Unix) PHP/4.3.2'
SHELL=/sbin/nologin
SHELLOPTS=braceexpand:hashall:interactive-comments
SHLVL=2
TERM=dumb
UID=99
_=/bin/bash

The important lines I have made bold.

We can see these things:

The cookies: HTTP_COOKIE='foo=bar; wonder=always'

We can see both cookies in a single enviroment variable, we will want a way to split those up.
Extra path information: PATH_INFO=/sausage/machine

This was extra information added after "test2.lua" in my URL path. It appears that Apache has realised that test2.lua is the executable file, and that the rest of the path is collected into PATH_INFO.
The query string: QUERY_STRING='type=meat&date=Friday'

This is the part following the "?" in the URL. Both of the variables here (meat and date) are in a single line, we need to split those up as well.
How the page was sent: REQUEST_METHOD=GET

This tells us that the URL was entered into the Browser (or a link was clicked on), rather than a form being filled in. (If a form was filled in then the request method would be "POST" rather than "GET").

Nick Gammon Australia Forum Administrator Fri 28 Apr 2006 04:16 AM #2

Some simple utilities for use with dynamic web pages

To help process and write dynamic web pages I have developed a few routines in Lua which I will place in a separate file, which can be easily included into our main page.


-- Lua utilities for CGI web use

-- split a string with delimiters into a table (reverse of table.concat)

function split (s, delim)

  assert (type (delim) == "string" and string.len (delim) > 0,
          "bad delimiter")

  local start = 1
  local t = {}  -- results table

  -- find each instance of a string followed by the delimiter

  while true do
    local pos = string.find (s, delim, start, true) -- plain find

    if not pos then
      break
    end

    table.insert (t, string.sub (s, start, pos - 1))
    start = pos + string.len (delim)
  end -- while

  -- insert final one (after last delimiter)

  table.insert (t, string.sub (s, start))

  return t
 
end -- function split

-- trim leading and trailing spaces from a string
function trim (s)
  return (string.gsub (s, "^%s*(.-)%s*$", "%1"))
end -- trim

mysql_replacements = { 
   ["\0"] = "\\0",
   ["\n"] = "\\n",
   ["\r"] = "\\r",
   ["\'"] = "\\\'",
   ["\""] = "\\\"",
   ["\026"] = "\\Z",
   ["\b"] = "\\b",
   ["\t"] = "\\t",
   }

-- Fix SQL text by converting various characters to the format MySQL 
--  will recognise in its string processor
--
-- Note that not all the escapes are necessary for internal SQL use, 
-- however if data is being dumped to disk (eg. as SQL statements) 
-- then it is handy for have things like \n and \r made more readable
--   See: http://dev.mysql.com/doc/refman/5.1/en/string-syntax.html

function fixsql (s)

  return (string.gsub (tostring (s), "[%z\n\r\'\"\026\b\t]", 
    function (str)
      return mysql_replacements [str] or str
    end ))

end -- fixsql

html_replacements = { 
   ["<"] = "&lt;",
   [">"] = "&gt;",
   ["&"] = "&amp;",
   }

-- fix text so that < > and & are escaped
function fixhtml (s)

return (string.gsub (tostring (s), "[<>&]", function (str)
  return html_replacements [str] or str
  end ))

end -- fixhtml

-- convert + to space
-- convert %xx where xx is hex characters, to the equivalent byte
function urldecode (s)
  return (string.gsub (string.gsub (s, "+", " "), 
          "%%(%x%x)", 
         function (str)
          return string.char (tonumber (str, 16))
         end ))
end -- function urldecode

-- process a single key=value pair from a GET line (or cookie, etc.)
function assemble_value (s, t)
  assert (type (t) == "table")
  local _, _, key, value = string.find (s, "(.-)=(.+)")

  if key then
    t [trim (urldecode (key))] = trim (urldecode (value))
  end -- if we had key=value

end -- assemble_value

-- output a Lua table as an HTML table
function show_table (t)
  local k, v
  assert (type (t) == "table")
  print "<table border=1 cellpadding=3>"
  for k, v in pairs (t) do
    print "<tr>"
    print ("<th>" .. fixhtml (k) .. "</th>" .. 
           "<td>" .. fixhtml (v) .. "</td>")
    print "</tr>"
  end -- for
  print "</table>"
end -- show_table

In here we have:

split - split a string at a delimiter, into a table
trim - remove leading and trailing spaces from around a string
fixsql - converts a string for use with MySQL (and similar) by escaping out things like quotes by prepending a backspace
fixhtml - converts the symbols < > and & into the HTML equivalents (< >, & )
urldecode - converts special character in a URL (like %20) into their equivalents (space in this case)
assemble_value - takes a key=value string from a URL or cookie, and makes a key/value table entry from it
show_table - converts a simple Lua table into an HTML table

Nick Gammon Australia Forum Administrator Fri 28 Apr 2006 04:28 AM #3

Extracting out the GET and COOKIE data

This example decodes the query data (the stuff after the "?") into individual table entries, and also the individual cookies.

We are also now using HTML rather than straight text, so we need to use fixhtml to escape out any <, > or & symbols:


#! /usr/local/bin/lua

dofile "cgiutils.lua"

-- HTTP header
print [[
Content-Type: text/html; charset=iso-8859-1
Set-Cookie: foo=bar
Set-Cookie: wonder=always I think

]]

-- body of page

-- XML header, doctype, page header

print [[
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html lang="en">
<head> 
<title>My title</title>
</head>
<body>
]]

-- show data from the query string

QUERY_STRING = os.getenv ("QUERY_STRING")

get_data = {}

if QUERY_STRING then
  query = split (QUERY_STRING, "&")
  for _, v in ipairs (query) do
    assemble_value (v, get_data)
  end -- for
end -- if

print "<h1>GET data</h1>\n"

show_table (get_data)

-- show cookie data

COOKIE_STRING = os.getenv ("HTTP_COOKIE")

cookie_data = {}

if COOKIE_STRING then
  cookie = split (COOKIE_STRING, ";")
  for _, v in ipairs (cookie) do
    assemble_value (v, cookie_data)
  end -- for
end -- if

print "<h1>COOKIE data</h1>\n"

show_table (cookie_data)

-- end of body

print [[
</body>
</html>
]]

I'll run this with the URL:



http://10.0.0.2/cgi-bin/test2.lua?type=meat&date=Friday%20the%2013th

These are the results (except they actually appear as HTML tables):


GET data

date	Friday the 13th
type	meat

COOKIE data

wonder	always I think
foo	bar

Notice how the spaces in the cookies and the %20 in the URL have been correctly converted into actual spaces.

The file "cgiutils.lua" is described in the post immediately before this one.

Nick Gammon Australia Forum Administrator Fri 28 Apr 2006 04:40 AM #4

Getting form data

If your web page has forms in it (that is, you are using <form> ... </form> ) then we need another method to access that data. Form data can typically be quite long (such as a forum post), so rather than being stored in an environment variable, you can read it by reading from stdin. This example shows reading from stdin to access the form data:


#! /usr/local/bin/lua

dofile "cgiutils.lua"

-- HTTP header
print [[
Content-Type: text/html; charset=iso-8859-1

]]

-- body of page

-- XML header, doctype, page header

print [[
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html lang="en">
<head> 
<title>My title</title>
</head>
<body>
]]

--- form test

print [[
<form METHOD="post" ACTION=http://10.0.0.2/cgi-bin/test3.lua?humphrey=bogart>
<p>Enter data here: <input type=text Name="Widget" size=20 maxlength=20>
<input Type=hidden Name=date Value="2006-04-28">
<input Type=hidden Name=action Value=fubar>
<input Type=submit Name=Submit Value="Test posting">
</p>
</form>
]]

---- end form test

POST_DATA = io.read ("*a")  -- read all of stdin

post_data = {}

if POST_DATA then
  post = split (POST_DATA, "&")
  for _, v in ipairs (post) do
    assemble_value (v, post_data)
  end -- for
end -- if


print "<h1>POST data</h1>\n"

show_table (post_data)

print ("<p>Request method = ", os.getenv ("REQUEST_METHOD"), "</p>")

-- end of body

print [[
</body>
</html>
]]

The first time I simply enter the URL into my browser:



http://10.0.0.2/cgi-bin/test3.lua

No "POST" data is displayed and I see:



Request method = GET

However if I type "some data" into the text box, and click on the "Test posting" button, I now see:


POST data

action	fubar
date	2006-04-28
Widget	some data
Submit	Test posting

Request method = POST

Now the request method is POST, which tells me that they have filled in a form, and the various fields, including the hidden ones, are displayed.

Nick Gammon Australia Forum Administrator Fri 28 Apr 2006 05:29 AM #5

Putting it all together

We can simplify all the various ways data can be presented to the server from a web page by adding a new routine to the cgiutils.lua file:


-- get query data, cookie data, post data
function get_user_input ()
local _, v

  -- query data (stuff after the ? )
 
  local get_data = {}
  
  for _, v in ipairs (split (os.getenv ("QUERY_STRING"), "&")) do
    assemble_value (v, get_data)
  end -- for
  
  -- cookies
 
  local cookie_data = {}
  
  for _, v in ipairs (split (os.getenv ("HTTP_COOKIE"), ";")) do
    assemble_value (v, cookie_data)
  end -- for
  
  -- posted data (from a form)
  
  local post_data = {}
  
  for _, v in ipairs (split (io.read ("*a"), "&")) do
    assemble_value (v, post_data)
  end -- for
  
 return get_data, cookie_data, post_data, os.getenv ("REQUEST_METHOD")
 
 end -- function get_user_input

This will return four things:

The "get" data from the URL
The cookies data
The "post" data from any form
The request method (GET or POST)

We can use our simplified interface like this:


#! /usr/local/bin/lua

dofile "cgiutils.lua"

-- HTTP header
print [[
Content-Type: text/html; charset=iso-8859-1

]]

-- body of page

-- XML header, doctype, page header

print [[
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html lang="en">
<head> 
<title>My title</title>
</head>
<body>
]]

--- form test

print [[
<form METHOD="post" ACTION=http://10.0.0.2/cgi-bin/test4.lua?humphrey=bogart>
<p>Enter data here: <input type=text Name="Widget" size=20 maxlength=20>
<input Type=hidden Name=date Value="2006-04-28">
<input Type=hidden Name=action Value=fubar>
<input Type=submit Name=Submit Value="Submit">
</p>
</form>
]]

---- end form test

get_data, cookie_data, post_data, request_method = get_user_input ()

print "<h1>GET data</h1>\n"
show_table (get_data)

print "<h1>COOKIE data</h1>\n"
show_table (cookie_data)

print "<h1>POST data</h1>\n"
show_table (post_data)

print ("<p>Request method = ", request_method, "</p>")

-- end of body

print [[
</body>
</html>
]]

Here, a single call to get_user_input returns 3 tables, which is the data described above. Now we can simply index into the relevant tables if we want to.

For example, to see what cookie "foo" has in it:



print (cookie_data.foo)

Zeno USA Sun 30 Apr 2006 05:43 PM #6

Very nice. I need to display a local file (not located in the web dir) on my site, could I use Lua to do that?

David Haley USA Sun 30 Apr 2006 08:56 PM #7

If the user Lua is run as has permission to open the file, then yes. If the file and directories containing it are world-readable, then you should be fine.

Nick Gammon Australia Forum Administrator Mon 01 May 2006 06:19 AM #8

Indeed. My first example showed reading and displaying a file, which could have any name readable by the Apache CGI scripts:


f = io.open (fname, "r")  -- open it
s = f:read ("*a")  -- read all of it
print (s)
f:close ()  -- close it

Nick Gammon Australia Forum Administrator Mon 01 May 2006 07:14 AM #9

How to handle errors in scripts

An interesting problem arose when developing these demonstration pages:

A single syntax error (such as leaving off a quote or bracket) would make the page not display at all, as the Lua interpreter would reject it; and
A runtime error (like misspelling a function name) would cause the page to terminate prematurely

Both of these were frustrating to locate and correct. There must be an easier way, I thought, especially as Lua is good at letting you call functions in a "protected" mode, and finding out what went wrong.

As it turns out the, answer is simple - to turn the paradigm around. Rather than having:



My script file (eg. test5.lua) ===> calling cgiutils.lua (for useful subroutines)

Do this instead:



The common utilities file (cgiutils.lua) ===> calling my script file (test5.lua) in protected mode

We will assume that a single set of "core" utilites, once debugged, will need infrequent changes, and will be unlikely to have ongoing syntax or runtime problems. However our hundreds of other web pages we develop from time to time are much more likely to be sources of errors, if only because they will be larger.

But, if Apache is really running cgiutils.lua, how will that script know what page the user "really" wants? The key here is to use the "PATH_INFO" environment variable, passed down by Apache. This has the trailing part of the URL in it.

Thus, to run a script "test10.lua" the URL will now be:



http://10.0.0.2/cgi-bin/cgiutils.lua/testing/test10.lua

The two parts in bold above illustrate what is happening. Apache is running the script "cgiutils.lua". This script finds the contents of the "PATH_INFO" variable (which will be "/testing/test10.lua" in this case, prepends a dot to make it a relative pathname, and then does a "dofile" on that filename, thus running the desired file (test10.lua).

The main code to do this (simplified slightly) will be in cgiutils.lua, and look like this:


function load_page ()

  -- get GET, COOKIE, POST data, and request method
  
  get_data, cookie_data, post_data, request_method  =
     get_user_input ()
 
  local scriptname = "." .. os.getenv ("PATH_INFO")
  
  -- do checks here on validity of scriptname

  -- run the file specified in the rest of the URL
  
  dofile (scriptname)
  
end -- load_page


-- main code starts here

ok, error = xpcall (load_page, _TRACEBACK)

if not ok then
  major_problem (error)
end -- error in page

I have added a couple of handy routines to do the main legwork of doing the HTTP and HTML headers and footers, so our file test10.lua can look like this now:


  show_html_header ("my title")
  print ("hello, world<p>")
  show_html_footer ()

Security warning

You would need to be careful in implementing this scheme to make sure that the user of your web site cannot "break out" of your web page hierarchy by doing tricks like this:



http://10.0.0.2/cgi-bin/cgiutils.lua/../../../somedirectory/somefile.lua

You might want to build in a check that disallows the ".." characters in the pathname, for example. Another check might be that the resulting pathname matches a very specific pattern in a regular expression (eg. ends in .lua, starts with the directory where your scripts should be).

The enhanced cgiutils.lua file will appear in the next post.

Nick Gammon Australia Forum Administrator Mon 01 May 2006 07:16 AM #10

Enhanced cgiutils.lua file

Below is the improved cgiutils.lua file. It is similar to the earlier one, with extra functions for automating the production of the http and html headers and footers.


#! /usr/local/bin/lua

-- Lua utilities for CGI web use

-- Written by Nick Gammon - May 2006.

--[[
Copyright © 2006 Nick Gammon.

Permission is hereby granted, free of charge, to any person obtaining a copy 
of this software and associated documentation files (the "Software"), to deal 
in the Software without restriction, including without limitation the rights 
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
copies of the Software, and to permit persons to whom the Software is 
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all 
copies or substantial portions of the Software.    

The software is provided "as is", without warranty of any kind, express or 
implied, including but not limited to the warranties of merchantability, 
fitness for a particular purpose and noninfringement. In no event shall the 
authors or copyright holders be liable for any claim, damages or other 
liability, whether in an action of contract, tort or otherwise, arising from, 
out of or in connection with the software or the use or other dealings in the 
software. 
--]]

-- global flags

done_http_header = false  -- have we sent the HTTP header yet?
done_html_header = false  -- have we sent the HTML header yet?
done_html_footer = false  -- have we sent the HTTP footer yet?

-- split a string with delimiters into a table (reverse of table.concat)
function split (s, delim)

  local t = {}  -- results table

  if s then   
    assert (type (delim) == "string" and string.len (delim) > 0,
            "bad delimiter")
  
    local start = 1
    local pos
  
    -- find each instance of a string followed by the delimiter
  
    repeat
      pos = string.find (s, delim, start, true) -- plain find
  
      if pos then
        table.insert (t, string.sub (s, start, pos - 1))
        start = pos + string.len (delim)
      end
    until not pos
  
    -- insert final one (after last delimiter)
  
    table.insert (t, string.sub (s, start))
  end -- not nil string
  
  return t
 
end -- function split

-- trim leading and trailing spaces from a string
function trim (s)
  return (string.gsub (string.gsub (s, "%s+$", ""), "^%s+", ""))
end -- trim

mysql_replacements = { 
   ["\0"] = "\\0",
   ["\n"] = "\\n",
   ["\r"] = "\\r",
   ["\'"] = "\\\'",
   ["\""] = "\\\"",
   ["\026"] = "\\Z",
   ["\b"] = "\\b",
   ["\t"] = "\\t",
   }

-- Fix SQL text by converting various characters to the format MySQL 
--  will recognise in its string processor
--
-- Note that not all the escapes are necessary for internal SQL use, 
-- however if data is being dumped to disk (eg. as SQL statements) 
-- then it is handy for have things like \n and \r made more readable
--   See: http://dev.mysql.com/doc/refman/5.1/en/string-syntax.html

function fixsql (s)

  return (string.gsub (tostring (s), "[%z\n\r\'\"\026\b\t]", 
    function (str)
      return mysql_replacements [str] or str
    end ))

end -- fixsql

html_replacements = { 
   ["<"] = "&lt;",
   [">"] = "&gt;",
   ["&"] = "&amp;",
   ["\""] = "&quot;",
   }

-- fix text so that < > and & are escaped
function fixhtml (s)

  return (string.gsub (tostring (s), "[<>&\"]", 
    function (str)
      return html_replacements [str] or str
    end ))

end -- fixhtml

-- convert + to space
-- convert %xx where xx is hex characters, to the equivalent byte
function urldecode (s)
  return (string.gsub (string.gsub (s, "+", " "), 
          "%%(%x%x)", 
         function (str)
          return string.char (tonumber (str, 16))
         end ))
end -- function urldecode

-- reverse of urldecode - converts characters that are not alphanumeric
--  into the form %xx (for use in cookies etc.)
function urlencode (s)
 return (string.gsub (s, "%W", 
        function (str)
          return string.format ("%%%02X", string.byte (str))
        end  ))
end -- function urlencode
        
-- process a single key=value pair from a GET line (or cookie, etc.)
function assemble_value (s, t)
  assert (type (t) == "table")
  local _, _, key, value = string.find (s, "(.-)=(.+)")

  if key then
    t [trim (urldecode (key))] = trim (urldecode (value))
  end -- if we had key=value

end -- assemble_value

-- output a Lua table as an HTML table
function show_table (t)
  local k, v
  assert (type (t) == "table")
  print "<table border=1 cellpadding=3>"
  for k, v in pairs (t) do
    print "<tr>"
    print ("<th align=left >" .. fixhtml (k) .. "</th>" .. 
           "<td align=left >" .. fixhtml (v) .. "</td>")
    print "</tr>"
  end -- for
  print "</table>"
end -- show_table


-- get query data, cookie data, post data
function get_user_input ()
local _, v

  -- query data (stuff after the ? )
 
  local get_data = {}
  
  for _, v in ipairs (split (os.getenv ("QUERY_STRING"), "&")) do
    assemble_value (v, get_data)
  end -- for
  
  -- cookies
 
  local cookie_data = {}
  
  for _, v in ipairs (split (os.getenv ("HTTP_COOKIE"), ";")) do
    assemble_value (v, cookie_data)
  end -- for
  
  -- posted data (from a form)
  
  local post_data = {}

  local post_length = tonumber (os.getenv ("CONTENT_LENGTH")) or 0
  if os.getenv ("REQUEST_METHOD") == "POST" and post_length > 0 then
    for _, v in ipairs (split (io.read (post_length), "&")) do
      assemble_value (v, post_data)
    end -- for
  end -- if post
  
 return get_data, cookie_data, post_data, os.getenv ("REQUEST_METHOD")
 
 end -- function get_user_input
 
 -- show a standard HTTP header with optional cookies
function show_http_header (cookies)

  assert (not done_http_header, "Too many HTTP headers")
  
  print ("Content-Type: text/html; charset=iso-8859-1")
  print ("X-Powered-By: cgiutils.lua written by Nick Gammon")
  
  -- output a Set-Cookie line for each cookie in the table
  if cookies then
    assert (type (cookies) == "table")
    for k, v in pairs (cookies) do
      print ("Set-Cookie: " .. urlencode (k) .. "=" .. urlencode (v))
    end -- for each cookie
  end -- we have cookies

  print ("")  -- blank line separates header from body
  done_http_header = true

end -- function show_http_header 

-- show a standard HTML header with optional title, and other free-format things
--  the "other" stuff could be things like stylesheets, links, etc.

function show_html_header (title, other)

  assert (not done_html_header, "Too many HTML headers")

  if not done_http_header then
    show_http_header ()
  end 

  print ('<?xml version="1.0" encoding="iso-8859-1"?>')
  print ('<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">')
  print ('<html lang="en">')
  print ('<head>')
  if title then
    print ('<title>' .. fixhtml (title) .. '</title>')
  end -- having title
  if other then  
    print (other)
  end -- having other stuff
  print ('</head>')
  print ('<body>')

  done_html_header = true
end -- function show_html_header

-- wrap up with an HTML footer
function show_html_footer ()

  assert (not done_html_footer, "Too many HTML footers")

  print ('</body>')
  print ('</html>')
  done_html_footer = true
end -- function show_html_footer

-- helper function to show an error in inverse bold coloured text
function show_error (theerror)
 
  if not done_html_header then
    show_html_header ("Error on web page")
  end 
 
  local colour_error_text = "#FFFFFF"  -- white
  local colour_error_bgnd = "#A52A2A"  -- brown
  local family = "sans-serif"
  local size = "150%"
  local weight = "bolder"
  
  -- style string for the error message
  local style = string.format (
                'color: %s; ' ..
                'background: %s; ' ..
                'font-family: %s; ' ..
                'font-size: %s; ' ..
                'font-weight: %s;',
                
                  colour_error_text,
                  colour_error_bgnd,
                  family,
                  size,
                  weight)
  
  -- show error in desired style
  print (string.format ('<p><span style="%s">&nbsp;%s&nbsp;</span></p>',
         style,
         fixhtml (theerror)))
          
end -- end of show_error
  
-- Call this for things like script errors - it shows the error and exits
function major_problem (why)

  if not done_html_header then
    show_html_header ("Script error on web page")
  end 
  
  print ("<h3>We apologise that there has been a problem with the web server ...</h3>\n")
  print ("<hr>")
  print ("<code><pre>" .. fixhtml (why) .. "</pre></code>")
  print ("<p>Error occurred on " .. os.date ("%#x at %X") .. "</p>\n")
  print ("<hr>")
  
  show_html_footer ()
  
  os.exit (1)  -- let's stop while we are ahead :)
end -- function major_problem
 
-- this function is run in protected mode in case the script page has errors
function load_page ()

  -- get local locale
  
  os.setlocale ("", "time")
  
  -- get GET, COOKIE, POST data, and request method
  
  get_data, cookie_data, post_data, request_method  = get_user_input ()
 
  local scriptname = os.getenv ("PATH_INFO")
  
  -- validity check on requested filename
  if not scriptname or
         string.find (scriptname, "..", 1, true) or
         not string.find (scriptname, "/%a+/[%a%d_%-]+%.lua")
         then
    return show_error ("That URL is not valid")
  end -- if
     
  scriptname = "." .. scriptname  -- make local path
  
  -- check page exists, to avoid annoying user with Lua traceback message
  local f, err1, err2 = io.open (scriptname)
   
  if f then
    f:close ()  -- we found the file, close it now
  else
    return show_error ("The requested page does not exist")
  end -- if
      
  -- run the file specified in the rest of the URL
  dofile (scriptname)
  
end -- load_page


-- main code starts here

ok, error = xpcall (load_page, _TRACEBACK)

if not ok then
  major_problem (error)
end -- error in page

-- ensure footer shown
if done_html_header and not done_html_footer then
  show_html_footer ()
end -- if

[EDIT]

Changed on 4th May 2006, to add various extra functionality:

Checking inside show_http_header, show_html_header and show_html_footer to ensure none of these are done twice.
Automatic footer generation if not done by the called script.
New function urlencode - this encodes a URL (reverse of urldecode) so that things like ';' are converted to '%3B'. This is used internally when setting a cookie.
Improved show_error function - colours the error message using <span> rather than a single-item table.
Improvements to load_page to do further checks on the requested script file name (using a regular expression). It also checks if the requested script file exists and if not gives a specific error message to that effect.
Fixed bug in split function (pos cannot be a local variable)
Added license so that you may use this software on your own web pages.

[EDIT]

Changed around 12 September 2006 to fix problems with getting POST data from some web servers using certain browsers. *cough**cough*Safari*cough*

Sylaer Mon 01 May 2006 09:11 PM #11

First of all, it's extremely encouraging to share a forum with someone of your calibre. I've heard mention of you before and didn't realize until now that YOU were that guy. For what it's worth, congratulations on all your hard work :)
I have to ask though, with your extensive PHP background, why learn Lua at all? Admittedly I know little about Lua, but PHP seems to be an ideal dynamic web code. Had to wonder what attracted you to it.

David Haley USA Mon 01 May 2006 09:36 PM #12

Lua is a much more elegant language than PHP. In fact, PHP as a language has a number of issues with it; issues of inconsistency, hastily implemented features, etc. Of course, PHP has a much more extensive set of helper functions specifically designed for web scripting, but Lua is slowly developing its own set of modules.

In any case, a lot of it really is that Lua is a much nicer language, with much more precise, consistent, and elegant language structures.

Nick Gammon Australia Forum Administrator Mon 01 May 2006 11:24 PM #13

Quote:

For what it's worth, congratulations on all your hard work :)

Thanks for the complimentary remarks. I am pleased that the forum in general attracts a high standard of posts, there are quite a few knowledgeable and helpful people posting regularly here. :)

Quote:

I have to ask though, with your extensive PHP background, why learn Lua at all?

That is a good question. I should make it clear that so far my Lua web development is at an experimental stage, all of this site here is still in PHP. However, to answer in more detail, here are some considerations:

When I first started developing dynamic web pages, I hadn't heard of Lua (it may not have existed then), so I chose the language which was well-known, and fairly close in syntax to C, namely PHP.
There is extensive documentation about developing web pages in PHP, virtually none about doing it in Lua. There is indeed a project (now) for doing that, CGILua, however I found the lack of examples on their pages made it very hard to get things going. I am starting to understand more what they are doing, now that I have done a similar thing myself. To give credit to the "official" project, I refer you to this page:

http://www.keplerproject.org/cgilua/
The further I went into adding Lua into MUSHclient, and also using it as a stand-alone tool for other things (like converting old emails from one format to another), the more I liked Lua.
Lua has a fundamental simple elegance that appeals to me. The core language is simple (by design) and largely absent of frills. The designers have made a conscious effort to keep Lua minimal, as they have made it simple (and well documented) to add your own extensions by writing your own DLLs (or shared libraries in Unix).

On my Linux PC, the comparable sizes of Lua to PHP are:
- Lua: 109 Kb PHP: 1.3 Mb
On my Windows PC, the comparable sizes of Lua to PHP are:
- Lua: 104 Kb PHP: 4.07 Mb
Lua can be extended easily (for example, you can add MySQL database queries by simply loading a shared library).
Various design decisions about PHP that annoy me, but I had learnt to live with, see below.

I don't think PHP is going to be replaced in a long time, if ever. Hundreds of thousands of web pages are developed in it, it has extensive helper functions built in, lots of documentation, and a very good support web site.

Things about PHP that annoy me

The things described below are not bugs, they are design decisions by the PHP developers. As such, PHP is working as designed, and there are always arguments about whether or not a design decision is correct. Much can probably be said in defence of the original design. However, here are my comments about them:

The decision (initially) to dump all of the get, post, and cookie variables into the PHP global address space. This was for the ease of programming, so that, if we take this URL as an example:

http://10.0.0.2/myfile.php?type=meat&date=Friday

PHP would set global variables:

$type = "meat"; $date = "Friday";

This was certainly convenient, but it could be confusing if you needed to distinguish a variable that came from a cookie, to one that came from the URL. Also, it made scripts a bit more vulnerable to attacks like this:

http://10.0.0.2/myfile.php?administrator=1

If the script incautiously assumed that administrator would default to 0, and only set it to 1 if certain checks were passed, then users could assume administrator status (set the global administrator variable) by simply setting it on their URL.
The decision to make all variables in a function local, unless specified as global, which is the reverse to what most languages do. For example:
```
Lua

function test ()
  foo = 22 -- sets global foo
  
  local bar
  
  bar = 42  -- sets local bar
end -- test

PHP

function test ()
  {
  $foo = 22; // sets local foo, global foo is unchanged
  
  global $bar;
  
  $bar = 42; // sets global bar
  }
```
Personally I think this is messy. Most functions will want to access lots of global variables, but may occasionally use a local variable for looping, or calculations. This decision forces PHP developers to put heaps of "global this-and-that" at the start of each function. If you forget one, or add a new line to a script later on, it will not work correctly. Also, doing something simple like taking a piece of a function and making it into its own function (for readability) won't work by doing a simple copy and paste, unless you carefully inspect it for any references to global variables.
The decision (initially) to add slashes automatically to variables from web pages. Thus if you had in a form:

Nick's cat

The script actually received:

Nick\'s cat

This extra backslash was intended to be helpful if you were going to write the variable to an SQL database, where it would be needed to avoid premature closing of quotes. However if you were not going to do that, you needed to "strip the slashes", and then add them back later.

With extra slashes being added "behind the scenes" scripts soon take on a nightmare proportion of adding and subtracting slashes, in ways which seem unbalanced (because some are implicit, not explicit).

Making things worse, this became an option, so that scripts that work on one site stop working on another because the option was reversed.
The requirement that all variables must start with a $, which grates with programmers who are used to C. In other words you must do:

$foo = $bar; // not --> foo = bar;

Symbols without the $ prefix are also, rather bizarrely, treated as strings, like this:

echo hello; // echoes "hello"

This leads to rather confusing error messages if you forget the $ prefix.
The decision to allow variables inside strings, like this:

$foo = "world"; $bar = "hello, $foo";

Again, this is convenient, but what if:
- You need to put a $ sign inside the string?
- You need to follow the variable directly by a letter (eg. "hello, worlds"), then this won't work:
$foo = "world"; $bar = "hello, $foos";

In this example we don't get "hello, worlds", because it is trying to find the variable "foos".

I know you can use single-quoted strings to avoid this behaviour, but it is still a bit messy.

Sylaer Tue 02 May 2006 05:20 AM #14

Nick-
I grinned when you mentioned the same issues that griped me when I was learning PHP (which, incidentally, was before I started learning C). A year or two ago I could be seen at my desk with my nose to the screen in frustration trying to figure out how many escape characters to put into a string to store it in a mySQL database. (I got up to eight once, no kidding. Strangely, it worked.) Dumping cookie and form variables into the url was a pain too, and ultimately, I think, what caused me to give up PHP as a gaming language. Thus I find myself here.
Anyway, I suspect you'll one day be an authority on Lua, and again it's great to have you here. In that spirit, please continue your discussion on Lua :D

Nick Gammon Australia Forum Administrator Tue 02 May 2006 05:54 AM #15

Well, I try *grin*.

I have done my own documentation on Lua (version 5.0.2) base, coroutines, debug, io, math, os, string, and table functions.

Nick Gammon Australia Forum Administrator Wed 03 May 2006 08:55 PM #16

Accessing the MySQL database from the dynamic web page

Now let's look at accessing a database. This is simple enough, using the luasql library available from the LuaForge site. Assuming we have installed the shared library, we use loadlib to load it into our dynamic page. I have made up an example database using these SQL commands:


CREATE TABLE widgets (
  widget_id int NOT NULL auto_increment primary key,
  widget_name varchar(64) NOT NULL,
  weight int,
  price double
  ) 

INSERT INTO widgets VALUES (1,'Gyro spanner',15,102);
INSERT INTO widgets VALUES (2,'Packet of things',22,14);
INSERT INTO widgets VALUES (3,'Roll of paper',42,95);

The page below will display the first 100 records as a table.

This page makes the widget_id a hyperlink, so we can see more details about an individual widget if we click on the link. The page assumes you are calling it using the method described earlier using the cgiutils.lua as the calling module. The URL I used was:



http://10.0.0.2/cgi-bin/cgiutils.lua/testing/test11.lua

Thus, the file below was called test11.lua and placed in the "testing" subdirectory below where cgiutils.lua resides.


-- load the MySQL dll
assert (loadlib ("/usr/local/lib/libmysql.2.0.1.so", "luaopen_luasqlmysql")) ()

-- HTTP and HTML headers
show_html_header ("Database example")
  
-- Page header
print "<h1>Listing of Widgets</h1>"
  
-- create environment object
env = assert (luasql.mysql())

-- connect to data source
con = assert (env:connect ("databasename", "username", "password", "localhost"))

-- retrieve a cursor
cur = assert (con:execute ("SELECT * from widgets LIMIT 100" ))

-- table header row
print [[
<table border=1 cellpadding=3>
<tr>
<th>ID</th><th>Name</th>
</tr>
]]


-- print all rows, the rows will be indexed by field names
row = cur:fetch ({}, "a")

while row do
  
  -- start new row
  print "<tr>"

  print (string.format (
          '<td><a href="test12.lua?id=%i">%i</a></td>', 
          row.widget_id, 
          row.widget_id))
   
  -- Widget name
  print (string.format (
         '<td>%s</td>', 
         fixhtml (row.widget_name)))

  -- end of row
  print "</tr>"
         
  -- reusing the table of results
  row = cur:fetch (row, "a")
  
end

-- end of table
print "</table>"

-- close everything
cur:close()
con:close()
env:close()  
  
-- wrap up page 
show_html_footer ()

Output was (however displayed as an HTML table):


Listing of Widgets

ID	Name
1 	Gyro spanner
2 	Packet of things
3 	Roll of paper

Nick Gammon Australia Forum Administrator Wed 03 May 2006 09:49 PM #17

Accessing the GET and POST data to display an individual record

The next example (which implements test12.lua which is linked to in the previous post) shows how you can access the variable data which the user chooses.

The important line in this file is:



show_widget (post_data.id or get_data.id)

The tables post_data and get_data have been automatically set up by cgiutils.lua before loading this page. If the user clicks on the "Gyro spanner", then this is the hyperlink:



http://10.0.0.2/cgi-bin/cgiutils.lua/testing/test12.lua?id=1

The important part here is "?id=1" which puts the value "1" into the key "id" in the get_data table. Recall that get_data is the data from the URL.

However the displayed page also lets the user enter another widget ID by entering the number into the form box and clicking on the "Show" button. In this case the widget id ends up in the post_data table. The expression "post_data.id or get_data.id" used above means we prefer the post_data to the get_data, if there is a clash.

The function show_widget:

Exits if no widget supplied (ie test12.lua called with no id on the URL or form data)
Checks the widget id is numeric (to avoid problems with the SQL syntax)
Opens the database
Locates the desired widget with a SELECT statement
Displays an error if that widget is not on file
Otherwise displays its data
Closes the database

The main script calls show_widget and then displays a form to let the user choose a different widget ID (even if the previous one was not there).


function show_widget (which)
  
  -- exit if nothing
  if not which then
    return
  end -- no widget ID supplied

  -- check a number
  local id = tonumber (which)
  
  if not id then
    show_error (string.format ("Widget ID '%s' is not numeric", tostring (which)))
    return
  end -- not numeric
  
  -- load the MySQL dll
  assert (loadlib ("/usr/local/lib/libmysql.2.0.1.so", "luaopen_luasqlmysql")) ()
  
  -- create environment object
  local env = assert (luasql.mysql())
  
  -- connect to data source
  local con = assert (env:connect ("databasename", "username", "password", "localhost"))
  
  -- retrieve a cursor
  local cur = assert (con:execute ("SELECT * from widgets WHERE widget_id = " .. id))
  
  -- fetch the row
  local row = cur:fetch ({}, "a")
  
  if row then
    show_table (row)
  else
    show_error (string.format ("Widget %i not on file", id))
  end -- if
  
  -- close everything
  cur:close()
  con:close()
  env:close()  

end -- show_widget

-- HTTP and HTML headers
show_html_header ("Form example")
  
-- Page header
print "<h1>Display one widget</h1>"

-- show requested record  
show_widget (post_data.id or get_data.id)
 
-- let them enter a new ID
print [[
<form METHOD="post" ACTION="test12.lua">
<p>Enter Widget ID: <input type=text Name="id" size=6 maxlength=6>
<input Type=submit Name=Submit Value="Show">
</p>
</form>
]]

-- wrap up page 
show_html_footer ()

Example output:


Display one widget

widget_id	1
price	102
widget_name	Gyro spanner
weight	15

Enter Widget ID: ______ [Show]

Nick Gammon Australia Forum Administrator Thu 04 May 2006 01:19 AM #18

Example of using cookies

This page (test13.lua) demonstrates getting a cookie value, and setting it from a form. This is a simple demonstration of how you can use a cookie to remember a value from one time the user goes to a page, to another (like in this forum, when you log on).


-- Example of using cookies

-- if they changed it use the new value, otherwise take the cookie value
foo = post_data.foo or cookie_data.foo

-- HTTP header
show_http_header { foo = foo }

-- HTML header
show_html_header ("Cookie example")
  
-- Page header
print "<h1>Testing setting a cookie</h1>"
 
-- let them enter a new cookie value
print [[
<form METHOD="post" ACTION="test13.lua">
<p>Enter cookie value: <input type=text Name="foo" size=20 maxlength=20
]]
if foo then
  print ('value="' .. foo .. '"')
end -- if cookie already set
print [[
> 
<input Type=submit Name=Submit Value="Show">
</p>
</form>
]]

Since cookies are sent as part of the HTTP header, they must be changed (if you are planning to set or change them) as the very first thing you do, before sending any other HTML code. Thus the first thing this script does is set the variable "foo" to be either the post data (in case it has just been changed) or the cookie data (from last time).

Then we call:



show_http_header { foo = foo }

The optional argument to show_http_header lets you specify a table of cookie name/value pairs. You could enhance this by adding expiry dates etc., but for simplicity I haven't here.

Then the page displays a form showing the current value and lets you change it.

Nick Gammon Australia Forum Administrator Thu 04 May 2006 11:31 PM #19

Making the URL look nicer

One minor problem with the technique described above is that the URL that the web page user sees looks a bit strange, like this:



http://www.yoursite.com/cgi-bin/cgiutils.lua/testing/test12.lua

The "problem" is the two lots of ".lua" items in the URL. It works, but looks strange. The simple solution is to simply rename cgiutils.lua as something that looks like a directory, eg. "scripts". Then the URL becomes:



http://www.yoursite.com/cgi-bin/scripts/testing/test12.lua

Now you are hiding from the users of your web site that the script is really the file "scripts". It just makes the URL look a bit more natural.

Nick Gammon Australia Forum Administrator Fri 05 May 2006 04:55 AM #20

Making writing HTML easier

Now that we have got this far, let's make generating the actual HTML easier. One thing page designers do an awful lot of is stuff like this:



<h1> My page heading &amp; other stuff </h1>

It would be nice to simplify this. We need to group together:

The opening tag (eg. <h1> )
The text in the middle, escaping out things like <, > and &
The closing tag (eg. </h1> )

A simple function will do this:


function tag (name, contents)

  print ("<" .. name .. ">" .. 
         fixhtml (contents) .. "</" .. name .. ">")

end -- function tag

We can call it like this:

tag ("h1", "My page heading & other stuff")

However, this function make a couple of assumptions:

That the tag doesn't have arguments (like align="left")
That the contents are a simple string

For more complex things like tables, this isn't really true.

Let's make a more complex function, that will still handle the simple case, but also handles tag arguments, nested tags, and so on.

This is my next attempt:


function tag (name, ...)

local i = 1  -- 2nd argument may be parameters, or simply the thing to be displayed

  io.write ("<" .. name)

  -- handle parameters (like align="left")
  if type (arg [i]) == "table" then
    for k, v in pairs (arg [i]) do
      -- boolean attributes (eg. selected=true) have to be shown simply as "selected" 
      -- or omitted if false
      if type (v) == "boolean" then
        if v then
          io.write (' ' .. k)
        end -- value is true
      else      
        io.write (' ' .. k .. '="' .. fixhtml (v) .. '"')
      end -- if boolean or not
    end -- for each parameter 
    i = i + 1
  end -- tag parameters supplied
  
  -- if no contents assume closed tag
  if not arg [i] then
    print ("/>")
    return
  end -- no argument - must be tag with no contents (eg. <br/>)

  -- finished the opening tag
  io.write (">")
  
  -- now the contents of the tag
  -- if a function, call it, passing it any further arguments
  
  if type (arg [i]) == "function" then
    local f = arg [i]
    local j
    -- remove elements up to the function arguments
    for j = 1, i do
      table.remove (arg, 1)
    end -- removing argument
    print ()  -- may as well have a newline for neatness
    f (unpack (arg))
  else
    io.write (fixhtml (arg [i]))
  end -- if
  
  -- close the tag (and do the newline)
  print ("</" .. name .. ">")

end -- function tag

The first argument to "tag" is still the name of the tag (eg. hr, table, tr, th).

The next argument can optionally be a table of parameters. If the second argument is a table, it is expanded out, as arguments to the tag. For example:



tag ("hr", {size=10, width=110, align="left"} )

This function call generates this line:



<hr align="left" width="110" size="10"/>

In this case the absence of any further arguments generates an "empty" tag, that is one with no contents.

Now let's try something with contents.



tag ("p", {align="center"}, "This is my paragraph" )

This function call generates this line:



<p align="center">This is my paragraph</p>

Now we have the opening and closing tag.

Finally we can call a function rather than have straight text. This allows us to nest calls, to do something like iterate through a table. Here is an example:


function showrow (k, v) 
  tag ("th", k)
  tag ("td", v)
end -- showrow

function showtable (t)
  for k, v in ipairs (t) do
    tag ("tr", {align="left"}, showrow, k, v)  
  end -- for
end -- showtable

tag ("table",  {border = 1, cellpadding = 5}, 
     showtable, 
     { "every", "good", "boy", "deserves", "fruit" }
    )

This example uses the tag function a number of times. The last one (the one that is called first) sets up a table with the "table" tag, and requests that the "showtable" function be called. The arguments to showtable are the inlined table.

The function showtable then iterates through each table item, doing a "tr" tag, and for each one calls "showrow", passing to that two arguments, the key and value for that particular row.

Finally the function showrow calls tag twice, to do the "th" and "td" data elements. The generated HTML from all this is;


<table border="1" cellpadding="5">
<tr align="left">
<th>1</th>

<td>every</td>
</tr>
<tr align="left">
<th>2</th>
<td>good</td>
</tr>
<tr align="left">
<th>3</th>
<td>boy</td>
</tr>
<tr align="left">
<th>4</th>

<td>deserves</td>
</tr>
<tr align="left">
<th>5</th>
<td>fruit</td>
</tr>
</table>

Overall I think that the tag function simplifies the generation of HTML, including complex nested calls, and ensures that each opening tag is balanced by a closing tag.

Nick Gammon Australia Forum Administrator Wed 10 May 2006 10:42 PM #21

How to write a page that executes user-entered Lua code

This is an interesting project. I often want to experiment with Lua snippets but am not sitting at my PC with Lua installed. This page here demonstrates how you can let someone execute Lua code from their web browser.

To make it fairly robust, it has a number of interesting features:

Each time you run it, it remembers what code you entered last time (so you can fix syntax errors)
Any output from the executed code (via the 'print' statement) is output to the page, however with HTML properly escaped (that is, "<" turned into "<" and so on)
A syntax error during the compile phase is shown as such
Runtime errors are detected and shown with a stack traceback
To hopefully stop malicious code, the entire code is executed in a specially-constructed "sandbox" which has most of the library Lua functions in it, however omitting things like "os.execute" and "os.remove" which might cause grief.
To guard against runaway code (like "repeat until false") hanging the web server, there is code to detect that.
To guard against someone writing thousands of lines of output, and making a web page that browsers probably could not load, there is code to stop that (built into the "print" function)
To guard against someone writing thousands of bytes, without starting a new line, there is also a test on the number of bytes printed.
Added a simulated io.write function to help people who want to build up an output line in pieces.

Here is the code (I called it "execute.lua" which is referred to internally in the <form> data) ...


-- limits
runaway_instruction_limit = 100000  -- how many instructions
linelimit = 500                     -- how many lines can be printed
outputlimit = 10000                 -- how many bytes can be printed

-- this is called if 'runaway_instruction_limit' instructions are done
function hook ()
  error ("Runaway instruction limit reached")
end -- hook

-- heading for errors
function heading (s)
  print '<p><span style="font-family: sans-serif; font-weight: bolder;">'
  print (fixhtml (s))
  print '</span></p>'
end -- heading

linecount = 0    -- how many lines they have printed so far
outputcount = 0  -- how many bytes they have printed so far

-- local print function for use inside executed code, uses fixhtml on the string
-- does a newline at the end, adds a space between arguments
function myprint (...)
  local i
  
  -- check for runaway outputting
  linecount = linecount + 1  
  assert (linecount <= linelimit, "too many lines printed")
  
  -- I am not using ipairs because that stops on a nil argument
  for i = 1, arg.n do
    local s = fixhtml (tostring (arg [i]))
    outputcount = outputcount + string.len (s) + 1
    assert (outputcount <= outputlimit, "too many bytes printed")
    io.write (s .. " ")
  end -- for
  print ""  -- final end of line
  
end -- myprint

-- local io.write function for use inside executed code, uses fixhtml on the string
-- doesn't do a newline, doesn't add a space between arguments
function mywrite (...)
  local i

  -- I am not using ipairs because that stops on a nil argument
  for i = 1, arg.n do
    local s = fixhtml (tostring (arg [i]))
    outputcount = outputcount + string.len (s)
    assert (outputcount <= outputlimit, "too many bytes printed")
    io.write (s)
  end -- for
end -- mywrite

-- makes a sandbox to make execution safer
function MakeSandbox ()
  local box = {}
  local _, name

  -- add in global functions we consider safe
  for _, name in ipairs 
  {
   -- functions 
   "_TRACEBACK", "__pow", -- helper functions
   "assert", "error", -- error management
   "collectgarbage", "gcinfo", -- garbage collection
   "getmetatable", "setmetatable", -- metatables
   "ipairs", "pairs", "next", -- looping through tables
   "loadstring", "pcall", "xpcall", -- calling functions
   "rawequal", "rawget", "rawset", -- raw table management 
   "tonumber", "tostring", "type", -- type management
   "unpack", -- argument management
   
   -- version string
   "_VERSION",
   }
   do
    box [name] = _G [name]
   end -- for each global function
    
   box._G = box  -- _G points to itself
   box.print = myprint -- special print function escapes HTML codes
   box.io = {}
   box.io.write = mywrite -- my special write function
   
  -- now other libraries
  -- we will omit io - don't want file io
  -- also omit debug, that could be dangerous to play with
  for _, name in ipairs 
  { "coroutine", "math", "os", "string", "table", }
   do
     box [name] = {}  -- make library sub-table
     
     -- copy functions into the table
     local k, v
     for k, v in pairs (_G [name]) do
       box [name] [k] = v
     end -- for each entry in the library table 
   end -- for each library
  
  -- omit a couple of the os functions we don't want them to play with
  box.os.execute = nil  -- no executing arbitrary code
  box.os.exit = nil     -- don't terminate us
  box.os.remove = nil   -- don't remove files
  box.os.rename = nil   -- don't rename files
  box.os.getenv = nil   -- don't leak information about our environment
    
  return box

end -- MakeSandbox

-- executes the code they entered, in a "sandbox" environment
function Execute (code)

  local ok
  
  -- this is called for a runtime error
  local function error_handler (err)
    heading ("Runtime error")
    print (fixhtml (err))
    local s = debug.traceback ()
    -- strip traceback from the xpcall upwards
    s = string.gsub (s, "%[C%]%: in function `xpcall'.*", "")
    print (fixhtml (s))
  end -- function error_handler
  
  -- try compiling the code
  local f, err = loadstring (code, "Code")
  
  -- error on compile
  if not f then
    heading ("Syntax error")
    print ("<pre><code>" .. fixhtml (err) .. "</code></pre>")
    return  -- cannot execute it
  end -- syntax error

  -- sandbox them so they don't start deleting files
  setfenv (f, MakeSandbox ())
  
  -- put limit on runaway loops
  debug.sethook (hook, "", runaway_instruction_limit)
  
  -- try running the code
  heading ("Output")
  print ("<pre><code>")
  ok = xpcall (f, error_handler)
  print ("</code></pre>")

  return ok
end -- function Execute

-->>>> CODE STARTS HERE <<<<--

-- HTTP and HTML headers
show_html_header ("Execute Lua code")
  
print [[
<form METHOD="post" ACTION="execute.lua">
<p>Enter Lua code to be executed:</p> 
<p>
<textarea name="code" wrap="virtual" rows="20" cols="80" >]]
io.write (fixhtml (post_data.code or ""))  -- what they had last time
print [[
</textarea>
</p>
<input Type=submit Name=Submit Value="Execute">
</form>
]]

-- now execute it

if post_data.code then
  if Execute (string.gsub (post_data.code, "\r", "")) then
    heading "Completed OK"
  end -- if finished ok
end -- if

show_html_footer ()

This code assumes you are using the method described earlier in this thread of executing using cgiutils.lua. This is the URL I used to test it:



http://10.0.0.2/cgi-bin/cgiutils.lua/testing/execute.lua

If the executed code completes successfully you see the heading "Completed OK", to confirm the script finished.

I have made a couple of edits to the code, taking out getfenv and setfenv, on the grounds that they might be used for escaping from the sandbox environment.

Nick Gammon Australia Forum Administrator Thu 11 May 2006 12:04 AM #22

This test script can be used in conjunction with the "Lua execute" page presented above. It is a slight variation on the "tprint" utility which is included with MUSHclient. It is a recursive table-printer. In this case it prints the _G table, showing all of the functions and variables made available in the sandbox to the script writer.


function tprint (t, indent, done)
  -- show strings differently to distinguish them from numbers
  local function show (val)
    if type (val) == "string" then
      return '"' .. val .. '"'
    else
      return tostring (val)
    end -- if
  end -- show
  -- entry point here
  done = done or {}
  indent = indent or 0
  for key, value in pairs (t) do
    io.write (string.rep (" ", indent)) -- indent it
    if type (value) == "table" and not done [value] then
      done [value] = true
      print(show (key), ":");
      tprint (value, indent + 2, done)
    else
      io.write (show (key), "=")
      print (show (value))
    end
  end
end


tprint (_G)

Output for me was:


Output


"string" : 
  "sub"=function: 0x8066758 
  "gfind"=function: 0x80668e8 
  "rep"=function: 0x80667c0 
  "find"=function: 0x80668b0 
  "char"=function: 0x8066790 
  "dump"=function: 0x8066878 
  "byte"=function: 0x8066960 
  "upper"=function: 0x8066840 
  "gsub"=function: 0x8066920 
  "format"=function: 0x8066998 
  "len"=function: 0x8066728 
  "lower"=function: 0x8066808 
"xpcall"=function: 0x8063d18 
"tprint"=function: 0x8072f80 
"table" : 
  "setn"=function: 0x8065308 
  "insert"=function: 0x80654d8 
  "getn"=function: 0x80653b8 
  "foreachi"=function: 0x8065380 
  "remove"=function: 0x8065510 
  "sort"=function: 0x8065340 
  "concat"=function: 0x80650b8 
  "foreach"=function: 0x80652d0 
"rawset"=function: 0x8063ca8 
"gcinfo"=function: 0x8063d90 
"tonumber"=function: 0x8064838 
"print"=function: 0x806b298 
"coroutine" : 
  "resume"=function: 0x80650f0 
  "yield"=function: 0x8065128 
  "status"=function: 0x8065078 
  "wrap"=function: 0x8065040 
  "create"=function: 0x8064fc8 
"os" : 
  "setlocale"=function: 0x8064aa8 
  "getenv"=function: 0x80649d8 
  "difftime"=function: 0x8064a38 
  "time"=function: 0x8064ae0 
  "clock"=function: 0x8065000 
  "date"=function: 0x8065470 
  "tmpname"=function: 0x8064b18 
"__pow"=function: 0x80671e0 
"math" : 
  "log"=function: 0x8066058 
  "atan"=function: 0x8067010 
  "ldexp"=function: 0x8067248 
  "deg"=function: 0x80670e0 
  "tan"=function: 0x8064e78 
  "cos"=function: 0x8064e48 
  "abs"=function: 0x8066bd8 
  "random"=function: 0x8067178 
  "rad"=function: 0x8067140 
  "frexp"=function: 0x8064f50 
  "pi"=3.1415926535898 
  "floor"=function: 0x8064ee8 
  "acos"=function: 0x8064df0 
  "max"=function: 0x80672e0 
  "sqrt"=function: 0x8067280 
  "atan2"=function: 0x8067048 
  "asin"=function: 0x8064db8 
  "min"=function: 0x80672b0 
  "mod"=function: 0x8064f18 
  "log10"=function: 0x8067080 
  "exp"=function: 0x80670b0 
  "pow"=function: 0x8067110 
  "randomseed"=function: 0x80671b0 
  "sin"=function: 0x8064da0 
  "ceil"=function: 0x8064eb0 
"_G" : 
  "string"=table: 0x8072f30 
  "xpcall"=function: 0x8063d18 
  "tprint"=function: 0x8072f80 
  "table"=table: 0x8072f58 
  "rawset"=function: 0x8063ca8 
  "gcinfo"=function: 0x8063d90 
  "tonumber"=function: 0x8064838 
  "print"=function: 0x806b298 
  "coroutine"=table: 0x8063b78 
  "os"=table: 0x8063c08 
  "__pow"=function: 0x80671e0 
  "math"=table: 0x8063be0 
  "_G"=table: 0x80686f0 
  "pcall"=function: 0x8063ce0 
  "_VERSION"="Lua 5.0.2" 
  "unpack"=function: 0x8064918 
  "type"=function: 0x80648a8 
  "pairs"=function: 0x80647c8 
  "next"=function: 0x8064758 
  "_TRACEBACK"=function: 0x8067898 
  "assert"=function: 0x80648e0 
  "tostring"=function: 0x8064870 
  "ipairs"=function: 0x8064790 
  "loadstring"=function: 0x8063e38 
  "io" : 
    "write"=function: 0x806b2b8 
  "rawequal"=function: 0x8064950 
  "rawget"=function: 0x8064988 
  "collectgarbage"=function: 0x8063d58 
  "setmetatable"=function: 0x80646b0 
  "getmetatable"=function: 0x8064670 
  "error"=function: 0x8064630 
"pcall"=function: 0x8063ce0 
"_VERSION"="Lua 5.0.2" 
"unpack"=function: 0x8064918 
"type"=function: 0x80648a8 
"pairs"=function: 0x80647c8 
"next"=function: 0x8064758 
"_TRACEBACK"=function: 0x8067898 
"assert"=function: 0x80648e0 
"tostring"=function: 0x8064870 
"ipairs"=function: 0x8064790 
"loadstring"=function: 0x8063e38 
"io"=table: 0x8065d58 
"rawequal"=function: 0x8064950 
"rawget"=function: 0x8064988 
"collectgarbage"=function: 0x8063d58 
"setmetatable"=function: 0x80646b0 
"getmetatable"=function: 0x8064670 
"error"=function: 0x8064630 

Completed OK

Nick Gammon Australia Forum Administrator Fri 08 Sep 2006 04:30 AM #23

I have made a minor change to the cgiutils.lua file presented further back.

When testing it with the thttpd server it hung on the line which retrieved the post data. It seems that I should only be reading from stdin if the method is actually POST, otherwise it can just hang. This fixes it:


  if os.getenv ("REQUEST_METHOD") == "POST" then
    for _, v in ipairs (split (io.read ("*a"), "&")) do
      assemble_value (v, post_data)
    end -- for
  end -- if post

Nick Gammon Australia Forum Administrator Sat 09 Sep 2006 10:03 PM #24

I also found, using thttpd, that with some clients they hung with large amounts being returned, if you didn't send another header (Connection: close), plus thttpd didn't identify itself or send the date. A couple of extra lines fixes that:

In cgiutils.lua, after the line:


 print ("Content-Type: text/html; charset=iso-8859-1")

... add these lines:


  print ("Date: " .. os.date ("%a, %d %b %Y %H:%M:%S GMT"))
  print ("Server: " .. "thttpd/2.25b")  --> or whatever it is
  print ("Connection: close")