Python CGI Programming

What is CGI

CGI is currently maintained by NCSA, and NCSA defines CGI as follows:

CGI (Common Gateway Interface), a general gateway interface, is a program that runs on a server such as an HTTP server, providing an interface to client HTML pages.

Web Browsing

To better understand how CGI works, we can look at the process of clicking a link or URL on a web page:

  • 1. Use your browser to access the URL and connect to the HTTP web server.
  • 2. After receiving the request, the web server parses the URL and checks whether the accessed file exists on the server. If it exists, it returns the file content; if not, it returns the corresponding error message.
  • 3. The browser receives the information from the server and displays the received file or error message.

CGI programs can be Python scripts, PERL scripts, SHELL scripts, C or C++ programs, etc.

CGI Architecture Diagram

cgiarch

Web Server Support and Configuration

Before you start CGI programming, make sure your web server supports CGI and has a CGI handler configured.

Apache CGI Configuration:

Set up the CGI directory:

ScriptAlias /cgi-bin/ /var/www/cgi-bin/

All CGI programs executed by HTTP servers are stored in a pre-configured directory. This directory is called the CGI directory and, by convention, it is named /var/www/cgi-bin.

CGI files have the extension .cgi, and Python can also use the .py extension.

By default, Linux servers are configured to run the cgi-bin directory in /var/www.

If you want to specify another directory for running CGI scripts, you can modify the httpd.conf configuration file as follows:

<Directory "/var/www/cgi-bin">
   AllowOverride None
   Options +ExecCGI
   Order allow,deny
   Allow from all
</Directory>

Add the .py suffix to AddHandler, so we can access Python script files ending in .py:

AddHandler cgi-script .cgi .pl .py

First CGI Program

We use Python to create our first CGI program, named hello.py, located in the /var/www/cgi-bin directory, with the following content:

Example

#!/usr/bin/python3

print ("Content-type:text/html")
print ()                             # Blank line, tells the server that the header is finished
print ('<html>')
print ('<head>')
print ('<meta charset="utf-8">')
print ('<title>Hello Word - My first CGI program!</title>')
print ('</head>')
print ('<body>')
print ('<h2>Hello Word! I am the first CGI program from Example Tutorial</h2>')
print ('</body>')
print ('</html>')

After saving the file, modify hello.py and set the file permissions to 755:

chmod 755 hello.py

The result of accessing the above program in a browser is as follows:

The hello.py script is a simple Python script. The first line of output, "Content-type:text/html", is sent to the browser to tell the browser that the displayed content type is "text/html".

Use print to output a blank line to tell the server that the header information is complete.

HTTP Headers

"Content-type:text/html" in the hello.py file is part of the HTTP header. It is sent to the browser to tell the browser the content type of the file.

The format of the HTTP header is as follows:

HTTP 字段名: 字段内容

For example:

Content-type: text/html

The following table describes the information commonly used in HTTP headers in CGI programs:

headDescription
Content-type: The MIME type corresponding to the request entity. For example: Content-type:text/html
Expires: Date The date and time for which the response expires
Location: URL Used to redirect the receiver to a location other than the requested URL to complete the request or identify a new resource
Last-modified: DateThe last modified time of the requested resource
Content-length: NThe content length of the request
Set-Cookie: String Set HTTP Cookie

CGI Environment Variables

All CGI programs receive the following environment variables, which play an important role in CGI programs:

Variable NameDescription
CONTENT_TYPEThe value of this environment variable indicates the MIME type of the information being passed. Currently, the CONTENT_TYPE environment variable is generally: application/x-www-form-urlencoded, which means the data comes from an HTML form.
CONTENT_LENGTHWhen the server communicates with the CGI program via POST, this environment variable indicates the number of valid data bytes that can be read from standard input (STDIN). This variable must be used when reading input data.
HTTP_COOKIEThe COOKIE content on the client machine.
HTTP_USER_AGENTProvides client browser information including version numbers or other proprietary data.
PATH_INFOThe value of this environment variable represents additional path information that follows the CGI program name. It is often used as a parameter to the CGI program.
QUERY_STRINGIf the server passes information to the CGI program using the GET method, the value of this environment variable is the information being passed. This information follows the CGI program name, separated by a question mark '?'.
REMOTE_ADDRThe value of this environment variable is the IP address of the client making the request, for example 192.168.1.67 above. This value is always present. It is the only identifier that the web client needs to provide to the web server, and it can be used in CGI programs to distinguish different web clients.
REMOTE_HOSTThe value of this environment variable contains the hostname of the client sending the CGI request. If you do not wish to query it, this environment variable does not need to be defined.
REQUEST_METHODProvides the method by which the script was called. For scripts using the HTTP/1.0 protocol, only GET and POST are meaningful.
SCRIPT_FILENAMEThe full path of the CGI script
SCRIPT_NAMEThe name of the CGI script
SERVER_NAMEThis is the hostname, alias, or IP address of your web server.
SERVER_SOFTWAREThe value of this environment variable contains the name and version number of the HTTP server that called the CGI program. For example, the value above is Apache/2.2.14(Unix)

The following is a simple CGI script that outputs CGI environment variables:

Example

#!/usr/bin/python3

import os

print ("Content-type: text/html")
print ()
print ("<meta charset=\"utf-8\">")
print ("<b>Environment Variables</b><br>")
print ("<ul>")
for key in os.environ.keys():
    print ("<li><span style='color:green'>%30s </span> : %s </li>" % (key,os.environ[key]))
print ("</ul>")

Save the above as test.py, set the file permissions to 755, and the execution result is as follows:

GET and POST Methods

Browser clients pass information to the server using two methods: the GET method and the POST method.

Passing Data Using the GET Method

The GET method sends encoded user information to the server. The data is contained in the URL of the requested page, separated by a "?" sign, as shown below:

http://www.test.com/cgi-bin/hello.py?key1=value1&key2=value2
Some other notes about GET requests:
  • GET requests can be cached
  • GET requests remain in browser history
  • GET requests can be bookmarked
  • GET requests should not be used when handling sensitive data
  • GET requests have length limitations
  • GET requests should only be used to retrieve data

Simple URL Example: GET Method

The following is a simple URL that uses the GET method to send two parameters to the hello_get.py program:

/cgi-bin/hello_get.py?name=Example&url=http://www.example.com

The following is the code of the hello_get.py file:

Example

#!/usr/bin/python3

# CGI processing module
import cgi, cgitb

# Create an instance of FieldStorage
form = cgi.FieldStorage()

# Get data
site_name = form.getvalue('name')
site_url  = form.getvalue('url')

print ("Content-type:text/html")
print ()
print ("<html>")
print ("<head>")
print ("<meta charset=\"utf-8\">")
print (<title>Example CGI Test Example</title>)
print ("</head>")
print ("<body>")
print (<h2>%s official website: %s</h2> % (site_name, site_url))
print ("</body>")
print ("</html>")

After saving the file, modify hello_get.py, and change the file permission to 755:

chmod 755 hello_get.py

Browser request output result:

Simple Form Example: GET Method

The following is an HTML form that uses the GET method to send two data items to the server. The submitted server script is also the hello_get.py file. The hello_get.html code is as follows:

Example

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Example (example.com)</title>
</head>
<body>
<form action="/cgi-bin/hello_get.py" method="get">
Site Name:<input type="text" name="name">  <br />

Site URL:<input type="text" name="url" />
<input type="submit" value=Submit />
</form>
</body>
</html>

By default, the cgi-bin directory can only store script files. We store hello_get.html in the test directory and change the file permission to 755:

chmod 755 hello_get.html

GIF demo is as shown below:

Passing Data Using the POST Method

Using the POST method to pass data to the server is safer and more reliable. For sensitive information such as user passwords, POST should be used to transmit data.

The following is also hello_get.py. It can also handle POST form data submitted by the browser:

Example

#!/usr/bin/python3

# CGI processing module
import cgi, cgitb

# Create an instance of FieldStorage
form = cgi.FieldStorage()

# Get data
site_name = form.getvalue('name')
site_url  = form.getvalue('url')

print ("Content-type:text/html")
print ()
print ("<html>")
print ("<head>")
print ("<meta charset=\"utf-8\">")
print (<title>Example CGI Test Example</title>)
print ("</head>")
print ("<body>")
print (<h2>%s official website: %s</h2> % (site_name, site_url))
print ("</body>")
print ("</html>")

The following is a form using the POST method (method="post") to submit data to the server script hello_get.py:

Example

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Example (example.com)</title>
</head>
<body>
<form action="/cgi-bin/hello_get.py" method="post">
Site Name:<input type="text" name="name">  <br />

Site URL:<input type="text" name="url" />
<input type="submit" value=Submit />
</form>
</body>
</html>
</form>

GIF demo is as shown below:

Passing Checkbox Data via CGI Program

Checkbox is used to submit one or more option data. The HTML code is as follows:

Example

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Example (example.com)</title>
</head>
<body>
<form action="/cgi-bin/checkbox.py" method="POST" target="_blank">
<input type="checkbox" name="example" value="on" />Example
<input type="checkbox" name="google" value="on" /> Google
<input type="submit" value=Select site />
</form>
</body>
</html>

The following is the code of the checkbox.py file:

Example

#!/usr/bin/python3

# Import the CGI processing module
import cgi, cgitb

# Create an instance of FieldStorage
form = cgi.FieldStorage()

# Receive field data
if form.getvalue('google'):
   google_flag = Yes
else:
   google_flag = No

if form.getvalue('example'):
   example_flag = Yes
else:
   example_flag = No

print ("Content-type:text/html")
print ()
print ("<html>")
print ("<head>")
print ("<meta charset=\"utf-8\">")
print (<title>Example CGI Test Example</title>)
print ("</head>")
print ("<body>")
print (<h2> Whether Example is selected: %s</h2> % example_flag)
print (<h2> Whether Google is selected: %s</h2> % google_flag)
print ("</body>")
print ("</html>")

Change the permission of checkbox.py:

chmod 755 checkbox.py

Browser access GIF demo:

Passing Radio Data via CGI Program

Radio only sends one piece of data to the server. The HTML code is as follows:

Example

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Example (example.com)</title>
</head>
<body>
<form action="/cgi-bin/radiobutton.py" method="post" target="_blank">
<input type="radio" name="site" value="example" />Example
<input type="radio" name="site" value="google" /> Google
<input type="submit" value=Submit />
</form>
</body>
</html>

The radiobutton.py script code is as follows:

Example

#!/usr/bin/python3

# Import the CGI processing module
import cgi, cgitb

# Create an instance of FieldStorage
form = cgi.FieldStorage()

# Receive field data
if form.getvalue('site'):
   site = form.getvalue('site')
else:
   site = Submitted data is empty

print ("Content-type:text/html")
print ()
print ("<html>")
print ("<head>")
print ("<meta charset=\"utf-8\">")
print (<title>Example CGI Test Example</title>)
print ("</head>")
print ("<body>")
print (<h2> The selected website is %s</h2> % site)
print ("</body>")
print ("</html>")

Change the permission of radiobutton.py:

chmod 755 radiobutton.py

Browser access GIF demo:

Passing Textarea Data via CGI Program

Textarea sends multi-line data to the server. The HTML code is as follows:

Example

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Example (example.com)</title>
</head>
<body>
<form action="/cgi-bin/textarea.py" method="post" target="_blank">
<textarea name="textcontent" cols="40" rows="4">
Enter content here...
</textarea>
<input type="submit" value=Submit />
</form>
</body>
</html>

The textarea.py script code is as follows:

Example

#!/usr/bin/python3

# Import the CGI processing module
import cgi, cgitb

# Create an instance of FieldStorage
form = cgi.FieldStorage()

# Receive field data
if form.getvalue('textcontent'):
   text_content = form.getvalue('textcontent')
else:
   text_content = No content

print ("Content-type:text/html")
print ()
print ("<html>")
print ("<head>")
print ("<meta charset=\"utf-8\">")
print (<title>Example CGI Test Example</title>)
print ("</head>")
print ("<body>")
print (<h2> The input is: %s</h2> % text_content)
print ("</body>")
print ("</html>")
>

Change the permission of textarea.py:

chmod 755 textarea.py

Browser access GIF demo:

Passing Dropdown Data via CGI Program.

The HTML dropdown box code is as follows:

Example

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Example (example.com)</title>
</head>
<body>
<form action="/cgi-bin/dropdown.py" method="post" target="_blank">
<select name="dropdown">
<option value="example" selected>Example</option>
<option value="google">Google</option>
</select>
<input type="submit" value=Submit/>
</form>
</body>
</html>

The dropdown.py script code is as follows:

Example

#!/usr/bin/python3

# Import the CGI processing module
import cgi, cgitb

# Create an instance of FieldStorage
form = cgi.FieldStorage()

# Receive field data
if form.getvalue('dropdown'):
   dropdown_value = form.getvalue('dropdown')
else:
   dropdown_value = No content

print ("Content-type:text/html")
print ()
print ("<html>")
print ("<head>")
print ("<meta charset=\"utf-8\">")
print (<title>Example CGI Test Example</title>)
print ("</head>")
print ("<body>")
print (<h2> The selected option is: %s</h2> % dropdown_value)
print ("</body>")
print ("</html>")

Change the permission of dropdown.py:

chmod 755 dropdown.py

Browser access GIF demo:

Using Cookies in CGI

A major shortcoming of the HTTP protocol is that it does not determine the user's identity, which brings great inconvenience to programmers. The emergence of the cookie function makes up for this deficiency.

A cookie is to write record data to the client's hard disk through the client's browser while the client accesses a script. When the client accesses the script next time, the data information is retrieved, thereby achieving the function of identity discrimination. Cookies are commonly used in identity verification.

　

Cookie Syntax

HTTP cookies are sent through the HTTP header, which precedes file transfer. The syntax of the Set-Cookie header is as follows:

Set-cookie:name=name;expires=date;path=path;domain=domain;secure
  • name=name:The value of the cookie needs to be set (the name cannot use ";and,sign), when there are multiple name values, use;; to separate, for example:name1=name1;name2=name2;name3=name3。
  • expires=date:Cookie validity period, format: expires="Wdy,DD-Mon-YYYY HH:MM:SS"
  • path=path: Set the path supported by the cookie. If path is a directory, the cookie takes effect for all files and subdirectories under this directory, for example: path="/cgi-bin/". If path is a file, the cookie only takes effect for this file, for example: path="/cgi-bin/cookie.cgi".
  • domain=domain:The domain for which the cookie is valid, for example: domain="www.example.com"
  • secure:If this flag is given, it means the cookie can only be transmitted through an HTTPS server using the SSL protocol.
  • Cookie reception is achieved by setting the environment variable HTTP_COOKIE. CGI programs can obtain cookie information by querying this variable.

Setting Cookies

Setting cookies is very simple. Cookies are sent separately in the HTTP header. The following example sets name and expires in the cookie:

Example

#!/usr/bin/python3

print ('Set-Cookie: name="Example";expires=Wed, 28 Aug 2016 18:30:00 GMT')
print ('Content-Type: text/html')

print ()
print ("""
<html>
  <head>
    <meta charset="utf-8">
<title>Example (example.com)</title>
  </head>
    <body>
        <h1>Cookie set OK!</h1>
    </body>
</html>
""")

Save the above code to cookie_set.py and change the permission of cookie_set.py:

chmod 755 cookie_set.py

The above example uses the Set-Cookie header information to set Cookie information. The optional items set other Cookie attributes, such as expiration time Expires, domain Domain, path Path. This information is set"Content-type:text/html"before.

Retrieving Cookie Information

Retrieving Cookie information is very simple. The Cookie information is stored in the CGI environment variable HTTP_COOKIE. The storage format is as follows:

key1=value1;key2=value2;key3=value3....

The following is a simple CGI program to retrieve cookie information:

Example

#!/usr/bin/python3

# Import module
import os
import http.cookies

print ("Content-type: text/html")
print ()

print ("""
<html>
<head>
<meta charset="utf-8">
<title>Example Tutorial (example.com)</title>
</head>
<body>
<h1>Read cookie information</h1>
""")

if 'HTTP_COOKIE' in os.environ:
    cookie_string=os.environ.get('HTTP_COOKIE')
    c= http.cookies.SimpleCookie()
   # c=Cookie.SimpleCookie()
    c.load(cookie_string)

    try:
        data=c['name'].value
        print ("cookie data: "+data+"<br>")
    except KeyError:
        print ("Cookie not set or expired<br>")
print ("""
</body>
</html>
""")

Save the above code to cookie_get.py, and modify the permissions of cookie_get.py:

chmod 755 cookie_get.py

The GIF demonstrating the cookie setting above is as follows:

File Upload Example

HTML forms for uploading files need to setenctypethe attribute tomultipart/form-data, the code is as follows:

Example

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Example Tutorial (example.com)</title>
</head>
<body>
 <form enctype="multipart/form-data"
                    action="/cgi-bin/save_file.py" method="post">
   <p>Select file:<input type="file" name="filename" /></p>
   <p><input type="submit" value="Upload" /></p>
   </form>
</body>
</html>

The code for the save_file.py script file is as follows:

Example

#!/usr/bin/python3

import cgi, os
import cgitb; cgitb.enable()

form = cgi.FieldStorage()

# Get the file name
fileitem = form['filename']

# Check if the file has been uploaded
if fileitem.filename:
   # Set the file path
   fn = os.path.basename(fileitem.filename)
   open('/tmp/' + fn, 'wb').write(fileitem.file.read())

   message = 'File "' + fn + '" uploaded successfully'
   
else:
   message = 'File not uploaded'
   
print ("""\
Content-Type: text/html\n
<html>
<head>
<meta charset="utf-8">
<title>Example Tutorial (example.com)</title>
</head>
<body>
   <p>%s</p>
</body>
</html>
""" % (message,))

Save the above code to save_file.py, and modify the permissions of save_file.py:

chmod 755 save_file.py

The GIF demonstrating the cookie setting above is as follows:

If your system is Unix/Linux, you must replace the file separator; under Windows, you only need to use the open() statement:

fn = os.path.basename(fileitem.filename.replace("\\", "/" ))

File Download Dialog

First, create a foo.txt file in the current directory for the program to download.

File download is implemented by setting HTTP header information. The functional code is as follows:

Example

#!/usr/bin/python3

# HTTP headers
print ("Content-Disposition: attachment; filename=\"foo.txt\"")
print ()
# Open file
fo = open("foo.txt", "rb")

str = fo.read();
print (str)

# Close file
fo.close()
Other extensions