Use Python3 and the http.server Library to Create a Development Server / Framework

A discussion and example of the use of Python3 and the http.server library to create a simple web server / framework / application for development and testing

updated: Mar 17, 2019

Overview

Note: If you're looking for an example of how to use http.server, this article may help you. However, please use it only for testing and development. The code shown here is definitely not suitable for a public facing port.

We need a simple and flexible method to test, share & deliver Python web server-side code. Although there are many great web frameworks and servers out there (e.g., Django, CherryPy, Waitress, etc), they all come with a good deal of dependencies and inner complexities. Therefore, we have decided to see how far we can go with just using Python3 and its http.server library to build up our own development / test server. Other Python3 web server related articles on this site (will) make use of this framework, and it is our goal that they be as self-contained as possible.

You're probably already familiar with http.server on the command line. We have been using it for many years to serve up files in a directory (e.g., *.html):

	$ python3 -m http.server --bind < interface > < port >

However, we're now importing the library, subclassing base classes, and adding methods to create our own test web server / framework / application.

The Python3 Documentation for http.server is good, but there are also a lot of great notes in the source file server.py, plus we found that reviewing the code was essential to actually understanding how to use it. For example, it required some code analysis to determine where the do_GET() and do_POST() methods are defined. It turned out that these methods are the result of a concatenation of the string "do_" and the HTTP command sent on request (e.g., GET). This concatentation is performed dynamically inside handle_one_request()

On Ubuntu Linux, server.py can found at:

	/usr/lib/python3.<version>/http/server.py

You can also view it on Github.

http.server defined classes

The main class defined in http.server is HTTPServer, and its base class is socketserver.TCPServer. A request handler class must be passed to HTTPServer when instantiating it.

http.server defines three handler classes that can be used to handle HTTP requests (e.g., GET and POST). We make use of the BaseHTTPRequestHandler handler class for our test server.

BaseHTTPRequestHandler is inherited from socketserver.StreamRequestHandler. This class defines the core attributes and methods, and we put it to use by subclassing it and defining methods to respond to requests (e.g., do_GET()).

Note that when running http.server from the command line, the handler class is SimpleHTTPRequestHandler unless the -cgi option is passed. CGIHTTPRequestHandler is used for serving CGI applications.

Responding to a request

The quickest way to learn how http.server works is to load working code in a debugger, set a breakpoint, and inspect the backtrace. We show the backtrace below from our supplied example script using the Eclipse IDE with a break set on do_GET()

httpsrv backtrace — httpsrv.py do_GET() backtrace

The backtrace shows that the methods within socketserver.py do a good deal of the processing before handing it off to the http.server methods.

In contrast, take a look at the backtrace after a break on a Django request handler when running "manage.py runserver". That's a deep stack to deal with when prototyping & testing.

Note that we test our server inside a browser or from the command line using wget:

$ wget localhost:8000   # for GET

$ wget localhost:8000 --post-data=<data>  # for POST

Screenshots from our example httpsrv.py

HTTP Standards

As this simple test framework/server evolves, we plan to reference the IETF HTTP specifications / RFCs as appropriate. Note that http.server is an HTTP/1.1 Server (not HTTP/2)

As you may know, work within the IETF is underway to consolidate the existing HTTP/1.1 specifications into a smaller set of documents that are more concise and consistent. Listed below are links to the draft documents, and this is what we plan to use for reference as we move forward.

And for the sake of completeness, here is a list of the existing standards (RFCs)

Source

The Python script httpsrv.py is provided below. Note the disclaimer in the header of the file.

python3 script, httpsrv.py

##############################################################################
#
# Copyright (c) 2019 Mind Chasers Inc.
# All Rights Reserved.
#
#    file: httpsrv.py
#
#    experimental HTTP/1.1 web server / framework example using http.server
#
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################

from http.server import HTTPServer, BaseHTTPRequestHandler
import urllib

HOST_ADDRESS = ""
HOST_PORT = 8000

class RequestHandler(BaseHTTPRequestHandler):
    """ Our custom, example request handler """
    
    def send_response(self, code, message=None):
        """ override to customize header """
        self.log_request(code)
        self.send_response_only(code)
        self.send_header('Server','python3 http.server Development Server')     
        self.send_header('Date', self.date_time_string())
        self.end_headers()  
    
    def do_GET(self):
        """ response for a GET request """
        self.send_response(200)
        self.wfile.write(b'<head><style>p, button {font-size: 1em}</style></head>')
        self.wfile.write(b'<body>')
        self.wfile.write(b'<form method="POST" enctype="application/x-www-form-urlencoded">')
        self.wfile.write(b'<span>Enter something:</span>\
                            <input name="test"> \
                            <button style="color:blue">Submit</button>')
        self.wfile.write(b'</form>')    
        self.wfile.write(b'</body>')
        
    def do_POST(self):
        """ response for a POST """
        content_length = int(self.headers['Content-Length'])
        (input,value) = self.rfile.read(content_length).decode('utf-8').split('=')
        value = urllib.parse.unquote_plus(value)
        self.send_response(200)
        self.wfile.write(b'<head><style>p, button {font-size: 1em}</style></head>')
        self.wfile.write(b'<body>')
        self.wfile.write(b'<p>You submitted ' + bytes(value,'utf-8') + b'</p>')
        self.wfile.write(b'</body>')    
  
def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler):
    """ follows example shown on docs.python.org """
    server_address = (HOST_ADDRESS, HOST_PORT)
    httpd = server_class(server_address, handler_class)
    httpd.serve_forever()

if __name__ == '__main__':
    run(handler_class=RequestHandler)

Download httpsrv.py

What's next:

We plan to parameterize and generalize the example shown above to migrate it into a very minimal framework and server for the purpose of sharing & delivering working server side code. Some of the things we're working on include incorporating a TLS front end using the OpenSSL library, daemonizing the framework, and providing various examples for network monitoring and probing.

Additional References

Wikipedia: List of HTTP header fields
IANA: Message Headers

Didn't find an answer to your question? Post your issue below or in our new FORUM, and we'll try our best to help you find a solution.

And please note that we update our site daily with new content related to our open source approach to network security and system design. If you would like to be notified about these changes, then please join our mailing list.

Create your own Web Content Filter with Squid and Linux	Article discusses a Python3 redirector for Squid for the purpose of filtering content. Example provides a starting point for developing a full content filter / ad blocker
Use Python and Sockets to Create a Honey Pot and Catch Bad Network Actors	Article discusses using Python with sockets to create and deploy a honey pot for the purpose of detecting malicious activity on a network. A sample Python 3 script is provided.
Use Python3 to Model and Visualize Harmonic Waveforms	We provide a simple Python script and examples to model and visualize square and triangle waves with applications to circuit design and test.
Remote Oscilloscope Operation with Python and VISA	Example Python code is provided to perform basic remote operations with a Rohde and Schwarz RTO1044 Oscilloscope including waveform capture, display, and FFT.