Mind Chasers Inc.
Mind Chasers Inc.

Use Python3 and the http.server Library to Create a Development Server / Framework

A discussion and example of the use of Python3 and the http.server library to create a simple web server / framework / application for development and testing

Overview

Note: If you're looking for an example of how to use http.server, this article may help you. However, please use it only for testing and development. The code shown here is definitely not suitable for a public facing port.

We need a simple and flexible method to test, share & deliver Python web server-side code. Although there are many great web frameworks and servers out there (e.g., Django, CherryPy, Waitress, etc), they all come with a good deal of dependencies and inner complexities. Therefore, we have decided to see how far we can go with just using Python3 and its http.server library to build up our own development / test server. Other Python3 web server related articles on this site (will) make use of this framework, and it is our goal that they be as self-contained as possible.

You're probably already familiar with http.server on the command line. We have been using it for many years to serve up files in a directory (e.g., *.html):

	$ python3 -m http.server --bind < interface > < port >

However, we're now importing the library, subclassing base classes, and adding methods to create our own test web server / framework / application.

The Python3 Documentation for http.server is good, but there are also a lot of great notes in the source file server.py, plus we found that reviewing the code was essential to actually understanding how to use it. For example, it required some code analysis to determine where the do_GET() and do_POST() methods are defined. It turned out that these methods are the result of a concatenation of the string "do_" and the HTTP command sent on request (e.g., GET). This concatentation is performed dynamically inside handle_one_request()

On Ubuntu Linux, server.py can found at:

	/usr/lib/python3.<version>/http/server.py

You can also view it on Github.

http.server defined classes

The main class defined in http.server is HTTPServer, and its base class is socketserver.TCPServer. A request handler class must be passed to HTTPServer when instantiating it.

http.server defines three handler classes that can be used to handle HTTP requests (e.g., GET and POST). We make use of the BaseHTTPRequestHandler handler class for our test server.

BaseHTTPRequestHandler is inherited from socketserver.StreamRequestHandler. This class defines the core attributes and methods, and we put it to use by subclassing it and defining methods to respond to requests (e.g., do_GET()).

Note that when running http.server from the command line, the handler class is SimpleHTTPRequestHandler unless the -cgi option is passed. CGIHTTPRequestHandler is used for serving CGI applications.

Responding to a request

The quickest way to learn how http.server works is to load working code in a debugger, set a breakpoint, and inspect the backtrace. We show the backtrace below from our supplied example script using the Eclipse IDE with a break set on do_GET()

httpsrv backtrace
httpsrv.py do_GET() backtrace

The backtrace shows that the methods within socketserver.py do a good deal of the processing before handing it off to the http.server methods.

In contrast, take a look at the backtrace after a break on a Django request handler when running "manage.py runserver". That's a deep stack to deal with when prototyping & testing.

Django backtrace
Django backtrace

Note that we test our server inside a browser or from the command line using wget:

$ wget localhost:8000   # for GET

$ wget localhost:8000 --post-data=<data>  # for POST

Screenshots from our example httpsrv.py

GET request response
httpsrv get
POST response
httpsrv post

HTTP Standards

As this simple test framework/server evolves, we plan to reference the IETF HTTP specifications / RFCs as appropriate. Note that http.server is an HTTP/1.1 Server (not HTTP/2)

As you may know, work within the IETF is underway to consolidate the existing HTTP/1.1 specifications into a smaller set of documents that are more concise and consistent. Listed below are links to the draft documents, and this is what we plan to use for reference as we move forward.

And for the sake of completeness, here is a list of the existing standards (RFCs)

Source

The Python script httpsrv.py is provided below. Note the disclaimer in the header of the file.

python3 script, httpsrv.py
##############################################################################
#
# Copyright (c) 2019 Mind Chasers Inc.
# All Rights Reserved.
#
#    file: httpsrv.py
#
#    experimental HTTP/1.1 web server / framework example using http.server
#
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################

from http.server import HTTPServer, BaseHTTPRequestHandler
import urllib

HOST_ADDRESS = ""
HOST_PORT = 8000

class RequestHandler(BaseHTTPRequestHandler):
    """ Our custom, example request handler """
    
    def send_response(self, code, message=None):
        """ override to customize header """
        self.log_request(code)
        self.send_response_only(code)
        self.send_header('Server','python3 http.server Development Server')     
        self.send_header('Date', self.date_time_string())
        self.end_headers()  
    
    def do_GET(self):
        """ response for a GET request """
        self.send_response(200)
        self.wfile.write(b'<head><style>p, button {font-size: 1em}</style></head>')
        self.wfile.write(b'<body>')
        self.wfile.write(b'<form method="POST" enctype="application/x-www-form-urlencoded">')
        self.wfile.write(b'<span>Enter something:</span>\
                            <input name="test"> \
                            <button style="color:blue">Submit</button>')
        self.wfile.write(b'</form>')    
        self.wfile.write(b'</body>')
        
    def do_POST(self):
        """ response for a POST """
        content_length = int(self.headers['Content-Length'])
        (input,value) = self.rfile.read(content_length).decode('utf-8').split('=')
        value = urllib.parse.unquote_plus(value)
        self.send_response(200)
        self.wfile.write(b'<head><style>p, button {font-size: 1em}</style></head>')
        self.wfile.write(b'<body>')
        self.wfile.write(b'<p>You submitted ' + bytes(value,'utf-8') + b'</p>')
        self.wfile.write(b'</body>')    
  
def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler):
    """ follows example shown on docs.python.org """
    server_address = (HOST_ADDRESS, HOST_PORT)
    httpd = server_class(server_address, handler_class)
    httpd.serve_forever()

if __name__ == '__main__':
    run(handler_class=RequestHandler)

What's next:

We plan to parameterize and generalize the example shown above to migrate it into a very minimal framework and server for the purpose of sharing & delivering working server side code. Some of the things we're working on include incorporating a TLS front end using the OpenSSL library, daemonizing the framework, and providing various examples for network monitoring and probing.

Additional References

Didn't find an answer to your question? Post your issue below or in our new FORUM, and we'll try our best to help you find a solution.

And please note that we update our site daily with new content related to our open source approach to network security and system design. If you would like to be notified about these changes, then please follow us on Twitter and join our mailing list.

Related articles on this site:

share
subscribe to mailing list:

Please help us improve this article by adding your comment or question:

your email address will be kept private
authenticate with a 3rd party for enhanced features, such as image upload
previous month
next month
Su
Mo
Tu
Wd
Th
Fr
Sa
loading