Overview
Note: If you're looking for an example of how to use http.server, this article may help you. However, please use it only for testing and development. The code shown here is definitely not suitable for a public facing port.
We need a simple and flexible method to test, share & deliver Python web server-side code. Although there are many great web frameworks and servers out there (e.g., Django, CherryPy, Waitress, etc), they all come with a good deal of dependencies and inner complexities. Therefore, we have decided to see how far we can go with just using Python3 and its http.server library to build up our own development / test server. Other Python3 web server related articles on this site (will) make use of this framework, and it is our goal that they be as self-contained as possible.
You're probably already familiar with http.server on the command line. We have been using it for many years to serve up files in a directory (e.g., *.html):
$ python3 -m http.server --bind < interface > < port >
However, we're now importing the library, subclassing base classes, and adding methods to create our own test web server / framework / application.
The Python3 Documentation for http.server is good, but there are also a lot of great notes in the source file server.py, plus we found that reviewing the code was essential to actually understanding how to use it. For example, it required some code analysis to determine where the do_GET() and do_POST() methods are defined. It turned out that these methods are the result of a concatenation of the string "do_" and the HTTP command sent on request (e.g., GET). This concatentation is performed dynamically inside handle_one_request()
On Ubuntu Linux, server.py can found at:
/usr/lib/python3.<version>/http/server.py
You can also view it on Github.
http.server defined classes
The main class defined in http.server is HTTPServer, and its base class is socketserver.TCPServer. A request handler class must be passed to HTTPServer when instantiating it.
http.server defines three handler classes that can be used to handle HTTP requests (e.g., GET and POST). We make use of the BaseHTTPRequestHandler handler class for our test server.
BaseHTTPRequestHandler is inherited from socketserver.StreamRequestHandler. This class defines the core attributes and methods, and we put it to use by subclassing it and defining methods to respond to requests (e.g., do_GET()).
Note that when running http.server from the command line, the handler class is SimpleHTTPRequestHandler unless the -cgi option is passed. CGIHTTPRequestHandler is used for serving CGI applications.
Responding to a request
The quickest way to learn how http.server works is to load working code in a debugger, set a breakpoint, and inspect the backtrace. We show the backtrace below from our supplied example script using the Eclipse IDE with a break set on do_GET()
The backtrace shows that the methods within socketserver.py do a good deal of the processing before handing it off to the http.server methods.
In contrast, take a look at the backtrace after a break on a Django request handler when running "manage.py runserver". That's a deep stack to deal with when prototyping & testing.
Note that we test our server inside a browser or from the command line using wget:
$ wget localhost:8000 # for GET $ wget localhost:8000 --post-data=<data> # for POST
Screenshots from our example httpsrv.py
HTTP Standards
As this simple test framework/server evolves, we plan to reference the IETF HTTP specifications / RFCs as appropriate. Note that http.server is an HTTP/1.1 Server (not HTTP/2)
As you may know, work within the IETF is underway to consolidate the existing HTTP/1.1 specifications into a smaller set of documents that are more concise and consistent. Listed below are links to the draft documents, and this is what we plan to use for reference as we move forward.
And for the sake of completeness, here is a list of the existing standards (RFCs)
- RFC7230: Message Syntax and Routing
- RFC7231: Semantics and Content
- RFC7232: Conditional Requests
- RFC7233: Range Requests
- RFC7234: Caching
- RFC7235: Authentication
Source
The Python script httpsrv.py is provided below. Note the disclaimer in the header of the file.
############################################################################## # # Copyright (c) 2019 Mind Chasers Inc. # All Rights Reserved. # # file: httpsrv.py # # experimental HTTP/1.1 web server / framework example using http.server # # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS # FOR A PARTICULAR PURPOSE. # ############################################################################## from http.server import HTTPServer, BaseHTTPRequestHandler import urllib HOST_ADDRESS = "" HOST_PORT = 8000 class RequestHandler(BaseHTTPRequestHandler): """ Our custom, example request handler """ def send_response(self, code, message=None): """ override to customize header """ self.log_request(code) self.send_response_only(code) self.send_header('Server','python3 http.server Development Server') self.send_header('Date', self.date_time_string()) self.end_headers() def do_GET(self): """ response for a GET request """ self.send_response(200) self.wfile.write(b'<head><style>p, button {font-size: 1em}</style></head>') self.wfile.write(b'<body>') self.wfile.write(b'<form method="POST" enctype="application/x-www-form-urlencoded">') self.wfile.write(b'<span>Enter something:</span>\ <input name="test"> \ <button style="color:blue">Submit</button>') self.wfile.write(b'</form>') self.wfile.write(b'</body>') def do_POST(self): """ response for a POST """ content_length = int(self.headers['Content-Length']) (input,value) = self.rfile.read(content_length).decode('utf-8').split('=') value = urllib.parse.unquote_plus(value) self.send_response(200) self.wfile.write(b'<head><style>p, button {font-size: 1em}</style></head>') self.wfile.write(b'<body>') self.wfile.write(b'<p>You submitted ' + bytes(value,'utf-8') + b'</p>') self.wfile.write(b'</body>') def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler): """ follows example shown on docs.python.org """ server_address = (HOST_ADDRESS, HOST_PORT) httpd = server_class(server_address, handler_class) httpd.serve_forever() if __name__ == '__main__': run(handler_class=RequestHandler)
What's next:
We plan to parameterize and generalize the example shown above to migrate it into a very minimal framework and server for the purpose of sharing & delivering working server side code. Some of the things we're working on include incorporating a TLS front end using the OpenSSL library, daemonizing the framework, and providing various examples for network monitoring and probing.
Additional References
- Wikipedia: List of HTTP header fields
- IANA: Message Headers
