POST Requests

Unlike GET requests, where the web browser wants to download files, there are also POST requests that allow the browser to send data to the web server. You can think of this as like posting something on social media. You type some text, add images, or even add videos in a box provided for that purpose, and then press Post. The content is then uploaded to the server and subsequently displayed under your profile. Our simple server only uploads files from a browser and saves them in the uploads/ folder.

Again, the browser sends a header indicating that it wants to post something. You can easily find out what a post request looks like by running the command in Listing 6. In the browser, call the web form in the root folder and send a file (Figure 2). After a few seconds, interrupt the Netcat command by pressing Ctrl+C. The browser displays a File arrived message, and the file is where you redirected it. But this is not a displayable JPEG file, because the file saved here still contains the header, as shown in Figure 3.

$ echo "File arrived" | netcat -l 8081 > upload/filex.jpg

Figure 3: The upload request contains data that does not belong to the uploaded file.

Figure 2: Using the web form to upload files such as photos or texts to the web server.

Listing 7 shows how sed can get rid of the excess data that you do not want in the uploaded file. Sed handles this task in the while loop starting in line 9. Sed removes the header, boundary statements, the file name, and similar data. To compare this with what the data originally looked like, take a look at the cache file, which is also in the upload folder. If sed didn't remove all the ballast, the operating system would be unable to display the received files correctly.

01 function run_post_server () {
02
03 message_for_post='HTTP/1.1 200 OK
04 Content-Length: 13
05 Connection: close
06
07 File arrived
08
09 while true; do
10   cat <<< $message_for_post | netcat -l $HTTP_POST_PORT > "${CACHE_DATEI}"
11   new_name=$( sed -r -n '/filename/{ s/(.*)(filename=")(.+)(".*)/\3/; p}' ${CACHE_DATEI} )
12   upload_path="${HTTP_UPLOAD}/${new_name}"
13   sed '1,/filename/d;/Content-Type/{N;d};$d' ${CACHE_DATEI} > "${upload_path}"
14 done
15 }
16
17 run_post_server &

In the background, the routine also calls the run_post_server function (line 17). This function contains a matching response for POST requests stating the content length in bytes and containing instructions to break down the connection after reading. Without these instructions, Firefox would simply keep the connection option, although the data has already been sent. The function launches in the background (&) to avoid it blocking everything as soon as the files have been sent.

Unchecked

Even if the web browser explicitly requests the root directory or another file, the web server can basically return whatever you want – you just need to declare the returned content correctly for the browsers. Listing 8 shows an example of this where the browser immediately displays a JPEG file on calling localhost:8080 or IP_address:8080 without any complaints.

#!/bin/bash
header="HTTP/1.1 200 OK"
myfile="http_home/upload/IMG-20220213-WA0002.jpg"
content_length="Content-Length: $( cat $myfile | wc --bytes )"
content_type="Content-Type: image/jpeg"
cat $myfile | sed -r -e "1 i $header" -e \
    "1 i $content_length" -e \
    "1 i $content_type" -e \
    "1 i Connection: close\n" | netcat -l 8080

The interesting thing here is not just that this works, but that it also represents a potential vulnerability. Apparently, most web browsers don't bother checking whether the content of the GET request and the returned page actually match. In this case, the browser asked for the index page of the web server and was given a JPEG file instead. That's something like a tennis player getting a basketball thrown at them by their opponent all of a sudden.

From experience, these idiosyncrasies, and many other features you might want to implement, are not very well documented on the web or are not documented at all. That's why it could be useful to log what Firefox and other browsers request. The function in Listing 9 starts the server. You can see two tee redirects there that forward all of the data to a logfile for debugging. This log will then contain the date and time, what the web browser sent as a request, and what the server sent back as a response (Figure 4). Armed with these details, you can analyze each request and response and understand what exactly is going on when the client and server talk.

function run_server () {
  while true; do
    date | sed -r 's/^|$/\n/g' >> debug
    respond < $FIFO_GET | tee --append debug |
      netcat -l $HTTP_GET_PORT |
    tee --append debug > $FIFO_GET
  done
}

Figure 4: An excerpt from the debug file for port 8080.

For example, many browsers ask for the famous favicon.ico after they have talked to a server for a little while. This is the icon that you usually see at the top of the browsers' tabs. It is usually found in the web server's root folder.

If you want your own server to provide a favicon, you first need to find out what the browser request looks like and then tell the server to respond appropriately. You can tell that the web browser often asks for this file from the error message cat: http_home/favicon.ico: file or directory not found in the logfile.

Conclusions

As you can see, a rudimentary web server is quite easy to build yourself. The web server presented in this article has a plain and simple design, but it is not intended to compete with major league players like the Apache web server or NGINX. On the other hand, your homegrown web server does have some capabilities that a typical web server can't offer. For instance, you can access the whole repertoire of shell commands to display information locally with minimal overhead. The resources consumed by the small script are also minimal. This DIY server is quite useful as an info server on your own network, and you can also use it to transfer files from one computer to another – all told, not a bad solution for small tasks.

The Author

Goran Mladenovic is a hobby developer and inventor, who believes programming is a passion.