Building a dunderhead HTTP server from scratch

At the time that I started this project (September 2022) there were no examples online, that I could find, of a HTTP server that utilised the C SDK for the Pico W. I did however know from some prior adventures into ESP32, that you can remain very stupid as a microcontroller and still talk HTTP, so lets roll our own HTTP server!

The way I achieved this was by building on the tcp server example over on the official pico-examples github repository.

What even is HTTP anyway?

Hypertext Transfer Protocol (HTTP) at is core is just a TCP socket with fancy strings.
Theres a great crash course by @fasterthanlime over on his blog that I would recommend reading.

So how do we send fancy strings?

Let’s send a standard HTTP PUT request via the cli tool curl and see what it looks like, request:

curl -X PUT http://api.int/api/v1.0/pc\?code\=status

Response:

{"status": "on"}

Plucking the request out of wireshark, we can see that the actual TCP payload contains the following information:

PUT /api/v1.0/pc?code=status HTTP/1.1\r\n
Host: api.int\r\n
User-Agent: curl/7.68.0\r\n
Accept: */*\r\n
\r\n

Let’s disect this. The first line specifies a few things:

• Each line end is denoted with a carriage return \r and line feed \n.
• Each line after the first contains a header variable in the form of a key/value pair.
• The final line only contains \r\n to denote the end of the data
  
  

The response data looks like this:

HTTP/1.1 200 OK\r\n
Content-Length: 17\r\n
Content-Type: application/json\r\n
Date: Wed, 30 Nov 2022 10:40:24 GMT\r\n
Server: waitress\r\n
\r\n
{"status": "on"}

Again each preceding line is a key/value header variable. However it is worth noting that the HTTP server chose to send a JSON response.

To pull this off, the response had to contain the header Content-Length, to inform the client of how much data it was going to send. After which it appends the JSON string after the final \r\n line of the HTTP body.

This concept of setting the Content-Length header applies to the client too if we wish to send JSON in our request.

This protocol is actually simple enough that we can pick apart a HTTP request and construct responses just by doing some simple string manipulation. We don’t even need to understand anything about HTTP or JSON to speak the lingo!

Code

The first (and hardest) thing we need to do in our little HTTP server is process HTTP request messages.

We can achieve this by breaking the string up into it’s individual tokens. We are going to use the string function strpbrk to achieve this.

The man page has this to say on the strpbrk function:

SYNOPSIS
       #include <string.h>

       char *strpbrk(const char *s, const char *accept);

DESCRIPTION
       The  strpbrk() function locates the first occurrence in the string s of
       any of the bytes in the string accept.

RETURN VALUE
       The strpbrk() function returns a pointer to the byte in s that  matches
       one of the bytes in accept, or NULL if no such byte is found.

So this function takes a string and a set of deliminators, and outputs a pointer to the first deliminator it encounteres in the string.

If we look again at the top line of the HTTP request, there are indeed a fixed set of single character deliminators that denote certain parts of the line:

PUT /api/v1.0/pc?code=status HTTP/1.1\r\n

So we can extract a given token by performing a set of steps.

Define our message in a string (char *) and create a pointer to the first character:

char http_message[] = {
    "PUT /api/v1.0/pc?code=status HTTP/1.1\r\n"
    "Host: api.int\r\n"
    "User-Agent: curl/7.68.0\r\n"
    "Accept: */*\r\n"
    "\r\n"
};
char *p1 = http_message;

Call the strpbrk function to obtain a pointer to the first instance of one of our deliminators:

char *p2 = strpbrk(p1, " ?=\r");

Now lets null terminate p2. By placing \0 at the position earmarked by p2, we transform the deliminator into the last character of the string:

*p2 = '\0';

If we now print p1, it will only print up to the first \0, essentially turning it into our first token:

printf("%s\n", p1);
> PUT

We can further extend this technique to iterate over each token in the first line of the HTTP request, and by keeping track of the deliminators we can tell exactly which token we just extracted.

#include <string.h>
#include <stdio.h>


int main() {
  char http_message[] = {
    "PUT /api/v1.0/pc?other=monkey&code=status HTTP/1.1\r\n"
    "Host: api.int\r\n"
    "User-Agent: curl/7.68.0\r\n"
    "Accept: */*\r\n"
    "\r\n"
  };

  // Process HTTP message body, example: "PUT /api/v1.0/pc?code=status HTTP/1.1"
  // char *message_body = http_message;
  char *message_body = http_message;
  char *delim = " ?=&\r";
  char next_delim;
  char current_delim = '\0';
  char *token;

  while(1) {
    if (*message_body == '\0') { break; }
    token = message_body;
    message_body = strpbrk(message_body, delim);
    if (message_body == NULL) { break; }
    next_delim = *message_body;
    *message_body = '\0';

    switch(current_delim) {
      case '\0':
        printf("Method\t\t%s\n", token);
        break;
      case ' ':
        if (next_delim != '\r') {
          printf("Target\t\t%s\n", token);
        } else {
          printf("Version\t\t%s\n", token);
        }
        break;
      case '?':
      case '&':
        printf("Key\t\t%s\n", token);
        break;
      case '=':
        printf("Value\t\t%s\n", token);
        break;
    }
    current_delim = next_delim;
    message_body++;
    // Carriage return indicates we have reached the end of the first line of message body
    if (current_delim == '\r') { break; }
  }

  return 0;
}

Within the final program we actually take this one step further and use the same technique to step through JSON as a flat file if it is provided as part of the HTTP request.

Demo

Optional: Recycling unused pins for SWD debugging

If we take a look at the mainboard again, we can see there is a 4 pin “USB” header exposed on the board with unused pins:

I tested both NONE pins on the oscilloscope and determined that there was no activity on either pin during normal operation. As it turns out the panel on the back actually has a cut out for the USB header, which is only covered by a thin layer of mylar tape that can be cut away:

Due to this I decided to wire up the SWD debugging pins from the Pico to the unused pins so that I could perform debugging without having to re-open the unit:

I would highly recommend NOT doing this, as I managed to fry the SWD pins on one of my debuggers using these pins and I am not 100% as to the reason

Tying it all together

Throughout this adventure we have developed some interesting capabilities on our Pico, we can now:

• Get the current state of the Snowdon via the RGB LED
• Send our own NEC encoded commands to the Snowdon
• Decode arbitrary HTTP requests

To bring this all together we only really need a little bit of glue code to translate a HTTP request into a call to our PIO driver or LED decoder.

This is achieve in the program by extracting a value from one of the user provided parameters passed in the HTTP request (code) and passing this into a helper function. The function translates the user provided string into a 32 bit NEC code for our PIO driver, or a HTTP_CODE_LOOKUP_STATUS command for our LED decoder:

static uint32_t http_code_lookup(char *code) {
    if (!strcmp(code, "status"))        { return HTTP_CODE_LOOKUP_STATUS; }
    if (!strcmp(code, "power"))         {return 0x807F807F;}
    if (!strcmp(code, "input"))         {return 0x807F40BF;}
    if (!strcmp(code, "mute"))          { return 0x807FCC33; }
    if (!strcmp(code, "volume_up"))     { return 0x807FC03F; }
    if (!strcmp(code, "volume_down"))   { return 0x807F10EF; }
    if (!strcmp(code, "previous"))      { return 0x807FA05F; }
    if (!strcmp(code, "next"))          { return 0x807F609F; }
    if (!strcmp(code, "play_pause"))    { return 0x807FE01F; }
    if (!strcmp(code, "treble_up"))     { return 0x807FA45B; }
    if (!strcmp(code, "treble_down"))   { return 0x807FE41B; }
    if (!strcmp(code, "bass_up"))       { return 0x807F20DF; }
    if (!strcmp(code, "bass_down"))     { return 0x807F649B; }
    if (!strcmp(code, "pair"))          { return 0x807F906F; }
    if (!strcmp(code, "flat"))          { return 0x807F48B7; }
    if (!strcmp(code, "music"))         { return 0x807F946B; }
    if (!strcmp(code, "dialog"))        { return 0x807F54AB; }
    if (!strcmp(code, "movie"))         { return 0x807F14EB; }
    return HTTP_CODE_LOOKUP_UNKNOWN_VALUE;
}

We then just make the respective call in our program and return a nice hard coded JSON string in our HTTP response:

if (state->message_body->code == HTTP_CODE_LOOKUP_STATUS) {
    uint32_t gpio;
    do{
        gpio = (gpio_get_all() & RGB_MASK) >> RGB_BASE_PIN;
        switch(gpio) {
            case 0b110: // red
                http_generate_response(arg, "{\"onoff\": \"off\", \"input\": \"off\"}\n", "200 OK");
                break;
            case 0b100: // yellow
                http_generate_response(arg, "{\"onoff\": \"on\", \"input\": \"optical\"}\n", "200 OK");
                break;
            case 0b000: // white
                http_generate_response(arg, "{\"onoff\": \"on\", \"input\": \"aux\"}\n", "200 OK");
                break;
            case 0b101: // green
                http_generate_response(arg, "{\"onoff\": \"on\", \"input\": \"line-in\"}\n", "200 OK");
                break;
            case 0b011: // blue
                http_generate_response(arg, "{\"onoff\": \"on\", \"input\": \"bluetooth\"}\n", "200 OK");
                break;
            case 0b111: // off (likely in a transitioning state)
                busy_wait_ms(50);
                continue;
        }
    } while (gpio == 0b111);
    return;
}
if (state->message_body->code > HTTP_CODE_LOOKUP_NO_VALUE) {
    pio_sm_put_blocking(PIO_INSTANCE, 0, state->message_body->code);
    http_generate_response(arg, "{\"status\": \"ok\"}\n", "200 OK");
}

This is essentially all the building blocks required to roll our own RESTful API. With that out of the way, all that’s left to do now is wire it up and compile!

Wiring

SWDIO and SWCLK are not required for normal operations, they expose the SWD debug pins on the unused USB header for easy access

Compiling Guide

Please refer to the offical Getting Started guide for full details on getting a build environment setup

Due to the final software containing hardcoded WiFi credentials, it’s impossible to share a pre-built uf2. So we must install the Pico SDK and built it from scratch.

Install dependencies for SDK:

sudo apt update
sudo apt install git cmake gcc-arm-none-eabi libnewlib-arm-none-eabi build-essential

Clone SDK;

cd ~/
git clone https://github.com/raspberrypi/pico-sdk.git
cd pico-sdk
git submodule update --init

Export PICO_SDK_PATH variable:

export PICO_SDK_PATH=$HOME/pico-sdk

Clone Snowdon-II-WiFi:

git clone https://github.com/kennedn/snowdon-ii-wifi.git

Create a build directory and configure cmake with WiFi credentials:

cd snowdon-ii-wifi
mkdir build
cd build
# Replace <SSID> and <PASSWORD> with own values
cmake -DPICO_BOARD=pico_w -DWIFI_SSID="<SSID>" -DWIFI_PASSWORD="<PASSWORD>" ..

Compile the program:

cd src
make

If all goes well, a file named snowdon.uf2 should now exist under ~/snowdon-ii-wifi/build/src/.

The Pico can now be plugged in via USB whilst holding down the BOOTSEL button, and the uf2 file dropped in the volume mount.

API

The RESTful API is exposed on port 8080:

http://<ip_address>:8080

The endpoint expects a single code parameter, which can be sent via either url encoding or in the JSON body of the request, e.g:

curl -X PUT http://192.168.1.238:8080?code=status
# or
curl -X PUT http://192.168.1.238:8080 -H 'Content-Type: application/json' -d '{"code": "power"}'

And this is the full list of available code values:

power_state has the following possible values:

input_state has the following possible values:

Finish

With that we can now send HTTP commands to our Pico:

And it’s a win win because my remote control can go on its adventures into the deep recesses of the couch and I can still turn on my soundbar: