Wireless Embedded Internetworking Short Course

David E. Culler

University of California at Berkeley

Department of Electrical Engineering and Computer Science

 

 

Lab 9 Sensor Web

 

 


Now that we have the full suite of IP protocols, including TCP transport, it is natural to ask what it would mean to bring the broad body of Web technology down to these tier of low-power wireless embedded devices.  We saw in the first lab that this was possible.  There we had a very simple web server that listened on Port 80 and in response to any connection responded with HTML.  Since then we have developed sophisticated tools for sensing, sampling, alarms, and communication.  In this lab, we’ll pull them together into a reasonably complete embedded web server. This serves to illustrate the design concepts of robust application protocols on the Wireless Embedded Internet.

 

We will see that the TinyOS composition model extends naturally to provide an elegant means of layering services, where the implementation closely follows the conceptual structure, shown below.

 

 

The Hypertext Transport Protocol is specified in terms of request messages from the client to server that generate responses.  The request message consists of the following:

  • Request line, such as GET /images/logo.gif HTTP/1.1, which requests the file logo.gif from the /images directory
  • Headers, such as Accept-Language: en
  • An empty line
  • An optional message body

The request line and headers must all end with <CR><LF> (that is, a carriage return followed by a line feed). The empty line must consist of only <CR><LF> and no other whitespace. In the HTTP/1.1 protocol, all headers except Host are optional.

A request line containing only the path name is accepted by servers to maintain compatibility with HTTP clients before the HTTP/1.0 specification.

In fact, browsers are pretty sloppy and often the end of line is just a <LF>.  Like all things on the internet, you need to be generous in what you accept and parsimonious in what you produce.  The SimpleWeb we played with was a bit extreme in this regard. 

Exercise

To get a feeling for what HTTP really looks like, startup the TCPserver in a window on some unused port, eg. ./TCPserver 4444.  Open a browser and point it at this server, e.g., http://127.0.0.1:4444/foo.  The TCP server will print out the request it got.  (Wow, look at all those headers.) and since its response is not obeying the HTTP protocol, the browser will hang.

In the embedded context, we are going to need to consume the request, but we don’t need to store it all.  We can parse it on the fly and keep just what we are interested in.  Most servers don’t do it this way.  They open up a big receive buffer, suck in the whole request and set to parsing it.  The real issue here is limited memory, not low-power embedded networking per se.  Protocols are tricky to write regardless of what platform you are running on.  Since we are now working with essentially a sockets programming model, we can now develop the trickiest parts of the application on standard machines with plenty of visibility and then port the working code over to the mote.

Memory-Constrained HTTP

As an example of this, we’ve built a small footprint HTTP server for the sockets API.  Take a look in unix/http.  You can run this in linux or under cygwin in windows.  The first unusual thing in the HTTP implementation is that we have made the input buffer equal to the size of the segments that we typically received on the mote.

#define MAXRCV  48

void HTTP_TCP_server(int portno) {

  int newsockfd;

char inbuf[MAXRCV+1];      /* HTTP request rcv buffer*/

This is the size of the buffer that we provide to rcvfrom.  In datagram-oriented protocols, breaking things up into pieces is very natural.  In stream oriented protocols it can get tricky because they tend to be text based and a single item may get broken into pieces.  The buffer boundary might fall between the two end-of-line characters or in the middle of a header noun.  We deal with these issues by constructing a pump abstraction.  As the chunks flow in we pump them through a filter.  It maintains the state of the parser and picks out the fields that we are interested in.  In particular, it extracts the HTTP method (e.g., GET, POST) and the URL.  And it detects the blank line request terminator.  (Yes, the protocol says that the server is supposed to consume the entire request before it responds.)   

 

Exercise

 

Make http and run it on an unused port.  Point a browser to it and take a look at how the request stream is broken up.  http.c prints out quite a bit of information along the way.  Feel free to work with this.  You may want to extend the parser to pickout other headers or the body for post.  The main point is to see the value in being able to develop protocols in a familiar setting with a limit on footprint and then port them to the embedded environment.  You will notice in the course distribution that we have files in a shared directory so that the same exact code can be used in the two worlds.  We’ve used this primarily for a safer string library – another important aspect of embedded programming that is often overlooked.

 

Embedded Http

 

Let’s dive in to the embedded http and web services.  The application we are going to start with is in $TOS/SensorWeb.  You will find that that director has only the top level configuration and the web server.  We have built up several useful application level components in $TOS/lib.  The HTTP component is one of those.  Let’s start there.

 

Like all things in TinyOS, design focuses first on interface.  The http interface is in $tos/lib/Http.nc and reproduced here.  It provides commands to start and stop the service on a particular port, typically port 80.  Notice also that it adopts the same storage management position as the network stack – the application allocates the transmit buffer but the service manages it.  For an http server, the transmit buffer is for responses.  Requests are in the receive buffer.  The HTTP server parses the incoming request and signal http method events, GET or POST.  This could be extended to include the other HTTP methods.  These services are split-phase, but sort of in reverse.  The http server cannot assume that the method service is performed immediately. It may need to take samples or perform other operations.  When the service method is complete, it will issue the serviceDone command.  Finally, send allows the application to transmit responses through the HTTP service.  In fact, TinyOS’ compiler optimizations allow us to follow this strict layering – application over http over tcp – with little or no overhead.  The inlining eliminates the redundant calls as it passes through.

 

interface Http {

  command error_t Start(uint16_t port, uint8_t RespBuf[], uint16_t RespBufLen);

  command error_t Stop (uint16_t port);

 

  event   error_t GETservice (uint8_t url[], uint16_t urlLen);

  event   error_t POSTservice(uint8_t url[], uint16_t urlLen);

  command error_t send( const void *buf, uint16_t len );

  command error_t serviceDone();

}

 

The http server, found in lib/HttpC.nc, implements these commands and signals these events.  Of course, it uses the TCP interface, so it implements the HTTP version of the TCP events and calls the TCP commands.  If we walk through it in the sequence of the protocol, Http.start binds to the port using the application transmit buffer directly, no copies.

 

command error_t Http.Start(uint16_t HttpPort,

                   uint8_t RespBuf[], uint16_t RespBufLen) {

    /* Record service parameters for reopen */

    port    = HttpPort;

    resp    = RespBuf;

    respLen = RespBufLen;

 

    /* bind to port and setup response transmit buffer */

    if (call HttpTcp.bind(HttpPort, RespBuf, RespBufLen) == SUCCESS) {

      serverState = ACCEPTING;

      call Timer.startPeriodic( 2048 ); /* Start watchdog timeout */

      return SUCCESS;

    }

    return FAIL;

}

 

HTTP tracks the state of the protocol.  It also has its own watchdog timer.  This allows it to abort a connection that is stuck without making adequate progress.  Once the connection is made, HTTP starts the pump.

 

event bool HttpTcp.accept( sockaddr_in6_t *to ) {

    serverState = LISTENING;

    return TRUE;

  }

 

  event void HttpTcp.connected() {

    serverState = CONNECTED;   

    init_pump();

}

 

As segments are received, they are pumped through the HTTP parser until the end of request is found.  Then the appropriate HTTP method is signaled.

 

event uint16_t HttpTcp.recv( void *inbuf, uint16_t len )

  {

    if (serverState == CONNECTED) {

      serverState = GET_RECVD;   

    }

    if (serverState == GET_RECVD) {

       if (httppump(inbuf,len)) {

         switch (http_method) {

         case HTTP_GET : signal Http.GETservice (url, uindex); break;

         case HTTP_POST: signal Http.POSTservice(url, uindex); break;

         default:

              serverState = RESP_SENT;   

           call HttpTcp.close( FALSE );

         }

      }

    }

    return len;

  }

 

In processing the http method, lots of commands and events and tasks may be processed.  The node goes to sleep every chance it gets.  Response are sent.  Eventually, the HTTP protocol is informed of completion, whereup it closes the connection. 

 

command error_t Http.serviceDone() {

    serverState = RESP_SENT;   

    call HttpTcp.close( FALSE );

    return SUCCESS;

  }

 

With TCP, close involves a handshake too.  The closed event handler prepares the server to accept the next request.

 

event void HttpTcp.closed() {

    /* bind anew to port and setup response transmit buffer */

    if (resp) {

      call HttpTcp.bind(port, resp, respLen );

      serverState = ACCEPTING;

    } else {

      serverState = IDLE;

    }

    timeout = 0;

  }

Beautiful isn’t it?  The structure of the code directly reflects the HTTP state machine over the TCP state machine. 

Debugging

The actual code throws in some useful indicators on the LEDs.  It introduces another useful tool.  How do we deal with problems that come up on remote desktops, workstations and servers?  Syslog.  Here we use the UDP reporting that you developed earlier to provide debugging support.  While we are developing our web service applications on http over tcp, we can be transmitting udp packets that tells us what is going on.  This is part of the power of a true network in a true event-driven environment.

Embedded Web Server

Now we are ready to go up one level and build a real embedded web server.  The modularity allows us to be a highly application specific service with very little specialized code.  The entire systems is described in the configuration WebSense.nc. 

·        First, we define the services and how they connect together and to kernel resources.

·        Then we connect the web application to convenient data representation components for each logical input and output.

·        Finally, we connect those data components to the physical hardware resources that provide the information.

Here we utilize another powerful TinyOS composition tool that is provided by NesC – Generic Components.  Generic modules and generic configurations allow us to define a template and then instantiate one or more specialized instances of it at compile time.  Generic components are instantiated in configurations with the new keyword.

For example, look at how the User button is declared.

components new DigitalInC("User") as User;

  WebC.User   -> User.DigitalIn;

  User.Pin    -> Msp430PortsC.Port2[7];

  User.PinInt -> Msp430PortsC.Port2Interrupt[7];

The generic component here is DigitalInC.  You will find it in /lib/DigitanInC.nc.  We specialize it by providing it with a compile time parameter, which is a string describing the function of this particular digital input.  Here we are going to represent the date in a manner typical of the web.  The User digital input component plugs into the web server and attaches to the driver at Port 2, pin 7.

If we look further at the analog sensor inputs, some of them are specific and some are generic.  For example, Vcc is specific, but TrimPot is generic.  It is specialized with its reference voltage (this one is a ratiometric sensor) as well as its name.  Both of these provide the sensor interface, given by lib/Sensor.nc

components VccC;

WebC.Vcc -> VccC.Vcc; /* Specialized VCC sensor */

VccC.ADC       -> Msp430Adc12C.Msp430Adc12[INCH_11]; /* Internal voltage */

 

components new AnalogInC(REFVCC, “Pot”) as TrimPot;

WebC.Pot  -> TrimPot.Sensor;

TrimPot.ADC -> Msp430Adc12C.Msp430Adc12[INCH_4 ];

Exercise

·        Compare the generic module lib/AnalogInC with the specific ones, lib/VccV and lib/TempInt.  What are the differences? 

·        What occurs when a generic module is instantiated?

Now all that’s left if the web server itself.  Take a look at SensorWeb/WebC.nc. 

It provides two kinds of services. The human web, which provides documents full of information, and the programmable web, which provides information in a clean, self-documenting manner that programs can use to construct more valuable information.  serveHTMLindex does the first, serveREST does the latter.

Exercise

·        Make this web server and put it on your node.  Ping the node to make sure you can reach it.  Open a browser inside the Linux VM and point it at your node http://<v4nodeadress>/.  See what comes back.  Manipulate the inputs and refresh the page.  Compare what you see with the code.  Notice how the contents are being generated in a very uniform manner from the data object components.

Web services come in essentially two forms: REST and SOAP.  Both typically return XML, but can return other representations.  REST essentially treats the URI as programmatic request.  It is of the form http://<host>[:port]/<method>?<arglist>.  Here, our web server defines the read method and the names of each of the digital and sensor input are valid args.

Exercise

·        Try http://<v4nodeadress>/read?User.  See what comes back.  Try out the other options.  What happens if the args are bad.

·        What about the write method?

·        What is the XML schema for the results that are being produced?

·        How might you incorporate this into a rich web of physical information.

·        What sensors would you like to use to try out these ideas?

·        What would you like to turn into “an Internet Thing”?