This is probably the 9999th demonstration of a lightweight HTTP GET client, but if I can make someone else's life easier then so be it!

Our client needed a video playback device that could download new data and content via a massive distribution system that we had been contracted to build. Of course the decision was to use XML and HTTP for all of the data needs.

The client had built the target hardware. It was our task to implement the software. Our target platform was SigmaDesign's em8561 (ARM) architecture, 4MB of onboard flash and uClinux. So code space was a premium.

The XML implementation was a simple once I had found the TinyXML library (TinyXml Main Page). However, the HTTP client library ended up being a little more problematic than anticipated. I had researched libwww and libcurl. libcurl would compile but provided some fun segmentation faults that I was not interested in debugging. libcurl’s event handling mechanism seemed too pervasive to provide a simple blocking HTTP GET call. After linking with either library, the program file ended 200KB+ fatter. These options were not going to work.

After some thought, the solution was simpler than expected. After a few moments designing the components to build the functionality from scratch, the resulting code was fairly brief and clean. All that was needed was

  • Sockets class
  • String class
  • String Splitter class
  • URL class
  • HttpGet function implementation


I found a simple Sockets class (PracticalSockets @ Practical C++ Sockets). It is an elegant implementation, BUT it also uses STL. There nothing wrong with STL, but I need to reduce code where ever possible - So I have included a version that I have modified to use a custom lightweight String class. The Splitter class is used for parsing URLs strings and HTTP header data. The URL class is the container for the URL tokens. The HttpGet(char *url, char *file) function is a high level function to download the specified “char *url” to the “char *file”.

HttpGet function:


Code:
/*
 *   C++ HTTP GET Client Minimal Library
 *   Copyright (C) 2007 Venturality, Inc
 *	 www.Venturality.com
 *
 *   This program is free software; you can redistribute it and/or modify
 *   it under the terms of the GNU General Public License as published by
 *   the Free Software Foundation; either version 2 of the License, or
 *   (at your option) any later version.
 *
 *   This program is distributed in the hope that it will be useful,
 *   but WITHOUT ANY WARRANTY; without even the implied warranty of
 *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *   GNU General Public License for more details.
 *
 *   You should have received a copy of the GNU General Public License
 *   along with this program; if not, write to the Free Software
 *   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 */

#include "HttpGet.h"

//Buffer Size for handling incoming packets
#define RCVBUFSIZE 		1024    // Size of receive buffer

//Buffer Size for storing HTTP Header
#define MAXHEADERSIZE 	2048    // Size of header buffer


//Small helper function for TCPSocket class
void SendHttpMsg(TCPSocket *sock, char *msg) {
    sock->send(msg, strlen(msg));
}

//HttpGet(..) Simple HTTP file download function
//urlArg: URL for source content
//origFile: Target file to store content
int HttpGet(char *urlArg, char *origFile) {

	//Final result value to be returned at the end of the function
	//0 = No Error
	int ret=0;

	//TCP/IP Port
	unsigned short port = 80;

	//Buffer for incoming header from HTTP server
	char header[MAXHEADERSIZE];
	bzero(header,MAXHEADERSIZE);

	int headerSize = 0;

	//Pointer to end of header data to separate HTTP header from HTTP data
	char *endHeader = NULL;

	//Hold temp file in a temporary location until we are sure the download was successful
	//This was done for other needs - no reason not to write directly to target file
	char tempFile[400];

	//Buffer to hold simple HTTP header message to server
	char httpHeader[400];

  try {

	//Parse the URL string into protocol, host, and path
	Url url(urlArg);

	//printf("Making request to: %s %s\n", url.Host, url.Path);
    // Establish connection with the HTTP server
    TCPSocket sock(url.Host, port);


	//Create simple HTTP header
	//This can include other HTTP header pairs, but this gets the job done!
	if(url.Path == NULL) { //If no path then send the "/" for the ROOT
		strcpy(httpHeader,"GET / HTTP/1.1\r\nAccept: */*\r\nUser-Agent:Unknown\r\n\r\n");
	}
	else {
		sprintf(httpHeader,"GET /%s HTTP/1.1\r\nAccept: */*\r\nUser-Agent:Unknown\r\n\r\n", url.Path);
	}

	//Send simple HTTP header
	SendHttpMsg(&sock, httpHeader);

    char recvBuffer[RCVBUFSIZE];    // Buffer for recieving new data packets
    int bytesReceived = 0;              // Bytes read on each recv()
    int totalBytesReceived = 0;         // Total bytes read

    // Wait for the end of the network byte stream...
    while ((bytesReceived = (sock.recv(recvBuffer, RCVBUFSIZE))) > 0) {
      headerSize += bytesReceived;     // header bytes total
	  memcpy(header, recvBuffer, bytesReceived); //copy new network bytes to header buffer
	  if( (endHeader = strstr(header, "\r\n\r\n")) != NULL) { //did we get the end of the header delimiter yet?
		  endHeader += 4; //Found the end of the header need increment to HTTP data
		  break;
	  }
    }

	//Success? - if "200 OK" is in our header!
	if(strstr(header, "200 OK") != NULL) {
	    printf("Download was successful\n");

	    //Create temp file string
	    sprintf(tempFile, "%s.tmp", origFile);

	    //Open file...
		FILE *fp = fopen(tempFile, "w+");

		//Get final header size
		int finalHeaderSize = endHeader - header;
		//If the bytes copied to the header buffer are greater than the actual header
		//then some of the HTTP data is really in the header buffer
		//so we need to write it out then...
		if( finalHeaderSize < headerSize) {
			fwrite(endHeader-4, sizeof(char), (headerSize - finalHeaderSize), fp);
		}

		//Lets write everything we get from now on - out to the target file
		while ((bytesReceived = (sock.recv(recvBuffer, RCVBUFSIZE))) > 0) {
			fwrite(recvBuffer, sizeof(char), bytesReceived, fp);
		}

		fclose(fp);

		ret = 0;
	}
	else { //We must have got an HTTP Error msg 300s, 400s (like 404 file not found), 500s
		Splitter headerSplit(header, '\n'); //Look at the first line to determine which...
		//printf("Failed Response!\n");
		if(headerSplit.TokenCount > 0) {
			printf("Received error: %s\n", headerSplit.Tokens[0]);
		}

		ret = 1;
	}


    // Destructor closes the socket

  } catch(SocketException &e) {
    perror(e.what().c_str());
	ret = 2;
  }

	//Success so far?! - great, move tempfile to Target destination
	if(ret == 0) {
		rename(tempFile, origFile);
	}
  return ret;
}
Full source @ http://www.venturality.com/HttpGet.zip

Enjoy!!!