c++ socket issue

[sileacan]

I am sure there are a few things in this code that are not done as well as it could be, this is my first c++ application. I am looking to get help on an issue I am having.

Application: http proxy server
OS: Linux (fedora)
Code Language: c++
IDE: code blocks

My problem is that when I call recv on the socket that is connected to the http server (sending the browsers (client) request to the http server, and then getting the response back) it hangs.. to get around this i've tried using select.. but select always states that the socket is never ready to read.

One way around this is to parse the response header to find out the content length.. and keep track of how much data has come back from recv and while currentlen<contentlen call recv again..

If someone could please help me figure out why it's hanging on the recv (should it not return 0 ????) or why select is always coming back saying that the socket is not ready to read..

I've stripped my application down to the bare bones for this.. no threading etc .. hopefully this helps to figure out the problem.

#include <iostream>

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <arpa/inet.h>
#include <errno.h>
#include <pthread.h>

#define INVALID_SOCKET -1

using namespace std;

int main()
{
    int ret = 0;

    cout<<"proxy server: 0.1b"<<endl;

    cout<<"setting up listening socket"<<endl;

    int _listen, _browser, _server;

    if((_listen=socket(AF_INET, SOCK_STREAM, 0))!=(signed int)INVALID_SOCKET)
    {
        cout<<"listening socket created"<<endl;
        cout<<"setting up listening socket address information"<<endl;

        sockaddr_in _listen_addr;
        bzero((char *) &_listen_addr, sizeof(_listen_addr));
        _listen_addr.sin_family = AF_INET;
        _listen_addr.sin_port = htons(8100);
        _listen_addr.sin_addr.s_addr = INADDR_ANY;

        char msg[100]; sprintf(msg, "attemping to bind listening socket to port %i", 8100);
        cout<<msg<<endl;

        if((bind(_listen, (sockaddr*)&_listen_addr, sizeof(_listen_addr)))!=(signed int)INVALID_SOCKET)
        {
            cout<<"bind successful!"<<endl;
            cout<<"attemping to listen"<<endl;

            while(1)
            {
                if((listen(_listen, 10))!=(signed int)INVALID_SOCKET)
                {
                    cout<<"server is now listening"<<endl;

                    sockaddr_in _browser_addr;

                    socklen_t browsersize = sizeof(_browser_addr);
                    if((_browser=accept(_listen, (sockaddr*)&_browser_addr, &browsersize))!=(signed int)INVALID_SOCKET)
                    {
                        char *ip = inet_ntoa(_browser_addr.sin_addr);
                        sprintf(msg, "received connection from ip: %s", ip);
                        cout<<msg<<endl;

                        char buffer[1024];
                        int _browser_len, _server_len;

                        cout<<"creating socket to communicate to http server"<<endl;
                        if((_server = socket(AF_INET, SOCK_STREAM, 0))!=(signed int)INVALID_SOCKET)
                        {
                            sockaddr_in _server_addr;
                            hostent *hostInfo;
                            hostInfo = gethostbyname("www.google.ca");

                            bzero((char *) &_server_addr, sizeof(_server_addr));
                            _server_addr.sin_family = AF_INET;
                            _server_addr.sin_port = htons(80);
                            bcopy((char *)hostInfo->h_addr, (char *)&_server_addr.sin_addr.s_addr, hostInfo->h_length);

                            fd_set fread, fwrite, fex;

                            FD_ZERO(&fread);
                            FD_ZERO(&fwrite);
                            FD_ZERO(&fex);
                            FD_SET(_server, &fread);
                            FD_SET(_server, &fwrite);
                            FD_SET(_server, &fex);

                            cout<<"connecting http server socket to the http server"<<endl;
                            if((connect(_server, (sockaddr*)&_server_addr, sizeof(_server_addr)))!=(signed int)INVALID_SOCKET)
                            {
                                cout<<"connected to http server"<<endl;
                                cout<<"attempting to receive request from browser"<<endl;
                                while((_browser_len = recv(_browser, buffer, sizeof(buffer), 0))>0)
                                {
                                    buffer[_browser_len] = '\0';
                                    cout<<"received request from browser: "<<endl<<"*******************************"<<endl<<buffer<<endl<<"*******************************"<<endl;

                                    cout<<"sending request to http server"<<endl;
                                    send(_server, buffer, _browser_len, 0);
                                    cout<<"attemping to receive response from server"<<endl;

                                    select(_server + 1, &fread, &fwrite, &fex, NULL);
//this if condition is always false even though i know the socket has data in it's buffer to read
//removing this if block will result in the code hanging on recv
//this is what i need help on
                                    if(FD_ISSET(_server, &fread))
                                    {
                                        cout<<"select = server sent data back!"<<endl;
                                        while((_server_len = recv(_server, buffer, sizeof(buffer), 0))>0)
                                        {
                                            buffer[_server_len] = '\0';
                                            cout<<"received response from server: "<<endl<<"*******************************"<<endl<<buffer<<endl<<"*******************************"<<endl;
                                            cout<<"sending response back to browser"<<endl;
                                            send(_browser, buffer, _server_len, 0);
                                        }
                                    }
                                    else
                                    {
                                        cout<<"select = server did not send data back!"<<endl;
                                    }
                                    cout<<"attempting to receive request from browser"<<endl;
                                }
                                close(_server);
                                close(_browser);
                            }
                            else
                            {
                                cout<<"failed to connect to http server: terminating server"<<endl;
                                break;
                            }
                        }
                        else
                        {
                            cout<<"failed to create http server socket: terminating server"<<endl;
                            break;
                        }
                    }
                }
                else
                {
                    cout<<"failed to listen: terminating server"<<endl;
                    break;
                }
            }

        }
        else
        {
            cout<<"bind failed: terminating server"<<endl;
        }

        cout<<"closing server"<<endl;
        close(_listen);
    }
    else
    {
        cout<<"failed to setup listening socket"<<endl;
    }

    return ret;
}

[mitchpotter]

It's been a while since I did networking code, but if I remember correctly recv is a blocking call. That means when you call it your program will just sit there and wait until you get some activity on the port you are listening to. If that's what the app is meant for then recv will work just fine for you.

Select is the best option if you need the application to keep processing. If it always says not ready to read you may want to see how much data you sent from the server (run server in debug mode and break just before it sends the response). Also make sure you are interpreting the select function's return value correctly and understand exactly what it is doing for you. I assume from your post that you know how to interpret the data from the buffer correctly but you might want to step through in debug mode and just see what's missing, if anything, from the data.

[sileacan]

I am definately passing on the HTTP request properly as I do get a response back from the server. My problem is that even once I have finished receiving the entire response back from the web server, the next call to recv hangs because the connection is not being closed.. so it never returns 0, it doesn't error out giving me -1.. so it just sits and waits and waits.. even if i use setsockopt to give a very short timeout for the recv on the socket, it will never time out.. it just sits and waits and waits.. so far the only way i've been able to get around it is to first parse out the content-length from the response header and then keep track of how much data i have received from recv and if it's short of what i should have for the content length.. keep calling it.

that works.. but, why is it that select is not reporting the socket read attribute properly? why is it that setsockopt isn't working?

I don't mind parsing the header, and I am going to do that anyway, but I would like to understand why things don't work the way they .. appear to in theory.

[wje_lf]

the only way i've been able to get around it is to first parse out the content-length from the response header and then keep track of how much data i have received from recv and if it's short of what i should have for the content length.. keep calling it.

That's exactly what you're supposed to do. If there's a Content-Length header line, do not rely on the server to close the connection.

why is it that select is not reporting the socket read attribute properly?

Because with the presence of the Content-Length line, you're not supposed to rely on the server closing things first. That's part of the HTTP protocol.

setsockopt to give a very short timeout for the recv on the socket, it will never time out

What is the exact setsockopt call you're making?

[eggi]

I'm not a c++ expert, so please forgive me if my contribution is ignorant

I've done this in Perl and C and have always had to fork after binding to the port and then running the while (1) loop with the "listen" inside that.

Hope that helps point you in the right direction

Best wishes,

Mike

[wje_lf]

Quoth eggi:

I've done this in Perl and C and have always had to fork after binding to the port and then running the while (1) loop with the "listen" inside that.

Two comments.

1. That's not relevant here, because sileacan posted that he stripped out the threading stuff to make it easier for us to read the code.
2. The classic way is to bind once, listen once, have an accept loop, and after each successful accept do a fork to handle the request (or skip the fork if the request can be handled very quickly). Again, that's not relevant here.

[eggi]

That's what I meant - mistyped - forking on the accept. But, it's not relevant here. Apologies for putting that out there.

Best wishes,

Mike

[wje_lf]

So this leaves us with the previous question for sileacan:

What setsockopt call are you making in an attempt to set the timeout value?

[sileacan]

sorry for the delayed response!! and thanks for answering.

I will be using the content-length.. but the select method should have nothing to do with any protocol, it should only inform me as to whether the socket is ready to be read, and while this doesn't effectively mean that there is data present in the buffer, it should still report back to me that the socket COULD be read. It always returns false for my server socket, even though there is data present..

I understand why recv is hanging, the http server isn't closing the connection once it's finished transmitting the data, so the next call to recv waits for the next response.

The setsockopt I am calling is like:

timeval timeout;
timeout.tv_sec = 2;
setsockopt(_server, SQL_SOCKET, SO_RCVTIMEO, (timeval*)&timeout, sizeof(timeout));

Hoping that doing this would cause the recv to timeout.. but it doesn't? I can move on with my project and track the content-length, but this will still bother me.. not knowing why both the timeout and select options are not working?

thanks again!

[wje_lf]

the select method should have nothing to do with any protocol, it should only inform me as to whether the socket is ready to be read, and while this doesn't effectively mean that there is data present in the buffer, it should still report back to me that the socket COULD be read.

Properly used, select() should only return on one of three conditions:

1. there is, indeed, data in the buffer (at least one byte); or
2. there is no data now and there never will be any more data ("end of file"); or
3. there is an error.

The subsequent recv() call returns the number of bytes in the buffer, or 0 if there will never be more data, or -1 on error.

Full disclosure: I have never used recv(); I've only used read() in network programming, and with TCP/IP, that's all you ever need. But in theory what I say above is true. :)

Also, in the setsockopt() call, you don't seem to set the tv_usec field to zero. If it contains garbage, you might have a huge timeout value.

But you shouldn't have to bother with that setsockopt() call. Just make your socket use nonblocking I/O, and use select().

I'm surprised that your select() call doesn't work. It has always worked for me. You're setting the readfds, writefds, exceptfds, and timeout each time through the loop, right?