Find the answer to your Linux question:
Results 1 to 2 of 2
Hi, I have this old network emulator I wrote a couple of years back for simulating the behavior of wireless networks using libipq library. At that time, the bit rates ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Jan 2009
    Posts
    2

    iptables/libipq questions: buffering issues when copying packets to user space


    Hi,

    I have this old network emulator I wrote a couple of years back for simulating the behavior of wireless networks using libipq library. At that time, the bit rates we were considering were low (less than 1Mbps), so there were no performance penalty. Recently, I wanted to test some new things, and have to use some streaming data, which can reach peak rates of more than 12Mbps, and then I started running out of buffer (I was getting "Failed to received netlink message: No buffer space available")


    So I reread the libipq man pages and then I thought maybe
    I can avoid this problem if I stop reading the whole packet and only get the meta data, i.e. in ipq_set_mode i use IPQ_COPY_META instead of IPQ_COPY_PACKET, as after all, I don't have to manipulate the packet, I only care about the packet size for my emulator, which I can get from packet->data_len once I acquire the data via ipq_get_packet.

    So my first question is:

    1)If I just read the meta info and not the packet data into userspace, will I still be able to pass the verdict in the packet later on? i.e. is the packet queued in kernel space regardless whether we copy it to user space or not?


    So I wrote a simple test code to see if this works, as shown below. So basically what I did was
    ......ipq_set_mode(h, IPQ_COPY_META, BUFSIZE);// so that only meta data is copied
    ......ipq_read(h,buf, sizeof(struct ipq_packet_msg_t), 0); // i use the sizeof to make sure that I have enough buffer space to copy the meta data

    but I keep getting "Received message truncated" error.

    I increased the size parameter in ipq_read to a large value, then I don't get the errors, but my packets were not being accepted (ie. set_verdict was not called.) so I removed the if (packet->data_len > 0) check for calling the set_verdict method, and now my packets seem to be routed properly.

    So a look into the man pages again revealed to me that the packet->data_len is not zero only if IPQ_COPY_PACKET was used in ipq_set_mode. But I need to find out how big is the packet for my emulator to work.

    So you see my dilemma? my second question is

    2) If I use IPQ_COPY_PACKET mode, I run out of buffer space, and if I use IPQ_COPY_META, I am not able to access the packet size. So any suggestions to work around this? I thought of simply using a smaller value for BUFSIZE (a couple of hundrend bytes) and then ignore the "received message truncated" error, but the problem is in those cases (when error occurs), the length will not be the real length of the packet, but the length I specified. I thought not copying all the packet to user space would be the most optimal implementation for me as I don't manipulate the packet data, but if there is no other way, I might as well try to increase the buffer size used by ip_queue...but I don't know how to do that... any easy way to increase the buffer size?



    Thank you!
    Beat

    ------test code----------------

    #include <sys/time.h>
    #include <linux/netfilter.h>
    #ifdef __cplusplus
    extern "C" {
    #endif

    #include <libipq/libipq.h>

    #ifdef __cplusplus
    }
    #endif

    #include <pthread.h>
    #include <unistd.h>
    #include <sys/time.h>
    #include <signal.h>
    #include <string.h>
    #include <stdio.h>
    #include <stdlib.h>
    //#include "errors.h"
    #include <mcheck.h>
    #include <netinet/in.h>
    #include <iostream>
    #include <set>

    using namespace std;

    #define BUFSIZE 2048
    #define DROP 1

    struct ipq_handle *h;

    static void die(struct ipq_handle *h)
    {
    ipq_perror("ipq_error::");
    ipq_destroy_handle(h);
    exit(1);
    }

    int main(int argc, char **argv)
    {
    /* mtrace(); */
    int status;
    unsigned char buf[BUFSIZE];

    ipq_packet_msg_t *packet;

    h = ipq_create_handle(0, PF_INET);
    if (!h)
    die(h);
    status = ipq_set_mode(h, IPQ_COPY_META, BUFSIZE);
    if (status < 0)
    die(h);
    cout << "ipq_set_mode done" << endl;
    while (1)

    status = ipq_read(h,buf, sizeof(struct ipq_packet_msg_t), 0);

    cout << "ipq_read finished maybe status is not good?" << endl;

    if (status < 0)
    {
    die(h);
    }
    cout << "ipq_read done" << endl;

    switch (ipq_message_type(buf))
    {
    case NLMSG_ERROR:{
    fprintf(stderr, "Error! %s\n", ipq_errstr());
    break;
    }
    case IPQM_PACKET:
    {
    packet = ipq_get_packet(buf);
    cout << "ipq_get_packet done" << endl;
    if (packet->data_len > 0)
    {
    ipq_set_verdict(h,packet->packet_id,NF_ACCEPT, 0, NULL);
    cout << "ipq_set_verdict done" << endl;
    }
    break;
    }
    default:{
    fprintf(stderr,
    default:{
    fprintf(stderr, "Unknown message type!\n");
    break;
    }
    }
    }
    ipq_destroy_handle(h);
    return 0;
    }

  2. #2
    Just Joined!
    Join Date
    Jan 2009
    Posts
    2
    it seems I found a work around, this is what I did:
    -first I increased the receive buffer size, by doing

    int rcvbuf=0;/* Receive buffer size */
    socklen_t optlen; /* Option length */
    h = ipq_create_handle(0, PF_INET);
    if (!h)
    die(h);
    status = ipq_set_mode(h, IPQ_COPY_PACKET, BUFSIZE);
    if (status < 0)
    die(h);
    /* Set the SO_RCVBUF Size:*/
    rcvbuf = 131071;
    //I got the maximum possible value by checking /sbin/sysctl net.core.wmem_max
    //though setting very high receive size can be dangerous incase of packet loss, if SACK is used, it will be no problem
    status = setsockopt(h->fd,SOL_SOCKET,SO_RCVBUF,&rcvbuf,sizeof (rcvbuf));
    if ( status )
    bail("setsockopt(h->fd,SOL_SOCKET,SO_RCVBUF)");

    Just to make sure I don't copy data I don't need, I also stopped reading the whole buffer, and just read enough data to get the IP header, and from that header I can get the size of the packet

    packet = ipq_get_packet(buf);
    if (packet->data_len > 0)
    {
    data = (unsigned char*) (packet + 1);
    total_len= htons(*((unsigned short*)(data+2)));
    ip_header_len = (*data & 0xF) * 4;
    ......
    }

    Well, seems to work for now, I hope that not reading the whole packet will not cause any subtle side effects, though it makes no logical sense why it should

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •