Find the answer to your Linux question:
Results 1 to 2 of 2
I've been trying to get mmap to work on a 3GB file (Linux Ubuntu 12.04), but it keeps returning MAP_FAILED. If I ignore it, I eventually get a segmentation fault, ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Jun 2012
    Posts
    1

    Problem with mmap on files >2GB


    I've been trying to get mmap to work on a 3GB file (Linux Ubuntu 12.04), but it keeps returning MAP_FAILED. If I ignore it, I eventually get a segmentation fault, so I'm sure something is wrong somewhere.

    I'm compling with 64 bit file options:
    gcc -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 test.c

    Here's an example I made for testing:
    Code:
    #include <unistd.h>
    #include <fcntl.h>
    #include <stdio.h>
    #include <sys/stat.h>
    #include <sys/types.h>
    #include <inttypes.h>
    #include <stdlib.h>
    # include <sys/mman.h>
    # include <stdint.h>
    
    
    int main(int argc, char **argv)
    {
        uint8_t *base;
    
        int writable=0;
     
        if(argc != 2)    
            return 1;
     
        int file=0;
            if((file=open(argv[1],O_RDONLY)) < -1)
                return 1;
     
        struct stat fileStat;
        if(fstat(file,&fileStat) < 0)    
            return 1;
     
        printf("Information for %s\n",argv[1]);
        printf("---------------------------\n");
        printf("File Size: \t\t%" PRIuPTR " bytes\n", (uintptr_t) fileStat.st_size);
        printf("Number of Links: \t%d\n",fileStat.st_nlink);
        printf("File inode: \t\t%" PRIuPTR " bytes\n", (uintptr_t) fileStat.st_ino);
     
        printf("File Permissions: \t");
        printf( (S_ISDIR(fileStat.st_mode)) ? "d" : "-");
        printf( (fileStat.st_mode & S_IRUSR) ? "r" : "-");
        printf( (fileStat.st_mode & S_IWUSR) ? "w" : "-");
        printf( (fileStat.st_mode & S_IXUSR) ? "x" : "-");
        printf( (fileStat.st_mode & S_IRGRP) ? "r" : "-");
        printf( (fileStat.st_mode & S_IWGRP) ? "w" : "-");
        printf( (fileStat.st_mode & S_IXGRP) ? "x" : "-");
        printf( (fileStat.st_mode & S_IROTH) ? "r" : "-");
        printf( (fileStat.st_mode & S_IWOTH) ? "w" : "-");
        printf( (fileStat.st_mode & S_IXOTH) ? "x" : "-");
        printf("\n\n");
     
        printf("The file %s a symbolic link\n\n", (S_ISLNK(fileStat.st_mode)) ? "is" : "is not");
      
        fprintf(stderr,"file=%d flags=%d\n",file,writable);
        if (file == -1) {
            printf("Cannot open file %s\n", argv[1]);
            return 1;
        }
    
        if (fstat(file,&fileStat) == -1) {
            printf("Cannot get size of file %s\n", argv[1]);
            return 1;
        }
        if (fileStat.st_size == 0) {
            printf("file %s is an empty file\n", argv[1]);
            return 1;
        }
        
        base = mmap(NULL, fileStat.st_size, PROT_READ | (PROT_WRITE * writable),
                MAP_SHARED, file, 0);
        fprintf(stderr,"base: \t\t%" PRIuPTR " \n", (uintptr_t) base);
        if (base == MAP_FAILED) {
            fprintf(stderr, "Cannot map file %s into memory\n", argv[1]);
            return 1;
        }
    
        return 0;
    }
    Running it on a test file >2GB produces this example output
    HTML Code:
    Information for largefile.iso
    ---------------------------
    File Size: 		3019309056 bytes
    Number of Links: 	1
    File inode: 		14948955 bytes
    File Permissions: 	-rw-rw-r--
    
    The file is not a symbolic link
    
    file=3 flags=0
    base: 		4294967295 
    Cannot map file largefile.iso into memory
    The line base == MAP_FAILED is comparing the 4294967295 to (void *) -1

    Can someone clue me as to how to properly handle this large file with mmap? Am I running up against a physical memory limit or am I making some kind of interger size error?

    Thanks!

  2. #2
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,753
    You need to use mmap64() instead of mmap(). The off_t offset value used as the last argument in mmap() is a 32-bit value.
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •