FIles too big to open with fopen() in C

[SagaciousKJB]

Hey, I've been working on a small encryption program, and I ran into a snag on Linux: It seems that any time I try using the program on a file over about 2 gigs, I get a Segfault.

execve("/usr/bin/xor", ["xor", "./bigfile.iso", "-", "testing"], [/* 36 vars */]) = 0
brk(0)                                  = 0x804a000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f06000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=61789, ...}) = 0
mmap2(NULL, 61789, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7ef6000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\260a\1"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=1339816, ...}) = 0
mmap2(NULL, 1349136, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7dac000
mmap2(0xb7ef0000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x143) = 0xb7ef0000
mmap2(0xb7ef3000, 9744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7ef3000
close(3)                                = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7dab000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7dab6b0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xb7ef0000, 4096, PROT_READ)   = 0
munmap(0xb7ef6000, 61789)               = 0
brk(0)                                  = 0x804a000
brk(0x806b000)                          = 0x806b000
open("./bigfile.iso", O_RDONLY)           = -1 EFBIG (File too large)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV (core dumped) +++
Process 1372 detached

That is the strace I get from trying to run the program. I am fairly sure that this is an issue with Linux, because I don't have a problem with the version I compiled on Windows.

Here is the source just so you can give it a look over:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char** argv)
{
FILE *in=stdin,*out=stdout;

/*File handling scheme, if the first letters are - use stdin or stdout, if not use the stringe as filename*/
if(argc>1)in=(argv[1][0]!='-')?fopen(argv[1],"rb"):stdin;
if(argc>2)out=(argv[2][0]!='-')?fopen(argv[2],"wb"):stdout;
if(argc<4)
{
 printf("No key supplied.\n");
 return 0;
}

	char msg;
	unsigned int strl = strlen(argv[3]);
	/*Comarison between the elements in argv[3] makes mismatch between unsigned and signed, cause problems when using as function in other programs, but does not effect the encryption*/
	unsigned int i = 0;

	while(!feof(in)) {
		msg = fgetc(in);
		/*xor the character held in msg with the byte in the key stream*/
		msg = msg ^ argv[3][i];
		if(!feof(in))fprintf(out, "%c", msg);
		/*Looping structure to start at the first bit of the key once the last is reached*/
		if(i < strl) {
			/*Increment the key byte to the next set of bits in the key*/
			i++;
		}
		else if(i == strl) {
			/*Reset the incrementor to the first bit of the key*/ 
			i = 0;
		}
	}
	fclose(in);
	fclose(out);
	return 0;
}

Does anyone have any idea why this is happening? Is there some type of file size limit that fopen() can open in the Linux kernel or something?

At first I thought maybe my libraries were out-of-date, so I compiled using the DevC++ libraries that worked for the Windows version, and I still got the same results. It seems to have nothing to do with C and something to do with the Linux kernel, but I figured I'd ask some experts.

[wowbag1]

Hello, I found this at - Re: Large File in Linux

Large files are files that are larger than 2 gigabytes. This distinction
comes from the largest number that can be represented in the off_t type, which
is used to specify file offsets in system and library calls. On 32-bit
architectures (such as Intel), an off_t is commonly 32 bits. To avoid
conflicts with negative numbers, the largest offset that can be represented in
a 32-bit off_t variable is 2 gigabytes.

There is quite a bit more.

Hope this helps

[wje_lf]

wowbag1 nailed it:

#define _FILE_OFFSET_BITS 64

ought to fix it.

Oh. And. I know someone's going to scold you on this, so let me be the scolder, please:

When you do (or might do) an fopen(), always check the result for NULL, and give some informative message in that case. SIGSEGV is oh so very '50s.

[devnulled]

This is interesting, does that means that fopen() is a dangerous function and can be used to overwrite the buffer to gain control over the program to exec anything we wish with the program's privs?

[wje_lf]

Ummmm, no.

[devnulled]

How come? Why not?

[SagaciousKJB]

wowbag1 nailed it:

#define _FILE_OFFSET_BITS 64

ought to fix it.

Oh. And. I know someone's going to scold you on this, so let me be the scolder, please:

When you do (or might do) an fopen(), always check the result for NULL, and give some informative message in that case. SIGSEGV is oh so very '50s.

Ahh, thanks so much, that worked perfectly.

And yes I have thought several times to check for the files before opening them so I don't get a Segfault, but it has just been slipping my mind. Thanks for the reminder.

[Cabhan]

How come? Why not?

The reason for doing this bit of error checking is because if the pointer is NULL, you will get a segfault when you try and write to it later. Therefore, you can keep your program sane (and your debugging easier) by doing this bit of error checking.

My understanding of memory attacks is when you are referring to something in memory, generally a function pointer, but it is has somehow been changed from what you expect. However, referencing NULL is not such a thing.

[bnossum]

Thanks!

I had a problem with a > 2G textfile to be parsed by flex. Putting
%top{
#define _FILE_OFFSET_BITS 64
}
in the definitions part of the flex input file worked great!