Huge binary out of -static link?

**legendbb** · 10-03-2011

A not very complicated code: linked as
LDLIBS += -lpthread -static -L../_lib -lA -lB -lC -lrt

libA.a: 2.7K
libB.a: 72K
libC.a: 16K

.o is 50K

How come binary is around 720K

Is there any tool I can use to analyse such issue?

Code works fine, just can understand the size.

Thanks,

**drl** · 10-04-2011

Hi.

One way is to look over the link map from linking loader ld. Here's an example:

Code:

#!/usr/bin/env bash

# @(#) s1	Demonstrate details of static link using ld map.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C gcc ld specimen

FILE=${1-f1.c}

pl " Input data file $FILE:"
cat $FILE

gcc -c $FILE
object=${FILE%.*}.o
pl " Object $object before link is:"
ls -lgG $object

pl " Libraries consulted after non-static link of $object:"
gcc $object
file a.out
ls -lgG a.out
ldd a.out
./a.out

rm -f a.out
gcc -static -Wl,-M $object > f1.txt
s=$( wc -l < f1.txt )
pl " Sample of link map for static link for $object from $s lines:"
specimen 10:10:10 -n f1.txt
file a.out
ls -lgG a.out
./a.out

exit 0

producing:

Code:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
GNU bash 3.2.39
gcc (Debian 4.3.2-1.1) 4.3.2
GNU ld (GNU Binutils for Debian) 2.18.0.20080103
specimen (local) 1.17

-----
 Input data file f1.c:
#include <stdio.h>

int
main (int argc, char *argv[])
{
  printf (" Hello, world version from c.\n");
  return (0);
}

-----
 Object f1.o before link is:
-rw-r--r-- 1 1496 Oct  4 07:25 f1.o

-----
 Libraries consulted after non-static link of f1.o:
a.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped
-rwxr-xr-x 1 8784 Oct  4 07:25 a.out
	linux-vdso.so.1 =>  (0x00007fff389ff000)
	libc.so.6 => /lib/libc.so.6 (0x00007f5928cf1000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f5929044000)
 Hello, world version from c.

-----
 Sample of link map for static link for f1.o from 5641 lines:
Edges: 10:10:10 of 5641 lines in file "f1.txt"
     1	Archive member included because of file (symbol)
     2	
     3	/usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(libc-start.o)
     4	                              /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/crt1.o (__libc_start_main)
     5	/usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(check_fds.o)
     6	                              /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(libc-start.o) (__libc_check_standard_fds)
     7	/usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(libc-tls.o)
     8	                              /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(libc-start.o) (__pthread_initialize_minimal)
     9	/usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(elf-init.o)
    10	                              /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/crt1.o (__libc_csu_fini)
   ---
  2817	 *fill*         0x00000000004551a2        0xe 90909090
  2818	 .text          0x00000000004551b0        0x7 /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(prof-freq.o)
  2819	                0x00000000004551b0                __profile_frequency
  2820	 *fill*         0x00000000004551b7        0x9 90909090
  2821	 .text          0x00000000004551c0       0x30 /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(strtof.o)
  2822	                0x00000000004551c0                strtof
  2823	                0x00000000004551e0                __strtof_internal
  2824	 .text          0x00000000004551f0       0x30 /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/libc.a(strtod.o)
  2825	                0x0000000000455210                __strtod_internal
  2826	                0x00000000004551f0                strtod
   ---
  5632	 .debug_ranges  0x0000000000000000       0x50 /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/crti.o
  5633	 .debug_ranges  0x0000000000000050       0x40 /usr/lib/gcc/x86_64-linux-gnu/4.3.2/../../../../lib/crtn.o
  5634	
  5635	.gnu.attributes
  5636	 *(.gnu.attributes)
  5637	
  5638	/DISCARD/
  5639	 *(.note.GNU-stack)
  5640	 *(.gnu_debuglink)
  5641	OUTPUT(a.out elf64-x86-64)
a.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 2.6.8, not stripped
-rwxr-xr-x 1 641188 Oct  4 07:25 a.out
 Hello, world version from c.

This illustrates the space saving from using dynamic libraries as opposed to including everything into the final executable using a static link.

Note that the map listing is 5000+ lines long.

See man pages for details on features used, options, etc.

Best wishes ... cheers, drl (91)

**legendbb** · 10-04-2011

Great thanks for your brilliant script.
One thing stuck me is the "specimen 10:10:10 -n f1.txt"
I am running uBuntu 10.10, all I googled and search in ubuntu specimen is a MID controller sampler.
What exact package name is the specimen comes from?

Salute!

**drl** · 10-04-2011

Hi.

Looking over the display of the versions of utilities used in that demonstration script:

Code:

GNU bash 3.2.39
gcc (Debian 4.3.2-1.1) 4.3.2
GNU ld (GNU Binutils for Debian) 2.18.0.20080103
specimen (local) 1.17

you'll notice that specimen is local to our organization, so you won't find it in any package. You could write something similar by using the head and tail utilities (which is how specimen began life) ... cheers, drl

If you are interested in writing a work-alike for our code, then you might like to see all the options:

Code:

usage: specimen [options] -- [files]

Extract samples of the files for display, often as an
accompaniment in posts of solutions to problems in on-line
forums, as well as for written documentation. 

options:

--lines=head:middle:tail
  This is the primary control. It specifies how many lines from
the files are printed in the beginning ("head") segment, middle
segment, and final ("tail") segment.  They are generally
interpreted as line counts, but can also be percentages.  The
middle count also can be specified with an option (see --random)
so that a random sample of "middle" lines will be extracted from
the section between head and tail.  Default is set internally,
initially to "5::5". 

If only the head is specified, say as "6", then the sequence will
taken to be "6:0:6".

As a convenience, this option may appear without the "--lines="
sequence, but *only* if it is the first option. The sequence may
be preceded by a minus sign, "-", to make it look like a standard
*nix option, e.g "specimen -7 -- file".

--number
  Prefix each line with the line number, then a TAB (allowing
  command "cut" to be easily used on the output).  Default: off.

--random
  Instead of taking "middle" count line around the mid-point line
of the file, the count will be a number of randomly selected lines
from the entire middle section. Default: off.

--percent
  The line counts may be specified as percentages, so that "10:5:10"
would be interpreted as 10%:5%:10% of the total number of lines in
the file. Default: off.

--bare
  A short title for each file is printed, this option will cause
that, and other adornments to be omitted.

--help (or -h)
  print this message and quit.

--version
  print the version and quit.

**legendbb** · 10-05-2011

Is it possible to static link only my own libs, so the size won't be too big without -lrt or maybe -lpthread
LDLIBS += -lpthread -static -L../_lib -lA -lB -lC -lrt

Thanks,

**drl** · 10-05-2011

Hi.

There are options incremental, -i, and relocatable, -r, mentioned in man ld that may do what you are asking. I don't recall ever making use of it (they appear both to refer to the same behavior).

Please let us know the results of your experimentation ... cheers, drl

**drl** · 10-05-2011

Hi.

Here is a hastily-created example of the partial link option:

Code:

#!/usr/bin/env bash

# @(#) s2	Demonstrate collection of objects to a "partial" object.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C gcc nm ld

B1=main
B2=ourlib
for f in ${B1}.c ${B2}.c
do
pl " Input data file $f:"
cat $f
done

pl " Results from compile:"
db " B1 is :$B1:"
gcc -c main.c ourlib.c
file ${B1}.o ${B2}.o
ls -lgG ${B1}.o ${B2}.o

pl " Results from ar:"
rm -f ${B2}.a
ar cq ${B2}.a ${B2}.o
file ${B2}.a
ls -lgG ${B2}.a
nm ${B2}.a

pl " Results from partial link:"
ld -r -o partial ${B1}.o ${B2}.a
file partial
ls -lgG partial

pl " Results from shared link:"
gcc -o shared-link ${B1}.o ${B2}.a
file shared-link
ls -lgG shared-link
./shared-link

pl " Results for creating shared from partial:"
gcc -o absolute partial
file absolute
ls -lgG absolute
./absolute

pl " Results from a static link:"
rm -f a.out
gcc -static ${B1}.o ${B2}.a
file a.out
ls -lgG a.out
./a.out

exit 0

producing:

Code:

% ./s2

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
GNU bash 3.2.39
gcc (Debian 4.3.2-1.1) 4.3.2
GNU nm (GNU Binutils for Debian) 2.18.0.20080103
GNU ld (GNU Binutils for Debian) 2.18.0.20080103

-----
 Input data file main.c:
#include <stdio.h>

int main()
{
  void a(), b();
  printf("main -> a\n");
  a();
  printf("main -> b\n");
  b();
  return(0);
}

-----
 Input data file ourlib.c:
#include <stdio.h>

void a()
{
  printf("  a\n");
  return;
}
void b()
{
  printf("  b\n");
  return;
}

-----
 Results from compile:
main.o:   ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
ourlib.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
-rw-r--r-- 1 1656 Oct  5 08:48 main.o
-rw-r--r-- 1 1608 Oct  5 08:48 ourlib.o

-----
 Results from ar:
ourlib.a: current ar archive
-rw-r--r-- 1 1752 Oct  5 08:48 ourlib.a

ourlib.o:
0000000000000000 T a
0000000000000010 T b
                 U puts

-----
 Results from partial link:
partial: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
-rw-r--r-- 1 1991 Oct  5 08:48 partial

-----
 Results from shared link:
shared-link: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped
-rwxr-xr-x 1 9031 Oct  5 08:48 shared-link
main -> a
  a
main -> b
  b

-----
 Results for creating shared from partial:
absolute: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped
-rwxr-xr-x 1 9031 Oct  5 08:48 absolute
main -> a
  a
main -> b
  b

-----
 Results from a static link:
a.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 2.6.8, not stripped
-rwxr-xr-x 1 641291 Oct  5 08:48 a.out
main -> a
  a
main -> b
  b

Note that file partial is not a static link, but a mashed-together group of objects. However, the size is smaller than the constituent parts, and if your goal is to reduce size, then this may be appropriate ... cheers, drl

**drl** · 10-05-2011

Hi.

If you meant dynamic linking, here is an example of linking mathematical function cos at run-time:

Code:

#!/usr/bin/env bash

# @(#) s1	Demonstrate run-time linking, "dlopen", et al.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C gcc 

FILE=${1-main.c}

pl " Input data file $FILE:"
cat $FILE

pl " Results, shared, expecting similar to \"-0.416147\":"
rm -f main
gcc -rdynamic -o main main.c -ldl
file main
ls -lgG main
./main

pl " Results, static, expect warning:"
rm -f main
gcc -static -o main main.c -ldl
ls -lgG main
file main
./main

exit 0

producing:

Code:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
GNU bash 3.2.39
gcc (Debian 4.3.2-1.1) 4.3.2

-----
 Input data file main.c:
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>

int
main (int argc, char **argv)
{
  /* from: man dlopen */
  void *handle;
  double (*cosine) (double);
  char *error;

  handle = dlopen ("libm.so", RTLD_LAZY);
  if (!handle)
    {
      fprintf (stderr, "%s\n", dlerror ());
      exit (EXIT_FAILURE);
    }

  dlerror ();			/* Clear any existing error */

  /* Writing: cosine = (double (*)(double)) dlsym(handle, "cos");
     would seem more natural, but the C99 standard leaves
     casting from "void *" to a function pointer undefined.
     The assignment used below is the POSIX.1-2003 (Technical
     Corrigendum 1) workaround; see the Rationale for the
     POSIX specification of dlsym(). */

  *(void **) (&cosine) = dlsym (handle, "cos");

  if ((error = dlerror ()) != NULL)
    {
      fprintf (stderr, "%s\n", error);
      exit (EXIT_FAILURE);
    }

  printf ("%f\n", (*cosine) (2.0));
  dlclose (handle);
  exit (EXIT_SUCCESS);
}

-----
 Results, shared, expecting similar to "-0.416147":
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped
-rwxr-xr-x 1 10536 Oct  5 08:58 main
-0.416147

-----
 Results, static, expect warning:
/tmp/cc2hfGqZ.o: In function `main':
main.c:(.text+0x1b): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
-rwxr-xr-x 1 641321 Oct  5 08:58 main
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 2.6.8, not stripped
-0.416147

The documentation in man dlopen does not include an example of linking from a static object, so I included that as the second run above. Although it seemed to have worked -- albeit with a warning -- I have no idea whether that generally will work correctly.

Thanks for asking the question, I learned something from researching this and creating the demonstrations.

Good luck with your experiments and projects ... cheers, drl (141)

**legendbb** · 10-05-2011

Thank you for all these valuable example code, I will learn and test both options.
Will post result for future reference.

**legendbb** · 10-26-2011

I moved to dynamic linked .so solution for now.

One interesting find out was, .a can still be used for regular linking, "-llib_name" ld produces the same result as linking those -o.
But as long as -static is inserted, doing nothing code is also 700KB. Didn't get too deep in looking into the binary. But will do.

Thanks to all.

Thread Tools

Display

Huge binary out of -static link?

Posting Permissions