Why libraries and linking matter
3 min readLast lesson ended with a Makefile that compiles several .c files into several .o files and joins them into one program. That joining step was the linker, and you used it without looking inside. This lesson opens it. The reason to care is practical: nearly every confusing error you meet when a build works on your laptop and fails on the board is a linking error. "undefined reference to", "cannot open shared object file", "version GLIBC_2.38 not found" and "required file not found" all come from this layer.
Start with the pieces. The compiler turns each C file into an a file of machine code and data for one source file, with some addresses and function names still left blank. The blanks exist because a file that calls printf or a function in another file does not know where that code will live. A a named thing in an object file, such as a function or global variable, which can be defined in that file or merely referred to is how those names are recorded. The the program, ld on Linux, that combines object files and libraries into one executable or library by matching each reference to a definition fills in the blanks.
A library, meaning a collection of compiled functions and data that many programs link against instead of each carrying its own copy of the source, comes in two forms. A static library is copied into your program at link time. A shared library stays in its own file and is attached when the program starts. The rest of the lesson explains how each form works, how the files are laid out, and how the system finds and connects them.
The toolchain has two linking moments. Link time is when gcc runs ld on your machine. Load time is when the kernel starts your program and a second program, the dynamic linker, finishes the job before main runs. Keeping those two moments separate in your head answers most questions in this lesson.
On an embedded board this separation has extra weight. You build on a PC with a cross-compiler, but the libraries are found by a loader running on the board, from the board's filesystem. A mismatch between the sysroot you linked against and the root filesystem you deploy is the classic embedded failure, and the tools below are how you catch it.
Static libraries and the link line
4 min readWe use a small running example. A file stats_sum.c defines stats_sum, stats_mean.c defines stats_mean (which calls stats_sum), and main.c calls stats_mean on three numbers. A header stats.h declares both functions.
/* stats_sum.c */
#include "stats.h"
int stats_sum(const int *v, int n)
{
int s = 0;
for (int i = 0; i < n; i++)
s += v[i];
return s;
}
/* stats_mean.c */
#include "stats.h"
double stats_mean(const int *v, int n)
{
return n ? (double)stats_sum(v, n) / n : 0.0;
}
An a single file that bundles many object files, called members, plus an index of the symbols they define; the usual form of a static library is built with ar:
gcc -c -Wall stats_sum.c stats_mean.c main.c
ar rcs libstats.a stats_sum.o stats_mean.o
ar t libstats.a # list the members
nm -s libstats.a # members, their symbols and the index
The letters in rcs are options: r inserts or replaces members, c creates the archive without a warning, and s writes the symbol index, which is what ranlib does. By convention the file is named lib plus a name plus .a. To use it, write -lstats and tell the linker where to look with -L:
gcc main.o -L. -lstats -o app_static
The -lstats option means "search for libstats.so and then libstats.a in each -L directory and then the default directories". For one directory the shared version wins if both exist. You can force the archive with -l:libstats.a or -Wl,-Bstatic.
1The linker keeps a list of unresolved names
The linker reads files strictly from left to right. It keeps a set of symbols that have been referred to but not yet defined. After main.o is read, that set holds stats_mean.
2An archive supplies only the members that help
When the linker reaches libstats.a, it looks in the archive index for definitions of anything in the unresolved set. It pulls in stats_mean.o, which adds a new unresolved name, stats_sum, so it pulls in stats_sum.o as well. A member that defines nothing the program needs is never copied. This is why a static link of a large library can still produce a small program.
3Order matters
Because the scan is left to right, an archive listed before the object that needs it supplies nothing, since nothing was unresolved yet when it was read. The result is an undefined reference error even though the library is right there.
gcc -L. -lstats main.o -o app # wrong: library before the object that uses it
gcc main.o -L. -lstats -o app # right: objects first, libraries after
Two archives that call each other need either a repeated name (-lA -lB -lA) or a group, -Wl,--start-group -lA -lB -Wl,--end-group, which makes the linker rescan until nothing new appears.
Ubuntu's gcc also passes --as-needed to the linker by default. With that option a shared library is recorded as a dependency only if it satisfies a symbol that was already unresolved at that point, so the same left-to-right rule bites shared libraries on Ubuntu too.
In a Makefile, keep LDFLAGS (such as -L.) before the objects and LDLIBS (such as -lstats) after them. The built-in make rule for linking already uses this order, so put -l options in LDLIBS, not LDFLAGS.
Shared libraries, sonames and ldconfig
4 min readA shared object is a library file, ending in .so, that is mapped into the address space of every process using it, so one copy of its code serves many processes. It must work wherever the loader places it. Its code therefore has to be position-independent, which gcc produces with -fPIC. Without it, instructions would hold absolute addresses and the loader would have to patch them in every process, defeating the sharing. Building ours:
gcc -fPIC -c -Wall stats_sum.c stats_mean.c
gcc -shared -Wl,-soname,libstats.so.1 -o libstats.so.1.2.3 stats_sum.o stats_mean.o
Always pass -fPIC explicitly. Ubuntu's gcc builds position-independent executables by default, which hides some mistakes, but that is a distribution choice and not something to rely on.
1The real name carries the full version
libstats.so.1.2.3 is the file that holds the code. The numbers are major, minor and patch by convention. The kernel and loader do not interpret them; people and tools do.
2The soname is the compatibility promise
The a name stored inside a shared library, usually the file name with only the major version, such as libstats.so.1, which programs record as their dependency is set with -Wl,-soname,libstats.so.1. When you link a program against the library, the linker copies the soname, not the real file name, into the program. You raise the major number only for an incompatible change, so programs built against libstats.so.1 keep working with 1.2.3 and with a later 1.3.0.
3Two symlinks complete the scheme
ldconfig -n . # creates libstats.so.1 -> libstats.so.1.2.3
ln -s libstats.so.1 libstats.so # the link name used by -lstats
ls -l libstats*
The soname link is what the loader looks for at run time. The unversioned libstats.so, the link name, is what -lstats finds at build time; development packages install it, which is why Debian splits libfoo1 (runtime) from libfoo-dev (headers plus that link). ldconfig creates the soname link by reading the soname stored in each library, but it does not create the link name.
Try it below: resolve the link name through both symlinks to the real file.
You can read the stored soname with readelf -d libstats.so.1.2.3 | grep SONAME. Now link and run:
gcc main.o -L. -lstats -o app_dyn
./app_dyn
Typical output (yours will differ in wording slightly):
./app_dyn: error while loading shared libraries: libstats.so.1: cannot open shared object file: No such file or directory
The link succeeded because -L. told the linker where to look. The run failed because the loader does not search the current directory. The fix is to place the library where the loader looks, which is what ldconfig and /etc/ld.so.conf are for.
ldconfig and the cache
The loader does not scan directories on every start. ldconfig reads /etc/ld.so.conf, which includes the files in /etc/ld.so.conf.d/, together with the default directories /lib and /usr/lib. It builds the binary cache /etc/ld.so.cache, which ld.so reads. ldconfig -p prints what the cache contains. To install a library system-wide:
sudo cp libstats.so.1.2.3 /usr/local/lib/
sudo ldconfig
ldconfig -p | grep stats
On Ubuntu /usr/local/lib is listed in a file under /etc/ld.so.conf.d/. For a library in some other directory, add a line to a new file there and run sudo ldconfig again. Forgetting that last step is the most common reason a freshly installed library is "not found".
Many small embedded images have no ldconfig and no cache, and some use musl, whose loader reads a search path from /etc/ld-musl-ARCH.path and defaults to /lib, /usr/local/lib and /usr/lib. On such targets, install libraries in a standard directory at image build time rather than relying on a cache update at run time.
ELF files, sections and symbols
4 min readThe Executable and Linkable Format, the standard file format for object files, executables, shared libraries and core dumps on Linux and most Unix systems format began in System V Unix and replaced the older a.out format on Linux in the mid-1990s. The same format covers four kinds of file, which you see in the header's Type field: a relocatable object (.o), an executable, a shared object (.so), and a core dump.
Start with file and the header:
file app_dyn libstats.so.1.2.3 stats_sum.o
readelf -h app_dyn
The header begins with the four bytes 0x7f 'E' 'L' 'F' and records the class (32 or 64 bit), the byte order, the machine (such as x86-64 or AArch64), the entry point, and where the program header table and section header table are. A default Ubuntu executable is a position-independent executable, so readelf shows type DYN for it, just as for a shared library. file distinguishes them for you.
Sections: the linker's view
A section is a named range such as .text (code), .rodata (read-only data), .data (initialised writable data) and .bss (zero-filled data that takes no space in the file). Others hold the symbol tables .symtab and .dynsym, the dynamic metadata .dynamic, the relocation tables, the constructor lists .init_array and .fini_array, and the debug data from last lesson. readelf -S lists them.
Segments: the loader's view
The loader does not care about sections. It reads the program header table, readelf -l, which lists segments. The important ones are PT_LOAD (map this range of the file into memory with these permissions), PT_INTERP (the path of the dynamic linker), PT_DYNAMIC (the dynamic metadata), PT_GNU_RELRO (a region to make read-only after relocation) and PT_GNU_STACK (whether the stack may be executable). The output ends with a section-to-segment mapping that shows which sections fall inside which segment.
Section headers are not needed to run a program; stripping tools may remove them. Segments are what the kernel and loader use.
Symbols
Three tools show symbols. nm is the quickest, readelf -s shows every field, and objdump -t is similar. For a shared library, nm -D and readelf --dyn-syms show the dynamic symbol table, the subset the loader uses. strip removes .symtab but must keep .dynsym, because dynamic linking needs it.
nm stats_mean.o
Typical output (yours may list extra entries):
0000000000000000 T stats_mean
U stats_sum
T means defined in the code section, and U means undefined: this object refers to stats_sum but does not define it, so the linker must find it elsewhere. Other letters include D (initialised data), B (bss), R (read-only data), W (weak) and V (weak object). A capital letter means a global symbol; a lowercase letter means local, visible only inside that file. Functions and variables declared static in C become local symbols.
A a global symbol that another definition may override without error, and that resolves to address zero when it is only referenced and has no definition is declared with __attribute__((weak)). Embedded code uses this for default interrupt handlers: the startup code defines every handler as a weak alias of an empty function, and your driver defines a normal function of the same name, which silently wins.
__attribute__((weak)) void board_hook(void) { } /* default: does nothing */
C++ complicates names. To allow overloading, the C++ compiler mangles a function name to encode its parameter types, so void f(int) becomes _Z1fi. A C library that C++ code must call is declared inside extern "C" { ... }, which turns mangling off. That is why stats.h should contain the __cplusplus guard shown earlier. nm -C or c++filt turns mangled names back into readable ones.
Going deeperWhy undefined symbols are allowed in a shared library
By default gcc -shared does not complain about undefined symbols, because a shared library may legitimately depend on symbols supplied by the program or by another library at load time. That also means a missing dependency shows up as a run-time failure, not a build error. Add -Wl,-z,defs (or --no-undefined) when building a library that should be complete on its own, and the linker refuses to finish if anything is unresolved.
The dynamic linker and where it looks
4 min readThe the program, ld.so or ld-linux, named in the executable's PT_INTERP segment, that the kernel runs first to load the needed shared libraries and connect symbols before main is itself a shared object. readelf -l app_dyn | grep interpreter shows its path. On x86-64 it is /lib64/ld-linux-x86-64.so.2; on 64-bit ARM it is /lib/ld-linux-aarch64.so.1; on 32-bit ARM with hardware floating point it is /lib/ld-linux-armhf.so.3. The kernel maps your program and that interpreter and starts the interpreter, which maps the rest.
If that path does not exist on the machine, the shell reports cannot execute: required file not found, a confusing message because the program file itself is present. The cause is usually a glibc-linked binary on a musl system, or an ARM binary built for the wrong ABI.
What the loader reads
readelf -d app_dyn prints the dynamic section. Each NEEDED entry is a soname the loader must find. Run only on trusted files, ldd app_dyn prints each needed library and where it resolved:
linux-vdso.so.1 (0x00007ffd...)
libstats.so.1 => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f...)
/lib64/ld-linux-x86-64.so.2 (0x00007f...)
The linux-vdso.so.1 line is a kernel-provided object, not a file. Some versions of ldd run the program's loader on the file, and the manual page warns against using it on untrusted binaries; readelf -d or objdump -p only read the file and are safe.
The search order
For a NEEDED name without a slash, glibc searches in this order, described in ld.so(8):
- The directories in
DT_RPATH, but only if the object has noDT_RUNPATH. - The directories in the environment variable
LD_LIBRARY_PATH, skipped in secure-execution mode. - The directories in
DT_RUNPATH. - The cache
/etc/ld.so.cache. - The default directories,
/liband/usr/lib.
You set an embedded run path at link time with -Wl,-rpath,DIR. The special token $ORIGIN means the directory containing the file itself, which makes a self-contained application folder relocatable:
gcc main.o -L. -lstats -Wl,-rpath,'$ORIGIN' -o app_dyn
readelf -d app_dyn | grep -E 'NEEDED|RUNPATH|RPATH'
Single quotes keep the shell from expanding $ORIGIN; inside a Makefile write $$ORIGIN. On Ubuntu 24.04 the linker emits DT_RUNPATH for -rpath. The older tag DT_RPATH is produced with -Wl,--disable-new-dtags. They differ in two ways: RPATH is searched before LD_LIBRARY_PATH, so the environment cannot override it, and it also applies to the dependencies of dependencies, while RUNPATH is searched after LD_LIBRARY_PATH and covers only the object's own direct needs.
Setting LD_LIBRARY_PATH globally changes which libraries every program loads and can cause a program to pick up the wrong library. For a device, use RUNPATH with $ORIGIN or install into a standard directory. For setuid and setgid programs, and programs with file capabilities, glibc enters secure-execution mode and ignores LD_LIBRARY_PATH and most other LD_ variables, because otherwise a user could inject code into a privileged process. sudo also strips these variables from the environment by default.
The loader loads needed libraries breadth-first, starting from the program's own NEEDED list. This load order also fixes which definition wins when two libraries export the same name, a point that returns with LD_PRELOAD later.
Relocations, the PLT and the GOT
5 min readA shared library can be mapped at a different address in every process, and ASLR randomises the address each run. Code therefore cannot hold the address of a function in another library. A a record in an ELF file telling the linker or loader to compute an address and write it into a specific location describes each place that needs a computed value. readelf -r app_dyn lists them. On x86-64 the common types are R_X86_64_RELATIVE (add the load address of this object), R_X86_64_GLOB_DAT (store the address of a global symbol) and R_X86_64_JUMP_SLOT (store a function address for a call). The AArch64 equivalents are R_AARCH64_RELATIVE, R_AARCH64_GLOB_DAT and R_AARCH64_JUMP_SLOT.
The code sits in read-only, shared pages, so the addresses live elsewhere. The Global Offset Table, a table of addresses in the writable data of a program or library, filled by the dynamic linker holds them. A call to a shared function does not jump to the library directly but to a stub in the Procedure Linkage Table, a table of small code stubs in the program, each of which jumps through one GOT entry. The code is position-independent and the GOT entry is the only thing the loader needs to write.
The classic mechanism, still used with the option -z lazy, is lazy binding. The GOT entry for each function starts out pointing back into the PLT, so the first call takes a detour through the resolver:
Call the stub: The program calls the PLT stub for the function, for example the one for
stats_sumin a shared library.Read the GOT entry: The stub jumps through its GOT entry. On the first call that entry does not hold the real address; it points to the next instruction in the stub.
Push the identity: The stub pushes an index naming which relocation it is, then jumps to the first PLT stub, which pushes a pointer to the library list and jumps into the dynamic linker's resolver.
Look up the symbol: The resolver searches the loaded libraries in load order for the definition of the symbol.
Patch the GOT: The resolver writes the real address into the function's GOT entry.
Jump to the function: The resolver transfers control to the real function, which returns straight to the original caller.
Later calls go direct: On every later call the stub reads the patched GOT entry and jumps to the function immediately.
Lazy binding and BIND_NOW
Lazy binding speeds up startup because functions never called are never looked up. But the first call to each function pays an unpredictable lookup cost, which is a poor fit for a real-time loop, and the GOT must stay writable for the life of the process, which gives an attacker with a memory-write bug a target. Ubuntu's toolchain therefore links binaries with -z now by default, which sets the BIND_NOW flag: the loader resolves every function at start and then makes the relocated area read-only. This combination is called full RELRO, and readelf -l shows the GNU_RELRO segment while readelf -d shows FLAGS: BIND_NOW. You can still ask for lazy binding with -Wl,-z,lazy, and force binding at start on any program with the environment variable LD_BIND_NOW=1.
With -fno-plt, gcc emits calls that load the function address from the GOT directly and call through it, so no PLT stub is used. That saves one jump per call at the price of larger code and combines well with -z now.
Going deeperWhy the first call is hard to see with ltrace and gdb
In gdb, disassemble on a call to a shared function shows call to a symbol such as stats_sum@plt, not the library function. Stepping into it with stepi on a program linked with -z lazy shows the push and the jump into the resolver, and after the first call the same stub jumps directly. With Intel branch-protection enabled (-fcf-protection), stubs live in a second section called .plt.sec. On a -z now binary there are no lazy entries to watch, which is why the demonstration needs the explicit -z lazy flag.
Loading plugins with dlopen
4 min readSo far every library was named when the program was linked. A plugin is a shared library loaded by a running program on demand. That architecture needs the opposite: the program discovers files and loads them while running. The interface is in <dlfcn.h>: dlopen loads a library and returns a handle, dlsym returns the address of a symbol in it, dlclose releases it, and dlerror returns a text description of the last failure. On glibc before 2.34 you also link with -ldl; since glibc 2.34 the functions live in libc itself, so on Ubuntu 24.04 (glibc 2.39) the flag is no longer needed. Older Yocto and Buildroot images based on glibc earlier than 2.34 still need it.
The contract between host and plugin is a header both include:
/* plugin.h */
typedef int (*plugin_run_fn)(int input);
struct plugin_desc {
const char *name;
int abi_version;
plugin_run_fn run;
};
#define PLUGIN_ABI 1
A plugin exports one symbol, plugin_desc, filled in:
/* twice.c */
#include "plugin.h"
static int twice_run(int x) { return 2 * x; }
__attribute__((visibility("default")))
const struct plugin_desc plugin_desc = { "twice", PLUGIN_ABI, twice_run };
gcc -fPIC -shared -fvisibility=hidden -o twice.so twice.c
The host loads a path given on the command line:
/* host.c */
#include <dlfcn.h>
#include <stdio.h>
#include "plugin.h"
int main(int argc, char **argv)
{
if (argc < 2) { fprintf(stderr, "usage: %s ./plugin.so\n", argv[0]); return 2; }
void *h = dlopen(argv[1], RTLD_NOW | RTLD_LOCAL);
if (!h) { fprintf(stderr, "dlopen: %s\n", dlerror()); return 1; }
dlerror(); /* clear any old error */
const struct plugin_desc *d = dlsym(h, "plugin_desc");
const char *err = dlerror();
if (err || d->abi_version != PLUGIN_ABI) {
fprintf(stderr, "bad plugin: %s\n", err ? err : "ABI mismatch");
dlclose(h);
return 1;
}
printf("%s(21) = %d\n", d->name, d->run(21));
dlclose(h);
return 0;
}
1dlopen finds and maps the file
If the name contains a slash, dlopen uses it as a path. If not, it searches like the loader: the caller's RPATH, LD_LIBRARY_PATH, the caller's RUNPATH, the cache and the default directories. Passing twice.so without ./ therefore fails unless that library is on the search path, a frequent surprise.
2Flags choose when and where
RTLD_NOW resolves all undefined symbols before returning, so errors appear at dlopen; RTLD_LAZY defers function resolution to first call. Setting LD_BIND_NOW forces NOW. RTLD_LOCAL, the default, keeps the library's symbols private to this handle; RTLD_GLOBAL makes them available to libraries loaded later. For plugins, prefer RTLD_LOCAL so two plugins that happen to export the same helper name cannot clash.
3dlsym returns an address, dlerror explains failure
dlsym returns NULL on failure, but NULL can also be a legal value for a symbol, so the safe pattern is to call dlerror to clear it, call dlsym, then call dlerror again. dlerror returns a string once and then NULL. Converting dlsym's void * to a function pointer is allowed by POSIX, but -Wpedantic warns, which is why this lesson passes a data symbol.
4dlclose drops a reference
Each library has a reference count. dlclose decrements it and unmaps the library only when it reaches zero. Destructors run then. A library that registered a callback or started a thread must undo that first, or the program will later call into unmapped memory.
If a plugin calls a function defined in the host program, the program must export it. By default the dynamic symbol table of an executable contains only symbols that shared libraries need, so link the host with -rdynamic (or -Wl,--export-dynamic) to make its symbols available.
A library's constructors run during dlopen, before your code gets control. Never dlopen a path that an unprivileged user can write, and never build the path from untrusted input; loading a library is running its code.
Visibility, versioning and constructors
4 min readBy default every global function in a shared library is exported: it goes into the dynamic symbol table and other objects may bind to it, and even calls inside the library to its own functions go through the PLT because another object could in principle override them. A large library with thousands of internal helpers pays for all of that at load time and in memory. Symbol visibility fixes it.
Compile with -fvisibility=hidden so everything is hidden by default, then mark the public API:
#define STATS_API __attribute__((visibility("default")))
STATS_API int stats_sum(const int *v, int n);
A hidden symbol is not in .dynsym, cannot be bound to from another object, cannot clash with a same-named symbol elsewhere, and can be called directly or inlined by the compiler. The effects are a smaller symbol table, faster symbol lookup and fewer relocations at load time, and a clean line between public API and implementation. Check the result with nm -D libstats.so.1.2.3, which now lists only the public names.
Version scripts and symbol versioning
Visibility is all-or-nothing per symbol. A a linker input file that lists which symbols a library exports, optionally grouped under named versions such as LIBSTATS_1 controls it at link time and adds versions:
LIBSTATS_1 {
global:
stats_sum;
stats_mean;
local:
*;
};
gcc -shared -fPIC -Wl,-soname,libstats.so.1 -Wl,--version-script=libstats.map \
-o libstats.so.1.2.3 stats_sum.o stats_mean.o
objdump -T libstats.so.1.2.3
readelf -V libstats.so.1.2.3
Every exported symbol now carries a version tag. A program linked later records which version it needed, for example stats_sum@LIBSTATS_1, and the loader checks it. This is how glibc keeps old binaries running: objdump -T prog | grep GLIBC shows tags like GLIBC_2.2.5, the baseline version of many x86-64 functions, and a binary built against a newer glibc may need a tag such as GLIBC_2.38. When the library offers the same name with two behaviours, the .symver assembler directive attaches both versions, and @@ marks the default for new links. An incompatible change to a function does not force a new soname.
The practical consequence for embedded work: a binary built on a newer glibc than the board's fails there with version GLIBC_2.xx not found (required by ./prog). Build against the same glibc, or an older one, as the target, which is what a proper cross-toolchain and sysroot provide.
Constructors and destructors
A function marked __attribute__((constructor)) runs before main when the object is loaded; one marked destructor runs at exit or at dlclose. The compiler places pointers to them in the .init_array and .fini_array sections, and the dynamic section lists their addresses. Older toolchains used .ctors.
#include <stdio.h>
__attribute__((constructor(101))) static void early(void) { puts("early"); }
__attribute__((constructor)) static void normal(void) { puts("normal"); }
__attribute__((destructor)) static void bye(void) { puts("bye"); }
int main(void) { puts("main"); return 0; }
Typical output: early, normal, main, bye, one per line. A lower priority number runs first, and priorities 0 to 100 are reserved. Across files, a shared library's initialisers run before those of the programs that need it, and destructors run in the reverse order.
Going deeperSelf-registering code and archives
A pattern used in drivers and test frameworks is a constructor that adds an object to a global list. In a shared library this works. In a static archive it can silently fail: the linker pulls a member only if it defines a needed symbol, and a registration-only member defines nothing the program references, so its constructor is never linked in. Link the archive with -Wl,--whole-archive -lfoo -Wl,--no-whole-archive to force every member in, or make the program reference a symbol from each member.
Interposing, tracing and choosing static or dynamic
4 min readInterposition, replacing a library function with your own function of the same name by making the dynamic linker find yours first, works because the dynamic linker resolves a symbol by searching a global list of libraries in order: the executable first, then any libraries in LD_PRELOAD, then the NEEDED libraries in load order, then anything loaded with RTLD_GLOBAL. The first definition found wins. A preloaded library therefore overrides open, malloc or write in every library the process uses, and it can call the original with dlsym(RTLD_NEXT, "open"), which means "the next definition after me".
A minimal file-access tracer wraps open:
#define _GNU_SOURCE
#include <dlfcn.h>
#include <fcntl.h>
#include <stdarg.h>
#include <string.h>
#include <sys/types.h>
#include <unistd.h>
static int (*real_open)(const char *, int, ...);
int open(const char *path, int flags, ...)
{
mode_t mode = 0;
if (flags & (O_CREAT | O_TMPFILE)) {
va_list ap;
va_start(ap, flags);
mode = va_arg(ap, mode_t);
va_end(ap);
}
if (!real_open)
real_open = (int (*)(const char *, int, ...))dlsym(RTLD_NEXT, "open");
write(2, "open: ", 6);
write(2, path, strlen(path));
write(2, "\n", 1);
return real_open(path, flags, mode);
}
gcc -fPIC -shared -o libtrace.so trace.c
LD_PRELOAD=$PWD/libtrace.so ./app_dyn
The wrapper uses write on file descriptor 2 instead of printf, because printf may allocate memory and a wrapped malloc that calls printf would recurse forever. The same hazard applies to dlsym inside a malloc wrapper. Use a re-entrancy guard in those cases.
There are limits. LD_PRELOAD has no effect on statically linked programs, which have no dynamic linker, and it is restricted for setuid programs. It also misses calls the library makes to itself: libc's fopen does not go through the exported open, so a complete trace wraps fopen, openat and the 64-suffixed names too. A compiler may also turn printf("text\n") into puts. When a trace must be complete, use strace (next lesson) which watches the system calls themselves. System-wide preloading through /etc/ld.so.preload exists and is a powerful hazard; a bad library there can stop every program from starting.
Tracing the loader itself
LD_DEBUG asks glibc's dynamic linker to explain itself on stderr. Useful values are libs (search and load), files, symbols and bindings (which definition each symbol bound to), versions, statistics, and all. LD_DEBUG=help ./app_dyn lists them.
LD_DEBUG=libs ./app_dyn 2>&1 | head -20
LD_DEBUG=bindings ./app_dyn 2>&1 | grep stats_
cat /proc/self/maps | grep '\.so'
The last line shows which libraries a running process really has mapped, which is the truth after any dlopen or preload. ld --verbose prints the default linker script and its SEARCH_DIR lines, and -Wl,-Map=app.map writes a link map showing where every input landed. Linker scripts are mostly invisible in user space but are the main tool for placing code and data when you build bare-metal firmware.
Static or dynamic?
Linking with -static copies everything needed into one file, and requires the .a archives of every library, including libc; on Ubuntu those come from the libc6-dev package. The trade-offs:
| Concern | Static | Dynamic |
|---|---|---|
| File size | Larger per program, but only used members are included | Small program, libraries shared on disk and in memory |
| Deployment | One file, no missing-library errors | Libraries and versions must match the target |
| Security updates | Rebuild and redeploy every program | Update one library file |
| Startup | No loader work | Loading and relocation cost |
| Plugins and NSS | Hard with glibc | Natural |
On an embedded device with one application and no package manager, a static or mostly static build removes a whole class of deployment errors, and with -ffunction-sections -Wl,--gc-sections the linker drops unused code. On a device with many programs sharing libc, shared libraries save flash and RAM. glibc is a poor static-linking target because functions such as name lookup and user lookup load libraries at run time and warn when linked statically. musl is designed for clean static linking, which is a main reason it is popular on small targets, though its loader and search paths differ from glibc's, and it does not implement glibc symbol versioning. If you ship under the LGPL, static linking has licence consequences you must check.
Common mistakes and how to fix them
2 min readThe messages look alike but come from different moments, so first decide whether the failure is at link time or at run time.
Undefined reference at link time means a symbol has no definition among the inputs read so far. Usual causes are libraries placed before the objects that use them, a missing -l, or a -L path that does not contain the library. Diagnose with nm -C file.o | grep ' U ' and nm -s libfoo.a | grep symbol. Fix the order and add the missing library. For C++ code that calls C, check for the missing extern "C": the unresolved name will look mangled.
error while loading shared libraries: libfoo.so.1: cannot open shared object file means the loader searched its list and found no file named by NEEDED. Diagnose with ldd, readelf -d, LD_DEBUG=libs and ldconfig -p. Fix by installing the library in a standard directory and running sudo ldconfig, or by linking with -Wl,-rpath,'$ORIGIN/lib'. A frequent cause on embedded targets is that the sysroot had the library but the root filesystem image did not.
cannot execute: required file not found is the missing interpreter. Run readelf -l prog | grep interpreter and compare with what the target has. Rebuild with a toolchain that matches the target's C library, for glibc or musl.
version GLIBC_2.xx not found is built-against-newer-glibc. objdump -T prog | grep GLIBC shows the required versions, and the fix is to build with an older sysroot or on the target.
Wrong library loaded silently happens when LD_LIBRARY_PATH or an old /usr/local/lib copy shadows the one you intended. LD_DEBUG=libs and /proc/PID/maps show the file actually used.
A plugin that loads on your PC and fails on the board is usually a mismatch of architecture or C library, or a hidden symbol. file plugin.so shows the architecture; dlerror text names the missing symbol.
You cannot run ldd on an ARM program from an x86 PC. Use the cross tool: aarch64-linux-gnu-readelf -d prog | grep NEEDED, or run it under qemu-aarch64 -L /path/to/sysroot prog.
For a deployed program, list readelf -d NEEDED entries, confirm every library exists on the target at the same soname, confirm the interpreter path exists, and confirm glibc's version tags are no newer than the target's. Four checks catch nearly all first-boot linking failures.
Try it
1 min readBuild libstats.a and a versioned shared libstats.so.1.2.3 from the same objects, link two programs, and compare them with ls -l, file, readelf -d, nm and ldd. Write the dlopen plugin loader from this lesson, load it by path, and then break it on purpose by hiding the symbol and by dropping the ./. Finally, wrap open and write in an LD_PRELOAD library to trace the file accesses of an existing program, and check the missing calls with strace. The practice pack has exercises that climb from reading symbol letters to cross-checking an ARM binary's dependencies; solutions are not given here.
Key takeaways
- Linking joins object files and libraries by matching undefined symbols to definitions; library order on the command line matters.
- A static archive is a bundle of object files copied into the program; a shared library is mapped at run time and shared by many processes.
- A shared library has three names: the real file, the soname that programs record, and the link name used at build time.
- ELF sections describe a file for the linker; segments describe the same bytes for the loader, and readelf shows both.
- ld.so searches RPATH, LD_LIBRARY_PATH, RUNPATH, the ld.so.cache and the default directories, and ldd shows the result.
- Calls to shared functions go through the PLT and GOT; lazy binding resolves them on first call, while BIND_NOW and RELRO resolve and lock them at load time.
- dlopen loads a library by path at run time, dlsym finds a symbol and dlerror reports failures, which makes plugins possible.
- Hidden visibility, version scripts and LD_PRELOAD control which symbols a library exports and which definition a program sees.
Reflect
- Explain to a colleague, in two sentences: build a static library with ar and a shared library with -fPIC, -shared and a soname, and explain the three-name symlink scheme.
- Which objective are you least sure about? Re-read that section, then do its exercise in the practice pack.
- What surprised you when you ran the commands yourself?
On a real board
Repeat the key commands on your Raspberry Pi or lab board over SSH or the serial console. Note anything that behaves differently from your PC, such as missing tools, a BusyBox shell or a read-only filesystem.
References and further reading
- Man pageld.so(8), the dynamic linker/loader
- Man pagedlopen(3)
- Man pageelf(5), format of ELF executable binary files
- Man pageldconfig(8)
- Man pageldd(1)
- Official docsGNU ld, the GNU linker manual
- Official docsGNU binutils, the ar program
- Official docsGCC manual, Function Attributes (constructor, visibility, weak)
- PaperHow to Write Shared Libraries, Ulrich Drepper
Official documentation first. Links checked when the lesson was published.