The three tables behind every file descriptor
4 min readThe previous lesson ended with a promise: two descriptors can share one file offset, and two opens of the same file do not. This lesson explains why, then builds on it: duplication, descriptor control, and the locks that keep concurrent writers from corrupting a file.
Start with a concrete puzzle. Two programs append to one log file. Each opens the file itself, seeks to the end, and writes. Sometimes lines overwrite each other. Now change one program to inherit its descriptor from its parent instead of opening the file. The overwrites stop. Same file, same writes, different result. The difference is not in the file. It is in the tables the kernel keeps.
The kernel maintains three levels of state for open files:
- The per-process file descriptor table. Each process has its own. Entry number n is what your program calls file descriptor n. Each entry holds one flag, which decides whether the descriptor survives into a new program (close-on-exec, covered below), and a pointer to the next level.
- The system-wide table of open file descriptions. An the kernel object created by one successful open, holding the current file offset, the status flags and a reference to the file itself is the thing an fd really points at. It stores the current offset, the access mode, the status flags such as O_APPEND and O_NONBLOCK, and a reference to the inode.
- The inode table, one per filesystem. An the kernel's record of a file's identity: type, permissions, size, timestamps, and where its data lives, independent of any name holds the file's metadata and the list of locks held on it. Many names can point at one inode; many open file descriptions can point at one inode.
Now the puzzle answers itself. When a process calls open(), the kernel creates a new open file description with its own offset, even if the file is already open elsewhere. Two independent opens mean two offsets, so two seek-to-end-then-write sequences can land on the same bytes. When a descriptor is duplicated with dup, or inherited across fork, both descriptors point at the same open file description. One offset moves for both, so the writes follow each other instead of colliding.
The sharing rules are worth memorising:
- dup, dup2, dup3 and fcntl F_DUPFD create a new fd pointing at the same open file description. Offset and status flags are shared.
- fork, the call that creates a child process, copies the fd table into the child. Parent and child share the same open file descriptions, so they share offsets too.
- A separate open() of the same path creates a separate open file description, with a separate offset, even in the same process.
- The close-on-exec flag is per descriptor, not per open file description. Duplicating an fd does not copy it.
- All threads in one process share the fd table, so they share every open file description as well.
You can watch these tables from the shell. ls -l /proc/$$/fd lists your shell's descriptors as symlinks to what they point at. cat /proc/$$/fdinfo/1 shows the offset and flags of descriptor 1. These are the same structures the kernel uses, exposed as files.
Each process has a limit on open descriptors, RLIMIT_NOFILE, often 1024 by default. Exceeding it makes open fail with EMFILE; exhausting the system-wide pool gives ENFILE. A daemon on a board that leaks one fd per request dies after a thousand requests. Check ulimit -n, and watch /proc/sys/fs/file-nr on the whole system.
Going deeperThe structures in kernel source
In the kernel, an open file description is struct file (include/linux/fs.h): f_pos is the offset, f_flags the status flags, f_count a reference count, f_inode the link to the inode. The per-process table is a struct files_struct holding an fdtable, an array of struct file pointers plus bitmaps for close-on-exec and busy fds (include/linux/fdtable.h). Allocation and duplication live in fs/file.c; the locking code you will meet later lives in fs/locks.c. An fd lookup is an array index under an RCU read lock, not a search, which is why system calls can translate your fd to a struct file so cheaply. When f_count drops to zero, the open file description is freed; the inode survives until its own reference count and link count allow release.
dup, dup2, dup3 and /dev/fd
4 min read#include <unistd.h>
int dup(int oldfd);
int dup2(int oldfd, int newfd);
#define _GNU_SOURCE
#include <unistd.h>
int dup3(int oldfd, int newfd, int flags);
dup returns the lowest unused descriptor. With only 0, 1 and 2 open, dup(1) returns 3. dup2 uses the number you pass: if newfd is already open, dup2 closes it first, then makes it point at the same open file description as oldfd. The close and the copy happen as one atomic step, which matters in threaded programs where another thread could otherwise grab the number in between. If oldfd equals newfd, dup2 does nothing and returns newfd. If oldfd is invalid, dup2 fails with EBADF and does not close newfd. dup3 is dup2 plus a flags argument; today the only flag is O_CLOEXEC, and passing equal numbers fails with EINVAL instead of being a no-op.
Here is the offset-sharing experiment in full. It writes through a duplicated descriptor and through a second, independent open of the same file:
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
int main(void)
{
int a = open("offsets.txt", O_RDWR | O_CREAT | O_TRUNC, 0644);
int b = dup(a); /* same open file description */
int c = open("offsets.txt", O_RDWR); /* a new, independent one */
write(a, "AAAA", 4); /* shared offset now 4 */
write(b, "BBBB", 4); /* continues at 4, ends at 8 */
write(c, "cccc", 4); /* c starts at 0, overwrites AAAA */
printf("a/b offset: %lld, c offset: %lld\n",
(long long)lseek(a, 0, SEEK_CUR),
(long long)lseek(c, 0, SEEK_CUR));
close(a); close(b); close(c);
return 0;
}
The file ends up containing ccccBBBB, and the program prints offset 8 for a and b, offset 4 for c. Error checks are omitted for brevity; the lab version adds them. The same sharing applies across fork: a child that writes through an inherited fd advances the parent's offset too, because both point at one open file description.
This is exactly how the shell implements redirection. In ./tool > out.log 2>&1, the shell does not open out.log twice. Opening twice would give two offsets, and stdout and stderr could overwrite each other. Instead it duplicates:
Open the target: the shell opens out.log for writing, getting some descriptor, say 3.
Redirect stdout: dup2(3, 1) closes descriptor 1 and makes it point at the same open file description as 3.
Drop the spare: close(3); descriptor 1 now carries the file alone.
Redirect stderr: dup2(1, 2) makes descriptor 2 point at the same open file description as 1.
Exec the tool: the new program is loaded with exec, the call that replaces a process's program; it inherits 0, 1 and 2, and both 1 and 2 share one offset, so output and errors interleave cleanly.
Order matters. 2>&1 > out.log first points stderr at the terminal (where stdout still goes), then redirects stdout to the file; stderr stays on the terminal. The shell evaluates redirections left to right, and each dup2 copies the current target.
Linux also exposes descriptors as filenames. /dev/fd is a symlink to /proc/self/fd, and opening /dev/fd/n for a regular file is equivalent to dup(n), provided the access mode matches. /dev/stdin, /dev/stdout and /dev/stderr are symlinks to /dev/fd/0, 1 and 2. This lets programs that only accept filenames work with pipelines: ls | diff /dev/fd/0 oldlist compares a pipe against a file. Bash process substitution, diff <(ls dir1) <(ls dir2), uses the same mechanism with higher descriptor numbers. Sockets cannot be reopened this way; the open fails with ENXIO.
Resolve those symlink chains yourself - trace /dev/stdin down to the per-process fd symlink it finally lands on:
fcntl: one call, many jobs
3 min read#include <fcntl.h>
int fcntl(int fd, int cmd, ...);
The most common job is reading and changing the open file status flags, the ones you passed to open. F_GETFL returns them; F_SETFL writes them back. Because the flags live in the open file description, a change through one duplicated descriptor is visible through all of them. The safe pattern is read, modify, write:
int flags = fcntl(fd, F_GETFL);
if (flags == -1) { perror("fcntl F_GETFL"); exit(1); }
if (fcntl(fd, F_SETFL, flags | O_APPEND) == -1) {
perror("fcntl F_SETFL");
exit(1);
}
This matters most for descriptors you did not open yourself: the three standard descriptors, or the ends returned by pipe() and socket(). You cannot pass flags to those calls, so fcntl is the only way to add O_NONBLOCK or O_APPEND afterwards.
Two details trip people up. First, the access mode is not a single bit: O_RDONLY is 0, O_WRONLY is 1, O_RDWR is 2. To test it, mask with O_ACCMODE and compare: if ((flags & O_ACCMODE) == O_WRONLY). Second, F_SETFL can only change some flags. On Linux these are O_APPEND, O_NONBLOCK, O_NOATIME, O_ASYNC and O_DIRECT. Attempts to change the access mode or other flags are ignored, not reported as errors.
You cannot turn a read-only descriptor into a writable one with F_SETFL. The access mode is fixed at open time. If you need write access, open the file again with the mode you need.
F_GETFD and F_SETFD read and set the descriptor flags, which today means only FD_CLOEXEC, the close-on-exec flag covered in the next section. Note the pairing: F_GETFL/F_SETFL act on the shared open file description; F_GETFD/F_SETFD act on the private fd table entry.
For duplication, fcntl(oldfd, F_DUPFD, start) returns the lowest free descriptor greater than or equal to start. It is dup with a floor, useful when you want to keep low numbers free. F_DUPFD_CLOEXEC does the same and sets close-on-exec on the new descriptor in the same step.
Finally, F_GETOWN and F_SETOWN get and set the process or process group that receives SIGIO and SIGURG for this descriptor. A negative value means a process group. Combined with O_ASYNC, this is signal-driven I/O: the kernel sends SIGIO when input arrives on a terminal, pipe or socket. Regular files never generate these signals. The newer F_SETOWN_EX and F_GETOWN_EX take a small structure and can target a single thread, removing the ambiguity between a PID and a process group ID. You will use this machinery again in the event-loop lessons.
Close-on-exec: stopping fd leaks at exec
4 min readWhen a process calls execve, the system call that loads a new program into the process, the kernel replaces the program but keeps the fd table, except for descriptors marked with a per-descriptor flag, FD_CLOEXEC, that makes the kernel close the descriptor automatically when an exec succeeds. This default is deliberate: the shell's redirections from the previous section only work because the descriptors survive exec. A pipe you built for a child only works because the child inherits its ends.
But every other open descriptor survives too. A log file, a database, a listening socket: the new program inherits them all, whether it wants them or not. On a long-running embedded system this is a real failure mode. A supervisor that opens a config file and then execs a helper hands the helper an open config fd; the helper execs something else; the chain grows. The file may stay unlinked-but-open, its space never freed, and a privileged descriptor may land in a less privileged program.
Reproduce the leak in three lines of substance:
#include <fcntl.h>
#include <unistd.h>
int main(void)
{
open("/home/learner/secret.log", O_WRONLY | O_CREAT | O_APPEND, 0600);
execlp("ls", "ls", "-l", "/proc/self/fd", (char *)NULL);
return 1; /* reached only if exec failed */
}
ls lists its own descriptors, which are the ones our program handed over. Among the standard descriptors you will see a line like this (typical output, yours will differ):
3 -> /home/learner/secret.log
The fix is to mark the descriptor. The modern way is at creation time, with O_CLOEXEC in the open flags:
int fd = open("/home/learner/secret.log",
O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0600);
Now exec closes it, and the listing shows no secret.log, only the standard descriptors plus ls's own temporary directory handle. The older way is a second call after open:
int flags = fcntl(fd, F_GETFD);
if (flags == -1) { perror("fcntl F_GETFD"); exit(1); }
if (fcntl(fd, F_SETFD, flags | FD_CLOEXEC) == -1) {
perror("fcntl F_SETFD");
exit(1);
}
Both work in a single-threaded program. In a threaded one, only the atomic forms are safe, as the deep block below explains. For duplicates, remember that dup, dup2 and F_DUPFD always clear close-on-exec on the new descriptor. If the duplicate must also be marked, use dup3 with O_CLOEXEC or F_DUPFD_CLOEXEC.
The flag's journey through process creation is easy to confuse, so pin it down. fork copies the flag as-is into the child; the descriptor stays open in both. exec closes exactly the descriptors whose flag is set. A descriptor without the flag survives any number of execs. One legitimate exception to marking everything: systemd socket activation hands a service pre-opened listening descriptors that must survive exec, so those are deliberately left unmarked.
Going deeperWhy the atomic forms exist
Between open() and fcntl(F_SETFD) there is a window. In a multithreaded program, another thread can fork and exec inside that window, and the child inherits the unmarked descriptor. No amount of care in your own thread closes it. The kernel added O_CLOEXEC in Linux 2.6.23, F_DUPFD_CLOEXEC in 2.6.24, and dup3 in 2.6.27 precisely to make marking atomic with creation. The rule in modern code: mark at creation, never as an afterthought. The same reasoning applies to accept4 with SOCK_CLOEXEC, pipe2 with O_CLOEXEC, and socket with SOCK_CLOEXEC, which you will meet in the sockets lessons.
Nonblocking I/O and files beyond 2 GB
4 min readO_NONBLOCK has two effects. At open time, it stops the open itself from waiting: this matters for FIFOs, named pipes that exist as files in the filesystem, where a normal open blocks until the other end appears. After open, it changes I/O: a read with no data available, or a write to a full buffer, returns -1 with errno EAGAIN instead of sleeping. On Linux, EAGAIN and EWOULDBLOCK are the same value, so checking either works; portable code checks both.
Regular files are the exception people misread. A read from a regular file is served from the page cache when possible, and when the data is not cached the kernel waits for the storage. That wait happens regardless of O_NONBLOCK, and it is uninterruptible. So O_NONBLOCK does not make regular-file I/O instant; it is effectively ignored there. Its historical effect on regular files was tied to mandatory locking, which no longer exists on current kernels. Where the flag earns its keep is pipes, sockets, terminals and device files:
int p[2];
if (pipe(p) == -1) { perror("pipe"); exit(1); }
int fl = fcntl(p[0], F_GETFL);
if (fl == -1 || fcntl(p[0], F_SETFL, fl | O_NONBLOCK) == -1) {
perror("fcntl");
exit(1);
}
char buf[64];
ssize_t n = read(p[0], buf, sizeof buf); /* -1, errno EAGAIN: pipe empty */
pipe() gives you no chance to pass flags, so F_SETFL is the standard route. In nonblocking mode a read or write may also transfer fewer bytes than requested, exactly like the short counts of the previous lesson; your loops must handle both. For FIFOs, open(path, O_RDONLY | O_NONBLOCK) succeeds immediately even with no writer, while open(path, O_WRONLY | O_NONBLOCK) fails with ENXIO if no reader has the FIFO open.
The second topic is size. The offset type off_t is signed, because -1 marks errors. On a 32-bit system a signed long is 32 bits, so offsets stop at 2^31 - 1 bytes, just under 2 GB. Disks outgrew that long ago. The Large File Summit of 1996 defined the fix, and Linux has supported it on 32-bit systems since kernel 2.4.
1compile with 64-bit offsets
Define _FILE_OFFSET_BITS to 64 before any header, or on the command line:
gcc -D_FILE_OFFSET_BITS=64 -o bigtool bigtool.c
The headers then redirect off_t to a 64-bit type and turn open, lseek, stat and friends into their 64-bit variants transparently. Your source does not change. Without it, a 32-bit program that stats a file over 2 GB fails with EOVERFLOW. The older transitional API, open64 and lseek64 with _LARGEFILE64_SOURCE, still exists but is obsolete; do not write new code against it.
2print offsets correctly
With the macro set, off_t can be wider than long, so the usual cast-to-long trick is not enough. Cast to long long and use %lld, as the dup example above already does. Also make sure every module that passes off_t around was compiled with the same setting, or the types will not match across the boundary.
On 64-bit systems, including your Ubuntu host and a 64-bit Raspberry Pi, off_t is already 64 bits and none of this is needed. It matters on 32-bit ARM boards, which are still common in products. Two limits sit below the API anyway: FAT32, often used on SD cards, caps a single file at 4 GiB minus one byte, and NFSv2 caps files at 2 GB regardless of your compile flags.
On a 32-bit Cortex-A board running a glibc Buildroot image, add -D_FILE_OFFSET_BITS=64 to the package's CFLAGS or a logger will silently fail on a 2 GB file. If the image uses musl instead of glibc, off_t is always 64 bits and the macro is a no-op; musl never had a 32-bit off_t. Know which libc your toolchain targets before you debug a size problem.
flock: whole-file locks
5 min readLocks exist because read-modify-write is not atomic. Imagine a file holding a sequence number. Two processes each read 1000, increment, and write back 1001. One increment vanished. The previous lesson's O_APPEND fixes concurrent appends, but it cannot protect a read-then-write cycle. For that, the processes must take turns, and a lock is the kernel's way of arranging the queue.
Unix file locking goes back to around 1980; fcntl a lock on a byte range of a file, requested through fcntl, with read (shared) and write (exclusive) types locking shipped in System V Release 2 in 1984, and flock came from BSD. Both are a lock the kernel records and enforces only between processes that ask for locks; a program that never calls the locking API can still read and write freely. The kernel will not stop a rogue writer. Cooperation is the contract: every program that touches the file must lock first. File permissions are the backstop against programs that do not cooperate.
#include <sys/file.h>
int flock(int fd, int operation);
operation is LOCK_SH (shared, for readers), LOCK_EX (exclusive, for writers), or LOCK_UN (release), optionally OR-ed with LOCK_NB to fail instead of waiting. A conflicting LOCK_NB request returns -1 with errno EWOULDBLOCK. Any number of shared locks coexist; an exclusive lock excludes both kinds. You can convert between shared and exclusive by calling flock again, but the conversion is not atomic: the old lock is dropped first, and with LOCK_NB you can lose your lock entirely if a waiter slips in.
The crucial design point: a flock lock is attached to the open file description, not to the descriptor number and not to the process. Everything follows from that:
- dup'd descriptors refer to the same lock. Unlocking through any of them releases it, and the lock survives until every descriptor of that open file description is closed.
- A second open() of the same file creates a second open file description, treated independently. A process can lock itself out: flock(fd1, LOCK_EX) then flock(fd2, LOCK_EX) on two opens of one file blocks forever.
- fork shares the lock with the child, because the child's fds point at the same open file descriptions. A child can release its parent's lock. This also gives a clean handoff: fork, then the parent closes its copy, and the child owns the lock alone.
- exec preserves the lock, as long as the descriptor survives (close-on-exec can remove it).
- flock has no a stand-off where each process holds a lock the other needs, so neither can ever proceed detection. Two processes waiting on each other's flock locks wait forever.
A classic use is ensuring a single instance of a daemon. The robust pattern locks a pid file, then writes the PID:
1open without truncating
Open the lock file with O_RDWR | O_CREAT, and O_CLOEXEC if the daemon execs helpers. Do not use O_TRUNC here: truncating before you hold the lock would erase the running instance's PID.
2lock, then truncate, then write
int fd = open("/run/mytool.pid", O_RDWR | O_CREAT | O_CLOEXEC, 0644);
if (fd == -1) { perror("open"); exit(1); }
if (flock(fd, LOCK_EX | LOCK_NB) == -1) {
fprintf(stderr, "mytool is already running\n");
exit(1);
}
if (ftruncate(fd, 0) == -1) { perror("ftruncate"); exit(1); }
/* now write getpid() as text into the file */
3hold it for life
Keep the descriptor open until exit. The kernel releases the lock when the process dies, even on a crash or a kill -9, so a stale lock file is never a problem: the file may remain, but the lock is gone. That is the weakness of the alternative, creating a file with O_CREAT | O_EXCL and treating its existence as the lock. The create is atomic, but a crash leaves the file behind and the next start refuses to run until someone deletes it. Prefer the lock over the file's existence.
From the shell, util-linux provides flock. The common idiom locks a descriptor the shell opened:
exec 9>/run/mytool.lock
flock -n 9 || { echo "already running"; exit 1; }
BusyBox includes a flock applet with the same descriptor form, so the pattern survives on a minimal board image.
Over NFS, history matters. flock was originally a local-only BSD mechanism, and many NFS implementations ignored it. Since Linux 2.6.12, the NFS client emulates flock as a whole-file fcntl lock, so it works against Linux servers, but mixing flock users on the server itself with clients can still produce surprises. fcntl record locks are the portable choice on network filesystems, and the next section covers them.
flock and fcntl record locks are independent systems on Linux. A flock lock does not conflict with an fcntl lock on the same file, and vice versa. If half your programs use one and half the other, they will happily corrupt the file together. Choose one per file and document it.
fcntl record locks: byte ranges and their traps
6 min readA record lock is described by a structure:
struct flock {
short l_type; /* F_RDLCK, F_WRLCK, F_UNLCK */
short l_whence; /* SEEK_SET, SEEK_CUR, SEEK_END */
off_t l_start; /* start offset, relative to l_whence */
off_t l_len; /* bytes to lock; 0 means to end of file */
pid_t l_pid; /* filled by F_GETLK: who blocks us */
};
The range works like lseek: l_whence plus l_start picks the first byte, l_len counts the bytes. l_len of 0 means from the start point to the end of the file, however large it grows later, which is how you lock a whole file: SEEK_SET, start 0, len 0. You may lock bytes past the current end; they are reserved for future writes. Since Linux 2.4.21 a negative l_len locks the bytes before the start point. The access mode must match: a read lock needs a descriptor open for reading, a write lock needs one open for writing, or fcntl fails with EBADF. Open O_RDWR when you need both.
Three commands drive it. F_SETLK acquires or releases immediately; a conflict fails with EAGAIN or EACCES, so portable code tests both. F_SETLKW waits until the lock is granted; a signal can interrupt it with EINTR, and SA_RESTART does not restart it, so you can build a timeout with alarm(). F_GETLK asks whether a lock would be granted without taking it: on conflict the kernel fills the structure with one blocking lock's range and l_pid. Treat F_GETLK as information, not a reservation; by the time you call F_SETLK the answer may be stale.
Here is the counter the lab builds, protected by a write lock on the first 8 bytes. The lock covers the whole read-modify-write cycle, which is the entire point:
static int lock_range(int fd, int type)
{
struct flock fl = {
.l_type = type, .l_whence = SEEK_SET,
.l_start = 0, .l_len = 8
};
return fcntl(fd, type == F_UNLCK ? F_SETLK : F_SETLKW, &fl);
}
int bump_counter(int fd)
{
long long value = 0;
if (lock_range(fd, F_WRLCK) == -1)
return -1;
if (pread(fd, &value, sizeof value, 0) != sizeof value)
value = 0; /* empty or short file: start at 0 */
value++;
if (pwrite(fd, &value, sizeof value, 0) != sizeof value) {
lock_range(fd, F_UNLCK);
return -1;
}
return lock_range(fd, F_UNLCK);
}
pread and pwrite keep the shared offset out of the picture, as the previous lesson taught. Unlocking is the same structure with l_type set to F_UNLCK; unlocking a range you do not hold is not an error.
Open once: open the counter file O_RDWR and keep the descriptor for the program's lifetime.
Lock: F_SETLKW with F_WRLCK on bytes 0 to 7; wait if another worker holds it.
Read: pread the 8-byte value at offset 0.
Modify: increment in a local variable.
Write: pwrite the new value back at offset 0.
Unlock: F_SETLK with F_UNLCK on the same range; the next waiter wakes.
Now the ownership rules, which differ sharply from flock. Record locks belong to the process, paired with the inode:
- All threads of a process share one set of record locks.
- fork does not inherit them: the child starts with no locks, even though it inherits the descriptors.
- exec preserves them.
- When the process exits, all its locks are released.
- The sharp edge: closing any descriptor that refers to the file releases all of the process's locks on that file, no matter which descriptor obtained them.
That last rule deserves its own example:
int fd1 = open("data", O_RDWR);
int fd2 = open("data", O_RDWR);
/* acquire a write lock through fd1 ... */
close(fd2); /* the lock obtained via fd1 is now gone */
A library that locks a file cannot protect its lock from a caller that closes some other descriptor of the same file. This is a historical design blemish, now frozen into POSIX, and it is the main reason open file description (OFD) locks exist. Within one process you can never lock yourself out, unlike flock: a second lock request on a held range converts or merges instead of blocking. Locking a different type in the middle of a held range splits it into three; adjacent locks of the same type merge into one.
Two processes can also deadlock: A holds range 1 and waits for range 2 while B holds range 2 and waits for range 1. The kernel detects such cycles for F_SETLKW, even across multiple files and rings of processes, and fails one request with EDEADLK; on Linux it is the most recent requester. Your code must handle EDEADLK, usually by releasing what it holds and retrying in a consistent order. flock offers no such detection. And even without deadlock, a stream of readers can a situation where a waiting process is repeatedly passed over, here a writer kept waiting by an endless series of new read locks a waiting writer indefinitely; Linux makes no FIFO or writer-priority promise for queued lock requests.
If you use fprintf on a locked file, the data may sit in the stdio buffer after you unlock, and the next writer reads stale content. Flush immediately after locking and immediately before unlocking, or do the locked I/O with read and write as the counter example does.
Going deeperHow the kernel stores record locks
Each inode carries a linked list of its locks, ordered by owner and then start offset, managed in fs/locks.c. A new request walks the list to find conflicts, and merges or splits neighbouring locks of the same owner. The cost of a lock operation therefore grows roughly linearly with the number of locks already on that one file, which is why databases lock few, coarse ranges rather than thousands of tiny ones. Linux sets no fixed cap on the number of record locks; ENOLCK appears only when memory runs out. The POSIX lockf() function is a thin wrapper over this machinery: it locks from the current offset, takes only write locks, and uses its own constants. SQLite on Unix is a well-known user of exactly these advisory record locks.
OFD locks, leases and the end of mandatory locking
4 min readLinux 3.15 (2014) added an fcntl byte-range lock owned by the open file description rather than the process, requested with F_OFD_SETLK, F_OFD_SETLKW or F_OFD_GETLK. The API is the record-lock API with new command names:
#define _GNU_SOURCE
#include <fcntl.h>
struct flock fl = {
.l_type = F_WRLCK, .l_whence = SEEK_SET,
.l_start = 0, .l_len = 0, .l_pid = 0 /* l_pid must be 0 */
};
if (fcntl(fd, F_OFD_SETLKW, &fl) == -1) {
perror("fcntl F_OFD_SETLKW");
exit(1);
}
The semantics change in exactly the places that hurt. The lock belongs to the open file description, so closing some other descriptor of the same file does not touch it; it is released only by an explicit unlock or when every descriptor of that open file description is closed. dup'd descriptors and a forked child share the same open file description and therefore the same lock, so either can release it, and the lock survives until all copies are gone. OFD locks conflict with traditional fcntl record locks, so old and new code can cooperate on one file, but they remain independent of flock. For libraries and multi-process services, OFD locks are the right default on any kernel since 3.15; glibc exposes the commands under _GNU_SOURCE.
A a Linux-specific fcntl arrangement that notifies the holder with a signal when another process tries to open or truncate the leased file solves a different problem: knowing when someone is about to disturb a file you cache. fcntl(fd, F_SETLEASE, F_WRLCK) takes a write lease on a file you hold open for writing; F_RDLCK takes a read lease, and F_GETLEASE reports what you hold. When another process opens the file in a conflicting way, or truncates it, the kernel blocks that process and sends the lease holder SIGIO (or the signal chosen with F_SETSIG). A write lease is disturbed by any open or truncate; a read lease only by an open for writing or a truncate. The holder is expected to flush its caches and downgrade to a read lease or release with F_UNLCK. If it does not respond within the lease break time, /proc/sys/fs/lease-break-time, 45 seconds by default, the kernel breaks the lease and lets the opener through. Leases exist to serve Samba's SMB oplocks and NFSv4 delegations, so a board acting as a small NAS uses them whether you write the code or not.
Mandatory locking is the historical third rail. An advisory lock only binds programs that ask; a mandatory lock would make the kernel itself block or fail reads and writes that conflict with a held lock. Linux supported it through a strange door: mount the filesystem with -o mand, then set the set-group-ID bit and clear group-execute on the file (chmod g+s,g-x), a combination ls shows as an S in the group permissions. Conflicting I/O then blocked, or failed with EAGAIN under O_NONBLOCK, and truncate and shared mmap were restricted too.
It never worked well. The implementation had races that let I/O slip past locks, a malicious user could deny service by holding a lock forever since not even root could override it, every I/O call paid a lock-check cost, and unlinking the file remained possible anyway. The kernel removed mandatory locking entirely in Linux 5.15 (2021). On Ubuntu 24.04, whose kernel is 6.8, mount -o mand simply fails. Treat mandatory locking as history you may meet in old books and old code, not as a tool; the modern answer is advisory locks plus correct file permissions.
Going deeperWhy the locking zoo looks like this
flock arrived from BSD with clean open-file-description semantics but whole-file granularity. System V's record locks added byte ranges but tied ownership to the process, producing the close-releases-all rule that POSIX then standardised. Linux carried both, plus leases for network servers, for decades. OFD locks are the modern reconciliation: byte ranges with open-file-description ownership, added without breaking either legacy API. When you read old code, identify which of the three systems it uses before reasoning about fork, exec or close; the inheritance rules are different in each, and mixing them on one file is undefined cooperation at best.
Common mistakes and how professionals debug them
4 min readThe lock that vanishes. Symptom: two writers corrupt a file even though both take fcntl locks. Cause: some code path closes a second descriptor of the same file, dropping all of the process's record locks. Diagnose: watch /proc/locks while the program runs; the lock entry disappears at the moment of the stray close. strace -e trace=fcntl,close shows the sequence. Fix: use OFD locks, or audit every close of that inode, including ones hidden in libraries.
The program that blocks itself. Symptom: a single process hangs in flock with no other process involved. Cause: it opened the file twice and took exclusive locks through both descriptors; flock treats them independently. Diagnose: /proc/locks shows two FLOCK entries with the same PID. Fix: reuse one descriptor, or dup it; never flock two opens of one file in the same process.
The descriptor that leaks past exec. Symptom: a child program holds files it never opened, disk space stays allocated after unlink, or a privileged fd reaches a helper. Cause: missing close-on-exec. Diagnose: ls -l /proc/PID/fd or lsof -p PID on the child shows the inherited descriptors. Fix: O_CLOEXEC at open, dup3 or F_DUPFD_CLOEXEC for duplicates; mark at creation, not afterwards.
The two workers that hang. Symptom: two processes stop making progress, each inside F_SETLKW. Cause: they take ranges in opposite orders, a deadlock; with flock there is not even an EDEADLK to break it. Diagnose: /proc/locks shows granted locks plus blocked requests, marked with an arrow. strace shows each process parked in fcntl. Fix: handle EDEADLK by releasing and retrying, take locks in one global order, and consider alarm()-based timeouts on F_SETLKW.
The counter that loses updates. Symptom: N workers increment M times each, but the final value is below N times M. Cause: the read-modify-write cycle is not fully inside the lock, or stdio flushes after the unlock. Diagnose: run two workers under strace and look for reads or writes outside the locked window. Fix: lock the whole cycle, use pread and pwrite, and flush before unlocking if stdio is involved.
The central tool is /proc/locks. Each line names the lock system (FLOCK, POSIX, OFDLCK or LEASE), ADVISORY, READ or WRITE, the holder's PID, the file as major:minor:inode, and the byte range. Typical output (yours will differ):
1: FLOCK ADVISORY WRITE 4321 08:01:131073 0 EOF
2: POSIX ADVISORY WRITE 4388 08:01:131074 0 7
2: -> POSIX ADVISORY WRITE 4402 08:01:131074 0 7
The arrow marks a blocked request waiting on lock 2. To find the file from the inode number, locate the mount from the device numbers, then find /mountpoint -xdev -inum 131074. To find the holder, ps -p 4388. lsof ties it together from the other direction: lsof /path/to/file lists every process holding it open.
For offsets and flags, /proc/PID/fdinfo/N shows pos, flags in octal, mnt_id and ino. Typical contents for an append-mode log descriptor (yours will differ):
pos: 128
flags: 0102001
mnt_id: 30
ino: 131073
The flags here decode to O_WRONLY with O_APPEND and O_LARGEFILE. If two descriptors you expected to share an offset show different pos values, they are separate open file descriptions, and the bug is usually a second open where a dup belonged.
lsof and strace are often absent from a stripped-down board image, but /proc is always there: /proc/locks, /proc/PID/fd and fdinfo need no packages. BusyBox provides an lsof applet if you enable it in the build. When you can, reproduce the bug on the host with full tools, then verify the fix on the board through /proc.
Try it
1 min readOpen the lab and build the three programs this lesson prepares: a dup2 demo that shows shared versus independent offsets across a fork, a multi-process counter protected by F_SETLKW record locks on an 8-byte range, and a program that leaks an fd past exec before you fix it with O_CLOEXEC. Run the counter with and without the lock and compare the final values, then watch the whole story in /proc/locks and strace. The practice pack has the warm-up, core, challenge and expert exercises, including the flock self-lockout and the close-releases-all trap. The next lesson builds on this with atomic file updates and inotify, and starts the mytools mini project.
Key takeaways
- An fd is an index into a per-process table; the offset and status flags live in the open file description behind it.
- dup, dup2, dup3 and fork make descriptors share one open file description; a second open() of the same file does not.
- fcntl reads and sets status flags with F_GETFL/F_SETFL and the per-descriptor close-on-exec flag with F_GETFD/F_SETFD.
- O_CLOEXEC, dup3 and F_DUPFD_CLOEXEC mark descriptors atomically; open-then-fcntl has a race in threaded programs.
- flock locks whole files and belongs to the open file description; fcntl record locks lock byte ranges and belong to the process.
- Closing any descriptor of a file drops all of that process's fcntl record locks on it; OFD locks fix this.
- File locks are advisory, so every program must cooperate; mandatory locking was removed in Linux 5.15.
- O_NONBLOCK changes pipes, sockets and FIFOs, not regular files; 32-bit systems need _FILE_OFFSET_BITS=64 for files over 2 GB.
Reflect
- Explain to a colleague, in two sentences: explain the per-process fd table, the open file description and the inode table, and predict which descriptors share an offset.
- Which objective are you least sure about? Re-read that section, then do its exercise in the practice pack.
- What surprised you when you ran the commands yourself?
On a real board
Repeat the key commands on your Raspberry Pi or lab board over SSH or the serial console. Note anything that behaves differently from your PC, such as missing tools, a BusyBox shell or a read-only filesystem.
References and further reading
- Man pagefcntl(2), manipulate file descriptor
- Man pageflock(2), apply or remove an advisory lock on an open file
- Man pagedup(2), dup2(2), dup3(2), duplicate a file descriptor
- Man pageopen(2), open and possibly create a file
- Man pageproc_locks(5), held and blocked file locks
- Man pageproc(5), process information pseudo-filesystem (fd and fdinfo)
- Official docsGNU C Library manual, File Locks
Official documentation first. Links checked when the lesson was published.