Module 2 · Lesson 4 of 30

File I/O, short reads/writes, buffering

open, read, write, close and lseek; why short reads and writes happen and how readn/writen fix them; page cache, fsync, stdio buffering and O_DIRECT.

  • Beginner to advanced
  • 26 min read
  • 45 min lab
What you will need

gcc and strace on Ubuntu 24.04 (WSL2, VM or container); a Raspberry Pi or other ARM board for the embedded notes

By the end you can 7 objectives
  • Explain the universal I/O model and the role of the file descriptor and the current file offset shared by read, write and lseek.
  • Open files with the correct flags and mode, and predict the effect of umask, O_EXCL, O_TRUNC and O_APPEND.
  • Write read and write loops that finish the job despite short counts and EINTR, using readn and writen helpers.
  • Use lseek and truncate/ftruncate, create a sparse file, and measure it with du and ls -s.
  • Use pread/pwrite and readv/writev and explain when they are safer or cheaper than lseek plus read and repeated write calls.
  • Explain the page cache, writeback, fsync/fdatasync/sync, and when O_SYNC, O_DSYNC or O_DIRECT make sense.
  • Choose a stdio buffering mode and flush points, and avoid corrupting or losing data when mixing stdio with raw system calls.
Read the lesson below; the video will appear here when published.
01

The universal I/O model

3 min read

The previous lesson showed you the machinery of a system call: the trap, the return value, errno and strace. This lesson puts that machinery to work on the most important job a program does: moving bytes. The interface is five calls that every Unix system since the early 1970s has kept. You open a file to get a , you read and write through it, you move the with lseek, and you release it with close.

The uniformity matters because it is not an accident of naming. The kernel implements read and write for a regular file, a pipe, a terminal and a socket through the same entry points, with each object type providing its own file operations. A program that copies a file with read and write works unchanged on a serial port, because the bytes are the point. This is the universal I/O model, and it is why cp, cat and tail are small programs.

Why an engineer should care comes down to two things. First, correctness: read and write do not guarantee they will move every byte you asked for, and programs that assume they do lose data in the field. Second, performance: the hook from the session is a copy tool that takes minutes to move a half-megabyte file because it reads and writes one byte at a time, and the same file copies instantly once the buffer is 4 KiB. The difference is the number of system calls, which you learned last lesson are expensive. A 512 KiB file copied one byte at a time makes about a million reads and a million writes; at 4 KiB it makes 128 of each.

The universal I/O modelProgramscat, cp, your codeFive callsopen read write close lseekVFSone interface, many objectsPage cacheRAM copy of file dataDevice driverDisk or flashrequests go down, data comes upProgramscat, cp, your codeFive callsopen read write close lseekVFSone interface, many objectsPage cacheRAM copy of file dataDevice driverDisk or flashrequests go down, data comes up
The five calls between a program and the storage underneath

The rest of the lesson climbs the stack this picture shows: the calls themselves, their edge cases, then the two buffers that sit between your program and the physical disk, and how to control them.

02

Opening a file with open()

4 min read
c
#include <fcntl.h>

int fd = open("/home/learner/notes.txt", O_RDWR | O_CREAT, 0644);
if (fd == -1) { perror("open"); /* handle error */ }

The prototype is int open(const char *pathname, int flags, mode_t mode), where mode is read only when a file is created. The flags fall into three groups.

1the access mode

Exactly one of the first three is required: O_RDONLY, O_WRONLY or O_RDWR. On Linux their values are 0, 1 and 2, and the kernel extracts the access mode with the mask O_ACCMODE (3); do not rely on the numeric values, which vary across systems. A file opened write-only cannot be read, and read on it fails with EBADF.

2the creation and truncation flags

O_CREAT creates the file if it does not exist; combined with O_EXCL it fails with EEXIST if the file already exists, and the check-and-create is atomic, which makes it the standard way to build a lock file. O_TRUNC sets an existing file's length to zero when it is opened for writing, so a write-only open followed by writes replaces the old contents. O_APPEND does something subtle: before every write, the kernel moves the offset to the current end of the file, and the move plus the write are done atomically, so several processes appending to a log cannot overwrite each other's lines. Without O_APPEND, two processes that both seek to the end and then write race.

3the status flags

O_CLOEXEC sets the close-on-exec flag atomically as part of open, so the descriptor does not leak into a program started later with exec; the next lesson covers the fd table and why this matters. O_NONBLOCK makes operations on pipes, sockets and devices return immediately with EAGAIN instead of waiting; it has no effect on regular files. O_SYNC makes every write synchronous, which is covered with the page cache later.

0x42 = O_RDWR | O_CREAT from the open() above: the low two bits (value 2) select read-write access, and the Flags field (value 1) marks O_CREAT. The kernel reads the access mode with flags & O_ACCMODE (3); the upper bits are independent flag bits.

  • fields: [{"name": "Access mode", "lsb": 0, "width": 2}, {"name": "Flags", "lsb": 6, "width": 5}]
  • value: 0x42
  • width: 16

The mode argument is subject to the process . The kernel computes the final permissions as mode & ~umask. A shell login sets umask to 022, so 0644 requested means 0644, and 0666 requested means 0644. See yours with umask, and change it with umask 002. Only newly created files are affected; the mode argument is ignored when the file already exists.

0666 = -rw-rw-rw-: everyone can read and write. With the usual umask 022, open() creates the file as 0666 & ~022 = 0644 = -rw-r--r--, clearing group and other write. Toggle the rwx bits to see how each position maps to its octal digit.

  • octal: 0666
Where openat fits

glibc implements open() with the openat() system call, which takes a directory file descriptor: openat(dirfd, pathname, flags, mode). A relative pathname is resolved from dirfd rather than the current working directory, and the constant AT_FDCWD means "the real current directory". This lets a program open files relative to a directory it already holds open, and avoids a race where the pathname is resolved against one directory and used against another. The next lessons use openat through strace constantly.

open() with its flagsopen(path,flags,mode)ResolvepathAccesscheckO_CREATcreate ifmissingO_TRUNClength tozeroO_APPENDwrites atendreturnlowest freefdopen(path,flags,mode)ResolvepathAccesscheckO_CREATcreate ifmissingO_TRUNClength tozeroO_APPENDwrites atendreturnlowest freefd
What open() does with its flags on the way to a file descriptor
Going deeperopenat and the time-of-check to time-of-use race

Before the *at family existed, a program that wanted to open a file inside a specific directory had to build an absolute path and open it. Between building the path and opening it, an attacker could rename the directory, so the program opened a file somewhere else. Holding the directory open and calling openat(dirfd, "config", ...) pins the directory, so the lookup cannot be redirected by a rename in between. The same idea recurs across the API: fchmodat, renameat, unlinkat, and the fstatat family all take a dirfd. These calls, and O_NOFOLLOW and O_PATH, come from POSIX.1-2008; openat arrived in Linux 2.6.16.

03

Reading, writing and the short count

3 min read
c
ssize_t read(int fd, void *buf, size_t count);
ssize_t write(int fd, const void *buf, size_t count);

read returns the number of bytes placed in buf. It is fewer than count in exactly the situations that trip people up: a pipe or socket may hold fewer bytes than you asked for, a terminal in canonical mode returns up to one line, and a signal can interrupt the call. write is symmetric: a pipe or socket buffer may have room for fewer bytes than count, and the call returns the number accepted. A is not an error and not a reason to report failure.

The three-way return from the last lesson applies directly. -1 is an error, with errno naming it, and EINTR is special: the call was interrupted before anything was moved, so retry it. 0 from read means end of file, the writer of a pipe closed it, or a peer closed a socket. A positive number is progress, possibly less than count. A subtle point: a 0 from write is not an end marker; write returns 0 only when count is 0, so a loop that treats 0 as "done" would loop forever on a short write only if you wrote it wrong.

Why short counts are normal, not exceptional, is worth pinning down. A regular file usually returns the full count in one read because the data is already in the and there is no natural boundary, but the guarantee is not there: a read can be short at end of file, and a signal can interrupt before the data arrives. Pipes, sockets, terminals and character devices return whatever is available now. The rule to internalise is that read returns what it can, not what you want.

A short read from a pipeProgramKernelPiperead(fd, buf, 8192)how much data?3 bytes availablereturns 3 (short)1. Program → Kernelread(fd, buf, 8192)2. Kernel → Pipehow much data?3. Pipe → Kernel3 bytes available4. Kernel → Programreturns 3 (short)
A read that asks for 8192 bytes and receives three from a pipe
Going deeperWhy a local disk read is almost never interrupted

A signal only interrupts a call while the process sleeps waiting for data. A read from a regular file is normally served from the page cache in one step, so there is no sleep to interrupt. When a regular file read does miss the cache and wait for disk, that wait is uninterruptible: the process sits in state D in ps, and a signal cannot cancel it. This is why a read on a stuck NFS mount can leave a process unkillable. Pipe, socket and terminal reads wait in an interruptible state, so they are the ones that return EINTR. The asymmetry is why retry loops matter most for non-file descriptors.

04

readn and writen: loops that finish

2 min read

The two helpers below are the workhorses of real code. You will find equivalents named readn, writen, read_full or write_all in every serious codebase, from network daemons to embedded loggers, because the raw calls refuse to promise completeness.

1readn

c
ssize_t readn(int fd, void *buf, size_t count)
{
    char *p = buf;
    size_t left = count;

    while (left > 0) {
        ssize_t n = read(fd, p, left);
        if (n == -1) {
            if (errno == EINTR)
                continue;          /* interrupted: try again */
            return -1;             /* a real error */
        }
        if (n == 0)
            break;                 /* end of file before count was filled */
        p += n;
        left -= n;
    }
    return count - left;           /* bytes actually read, may be < count */
}

readn returns the number of bytes read, which is less than count only at end of file. A caller that needs exactly count bytes must compare the result and treat a short return as end of input. The EINTR branch is the loop the previous lesson taught in miniature: copy errno, do nothing else, continue.

2writen

c
ssize_t writen(int fd, const void *buf, size_t count)
{
    const char *p = buf;
    size_t left = count;

    while (left > 0) {
        ssize_t n = write(fd, p, left);
        if (n == -1) {
            if (errno == EINTR)
                continue;
            return -1;
        }
        p += n;                    /* write never returns 0 while left > 0 */
        left -= n;
    }
    return count;
}

writen returns count on success or -1 on failure; there is no short-success case because write does not return 0 for a positive count. The loop absorbs a short write on a full pipe by calling again for the rest.

Step by stepreadn: one pass around the loop
  1. Call read: ask for the bytes still outstanding, starting at the current pointer.

  2. EINTR check: if the result is -1 and errno is EINTR, the call was interrupted, so loop without consuming anything.

  3. EOF check: if the result is 0, the file is exhausted, so break and report the bytes accumulated so far.

  4. Progress: a positive result advances the pointer and shrinks the remaining count.

  5. Done or loop: when the remaining count reaches zero, return the total; otherwise call read again.

The lab for this lesson drives these loops hard: a producer writes a pipe one byte at a time while a consumer readn's a large buffer, forcing many short reads, and a second exercise fills a pipe until write is short, then lets writen finish the job.

The readn loopcallread(fd, p,left)-1 andEINTR?retry0?end of filepositiveadvance p,shrink leftleft is 0?buffer fullreturntotal↻ Repeat from step 1callread(fd, p,left)-1 andEINTR?retry0?end of filepositiveadvance p,shrink leftleft is 0?buffer fullreturntotal↻ Repeat from step 1
The readn loop: retry on EINTR, stop at end of file, accumulate progress
05

Seeking, holes and truncation

2 min read
c
off_t lseek(int fd, off_t offset, int whence);

whence is SEEK_SET (from the start), SEEK_CUR (from the current offset) or SEEK_END (from the end), and lseek returns the new offset, so lseek(fd, 0, SEEK_END) returns the file size. It fails with ESPIPE on objects with no offset, such as pipes and sockets. The offset itself lives in the kernel's open-file description, which is why read, write and lseek all see the same position, and why two descriptors obtained by dup share one offset. That sharing is the subject of the next lesson.

A is the basis of a . Create one by seeking far ahead and writing a single byte:

c
#include <fcntl.h>
#include <unistd.h>

int main(void)
{
    int fd = open("holes.bin", O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (write(fd, "A", 1) == -1) { perror("write"); return 1; }
    if (lseek(fd, 1 << 20, SEEK_SET) == -1) { perror("lseek"); return 1; }
    if (write(fd, "B", 1) == -1) { perror("write"); return 1; }
    close(fd);
    return 0;
}

The file now has a logical size of 1048577 bytes but only two blocks of data. Reading the hole returns zeros without touching the disk. Filesystems such as ext4 store holes implicitly, which is why a virtual-machine disk image that was created with a tool that seeks past the end can look huge in ls yet occupy little storage until the guest actually writes into the hole. The reverse hazard matters on flash: a sparse file can stop being sparse as a database rewrites it in place, silently consuming the device.

du versus ls -s

ls -l reports the logical size in bytes. ls -s and du report allocated blocks. For holes.bin above, ls -l shows 1048577 while du shows a few kilobytes. Block sizes vary by filesystem and tool, so treat the exact numbers as "small" rather than precise.

Two more calls round out the position story. truncate(path, length) and ftruncate(fd, length) set the size directly: shrinking discards data beyond the new length, growing adds a hole of zeros. Databases use ftruncate to extend a file quickly, then fill the extension with writes.

Sparse file: logical size versus blocksls -l (bytes)du / ls -s(blocks)1-byte file1one block1 MiB hole + 1byte1048577two blocks
A sparse file: full logical size, almost no allocated blocks
06

pread, pwrite and scatter-gather I/O

3 min read
c
ssize_t pread(int fd, void *buf, size_t count, off_t offset);
ssize_t pwrite(int fd, const void *buf, size_t count, off_t offset);

pread and pwrite are exactly read and write except for the offset argument. The read or write happens at that offset, the shared file offset is left untouched, and the position plus the transfer are one atomic operation, so two threads in one process can read different records of a database without a race on the shared offset. The fd must be seekable, or the calls fail with ESPIPE. The atomicity is the point: an equivalent lseek then read would be two calls with a window in which another thread could move the offset.

Going deeperpwrite and O_APPEND: a known Linux deviation

POSIX says pwrite should write at the offset you pass regardless of O_APPEND. Linux historically ignored that: if the file was opened with O_APPEND, pwrite appended to the end instead, and the man page still documents this under BUGS. If your code opens with O_APPEND and calls pwrite with an explicit offset, test what actually happens on your kernel before relying on it. The safe reading of the standard is to use pwrite only on files opened without O_APPEND.

readv and writev extend the idea in another direction, scatter-gather I/O. Instead of one buffer you pass an array of structures:

c
#include <sys/uio.h>
#include <unistd.h>

int main(void)
{
    struct iovec iov[3];
    char a[] = "Hello", b[] = ", ", c[] = "world\n";
    iov[0] = (struct iovec){ a, 5 };
    iov[1] = (struct iovec){ b, 2 };
    iov[2] = (struct iovec){ c, 6 };
    return writev(1, iov, 3) == 13 ? 0 : 1;   /* one syscall, three buffers */
}

writev makes one system call and writes the buffers in order, returning the total bytes written, which may again be short. readv fills the buffers in order. The wins are fewer system calls when a message header and a payload live in separate buffers, and, on pipes, atomicity: a writev whose total length is at most PIPE_BUF (4096 on Linux) is written in one piece, so it cannot interleave with another writer's data the way two separate write calls could.

struct iovec array for writeviov[0]base:buf1len: 5iov[1]base:buf2len: 2iov[2]base:buf3len: 6one writev(fd, iov, 3) writes 13 bytes inone syscall
An iovec array describing three buffers for a single writev call
07

The page cache, writeback and fsync

5 min read

When you read a file, the kernel loads its data into the page cache, and later reads are served from it. When you write, the kernel copies your bytes into cache pages and marks them , then returns before the disk has seen anything. This is called write-back caching, and it is why writing is fast and why a power cut can lose data the program believed was saved.

A separate kernel mechanism, , runs later and writes dirty pages to disk. Since 2.6.32 this work is done by per-device flusher threads, and since 3.10 it runs in per-device workqueues, so you may see kworker threads doing the writes. The amount of waiting data is visible in /proc/meminfo as the Dirty and Writeback fields, in kilobytes. The kernel starts writing in the background when dirty memory passes vm.dirty_background_ratio and throttles writers at vm.dirty_ratio; vm.dirty_expire_centisecs and vm.dirty_writeback_centisecs control how old a page may get and how often the flush runs.

Step by stepLife of one buffered write
  1. Copy in: write() copies the bytes from user memory into a page-cache page.

  2. Mark dirty: the kernel records that the page no longer matches its copy on storage.

  3. Return: write() returns; the data exists in RAM but not yet on disk.

  4. Flush: a writeback thread picks the dirty page once it is old enough or memory pressure builds.

  5. Block I/O: the block layer issues a write to the device and the page becomes clean.

  6. Durable: the data is on the medium and will survive a power loss, barring device-cache lies.

To move data to disk on purpose you call the durability calls. fsync(fd) flushes the file's data and its metadata (size, timestamps) and returns only when the device has acknowledged the write. fdatasync(fd) flushes the data plus only the metadata needed to read it back, skipping timestamps, which is enough and cheaper for a database that just grew. sync() flushes everything on every filesystem, and syncfs(fd) flushes one filesystem. The per-write flags do the same work on every call: O_SYNC on open makes each write behave like write followed by fsync, and O_DSYNC like write followed by fdatasync. This is the write-back versus write-through trade-off: ordinary write-back caching makes writes cheap but leaves a window in which a crash loses them, while O_SYNC pays a full device round trip on every write to close that window, which is why it is reserved for the few writes that must survive a crash.

c
#include <fcntl.h>
#include <unistd.h>

int main(void)
{
    int fd = open("ledger.log", O_WRONLY | O_CREAT | O_APPEND, 0644);
    if (write(fd, "commit\n", 7) == -1) { perror("write"); return 1; }
    if (fsync(fd) == -1) { perror("fsync"); return 1; }
    close(fd);
    return 0;
}
Durability on flash

On an embedded board with eMMC or NAND, fsync matters twice. Power can vanish at any moment, so a logger that does not fsync can lose the last writes after an unclean shutdown. And flash wears out on write: O_SYNC on every byte turns a burst of small writes into a burst of full erase-block rewrites, accelerating wear. The usual design is to buffer in the page cache and fsync at a commit boundary, a rate the wear budget can afford.

You can advise the kernel about your access pattern with posix_fadvise(fd, offset, len, advice). POSIX_FADV_SEQUENTIAL tells it you will read front to back, so it doubles the readahead and frees pages behind you; POSIX_FADV_RANDOM disables readahead for a lookup-heavy workload; POSIX_FADV_WILLNEED asks it to readahead now; POSIX_FADV_DONTNEED drops the cached pages for a range, useful after a backup so the cache does not evict working data. Advice is not a contract, but the kernel does honour these on the page cache. madvise does the analogous job for memory mappings, with MADV_SEQUENTIAL, MADV_RANDOM and MADV_DONTNEED, which you will revisit with mmap in a later module.

Life of a buffered writewrite()copies intopage cachepage markeddirtywrite()returnswritebackthreadlaterblock I/Oto devicepage cleanwrite()copies intopage cachepage markeddirtywrite()returnswritebackthreadlaterblock I/Oto devicepage clean
A buffered write: into the page cache now, onto the device later
Going deeperReading the kernel's dirty accounting

The writeback policy lives in the sysctls under /proc/sys/vm/ and the counters in /proc/meminfo. A process that grows Dirty quickly while Writeback stays high is producing data faster than the device can flush it, a sign the storage is the bottleneck. The kernel source of the copying path is mm/filemap.c for reads and mm/page-writeback.c plus fs/fs-writeback.c for the writeback machinery. The address_space of each file tracks its pages and their dirty state, which is why a file and its page cache are one object from the kernel's point of view.

08

close(): failures that arrive late

2 min read

Most programs ignore the return value of close, and for a local filesystem that is usually harmless, but the call can fail, and on network filesystems it is where a failed write finally becomes visible. With write-back caching, write() returns before the device has seen the data, and nothing may have reported a problem by the time the program closes the file. The flush that close triggers can then fail, and close returns -1 with an errno such as EIO or ENOSPC. The rule is: check the return value of close when you cared that the data landed, the same way you check fsync.

The classic place this bites is NFS. A client writes to a file on an NFS server, write() succeeds because the data sits in the client's cache, then the server goes down before the flush. The failure surfaces at close() or fsync(), long after the code that "wrote" the data has moved on. Programs that copy files across NFS and only check write() will report success and then leave a truncated file on the server.

Two further traps. First, on Linux close() releases the file descriptor even when it returns an error, because the descriptor is already gone by the time the error is discovered. Retrying close(fd) is therefore wrong: the number may have been reused by another thread's open, and your retry would close someone else's file. Report the error and drop the descriptor. Second, calling close on an already-closed descriptor, or on a number that was never open, fails with EBADF, and a double close is a serious bug precisely because the number may now belong to a different file, so the second close silently closes the wrong thing. Initialise descriptors you intend to reuse, and null out pointers to them after closing.

A delayed error surfacing at closeProgramKernelNFS serverwrite(fd, data)returns n (cached)close(fd)flush dirty pagesserver errorclose() = -1 EIO1. Program → Kernelwrite(fd, data)2. Kernel → Programreturns n (cached)3. Program → Kernelclose(fd)4. Kernel → NFS serverflush dirty pages5. NFS server → Kernelserver error6. Kernel → Programclose() = -1 EIO
A write that reported success but whose failure surfaces at close over NFS
Never retry close

If close(fd) returns -1, do not call it again with the same fd. The descriptor is released regardless on Linux. A retry can close a descriptor that another thread has since obtained and corrupt an unrelated operation.

09

stdio buffering and mixing with system calls

3 min read

There are two buffers between your bytes and the disk, and this is the second one. printf and fwrite do not call write() for every character; they accumulate output in a user-space and flush it later. stderr is unbuffered by default, standard output is line-buffered when it is a terminal and fully buffered when it is a file or pipe, and files opened with fopen are fully buffered. That is why a program that prints a prompt without a newline can leave the user staring at nothing until the buffer fills or the program exits.

You control the policy with setvbuf. _IOFBF is full buffering, _IOLBF line buffering (flush on newline), and _IONBF no buffering; the size argument and a buffer you supply, or NULL to let the library allocate, complete the call.

c
#include <stdio.h>

int main(void)
{
    setvbuf(stdout, NULL, _IONBF, 0);   /* every putchar reaches write() now */
    putchar('x');
    return 0;
}

fflush(stream) forces the buffer to the kernel immediately, and fflush(NULL) flushes every open output stream. Flushing moves data from the stdio buffer into the page cache; it does not make it durable, for which you still need fsync. The default buffer size is BUFSIZ, which glibc defines as 8192, and a larger buffer amortises more calls.

The buffer is exactly where mixing stdio and raw system calls goes wrong. If you fwrite to a FILE and then read the underlying descriptor with read(), the read sees stale data because the fwrite data is still in the stdio buffer, not yet in the kernel. Flush first. If you fork a process with buffered output pending, the child inherits a copy of the buffer, and both parent and child later flush the same bytes, duplicating output; the fix is fflush(NULL) before fork. If you dup or redirect a descriptor, flush before the redirection so ordering is preserved. The rule that pairs with the lab: before any operation that bypasses stdio, fflush.

write() versus fwrite()write()fwrite() /printf()Buffernoneuser-space,BUFSIZSyscallsone per callone perbuffer fullReaches kernelimmediatelyat fflush
write() versus fwrite(): where the extra user-space buffer sits

stdio is also thread-safe, and that safety has a cost. Because a FILE may be shared by threads, glibc protects each fread, fwrite and printf with an internal lock per stream, so concurrent calls do not interleave. That lock is real work, and a tight loop of many small fprintf calls to one stream pays it on every call. The functions flockfile(stream) and funlockfile(stream) take and release the lock manually, and between them you may use the unlocked variants, fwrite_unlocked, fputs_unlocked and getc_unlocked, which skip the locking because you already hold it. A logging hot path can drop a measurable fraction of its CPU this way. The variants are POSIX, not ISO C, and the code is only correct while it holds the lock.

10

Common mistakes and how to diagnose them

3 min read

Ignoring short reads and writes. Symptom: a file transfer loses data, or a log line is truncated, and it works in the lab but fails in production. Cause: the code checks only for -1 and treats any positive count as complete, so a short read on a pipe or socket silently drops the rest, or a short write leaves the tail unwritten. Diagnose: strace shows a read returning fewer bytes than requested, followed by the program moving on. Fix: use readn and writen, or loop explicitly, and for write treat any count less than requested as "write the rest".

Trusting write() to be durable. Symptom: an embedded device loses the last few records after power is cut, or a database corrupts after a crash. Cause: write() returned but the data was only in the page cache. Diagnose: the loss is intermittent and correlates with unclean shutdown; /proc/meminfo shows Dirty pages that never got flushed. Fix: fsync or fdatasync at commit boundaries, or open with O_SYNC where every write must land, and weigh the flash-wear cost on an embedded board.

Reading stale data after stdio writes. Symptom: code that writes with fprintf and reads with read() sees old or missing data, or duplicated output appears after fork. Cause: the stdio buffer was not flushed before the raw operation. Diagnose: strace shows the write() call happening only at exit, not where the program expected it. Fix: fflush before read, dup or fork, and fflush(NULL) before fork in any program that mixes stdio with process creation.

Misaligned O_DIRECT. Symptom: a program that adds O_DIRECT for speed starts failing with EINVAL. Cause: direct I/O bypasses the page cache, so the buffer address, the transfer length and the file offset must each be a multiple of the logical block size of the underlying device, commonly 512 bytes and frequently 4096, and malloc's alignment is not enough. Diagnose: the EINVAL appears only with O_DIRECT and disappears without it. Fix: allocate with posix_memalign or an aligned buffer, round lengths and offsets to the block size, and confirm the numbers with blockdev --getss or the BLKSSZGET ioctl. Direct I/O trades away the cache, so it is right only for a workload that already manages its own cache, such as a database.

c
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
    void *buf;
    if (posix_memalign(&buf, 4096, 4096) != 0) return 1;   /* aligned address */
    int fd = open("big.bin", O_RDONLY | O_DIRECT);
    ssize_t n = pread(fd, buf, 4096, 0);                    /* aligned offset and length */
    free(buf);
    close(fd);
    return n < 0;
}
From symptom to fixData lostafter crashfsync()missing?only inpage cacheread() seesstale datafflush()missing?only instdiobufferfix: fflush+ fsyncData lostafter crashfsync()missing?only inpage cacheread() seesstale datafflush()missing?only instdiobufferfix: fflush+ fsync
From symptom to cause to fix for the common buffering failures
11

Try it

1 min read

Open the lab for this lesson and build readn and writen, then drive them against a producer that writes a pipe one byte at a time to force short reads, and a full pipe to force short writes. Benchmark a copy loop across buffer sizes from one byte up to 512 KiB, with and without fsync, and watch the syscall count in strace -c explain the wall time. Then create a sparse file with lseek, compare ls -l against du, and write a small program that mixes fprintf with read() to reproduce the stale-data bug before fixing it with fflush. The practice pack has the warm-up, core, challenge and expert exercises for this lesson.

Key takeaways

  • The same five calls, open, read, write, close and lseek, drive every file, device, pipe and socket.
  • read and write may transfer fewer bytes than you asked for; only a zero from read means end of file, and a short write still needs another call.
  • EINTR means try again, not fail; readn and writen hide both interruption and short counts behind one helper.
  • The current file offset is shared by read, write and lseek, while pread and pwrite use their own offset atomically.
  • A write returns once the data is in the page cache, not on disk; fsync and fdatasync are how you make it durable.
  • stdio adds a user-space buffer on top of the page cache, and fflush is the bridge between the two.
  • A sparse file stores holes as no blocks at all, so du is tiny while ls -l shows the full logical size.
  • O_DIRECT bypasses the page cache but demands aligned buffers and lengths; posix_fadvise only gives advice.

Reflect

  1. Explain to a colleague, in two sentences: explain the universal I/O model and the role of the file descriptor and the current file offset shared by read, write and lseek.
  2. Which objective are you least sure about? Re-read that section, then do its exercise in the practice pack.
  3. What surprised you when you ran the commands yourself?

On a real board

Repeat the key commands on your Raspberry Pi or lab board over SSH or the serial console. Note anything that behaves differently from your PC, such as missing tools, a BusyBox shell or a read-only filesystem.

Practise this lessonExercises, quiz, interview preparation and spaced review

References and further reading

Official documentation first. Links checked when the lesson was published.

Find your next step

↑ ↓ to move · Enter to open · Esc to close