Why Python for embedded tooling
4 min readEmbedded work produces a steady stream of small programs. You write a script to flash firmware, a parser to read a boot log over serial, a harness to exercise a driver before it ships, a wrapper around a vendor's build tool. For years these were written in Bash or C. Bash is fine for a three-command pipe but becomes unreadable as soon as logic and data structures appear. C is precise and fast but slow to write and unforgiving: one mistake and the program crashes or corrupts memory. Python is the middle path that has taken over the host side of embedded engineering.
To understand what Python is, you need the word a program that reads and executes source code directly, without a separate compile and link step. CPython, the reference implementation, is not a pure interpreter in the strictest sense. When you run a script it first compiles your source into a compact, portable instruction set that a virtual machine executes, one step removed from the source text, then a virtual machine executes that bytecode. What you notice in practice is the difference that matters: there is no compile, no link, and no reflash cycle between an edit and a result.
That short loop is most visible in the read-eval-print loop, the interactive prompt that reads one expression, evaluates it and prints the result. Type python3 with no arguments and you get the >>> prompt. Type 2 + 3, press Enter, and 5 appears immediately. For an engineer the REPL is a scratchpad for checking a regex, converting a hex value, or testing the bytes coming off a serial port before committing them to a script.
The trade is speed and predictability. Python is slower than C because every operation goes through the bytecode loop, and it cannot guarantee latency because a garbage collector and the global interpreter lock can pause execution at any moment. That is fine for tooling that runs on a workstation and bad for a hard real-time control loop running on the device itself. The division of labour is clean: C runs on the target, Python builds, flashes, tests and watches it from the host.
A 64 KB microcontroller cannot run CPython. Python's role in this program is on the development machine and the test rig: the build system, the flasher, the log parser, the CI job. When a lesson talks about running Python on a Raspberry Pi, the Pi is a full Linux computer, not a bare-metal target. (MicroPython and CircuitPython do run on larger microcontrollers, but they are a different runtime and are not covered here.)
Read the source: Python opens the .py file and reads the text.
Tokenise and parse: the parser turns the text into an abstract syntax tree, checking that the grammar is valid.
Compile to bytecode: the compiler lowers the tree into bytecode instructions for the Python virtual machine.
Cache it: for an imported module, CPython writes the bytecode to a .pyc file under pycache to skip recompiling next time.
Execute: the virtual machine steps through the bytecode, one instruction at a time, until the program ends.
A short history, and where Python runs in embedded products
4 min readGuido van Rossum began Python in December 1989 at the Centrum Wiskunde and Informatica in the Netherlands, and released the first public version, 0.9.0, in February 1991. He named it after the Monty Python comedy group, not the snake, and its early design goal was to be a scripting language that was easy to read and quick to learn. Version 1.0 followed in January 1994, adding lambda, map and filter. Version 2.0 arrived in October 2000 with list comprehensions, a cycle-detecting garbage collector, and Unicode support bolted on as a separate string type.
The turning point for this lesson is Python 3.0, released in December 2008. It was a deliberate break in compatibility, made to fix design flaws that could not be repaired while staying backward compatible. The most important change for you: text and bytes were split into two types. In Python 2, the str type held bytes and a separate unicode type held text, and code mixed them constantly. In Python 3, str holds Unicode text and bytes holds raw octets, and they never convert implicitly. Two other visible changes came with it: print became a function, and / became true division while // does floor division.
The transition took years because Python 2 kept running in parallel. Python 2.7, the last of the line, reached end of life on 1 January 2020, after which it received no more security fixes. That date matters when you meet an old tool that still says "Python 2". Everything new is Python 3, and the current 3.x line has been the only supported one since 2020.
This program uses Python 3.11 as its minimum. Later lessons use the match statement and the X | Y type-union syntax, both added in 3.10, and tomllib, added to the standard library in 3.11. Python 3.9 and 3.10 have also reached end of life (October 2025 and October 2026), so they no longer get security fixes. Ubuntu 24.04 ships 3.12, which covers everything.
The dates that matter for tooling:
Python's embedded footprint is on the host, not the device, and it is large. The a set of tools that turns source code and configuration into a bootable image or a set of binaries behind the Yocto Project and OpenEmbedded is BitBake, which is written in Python and whose recipes are a Python-based language. Several of Buildroot's helper scripts (its core is Make and Kconfig), the Zephyr project's west tool, the Yocto devtool, and kas, a wrapper for BitBake projects, are all Python. On the test side, pytest drives hardware-in-the-loop suites, labgrid manages boards in a lab, and pyserial reads and writes UARTs for log capture and firmware flashing. When you script a Yocto build or parse a boot log in a later module, you are using exactly the Python you learn here.
Installing and selecting Python
2 min readOn Linux the interpreter has two names, and they are not interchangeable. python3 is always the Python 3 interpreter. python is a legacy name that on some distributions points to Python 2 or to nothing at all. Ubuntu 24.04 ships only python3 by default. Make it a habit to type python3 in every command and script, so a machine that lacks the python alias does not surprise you.
Check what you have before you install anything:
python3 --version # typical output: Python 3.12.x (yours will differ)
which python3 # typical output: /usr/bin/python3
python3 -c "import sys; print(sys.version_info)"
There are four common ways to get Python, and they serve different needs.
apt(sudo apt install python3 python3-venv python3-pip) installs the distribution's own build. It is the lowest-friction option, it is updated by the system, and on Ubuntu 24.04 it is Python 3.12. The catch is that you get exactly the version the distribution chose, and you cannot easily switch it.python.orgoffers official binary installers and source tarballs. Building from source or installing a tarball gives you a specific, known version, but you manage upgrades yourself.pyenvinstalls several Python versions side by side in your home directory and selects one per directory or per shell. It is the standard tool when you must test code against a matrix of versions.uvis a fast, modern installer and package manager written in Rust. Itsuv python install 3.12fetches a pinned interpreter anduv venvcreates environments in milliseconds; it is replacing the older pip-and-virtualenv workflow in many teams.
For this program, apt on Ubuntu 24.04 is the simplest correct choice: it gives Python 3.12, which is above the 3.11 floor that the later lessons need. Whichever route you choose, record the version and use the same one in CI and with your team. Reproducibility comes from a pinned toolchain, and the pin is a one-line fact, not a guess.
Virtual environments and pip
3 min readThe system Python belongs to the operating system. Ubuntu's own tools are written in it, and installing your packages into it by hand can upgrade or remove something the system needs. Two of your projects can also conflict with each other: project A needs requests 2.31 and project B needs 2.28, and both want to put files in the same place. A a self-contained directory with its own copy of the interpreter, pip and a private site-packages, so installed packages stay inside the project solves both problems.
Create one per project, inside the project folder:
python3 -m venv .venv
This creates a directory called .venv (the name is convention, and the leading dot hides it from ls). The -m venv form runs the venv module, which is more reliable than the old virtualenv command because it uses the exact interpreter you invoked. On Ubuntu you may first need sudo apt install python3-venv python3-pip if the module is missing.
1create the environment
Run python3 -m venv .venv. Recent Python versions print nothing on success; the directory simply appears.
2activate it
source .venv/bin/activate # POSIX shells; on Windows use .venv\Scripts\activate
Activation does two small things: it prepends .venv/bin to your PATH and sets the VIRTUAL_ENV variable, and it changes your prompt to show the environment name. From now on python and pip resolve to the copies inside .venv, not the system ones. deactivate undoes it.
3install and pin
pip install requests # install a package into this environment only
pip freeze > requirements.txt # write the exact versions, one per line
pip install -r requirements.txt # recreate the same set elsewhere
pip uninstall requests # remove a package
pip freeze writes lines of the form requests==2.32.3, which is a pin: an exact version, so a teammate or a CI runner gets the same set you tested. Committing requirements.txt and recreating from it is how a project stays reproducible. The .venv directory itself must never be committed to git; it is rebuilt from the requirements file.
On Ubuntu 24.04, running pip install against the system interpreter fails with an "externally managed environment" error (PEP 668). That is deliberate: the distribution wants you to install into a venv, not into the OS copy. The error is not a bug. Create a venv, or use --break-system-packages only when you understand the risk.
Core scalar types, truthiness, and f-strings
4 min readThe integer type int is arbitrary precision. Unlike C's int, which is a fixed 32 or 64 bits, a Python int grows to hold the value you store, so it never overflows. You can write 2 ** 100 and get the exact 31-digit answer. Literals support the bases an engineer already thinks in: 42, 0x2A, 0b101010, and underscores for readability, 1_000_000.
Try the same value in every base below:
The floating-point type float is an IEEE-754 binary64 value, the same as a C double. It holds about 15 to 17 significant decimal digits, and most decimal fractions are not exact in it. The classic demonstration is 0.1 + 0.2, which evaluates to 0.30000000000000004, not 0.3. Treat floats as approximations; use int for exact counts and money-like values.
The boolean type bool has exactly two values, True and False, and it is a subclass of int: True == 1 and False == 0 both hold. Text is str, an immutable sequence of Unicode code points, and bytes is a separate type for raw octets that you will meet properly later in this lesson. None is the single value of its own type and means "no value here".
To discover or check a type you have two tools. type(x) returns the exact type, and isinstance(x, t) returns True if x is of type t or a subclass of it. The difference shows with booleans: type(True) is bool is true, but isinstance(True, int) is also true because bool subclasses int.
>>> type(42)
<class 'int'>
>>> isinstance(True, int)
True
>>> 0.1 + 0.2
0.30000000000000004
The operators are the familiar arithmetic set with two worth calling out. / is true division and returns a float, so 7 / 2 is 3.5. // is floor division, so 7 // 2 is 3, and % is the modulo, whose sign follows the divisor, so -7 % 3 is 2. Comparison operators are ==, !=, <, <=, >, >=, and the identity and membership tests are is, is not, in and not in.
Every value also has a truth value, and the rule is short: empty things and zeros are false, everything else is true. The falsy values are False, None, 0, 0.0, the empty string "", the empty bytes b"", the empty list [], the empty dict {} and the empty set set(). A list containing a zero, [0], is true because it is a non-empty container. bool(x) makes the test explicit.
an object whose value cannot be changed after it is created; operations return a new object instead scalars are all immutable, which is why a string method such as upper() returns a new string and leaves the original alone. For formatted output you use f-strings, which interpolate expressions and apply a format specifier:
>>> f"{0x1F:08x}"
'0000001f'
>>> f"booted in {4.213:.2f} s"
'booted in 4.21 s'
The specifier :08x is the embedded workhorse: it renders an integer as zero-padded lowercase hex, which is how you print a register value or a memory address in a log. :>8 right-aligns text, :.2f fixes two decimal places, and , adds thousands separators.
Going deeperHow an arbitrary-precision integer is stored
CPython's int is a variable-length object (PyLongObject) whose magnitude is stored as an array of digits. On a 64-bit build each digit is 30 bits wide, so a small integer fits in one or two digits and a thousand-digit integer uses more. The object grows by allocating a larger array when the value exceeds the current capacity, which is why there is no overflow and why arithmetic on huge integers gets slower as the numbers get longer. You can see the cost directly with sys.getsizeof: a small int is 28 bytes on a 64-bit build, and 2 ** 1000 is noticeably larger. A C int by contrast is always 4 bytes on these platforms and silently wraps past its range, which is one reason Python is safer for tooling that parses sizes and offsets from logs.
Collections: list, tuple, dict, set, and comprehensions
4 min readA list is an ordered, mutable sequence. You build it with [1, 2, 3], index it with xs[0], and change it with append, extend, insert, pop, remove and sort. A tuple is an ordered, immutable sequence written (1, 2). You cannot change its elements after creation, which makes it safe to share and eligible to be a dict key. A one-element tuple needs a trailing comma, (1,), because (1) is just the integer 1 in parentheses.
A dict maps keys to values. Keys must be a value with a stable hash and an equality test, so it can be used as a dict key or a set member; all immutable built-ins are hashable, and mutable ones are not, and the mapping preserves insertion order (guaranteed since Python 3.7). The everyday methods are get (with a default), keys, values, items, setdefault, update and pop. A set is an unordered collection of unique, hashable elements with constant-time membership tests, written {1, 2, 3}, and frozenset is its immutable cousin.
Sequences slice with the same syntax everywhere. xs[start:stop] is half-open, taking start up to but not including stop; negative indices count from the end, and a third value sets the step. xs[-1] is the last element, xs[1:4] takes indices 1 to 3, xs[::2] takes every second element, and xs[::-1] reverses.
xs = [3, 1, 2]
xs.append(4) # [3, 1, 2, 4]
xs.sort() # [1, 2, 3, 4]
record = ("uart0", 115200) # immutable tuple
boot = {"boot": 4200, "kernel": 1800}
boot["total"] = boot.get("total", 0) + boot["boot"]
seen = {3, 1, 2, 2} # {1, 2, 3}, duplicates dropped
The choice of collection is driven by what the data must guarantee. Use a tuple for a fixed record of heterogeneous fields and for dict keys; a list for a growing sequence of homogeneous items; a dict for lookup by a known key; a set for deduplication and fast membership. Because list and set are mutable they cannot be dict keys, which is the most common reason a program raises TypeError: unhashable type.
A an object that can produce its items one at a time, so a for loop or a comprehension can walk over it is the input to a comprehension, the idiomatic way to build one collection from another:
evens = [x for x in range(10) if x % 2 == 0] # list
squares = {x: x * x for x in range(4)} # dict
unique = {len(w) for w in ["a", "bb", "a"]} # set
summed = sum(x for x in range(1_000_000)) # generator expression
The generator expression in the last line is lazy: it produces one value at a time instead of building a million-element list first, so it costs almost no memory. enumerate pairs each item with its index, and zip pairs items from two iterables.
Going deeperWhat a dict looks like under the hood
A dict is a hash table: for each key CPython calls hash(key), an integer, and uses it to find a slot in a sparse array. The hash of an equal key is guaranteed equal, which is how lookup stays O(1) on average. The requirement that keys be hashable and immutable follows directly: if a key's hash changed after insertion, the table could no longer find it. That is why a list cannot be a key. Insertion order is preserved because CPython keeps a separate insertion-order index alongside the hash slots. Two consequences matter in practice: dict lookups are fast even for large mappings, and a set is built on the same machinery, which is why "is this value in the set" is O(1) while "is this value in the list" is O(n).
Functions, arguments, scope, and annotations
3 min readYou define a function with def, and it returns a value with return. A function with no return statement returns None. The first statement is conventionally a docstring, a string literal that documents the function and is shown by help().
def decode_hex(hex_str: str) -> int:
"""Convert a hex byte string such as 'ff' to an integer."""
return int(hex_str, 16)
Arguments come in four shapes. Positional arguments match by order, decode_hex("ff"). Keyword arguments match by name, decode_hex(hex_str="ff"). Default arguments fill in when omitted, def add(a, b=10). Default values are evaluated once, at definition time, not on every call, which is the root of the mutable-default trap you will meet in the advanced section. A bare * marks everything after it as keyword-only, and a bare / marks everything before it as positional-only.
*args collects extra positional arguments into a tuple, and **kwargs collects extra keyword arguments into a dict:
def log_fields(*names: str, **flags: bool) -> None:
print("fields:", names) # a tuple
print("flags:", flags) # a dict
log_fields("temp", "volt", verbose=True)
The mirror operation unpacks: f(*[1, 2, 3]) calls f(1, 2, 3) and f(**{"a": 1}) calls f(a=1).
When Python looks up a name it walks the LEGB scopes in order: Local (inside the function), Enclosing (an outer function that contains this one), Global (the module level) and Built-in (names such as print, len, range). Assignment inside a function creates a local variable by default, so to rebind a name in an outer scope you must say global or nonlocal explicitly. This is why a function can read a module-level constant but a plain assignment to it does not change the module copy.
Type annotations attach expected types to the signature, as in def parse(hex_str: str) -> int. They are hints, stored in the function's __annotations__ and not enforced at runtime, which is the point: a running script ignores them, but your editor and a static checker such as mypy use them to catch mistakes before the code runs. The common annotations are int, float, str, list[int], dict[str, int], Optional[int] for a value that may be None, and the shorthand int | None (Python 3.10+). Engineers annotate because the tooling pays back the effort on the first refactor, not because the interpreter requires it.
A function annotated -> int can still return a string at runtime and Python will not complain. Treat annotations as documentation that a machine can read, and rely on mypy in a later module to enforce them.
Control flow, exceptions, and the module execution model
3 min readDecisions use if / elif / else, and blocks are delimited by indentation rather than braces. Loops use for, which walks over any iterable, or while, which repeats while a condition holds. range(start, stop, step) produces a sequence of integers, and enumerate(seq) produces (index, item) pairs. Inside a loop, break exits it immediately and continue skips to the next iteration.
Two loop forms carry an else clause that beginners miss. for ... else and while ... else run the else suite only when the loop finishes without hitting break. That is exactly the shape of a search that may find nothing:
for name in ["uart", "i2c", "spi"]:
if name == "can":
break
else:
print("no can interface")
The match statement (Python 3.10+) is structural pattern matching. Where if tests a boolean, match compares a value against patterns and binds the parts that fit:
match cmd:
case ["read", addr]:
print(f"read from {addr}")
case ["write", addr, data]:
print(f"write {data} to {addr}")
case _:
print("unknown command")
Errors in Python are exceptions, and the idiom is try / except / else / finally. The try block runs the risky code. A matching except runs only if an exception of that type is raised; else runs only if no exception was raised; finally always runs, whether or not one was raised, which is where you release a resource or close a file. You raise an exception yourself with raise.
try:
value = int(raw)
except ValueError:
print("not an integer")
else:
print("parsed", value)
finally:
print("attempt done")
The built-in exceptions form a hierarchy under Exception, with ValueError, TypeError, KeyError and OSError among the common leaves. Python uses exceptions for error paths for two reasons. First, they separate the happy path from the failure path, so the main logic stays readable. Second, they cannot be ignored by accident the way a C return code can: an uncaught exception stops the program with a traceback. The community phrase for this style is EAFP, easier to ask forgiveness than permission, the opposite of checking every precondition first.
A .py file is a module, and one line decides its two possible lives. When a file is run as a script, Python sets its __name__ variable to the string "__main__". When the same file is imported, __name__ is the module's own name. The guard below runs the code only when the file is executed directly:
def main() -> None:
print("running as a script")
if __name__ == "__main__":
main()
The guard is why a tool can be both a script you run and a module you import in tests, without its top-level code firing on import.
bytes vs str in depth
4 min readA str holds text as a sequence of Unicode a number that Unicode assigns to one character or symbol, such as U+0065 for lowercase es. A bytes object holds a sequence of raw octets, integers from 0 to 255, with no meaning attached. The two are different types on purpose, and in Python 3 there is no silent conversion between them.
You convert text to bytes with encode() and bytes back to text with decode(), and both take an encoding. the variable-width Unicode encoding that represents every code point in one to four bytes and is identical to ASCII for the first 128 characters is the default and the right choice almost everywhere: it is ASCII-compatible, so an old boot log that is pure ASCII decodes as-is. latin-1 is the other encoding engineers reach for, because it maps each byte 0 to 255 to a code point directly and therefore decoding never fails.
text = "héllo" # str, 5 code points
raw = text.encode("utf-8") # bytes, 6 octets: é takes two bytes in UTF-8
raw.decode("utf-8") # back to "héllo"
bytes.fromhex("ff 0a") # b'\xff\n'
b"\xff".hex() # 'ff'
The conversion can fail, and how it fails is governed by the errors argument. The default is "strict", which raises. "ignore" drops the offending bytes, "replace" substitutes the replacement character U+FFFD, "backslashreplace" escapes them, and "surrogateescape" smuggles undecodable bytes through as private-use code points so a round trip is lossless.
The failure you will meet first is UnicodeDecodeError, and it is the single most common bug when talking to serial ports, sockets and files. A device sends bytes, not text. A boot log over a UART is usually ASCII, which decodes as UTF-8 without issue. A sensor or a vendor protocol, however, sends raw binary or latin-1, and decoding that as UTF-8 raises the moment a byte such as 0xff appears.
1reproduce the error in isolation
Keep the exact offending bytes. Decode them with a one-line command so you can see which byte breaks:
b"\xff\x00\x10".decode("utf-8") # raises UnicodeDecodeError
2identify the byte and the likely encoding
Print the bytes as hex with .hex(). If the stream is printable text in the 0x20 to 0x7E range it is ASCII, a UTF-8 subset. If every byte is a printable-looking value above 0x80 it may be latin-1. If it looks structured and arbitrary it is binary, and decoding is the wrong tool.
3pick the encoding or the errors policy
For a text stream in latin-1, decode with raw.decode("latin-1"). For binary you should not decode at all; unpack it with struct in a later lesson. For a lossy log where a bad byte is acceptable, use raw.decode("utf-8", errors="replace").
When you open /dev/ttyAMA0 with pyserial in a later module, read() returns bytes, never str. Printing the raw bytes or decoding them as UTF-8 without thought is the classic first-session mistake. Convert with an explicit encoding, or format with .hex() when you are inspecting a dump.
Going deeperHow CPython stores a string, and surrogateescape
Since PEP 393, a CPython string picks the narrowest internal representation that fits its content: one byte per character if it is all Latin-1, two bytes if all code points fit in 16 bits, and four bytes otherwise. This keeps strings compact while still giving O(1) indexing. The surrogateescape handler exists for a filesystem-specific reason: a filename on a POSIX system is a sequence of bytes, not text, and a filename may contain bytes that are not valid UTF-8. To represent such a name as a str without losing data, Python decodes the bad bytes into surrogate code points U+DC80 to U+DCFF, and re-encodes them on the way out. It is an escape hatch for round-tripping arbitrary bytes, and it is the reason you sometimes see lone surrogates in a string read from the operating system.
References, copies, closures, and the GIL
5 min readEvery value in Python is an object, and a name is a binding that points at an object. Assignment rebinds a name; it does not copy. So b = a makes b point at the same object a points at, and if that object is mutable, a change through one name is visible through the other:
a = [1, 2]
b = a # alias, not a copy
b.append(3)
print(a) # [1, 2, 3]
The identity test is checks whether two names point at the same object, while == checks whether two objects have equal values. They answer different questions, and id(x) returns the memory address of the object, which makes aliasing visible in a debug session.
The most famous consequence is the mutable default argument. Because a default value is evaluated once, at definition time, def f(items=[]) creates a single list shared by every call that omits the argument:
def bad(items=[]): # one shared list
items.append(1)
return items
bad(); bad() # [1, 1], not [1], [1]
The fix is the idiomatic None sentinel, creating the fresh object inside the function:
def good(items=None):
if items is None:
items = []
items.append(1)
return items
When you do need a real copy, copy.copy makes a shallow copy: a new outer container whose elements are still the same objects. copy.deepcopy copies recursively, so nested objects are duplicated too. For a flat list of immutable elements, list.copy() or the slice xs[:] is enough; for a list of lists, only deepcopy fully separates them.
Functions in Python are first-class objects: you can pass one as an argument, return one from a function, and store one in a variable. A lambda is a single-expression anonymous function, useful as a sort key: sorted(xs, key=lambda p: p[1]). A an inner function that remembers the variables of the enclosing function in which it was defined, even after that function returns captures its enclosing scope, and nonlocal lets the inner function rebind a captured name. This is how you build a counter or a configurable callback without a class.
The the global interpreter lock, a mutex inside CPython that allows only one thread to execute Python bytecode at a time is CPython's answer to the question of thread safety. Only one thread runs Python bytecode at any moment, and the lock is released during blocking I/O and after a short time slice (5 ms by default, adjustable with sys.setswitchinterval). The practical consequence is a clean split: I/O-bound work, such as talking to a serial port or waiting on a socket, benefits from threads because the waiting thread releases the GIL; CPU-bound work does not speed up with threads because only one thread computes at a time. For CPU-bound work you reach for the multiprocessing module or a C extension, and for many I/O-bound tasks you reach for asyncio, which the next session's second half covers.
Thread A holds the lock: it acquires the GIL and starts executing bytecode.
Thread A blocks on I/O: when it waits for a socket or file, it releases the GIL so others can run.
Thread B acquires the lock: it takes the GIL and starts executing its own bytecode.
Time slice ends: after 5 ms, a running thread releases the GIL even without I/O, so another thread gets a turn.
The interpreter alternates: threads interleave in small slices, but at most one runs Python bytecode at any instant.
Going deeperWhy the GIL exists, and where it is going
The GIL exists because CPython's memory management is simple by design. Reference counting, the primary way CPython frees objects, increments and decrements a counter on every assignment, and making that counter thread-safe without a global lock would add an atomic operation to every one of them, slowing single-threaded code. The GIL makes reference counting cheap and makes the C extensions that wrap libraries such as pyserial easier to write safely. The cost is that threads cannot run Python bytecode in parallel. PEP 703 accepted a plan to make the GIL optional, and CPython 3.13 ships an experimental free-threaded build without it, but that build is not the default and many extensions do not yet support it. For this program, assume the GIL is present and choose threads for I/O, multiprocessing or C for CPU work, and asyncio for many concurrent waits.
Common mistakes and how to debug them
A function that should remember a fresh list instead accumulates across calls. Cause: a mutable default argument is shared. Diagnose by checking the default with f.__defaults__, which shows the same list object growing. Fix: use the None sentinel and create the list inside.
Two names unexpectedly change together. Cause: aliasing, b = a bound both names to one list. Diagnose with b is a and id(a) == id(b). Fix: copy with b = list(a) or copy.copy, and copy.deepcopy for nested structures.
A log reader crashes with UnicodeDecodeError on a line of raw serial data. Cause: decoding binary or latin-1 bytes as UTF-8. Diagnose by printing the offending bytes with .hex(). Fix: decode with the correct encoding or errors="replace", or leave binary as bytes.
A copy of a nested list still shares its inner lists. Cause: a shallow copy duplicates only the outer container. Diagnose by mutating an inner list and watching both copies change. Fix: use copy.deepcopy.
Try it
1 min readThe lab for this lesson has three parts. First, create a venv, install a package with pip, and freeze a requirements.txt, then recreate the environment from the file in a clean directory. Second, write a function that decodes a list of hex byte strings into integers and formats a summary table, exercising bytes versus str and a comprehension. Third, solve the data-structure exercises: count word frequencies with a dict, deduplicate with a set, and transform a list with a comprehension. Before you start, re-read the bytes versus str section and the aliasing and GIL section, and try the mutable-default trap in the REPL yourself.
Key takeaways
- Python compiles source to bytecode and runs it on a virtual machine, which gives fast iteration but no hard real-time guarantees.
- Create one virtual environment per project with python3 -m venv .venv, activate it, and pin dependencies in requirements.txt.
- int, float, bool, str, bytes and None have distinct semantics; type() and isinstance() tell you which one you hold.
- list, tuple, dict and set differ in mutability, ordering and hashability; choose by what the data must guarantee.
- Comprehensions and generator expressions are the idiomatic, readable way to transform and filter collections.
- Python resolves names through the LEGB scopes; *args and **kwargs collect extra positional and keyword arguments.
- str holds Unicode code points and bytes holds raw octets; never mix them without an explicit encode or decode.
- The GIL serialises Python bytecode, so CPU-bound work needs multiprocessing or C, while I/O-bound work suits threads or asyncio.
Reflect
- Explain to a colleague, in two sentences: explain why Python suits embedded tooling and where it is a poor fit, using the interpreted versus compiled distinction.
- Which objective are you least sure about? Re-read that section, then do its exercise in the practice pack.
- What surprised you when you ran the commands yourself?
On a real board
Repeat the key commands on your Raspberry Pi or lab board over SSH or the serial console. Note anything that behaves differently from your PC, such as missing tools, a BusyBox shell or a read-only filesystem.
References and further reading
- Official docsThe Python Tutorial
- Official docsThe Python Language Reference
- Official docsvenv - Creation of virtual environments
- SpecificationPEP 484 - Type Hints
- SpecificationPEP 604 - Union types X | Y
- SpecificationPEP 636 - Structural pattern matching tutorial
- SpecificationPEP 393 - Flexible string representation
- SpecificationPEP 703 - Making the global interpreter lock optional
Official documentation first. Links checked when the lesson was published.