Page 2: Binary Representation: Bits, Bytes, Integers, and Character Encodings
Integer overflow bugs, corrupted international text (mojibake), and off-by-one buffer allocations all stem from misunderstanding how values map to bytes.
Computer memory does not store letters, colors, audio waves, or JSON objects directly—it stores only sequences of bits. A single bit can distinguish between $2^1 = 2$ states. Grouping 8 bits together forms a byte, which can distinguish between $2^8 = 256$ distinct bit patterns (from `00000000` to `11111111`). The meaning of any byte in memory depends entirely on the interpretation rule applied by the program reading it.
For example, the 8-bit pattern `01000001` represents the unsigned integer $65$ when interpreted as a number, the uppercase letter `"A"` when interpreted as ASCII or UTF-8 text, or a specific machine instruction opcode when fetched by a CPU. Understanding binary positional notation, signed integer encoding, and UTF-8 text representation is essential for working with network protocols, file formats, cryptography, and database storage.
2.1 Positional Notation: Binary and Hexadecimal
In base-10 (decimal), each digit position represents a power of 10 ($10^0, 10^1, 10^2, \dots$). Base-2 (binary) works identically, except each bit position represents a power of 2 ($2^0 = 1, 2^1 = 2, 2^2 = 4, 2^3 = 8, \dots$). To compute the value of an 8-bit unsigned byte, sum the powers of 2 where a `1` bit is present.
Because 32-bit or 64-bit binary strings are tedious for humans to read, engineers use base-16 (hexadecimal, digits `0–9` and `A–F`). Every single hexadecimal digit maps to exactly 4 bits (a nibble), meaning one 8-bit byte is always written as two hexadecimal digits (from `0x00` to `0xFF`).
2.2 Signed Integers and Two’s Complement
How should hardware represent negative integers? Early computers tried reserving the leftmost bit as a simple sign flag ("sign-magnitude"), but that approach created two representations for zero (`+0` and `-0`) and required complex circuitry to add positive and negative numbers.
Modern CPUs universally use Two’s Complement representation. In an $n$-bit Two’s Complement integer, the most significant bit has a negative weight of $-2^{n-1}$, while all remaining bits have positive weights. For an 8-bit signed integer (`Int8`), the leftmost bit has weight $-128$, giving a range of $-128$ (`10000000`) to $+127$ (`01111111`). To negate any number in Two’s Complement, invert all bits and add 1.
2.3 Character Encoding: ASCII, Unicode, and UTF-8
Originally, 7-bit ASCII mapped 128 English characters and control codes to numbers 0–127. To represent all human scripts, mathematical symbols, and emoji across the globe, the Unicode standard assigns every character a unique numeric code point (written `U+0041` for `"A"` or `U+1F680` for `"🚀"`).
UTF-8 is the dominant variable-width encoding on the web. It encodes ASCII characters (`U+0000` to `U+007F`) in a single byte—making it 100% backward-compatible with ASCII—while using 2, 3, or 4 bytes for non-ASCII characters.