SyntaxPilot
Courses/CS Fundamentals for Placements/Lesson 4

CS Fundamentals for Placements · Lesson 4 of 90 · 18 min

Floating point, character encoding and endianness

fixed-point representationfloating-point representationscientific notationIEEE 754 single and double precisionsign/exponent/mantissa fieldsexponent biasnormalized and denormalized numbershidden bitspecial values (zero, infinity, NaN)precision loss and rounding errorwhy 0.1 + 0.2 != 0.3ASCIIextended ASCIIUnicode

These are the notes for CS Fundamentals for Placements, episode 4. Until now every number we stored was a whole number. But a price like ₹3.75, the mass of an electron, the letter A and the word नमस्ते are not whole numbers. Here you invent how a computer stores all of them: fixed-point representation, scientific notation and floating-point representation, the IEEE 754 standard (single and double precision, sign / exponent / mantissa, the exponent bias, normalized numbers and the hidden bit), the special values (zero, infinity, NaN, denormalized numbers), precision loss and rounding error (why 0.1 + 0.2 != 0.3), then text: ASCII, extended ASCII and code pages, Unicode and code points, UTF-8, UTF-16 and UTF-32, and finally endianness (big-endian, little-endian) and network byte order (htonl, htons, ntohl, ntohs). Every term is explained in plain words, and there is a full glossary at the end. All outputs below are real: C compiled with gcc -O2 -Wno-unused-result main.c -o main -lm (gcc 13.3) and Python 3.13, on an x86-64 machine.

A quick refresher from earlier episodes: a bit is one 0 or 1, a byte is 8 bits, the MSB (most significant bit) is the leftmost bit, the one with the biggest place value, and the LSB (least significant bit) is the rightmost. Hexadecimal (hex) is base 16, and each hex digit stands for exactly 4 bits (0xB = 1011). A small ₂ after a number means it is written in binary: 101₂ = 5.

Binary fractions. Digits after a binary point are worth halves, quarters, eighths and so on: 0.1₂ = 1/2, 0.01₂ = 1/4, 0.001₂ = 1/8. So 101.11₂ = 4 + 1 + 1/2 + 1/4 = 5.75. To turn a decimal fraction into binary, multiply it by 2 again and again; each time the whole-number part (0 or 1) is the next bit, and you keep only the fraction.

The seven problems

Problem What it teaches
Store ₹3.75 in bits Fixed-point representation
Tiny and huge numbers Scientific notation, floating point, IEEE 754
1 ÷ 0 and 0 ÷ 0 Zero, infinity, NaN, denormalized numbers
0.1 + 0.2 ≠ 0.3 Precision loss, rounding error, tolerances
Store 'A' Character sets, ASCII
Store नमस्ते Extended ASCII, Unicode, UTF-8, UTF-16
Which byte goes first? Big-endian, little-endian, network byte order

Problem 1: a decimal point in bits (fixed point)

Real-life hook. A kirana shop sells a biscuit for ₹3.75. The register stores numbers in bits, and bits are only 0s and 1s: there is no bit for a decimal point.

The shopkeeper's trick. Write the price as 375 paise. Everyone agrees the decimal point is always two digits from the right. Storing a whole number and agreeing on a fixed position for the point is called fixed-point representation. Example: with 2 digits after the point, the integer 375 means 3.75 and the integer 5 means 0.05.

# attempt 1: fixed point, 2 digits after the point (store paise)
price = 375            # Rs 3.75 stored as 375 paise
print("price  =", price / 100)

electron_kg = 0.000000000000000000000000000000911
sun_km      = 149600000
print("electron as paise:", round(electron_kg * 100))
print("sun as paise:", sun_km * 100, "->", (sun_km * 100).bit_length(), "bits")

Output:

price  = 3.75
electron as paise: 0
sun as paise: 14960000000 -> 34 bits
  • The price is perfect: fixed point is exactly right for money.
  • The electron's mass became 0. Two digits after the point cannot see something that small.
  • The sun's distance (in km, times 100) needs 34 bits, more than a normal 32-bit integer holds. bit_length() tells how many bits a whole number needs.
  • Why it failed: fixed point has one precision for everything. The point never moves, so the same bits cannot cover the very tiny and the very huge at the same time.

Problem 2: floating point and IEEE 754

Scientific notation

Scientists already solved this. Scientific notation writes a number as a few significant digits times 10 to a power:

149,600,000        = 1.496 × 10⁸
0.000…000911       = 9.11  × 10⁻³¹

The digits part (1.496) is the significand (also called the mantissa), and the power (8) is the exponent. Changing the exponent slides the decimal point left or right.

The same idea in binary

5.75 = 101.11₂ = 1.0111₂ × 2². So we only need to store three things:

Field Meaning For 5.75
Sign 0 for plus, 1 for minus 0
Exponent the power of 2 2
Mantissa (significand) the binary digits 1.0111

The exponent moves the binary point, so the point floats. This is floating-point representation: a sign, an exponent and a mantissa. With the same number of bits, both 10⁻³¹ and 10⁸ fit, because the exponent does the scaling.

IEEE 754: the standard every CPU uses

If every company packed these bits its own way, a file written on one machine would be garbage on another. IEEE 754 is the standard (from the IEEE, the Institute of Electrical and Electronics Engineers) that almost every CPU follows for floating point.

Single precision (float) Double precision (double)
Total size 32 bits (4 bytes) 64 bits (8 bytes)
Sign 1 bit 1 bit
Exponent 8 bits, bias 127 11 bits, bias 1023
Mantissa (stored) 23 bits 52 bits
Precision (with hidden bit) 24 bits ≈ 7 decimal digits 53 bits ≈ 15-16 decimal digits
Largest value about 3.4 × 10³⁸ about 1.8 × 10³⁰⁸
Used by C float, Java float C double, Java double, Python float, every JavaScript number
single:  [S][EEEEEEEE][MMMMMMMMMMMMMMMMMMMMMMM]
          1     8                23              = 32 bits
double:  [S][EEEEEEEEEEE][MMMM ... 52 bits ... MMMM]
          1      11                 52           = 64 bits

Precision means how many correct significant digits a number can carry. More mantissa bits = more precision.

Trick 1: the exponent bias

Exponents can be negative (2⁻⁴), but we don't use 2's complement here. Instead, we add a fixed number and store the result, so the stored exponent is never negative. That number is the exponent bias: 127 for single, 1023 for double. In general, for a k-bit exponent the bias is 2ᵏ⁻¹ - 1.

real exponent  2  → stored  2 + 127 = 129 = 10000001
real exponent -4  → stored -4 + 127 = 123 = 01111011

Why bother? With a biased exponent, two positive floats compare in the same order as their bit patterns read as plain unsigned integers, so comparing floats is almost as cheap as comparing integers.

Trick 2: normalized numbers and the hidden bit

We always slide the point until exactly one 1 sits before it: 1.xxxx × 2ᵉ. This form is called normalized. Since the first digit of a normalized binary number is always 1, storing it would waste a bit. IEEE 754 leaves it out: that invisible leading 1 is the hidden bit (also called the implicit bit). So 23 stored bits give 24 bits of precision (52 give 53).

The formula for a normalized number:

value = (-1)^sign × 1.mantissa₂ × 2^(stored exponent − bias)

Seeing the real bits

Python's struct module packs a number into the 4 bytes of a 32-bit float; we print the sign, exponent and mantissa fields.

import struct

def bits32(x):   # the 32 bits of a float, as IEEE 754 stores them
    b = f"{struct.unpack('>I', struct.pack('>f', x))[0]:032b}"
    return b[0] + " " + b[1:9] + " " + b[9:]

for x in [5.75, -5.75, 0.1]:
    print(f"{x:>6} = {bits32(x)}")

e = int(bits32(5.75).split()[1], 2)
print("stored exponent =", e, "-> real exponent =", e - 127)

Output:

  5.75 = 0 10000001 01110000000000000000000
 -5.75 = 1 10000001 01110000000000000000000
   0.1 = 0 01111011 10011001100110011001101
stored exponent = 129 -> real exponent = 2
  • struct.pack('>f', x) gives the 4 bytes of x as a float (> = big-endian, see Problem 7); unpack('>I', ...) reads those bytes as an unsigned integer so we can print its bits.
  • 5.75: sign 0, exponent 10000001 = 129, minus 127 = 2. The mantissa begins 0111: exactly the digits after "1." in 1.0111. The hidden 1 is not stored.
  • -5.75 differs only in the sign bit.
  • 0.1: the mantissa is 1001 1001 1001 ... repeating, cut off (and rounded) at 23 bits. Remember this: it is coming back in Problem 4.

Worked example: -6.5 to IEEE 754 single, by hand

  1. Sign: negative, so sign = 1.
  2. Binary: 6 = 110₂, 0.5 = 0.1₂, so 6.5 = 110.1₂.
  3. Normalize: 110.1₂ = 1.101₂ × 2² (point moved 2 places left).
  4. Exponent: 2 + 127 = 129 = 10000001.
  5. Mantissa: drop the hidden 1, keep 101, pad to 23 bits: 10100000000000000000000.
  6. Result: 1 10000001 10100000000000000000000. Group into nibbles: 1100 0000 1101 0000 0000 ... = 0xC0D00000.

Worked example: 0.15625 to IEEE 754 single, by hand

  1. Sign: 0.
  2. Fraction to binary (multiply by 2, take the whole part):
Step × 2 Bit Keep
0.15625 0.3125 0 0.3125
0.3125 0.625 0 0.625
0.625 1.25 1 0.25
0.25 0.5 0 0.5
0.5 1.0 1 0 (stop)

So 0.15625 = 0.00101₂ (check: 1/8 + 1/32 = 0.15625). 3. Normalize: 0.00101₂ = 1.01₂ × 2⁻³. 4. Exponent: -3 + 127 = 124 = 01111100. 5. Mantissa: 01 then zeros: 01000000000000000000000. 6. Result: 0 01111100 01000000000000000000000 = 0x3E200000.

Handy patterns to remember: 1.0 = 0x3F800000 (exponent exactly 127) and -0.0 = 0x80000000 (just the sign bit). The mini project at the end decodes these and both worked examples from their bits, and prints the hex, in a real run.

Worked example: decoding a bit pattern

Decode 0 01111011 10011001100110011001101 (the 0.1 from above):

  1. Sign 0 → positive.
  2. Exponent 01111011 = 123 → 123 - 127 = -4.
  3. Mantissa with the hidden bit: 1.10011001100110011001101₂ ≈ 1.6000000238.
  4. Value = 1.6000000238 × 2⁻⁴ = 1.6000000238 / 16 ≈ 0.1000000015.

So a float that "holds 0.1" really holds 0.100000001490116... (the mini project below prints this).

Double precision layout

import struct

def bits64(x):
    b = f"{struct.unpack('>Q', struct.pack('>d', x))[0]:064b}"
    return b[0] + " " + b[1:12] + " " + b[12:]

for x in [1.0, -6.5, 0.1]:
    print(f"{x:>5} = {bits64(x)}")
print("0.1 as double hex:", struct.pack('>d', 0.1).hex())

Output:

  1.0 = 0 01111111111 0000000000000000000000000000000000000000000000000000
 -6.5 = 1 10000000001 1010000000000000000000000000000000000000000000000000
  0.1 = 0 01111111011 1001100110011001100110011001100110011001100110011010
0.1 as double hex: 3fb999999999999a
  • >d packs a double (8 bytes), >Q reads 8 bytes as an unsigned 64-bit integer.
  • -6.5 as a double: exponent 2 + 1023 = 1025 = 10000000001, mantissa 101 then zeros. Same sign and mantissa as the float; only the bias and field widths change.
  • 0.1's repeating 1001 pattern now runs for 52 bits, and the last bits ...1010 show the rounding up at the end. The hex 3fb999999999999a is the famous pattern: 9s (1001) ending in a.

Range and precision: float vs double

C's <float.h> header gives the limits of each type.

#include <stdio.h>
#include <float.h>

int main(void) {
    printf("float : %zu bytes, %d mantissa bits, %d digits\n", sizeof(float), FLT_MANT_DIG, FLT_DIG);
    printf("  FLT_MIN     = %g\n", FLT_MIN);
    printf("  FLT_MAX     = %g\n", FLT_MAX);
    printf("  FLT_EPSILON = %g\n", FLT_EPSILON);
    printf("  FLT_TRUE_MIN = %g\n", FLT_TRUE_MIN);
    printf("double: %zu bytes, %d mantissa bits, %d digits\n", sizeof(double), DBL_MANT_DIG, DBL_DIG);
    printf("  DBL_MIN     = %g\n", DBL_MIN);
    printf("  DBL_MAX     = %g\n", DBL_MAX);
    printf("  DBL_EPSILON = %g\n", DBL_EPSILON);
    printf("  DBL_TRUE_MIN = %g\n", DBL_TRUE_MIN);
    return 0;
}

Output:

float : 4 bytes, 24 mantissa bits, 6 digits
  FLT_MIN     = 1.17549e-38
  FLT_MAX     = 3.40282e+38
  FLT_EPSILON = 1.19209e-07
  FLT_TRUE_MIN = 1.4013e-45
double: 8 bytes, 53 mantissa bits, 15 digits
  DBL_MIN     = 2.22507e-308
  DBL_MAX     = 1.79769e+308
  DBL_EPSILON = 2.22045e-16
  DBL_TRUE_MIN = 4.94066e-324
float (single) double
Stored exponent range for normal numbers 1 to 254 1 to 2046
Real exponent range -126 to +127 -1022 to +1023
Smallest positive denormal (*_TRUE_MIN) 2⁻¹⁴⁹ ≈ 1.4013 × 10⁻⁴⁵ 2⁻¹⁰⁷⁴ ≈ 4.94066 × 10⁻³²⁴
Smallest positive normal (*_MIN) 2⁻¹²⁶ ≈ 1.17549 × 10⁻³⁸ 2⁻¹⁰²² ≈ 2.22507 × 10⁻³⁰⁸
Largest finite (*_MAX) (2 - 2⁻²³) × 2¹²⁷ ≈ 3.40282 × 10³⁸ (2 - 2⁻⁵²) × 2¹⁰²³ ≈ 1.79769 × 10³⁰⁸
Machine epsilon (*_EPSILON) 2⁻²³ ≈ 1.19209 × 10⁻⁷ 2⁻⁵² ≈ 2.22045 × 10⁻¹⁶
Precision bits (*_MANT_DIG, includes hidden bit) 24 53
Decimal digits 24 × log₁₀2 ≈ 7.22; guaranteed FLT_DIG = 6 53 × log₁₀2 ≈ 15.95; guaranteed DBL_DIG = 15
  • Machine epsilon is the gap between 1.0 and the next bigger number the type can store. It measures precision: below 1 part in about 10⁷, a float cannot tell numbers apart.
  • "About 7 digits" for float comes from 24 bits × log₁₀2 ≈ 7.22. FLT_DIG = 6 is the number of decimal digits that are guaranteed to survive a round trip (decimal → float → decimal).
  • The %g format prints a number in the shorter of normal and scientific style, with 6 significant digits.

Problem 3: zero, infinity and NaN (special values)

New problem. With a hidden 1 always in front, the smallest value is 1.0 × 2⁻¹²⁶. How do we store zero? We can't, with the normal rule. And what should 1 ÷ 0 or 0 ÷ 0 give? IEEE 754 reserves the two extreme exponent patterns, all 0s and all 1s, for these.

Exponent bits Mantissa Meaning Value
all 0s 0 Zero (+0 or -0) 0
all 0s not 0 Denormalized (subnormal) (-1)ˢ × 0.m × 2⁻¹²⁶
1 to 254 anything Normalized (-1)ˢ × 1.m × 2^(e-127)
all 1s 0 Infinity (+∞ or -∞) ±∞
all 1s not 0 NaN (Not a Number) NaN
  • Zero. Because there is a sign bit, there is +0 (0x00000000) and -0 (0x80000000). They compare as equal: -0.0 == 0.0 is true.
  • Denormalized numbers (also called subnormal numbers): when the exponent is all 0s, the hidden bit becomes 0 instead of 1, and the exponent is fixed at -126. This allows values smaller than the smallest normal number, down to 2⁻¹⁴⁹, at the cost of fewer correct digits. Numbers fade to zero step by step instead of jumping straight from 2⁻¹²⁶ to 0. This is called gradual underflow. (Underflow means a result too small to store as a normal number.)
  • Infinity: the result of overflow (a result too big, like 1e308 × 10 in a double) or of dividing a non-zero number by zero. It keeps its sign: +inf and -inf.
  • NaN, Not a Number: the answer to a question with no answer, like 0 ÷ 0, ∞ - ∞ or the square root of -1. Rule: NaN is not equal to anything, not even itself.
#include <stdio.h>
#include <float.h>

int main(void) {
    double zero = 0.0;
    printf("1/0    = %f\n", 1.0 / zero);
    printf("-1/0   = %f\n", -1.0 / zero);
    double n = zero / zero;
    printf("0/0    = %f\n", n);
    printf("n == n ? %d\n", n == n);
    printf("smallest normal float = %g\n", FLT_MIN);
    printf("half of it            = %g\n", FLT_MIN / 2);
    return 0;
}

Output:

1/0    = inf
-1/0   = -inf
0/0    = -nan
n == n ? 0
smallest normal float = 1.17549e-38
half of it            = 5.87747e-39
  • No crash: floating-point division by zero gives inf and -inf. (Integer division by zero is different: in C it is undefined behaviour and usually crashes the program.)
  • 0 ÷ 0 printed -nan. The minus is just the NaN's sign bit being set on this x86-64 machine; it means nothing.
  • n == n is 0 (false). That is the classic NaN test: x != x is true only for NaN. In C, use isnan(x) from <math.h>; in Python, math.isnan(x).
  • Half of FLT_MIN did not become 0: it is a denormal, 2⁻¹²⁷ ≈ 5.87747 × 10⁻³⁹.

Problem 4: why 0.1 + 0.2 != 0.3

total = 0.1 + 0.2
print(total)
print(total == 0.3)
print(f"0.1 is really {0.1:.25f}")

# the fixes
print(abs(total - 0.3) < 1e-9)           # compare with a tolerance
print((10 + 20) == 30)                   # money: count paise as integers

Output:

0.30000000000000004
False
0.1 is really 0.1000000000000000055511151
True
True

The first three lines are the bug: the sum is not 0.3, and the stored 0.1 is slightly more than 0.1. (The last two lines are the fixes, explained below.)

Why 0.1 can't be stored exactly

In decimal, 1 ÷ 3 = 0.3333... forever. You must stop somewhere, so it is never exact. 0.1 in binary is just like 1/3 in decimal. Convert it by repeated multiplication by 2:

Step × 2 Bit
0.1 0.2 0
0.2 0.4 0
0.4 0.8 0
0.8 1.6 1
0.6 1.2 1
0.2 0.4 0 ← back to 0.2: it repeats

So 0.1 = 0.0 0011 0011 0011 ...₂ forever. A double keeps only 52 mantissa bits, so 0.1 is rounded to the nearest value it can store.

  • Rounding error is the difference between the true value and the stored value (for 0.1 as a double: about 5.55 × 10⁻¹⁸).
  • Precision loss is the general effect: digits beyond what the mantissa can hold are lost, so results carry small errors that can grow.
  • A decimal fraction is exact in binary only if its denominator (in lowest terms) is a power of 2: 0.5, 0.25, 0.375 are exact; 0.1, 0.2, 0.3 are not.

Python's Decimal(x) shows the exact value a double really stores:

from decimal import Decimal

# Decimal(x) shows the exact value a double really stores
print("0.1       ->", Decimal(0.1))
print("0.2       ->", Decimal(0.2))
print("0.1 + 0.2 ->", Decimal(0.1 + 0.2))
print("0.3       ->", Decimal(0.3))

# a decimal type built from strings is exact
print(Decimal("0.1") + Decimal("0.2") == Decimal("0.3"))

Output:

0.1       -> 0.1000000000000000055511151231257827021181583404541015625
0.2       -> 0.200000000000000011102230246251565404236316680908203125
0.1 + 0.2 -> 0.3000000000000000444089209850062616169452667236328125
0.3       -> 0.299999999999999988897769753748434595763683319091796875
True

0.1 and 0.2 are both stored slightly high; their sum is rounded to the double just above 0.3, while the literal 0.3 is stored slightly low. Two different doubles, so == says False. Python prints the sum as 0.30000000000000004 because it prints the shortest decimal that maps back to the same double.

Does a tiny error matter? A drifting clock

A clock stored in a 32-bit float adds 0.1 seconds every tick. After one hour (36,000 ticks) it should say 3600.

#include <stdio.h>

int main(void) {
    float t = 0.0f;                     // 32-bit float clock
    for (int tick = 0; tick < 36000; tick++)
        t = t + 0.1f;                   // add one tenth of a second
    printf("after 1 hour: %f seconds\n", t);
    printf("error: %f seconds\n", t - 3600);
    return 0;
}

Output:

after 1 hour: 3601.162354 seconds
error: 1.162354 seconds

More than a second wrong after one hour. Every addition rounds a little, and the errors accumulate (pile up). The bigger t gets, the fewer mantissa bits are left for the 0.1, so each step is rounded more coarsely.

This really happened. According to the GAO report IMTEC-92-26, on 25 February 1991 a Patriot missile battery at Dhahran, Saudi Arabia, failed to intercept an incoming Scud missile, which hit an Army barracks and killed 28 Americans. The system kept time in tenths of seconds in 24-bit registers, so 0.1 was never exact. After 100 hours of continuous operation the timing error was 0.3433 seconds, which shifted the range gate (the patch of sky the radar checks) by about 687 metres. The radar looked in the wrong place.

The fixes

Fix How Example
Never test floats with == Check whether they are close: abs(a - b) < tolerance abs(total - 0.3) < 1e-9 → True; or math.isclose(a, b)
Money: count paise as integers Go back to fixed point, or use a decimal type 10 + 20 == 30; Python Decimal, Java BigDecimal
Clocks: count ticks as integers Keep an integer counter, multiply once at the end seconds = ticks / 10
Need more correct digits Use double instead of float ~15-16 digits instead of ~7
  • A tolerance (often called epsilon in code) is the largest difference you are willing to call "equal". Pick it to suit the size of your numbers; a fixed 1e-9 is fine near 1 but useless near 10²⁰. math.isclose uses a relative tolerance (a fraction of the numbers' size) by default.
  • Do not confuse this everyday epsilon with machine epsilon (DBL_EPSILON), which is a fixed property of the type.

Problem 5: storing text (ASCII)

Real-life hook. Bits can only store numbers. So how do we store the letter A? Think of a cinema: every seat has a number, and your ticket just says seat 65. If everyone uses the same seating chart, a number can stand for a seat.

A chart that gives every character a number is a character set; the rule that turns those numbers into bytes is a character encoding. For simple 1-byte sets the two are the same thing, so people use the words loosely. The first famous one is ASCII (American Standard Code for Information Interchange): 7 bits, 128 characters (0 to 127).

Codes Characters
0 - 31 (and 127) Control codes, invisible commands: tab = 9, newline \n = 10, carriage return \r = 13
32 space
48 - 57 digits '0' to '9'
65 - 90 'A' to 'Z'
97 - 122 'a' to 'z'
  • Lowercase = uppercase + 32. 32 = 0100000, so case differs in one bit: 'A' = 1000001, 'a' = 1100001.
  • The character '7' is the number 55, not 7. Digit value = c - '0'.
#include <stdio.h>

int main(void) {
    char c = 'A';
    printf("'%c' is stored as %d\n", c, c);
    printf("'%c' is stored as %d\n", 'a', 'a');
    printf("'%c' is stored as %d\n", '0', '0');
    printf("'a' - 'A' = %d\n", 'a' - 'A');
    printf("%d + 1 = '%c'\n", c, c + 1);
    return 0;
}

Output:

'A' is stored as 65
'a' is stored as 97
'0' is stored as 48
'a' - 'A' = 32
65 + 1 = 'B'

In C, a char is just a small integer. %c prints it as a character, %d as a number. 65 + 1 printed with %c is B: text is just numbers in an agreed chart.

Problem 6: every language (Unicode, UTF-8, UTF-16)

Attempt: extended ASCII and code pages

ASCII only has English. A byte has 8 bits, so the 8th bit was used for 128 more characters (128 to 255). This is extended ASCII. But every region filled those 128 slots differently, and each version is called a code page. Example: the byte 0xE9 is é in Windows-1252 (Western Europe), Θ in code page 437, and щ in code page 866 (Cyrillic). Same byte, three characters.

word = "नमस्ते"
data = word.encode("utf-8")              # saved as UTF-8 bytes

# opened with an old 1-byte code page (Windows-1252)
print(data.decode("cp1252", errors="replace"))
# opened as UTF-8
print(data.decode("utf-8"))

Output:

नमस�ते
नमस्ते
  • encode turns text into bytes; decode turns bytes back into text using a chosen chart.
  • The 18 bytes of नमस्ते (e0 a4 a8 e0 a4 ae ...) were read one byte = one character with the Windows-1252 chart: 0xE0 = à, 0xA4 = ¤, 0xA8 = ¨, and so on. Byte 0x8D has no character in Windows-1252, so errors="replace" shows �, the replacement character.
  • Same bytes, wrong chart: garbled text like this is called mojibake (a Japanese word for "character transformation").
  • Why it failed: 256 slots can never hold Hindi, Tamil, Chinese and emoji together, and a file does not say which code page it used.

The fix: Unicode and code points

Think of phone numbers: every phone in the world has one unique number. Unicode is one chart for every script in the world: it gives every character its own number, called a code point, written U+ and at least 4 hex digits.

Character Code point
A U+0041 (65, same as ASCII)
é U+00E9
न (Hindi "na") U+0928
₹ (rupee sign) U+20B9
😀 U+1F600

Code points run from U+0000 to U+10FFFF. According to the Unicode blog (9 September 2025), Unicode 17.0 added 4,803 characters for a total of 159,801.

A code point is just a number. How that number becomes bytes is a separate choice, called the encoding, like writing the same phone number on a form in different styles.

Attempt: UTF-32

The obvious encoding: give every character 4 bytes. That is UTF-32. It works, and every character has the same size, but plain English text becomes four times bigger than ASCII. Wasteful.

UTF-8: a variable-length encoding

UTF-8 is a variable-length encoding: different characters take different numbers of bytes (1 to 4). The first bits of the first byte say how many bytes the character uses, and every follow-up byte (a continuation byte) starts with 10.

Code point range Bytes Byte pattern (x = bits of the code point) Free bits
U+0000 - U+007F 1 0xxxxxxx 7
U+0080 - U+07FF 2 110xxxxx 10xxxxxx 11
U+0800 - U+FFFF 3 1110xxxx 10xxxxxx 10xxxxxx 16
U+10000 - U+10FFFF 4 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx 21
  • Every ASCII character is still the same single byte, so old English files are already valid UTF-8. This is called being ASCII-compatible.
  • A byte starting 10 is never the start of a character, so a program can jump into the middle of a file and find the next character boundary.
  • According to W3Techs (28 October 2026), "UTF-8 is used by 99.1% of all the websites whose character encoding we know."

UTF-16 and surrogate pairs

UTF-16 uses 2 bytes (one 16-bit unit) for every code point up to U+FFFF, and 4 bytes for the rest (like most emoji). Those 4 bytes are two special 16-bit units called a surrogate pair: a high surrogate (D800-DBFF) followed by a low surrogate (DC00-DFFF). Java and JavaScript strings use UTF-16 inside, which is why "😀".length is 2 in JavaScript.

for ch in ["A", "é", "न", "😀"]:
    u8 = ch.encode("utf-8")
    u16 = ch.encode("utf-16-be")
    print(f"{ch}  U+{ord(ch):04X}  UTF-8: {len(u8)} bytes {u8.hex(' ')}"
          f"   UTF-16: {len(u16)} bytes")

word = "नमस्ते"
print(word, len(word), "code points,", len(word.encode("utf-8")), "bytes")

Output:

A  U+0041  UTF-8: 1 bytes 41   UTF-16: 2 bytes
é  U+00E9  UTF-8: 2 bytes c3 a9   UTF-16: 2 bytes
न  U+0928  UTF-8: 3 bytes e0 a4 a8   UTF-16: 2 bytes
😀  U+1F600  UTF-8: 4 bytes f0 9f 98 80   UTF-16: 4 bytes
नमस्ते 6 code points, 18 bytes
  • ord(ch) gives the code point; utf-16-be is UTF-16 with big-endian byte order (Problem 7) and no extra marker bytes.
  • A is the same 41 as ASCII. é is 2 bytes. न is 3 bytes starting e0 = 1110 0000: the 1110 says "3 bytes". The emoji is 4 bytes in both UTF-8 and UTF-16.
  • नमस्ते is 6 code points but 18 bytes (न म स ् त े: the virama ् and the vowel sign े are code points of their own). "Length" depends on whether you count characters or bytes.

Worked example: U+20B9 (₹) to UTF-8, by hand

  1. U+20B9 is between U+0800 and U+FFFF → 3 bytes: 1110xxxx 10xxxxxx 10xxxxxx (16 free bits).
  2. 0x20B9 in 16 bits: 0010 0000 1011 1001.
  3. Split into 4 | 6 | 6 bits: 0010 | 000010 | 111001.
  4. Fill the template: 11100010 10000010 10111001.
  5. Bytes: 11100010 10000010 10111001 = E2 82 B9.

A real run of "₹".encode("utf-8").hex(" ") in Python prints e2 82 b9. The same character is 20 b9 in UTF-16BE, b9 20 in UTF-16LE and 00 00 20 b9 in UTF-32BE.

Surrogate pair by hand for U+1F600: subtract 0x10000 → 0xF600 = 20 bits 0000111101 1000000000. High 10 bits = 0x03D → D800 + 3D = D83D. Low 10 bits = 0x200 → DC00 + 200 = DE00. Python's "😀".encode("utf-16-be").hex(" ") prints d8 3d de 00: it matches.

Notice that UTF-16BE and UTF-16LE give the same bytes in opposite order. That is our last problem.

Problem 7: which byte goes first? (endianness)

Real-life hook. An int is 4 bytes, but memory is a row of single bytes, each with its own address (its position number). Which byte goes at the lowest address? Think of dates: 2026-10-28 writes the biggest part first; 28-10-2026 writes the smallest part first. Both are correct, as long as the reader knows which one it is.

  • The most significant byte (MSB, here meaning byte) is the one with the biggest place value: in 0x12345678 it is 0x12. The least significant byte (LSB) is 0x78.
  • Big-endian: the most significant byte is stored at the lowest address ("big end first", like 2026-10-28).
  • Little-endian: the least significant byte is stored at the lowest address ("little end first", like 28-10-2026). The x86-64 machine used for these notes is little-endian, as the run below shows.

0x12345678 stored at address 0x100:

Address 0x100 0x101 0x102 0x103
Big-endian 12 34 56 78
Little-endian 78 56 34 12

Endianness is about the order of bytes, not bits: the bits inside each byte are not reversed. A single byte (a char, a UTF-8 byte stream) has no endianness problem.

#include <stdio.h>
#include <arpa/inet.h>

int main(void) {
    unsigned int x = 0x12345678;
    unsigned char *p = (unsigned char *)&x;   // look at it byte by byte
    printf("x = 0x%X\n", x);
    printf("in memory: %02X %02X %02X %02X\n", p[0], p[1], p[2], p[3]);

    unsigned int n = htonl(x);                // host to network byte order
    p = (unsigned char *)&n;
    printf("network:   %02X %02X %02X %02X\n", p[0], p[1], p[2], p[3]);
    return 0;
}

Output:

x = 0x12345678
in memory: 78 56 34 12
network:   12 34 56 78
  • unsigned char *p = (unsigned char *)&x points at the first (lowest-address) byte of x, so p[0]..p[3] read its bytes in memory order.
  • 78 56 34 12: the smallest byte sits first. This machine is little-endian.
  • Two machines talking over a network must agree. The internet's rule is big-endian, called network byte order. The machine's own order is the host byte order.

htonl, htons, ntohl, ntohs

From <arpa/inet.h> (h = host, n = network, l = long 32-bit, s = short 16-bit):

Function Converts Typical use
htonl(x) host → network, 32-bit IPv4 address before sending
htons(x) host → network, 16-bit port number before sending
ntohl(x) network → host, 32-bit IPv4 address after receiving
ntohs(x) network → host, 16-bit port number after receiving

On a big-endian machine they do nothing; on a little-endian machine they swap the bytes.

#include <stdio.h>
#include <arpa/inet.h>

int main(void) {
    unsigned short port = 8080;
    unsigned short wire = htons(port);           // host to network short
    printf("port         = %u (0x%04X)\n", port, port);
    printf("htons(port)  = %u (0x%04X)\n", wire, wire);
    printf("ntohs(wire)  = %u\n", ntohs(wire));  // network to host short
    unsigned int ip = 0xC0A80001;                // 192.168.0.1
    printf("htonl(0x%08X) = 0x%08X\n", ip, htonl(ip));
    printf("ntohl back       = 0x%08X\n", ntohl(htonl(ip)));
    return 0;
}

Output:

port         = 8080 (0x1F90)
htons(port)  = 36895 (0x901F)
ntohs(wire)  = 8080
htonl(0xC0A80001) = 0x0100A8C0
ntohl back       = 0xC0A80001

If you forget htons, the other side reads the bytes 90 1F as port 36895 instead of 8080: a completely different number.

Worked example: 0xA1B2C3D4, byte by byte

Address 0x100 0x101 0x102 0x103
Big-endian A1 B2 C3 D4
Little-endian D4 C3 B2 A1

Endianness also shows up in UTF-16 and UTF-32 files (20 b9 vs b9 20 above). Such files often begin with a BOM (byte order mark), the character U+FEFF, whose bytes (FE FF or FF FE) tell the reader which order was used.

The whole chain: you invented it

  1. ₹3.75 → fixed point (count paise).
  2. Tiny + huge → scientific notation → floating point, IEEE 754 (sign, biased exponent, mantissa with a hidden bit).
  3. Zero, 1 ÷ 0, 0 ÷ 0 → ±0, ±infinity, NaN, denormals.
  4. 0.1 + 0.2 ≠ 0.3 → rounding error → tolerance, integers, decimal types.
  5. 'A' → ASCII.
  6. नमस्ते → extended ASCII fails (mojibake) → Unicode code points → UTF-8, UTF-16.
  7. Which byte first → big- and little-endian, network byte order.

Common mistakes (and the real output)

Mistake Code Real output Fix
Comparing floats with == print(0.1 + 0.2 == 0.3) False abs(a - b) < 1e-9 or math.isclose(a, b)
Adding 0.1 again and again in a float t = t + 0.1f; 36,000 times after 1 hour: 3601.162354 seconds Count ticks as an integer; use double
Testing for NaN with == x == NAN 0 (never true) isnan(x), or x != x
Comparing a float with a double literal float f = 0.1f; f == 0.1 0 Compare with 0.1f, or use double throughout
Integer division before storing in a double double half = 1 / 2; 0.000000 Write 1.0 / 2
Using a digit character as its value '7' + 1 56 '7' - '0' gives 7
Expecting float to hold every integer 16777216.0f + 1.0f (2²⁴ + 1) 16777216.0 Floats hold every integer only up to 2²⁴; use int, long or double
Expecting Python float ÷ 0 to give inf 1 / 0.0 ZeroDivisionError: float division by zero Python raises instead of returning inf; check the divisor
Summing 0.1 ten times in Python loop total += 0.1 0.9999999999999999 False math.isclose(total, 1.0) → True
Expecting round(2.675, 2) = 2.68 round(2.675, 2) 2.67 2.675 is stored as 2.67499...; use Decimal("2.675")
Opening UTF-8 bytes with the wrong code page data.decode("cp1252", errors="replace") नमस�ते Always say encoding="utf-8"
Decoding non-ASCII bytes as ASCII data.decode("ascii") UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128) Decode with the encoding that wrote the bytes
Encoding Hindi as ASCII "नमस्ते".encode("ascii") UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128) Use UTF-8
Assuming 1 character = 1 byte len("नमस्ते") vs len("नमस्ते".encode("utf-8")) 6 18 Count bytes when sizing buffers
Forgetting htons on a port send 8080 raw from a little-endian machine read back as 36895 htons(port) before sending

Notes on three rows:

  • f == 0.1 is false because f is 0.1 rounded to 24 bits and 0.1 is 0.1 rounded to 53 bits; f is widened to double for the comparison, but the lost bits do not come back. f == 0.1f printed 1.
  • 2²⁴ + 1 = 16,777,217 needs 25 significant bits, one more than a float has, so it rounds back to 16,777,216.
  • -0.0 == 0.0 printed 1: the two zeros have different bits but compare equal.

Mini project: an IEEE 754 float decoder

Give it 32 bits; it splits them into sign, exponent and mantissa, says which kind of value it is, and computes the value with the right formula (hidden bit 1 for normal numbers, 0 for denormals).

# IEEE 754 single-precision decoder: bits -> sign, exponent, mantissa, value
import struct

def decode(bits):
    bits = bits.replace(" ", "")
    s, e, m = int(bits[0]), int(bits[1:9], 2), int(bits[9:], 2)
    sign = -1 if s else 1
    if e == 255:                                   # exponent all 1s
        kind = "NaN" if m else "infinity"
        value = float("nan") if m else sign * float("inf")
    elif e == 0:                                   # exponent all 0s
        kind = "zero" if m == 0 else "denormal"
        value = sign * (m / 2**23) * 2**-126       # hidden bit is 0
    else:
        kind = "normal"
        value = sign * (1 + m / 2**23) * 2**(e - 127)   # hidden bit is 1
    return s, e, m, kind, value

def float_to_bits(x):
    return f"{struct.unpack('>I', struct.pack('>f', x))[0]:032b}"

tests = ["0 10000010 01000000000000000000000",      # the quiz
         float_to_bits(-6.5), float_to_bits(0.15625), float_to_bits(0.1),
         float_to_bits(1.0), float_to_bits(-0.0),
         "0 00000000 00000000000000000000000",      # +0
         "0 00000000 00000000000000000000001",      # smallest denormal
         "0 00000001 00000000000000000000000",      # smallest normal
         "0 11111110 11111111111111111111111",      # largest float
         "1 11111111 00000000000000000000000",      # -infinity
         "0 11111111 10000000000000000000000"]      # NaN

print(f"{'hex':<10}{'sign':<5}{'exp':>4} {'real e':>7}  {'kind':<9} value")
for t in tests:
    s, e, m, kind, v = decode(t)
    real = "-" if e in (0, 255) else e - 127
    hx = f"{int(t.replace(' ', ''), 2):08X}"
    print(f"{hx:<10}{s:<5}{e:>4} {real:>7}  {kind:<9} {v:.10g}")

Output:

hex       sign  exp  real e  kind      value
41200000  0     130       3  normal    10
C0D00000  1     129       2  normal    -6.5
3E200000  0     124      -3  normal    0.15625
3DCCCCCD  0     123      -4  normal    0.1000000015
3F800000  0     127       0  normal    1
80000000  1       0       -  zero      -0
00000000  0       0       -  zero      0
00000001  0       0       -  denormal  1.401298464e-45
00800000  0       1    -126  normal    1.175494351e-38
7F7FFFFF  0     254     127  normal    3.402823466e+38
FF800000  1     255       -  infinity  -inf
7FC00000  0     255       -  NaN       nan
  • m / 2**23 turns the 23 mantissa bits into the fraction after the binary point.
  • The hex column matches the hand-worked answers (-6.5 = C0D00000, 0.15625 = 3E200000, the quiz's 10.0 = 41200000), and 0.1 is 3DCCCCCD: the repeating bits show up as repeating hex Cs, rounded up at the end.
  • The rows match the <float.h> limits: smallest denormal 1.4013 × 10⁻⁴⁵, smallest normal 1.17549 × 10⁻³⁸, largest 3.40282 × 10³⁸.
  • Try extending it: accept a hex string like C0D00000, or decode 64-bit doubles (1 / 11 / 52 bits, bias 1023).

Practice

  1. Convert 12.375 to IEEE 754 single precision (bits and hex).
  2. Convert -0.75 to IEEE 754 single precision.
  3. Decode the float 0xC1480000.
  4. Decode 0 01111110 10000000000000000000000.
  5. What is the exponent bias of a format with a 5-bit exponent? With an 11-bit exponent?
  6. Write the smallest positive normal and the smallest positive denormal float as powers of 2.
  7. What is the largest finite float, as a formula?
  8. Why does a float have "about 7" decimal digits and a double "about 16"?
  9. How many different bit patterns are NaN in single precision?
  10. Which of these are stored exactly in a float: 0.5, 0.1, 0.25, 0.2, 0.375?
  11. Encode U+00F1 (ñ) and U+0905 (अ) in UTF-8 by hand.
  12. Encode U+1F600 (😀) in UTF-8 by hand.
  13. Decode the UTF-8 bytes E0 A4 95.
  14. How many bytes is "Hello₹" in UTF-8, UTF-16 and UTF-32 (no BOM)?
  15. Store 0xA1B2C3D4 at address 0x200 in little-endian. What byte is at 0x202?
  16. On a little-endian machine, what is htons(0x1F90)?
  17. Convert 'a' to uppercase and 'Q' to lowercase using a single bit operation. What is 'Z' - 'A'?
  18. What does a float store when you assign it 16777217?
Solutions
  1. 12.375 = 1100.011₂ = 1.100011 × 2³. Sign 0, exponent 3 + 127 = 130 = 10000010, mantissa 100011 + zeros → 0 10000010 10001100000000000000000 = 0x41460000.
  2. 0.75 = 0.11₂ = 1.1 × 2⁻¹. Sign 1, exponent 126 = 01111110, mantissa 1 + zeros → 1 01111110 10000000000000000000000 = 0xBF400000.
  3. C1480000 = 1100 0001 0100 1000 0... → sign 1, exponent 10000010 = 130 → 3, mantissa 1001 → 1.1001₂ = 1.5625. Value = -1.5625 × 8 = -12.5.
  4. Sign 0, exponent 126 → -1, mantissa 1.1₂ = 1.5. Value = 1.5 × 0.5 = 0.75 (0x3F400000).
  5. Bias = 2ᵏ⁻¹ - 1: 5 bits → 15 (the half-precision format), 11 bits → 1023.
  6. Smallest normal: 1.0 × 2⁻¹²⁶ ≈ 1.17549 × 10⁻³⁸. Smallest denormal: 2⁻²³ × 2⁻¹²⁶ = 2⁻¹⁴⁹ ≈ 1.4013 × 10⁻⁴⁵.
  7. Exponent 254 → 127, mantissa all 1s → (2 - 2⁻²³) × 2¹²⁷ ≈ 3.40282 × 10³⁸.
  8. Precision is 24 bits (23 + hidden bit) and 24 × log₁₀2 ≈ 7.22 digits; double has 53 bits, 53 × log₁₀2 ≈ 15.95.
  9. Exponent all 1s (fixed), sign 0 or 1, mantissa any non-zero 23-bit value: 2 × (2²³ - 1) = 16,777,214.
  10. Exact: 0.5 (1/2), 0.25 (1/4), 0.375 (3/8). Not exact: 0.1, 0.2 (denominators with a factor 5).
  11. U+00F1 = 241 = 00011 110001 → 110 00011 10 110001 = C3 B1. U+0905 = 0000 100100 000101 → 1110 0000 10 100100 10 000101 = E0 A4 85.
  12. U+1F600 = 000 011111 011000 000000 (21 bits) → 11110 000 10 011111 10 011000 10 000000 = F0 9F 98 80.
  13. 1110 0000, 10 100100, 10 010101 → bits 0000 100100 010101 = 0x0915 = U+0915, क.
  14. UTF-8: 5 × 1 + 3 = 8. UTF-16: 6 × 2 = 12. UTF-32: 6 × 4 = 24.
  15. Little-endian: 0x200 = D4, 0x201 = C3, 0x202 = B2, 0x203 = A1.
  16. Bytes swapped: 0x901F (36895), as the htons run above printed.
  17. 'a' & 0xDF = 'A' (clear bit 5, value 32); 'Q' | 0x20 = 'q' (set bit 5); 'Z' - 'A' = 25.
  18. 16777216.0: 16,777,217 needs 25 significant bits; it rounds to the nearest float.

Every answer above was checked by running it in Python 3.13 (struct packing, encode, to_bytes).

Quick quiz

A 32-bit IEEE 754 float holds 0 10000010 01000000000000000000000. What number is it?

  • A. 5.0
  • B. 10.0
  • C. 2.5
  • D. 130.25
Answer

B, 10.0. Sign 0 → positive. Exponent 10000010 = 130, minus the bias 127 = 3. Mantissa with the hidden bit: 1.01₂ = 1.25. Value = 1.25 × 2³ = 10.0 (0x41200000; the decoder above prints it as the first row).

Interview questions

  • Fixed point vs floating point? Fixed point stores an integer with an agreed position for the point (same precision everywhere, exact for money). Floating point stores sign, exponent and mantissa, so the point moves; huge range, but most decimals are approximate.
  • Layout of IEEE 754 single and double? Single: 1 / 8 / 23 bits, bias 127. Double: 1 / 11 / 52 bits, bias 1023.
  • Why a biased exponent instead of 2's complement? Stored exponents are never negative, and positive floats then compare like unsigned integers.
  • What is the hidden bit? The leading 1 of a normalized mantissa, which is always 1 and therefore not stored; it gives 24 (53) bits of precision from 23 (52) stored bits.
  • What are denormalized numbers for? Values below the smallest normal number, with a hidden bit of 0, giving gradual underflow instead of a sudden jump to zero.
  • How are 0, infinity and NaN encoded? Exponent all 0s and mantissa 0 → ±0; exponent all 1s and mantissa 0 → ±inf; exponent all 1s and mantissa ≠ 0 → NaN.
  • Why is 0.1 + 0.2 != 0.3? 0.1, 0.2 and 0.3 repeat forever in binary, so each is rounded; the rounded sum is a different double from the rounded 0.3.
  • How do you compare floats safely? abs(a - b) <= tolerance, preferably relative to the size of the numbers (math.isclose).
  • How do you check for NaN? isnan(x), or x != x. x == NAN is always false.
  • How should money be stored? As integers in the smallest unit (paise), or a decimal type such as BigDecimal; never binary float.
  • ASCII vs Unicode vs UTF-8? ASCII: 128 characters, 7 bits. Unicode: one code point for every character. UTF-8: a variable-length (1 to 4 bytes) encoding of Unicode code points, ASCII-compatible.
  • UTF-8 vs UTF-16? UTF-8 uses 1-4 bytes and has no byte-order issue; UTF-16 uses 2 or 4 bytes (surrogate pairs) and comes in big- and little-endian forms.
  • What is mojibake? Garbled text from decoding bytes with a different encoding than the one that wrote them.
  • Big-endian vs little-endian? How do you detect it in C? Big: most significant byte at the lowest address; little: least significant first. Store 1 in an int and read its first byte through an unsigned char *: 1 means little-endian.
  • What is network byte order and why is htons needed? Big-endian, agreed for network protocols; htons/htonl convert from host order before sending, ntohs/ntohl after receiving.

Glossary

Term Plain meaning Example
Bit / byte One 0 or 1 / a group of 8 bits 01000001 is one byte
MSB / LSB Most / least significant bit (or byte): the one with the biggest / smallest place value In 0x12345678, MSB byte 12, LSB byte 78
Hexadecimal Base 16; each digit stands for 4 bits 0xC0D00000
Binary fraction Bits after the binary point worth 1/2, 1/4, 1/8 ... 0.101₂ = 0.625
Fixed-point representation An integer with an agreed, fixed position for the point 375 paise = ₹3.75
Scientific notation Significant digits × a power of the base 1.496 × 10⁸
Floating-point representation Store sign, exponent and mantissa, so the point can move 5.75 = +1.0111₂ × 2²
Sign field 1 bit: 0 plus, 1 minus -5.75 has sign 1
Exponent field Bits storing the power of 2 (with the bias added) 10000001 = 129 → 2
Mantissa / significand The significant binary digits of the number 1.0111 for 5.75
IEEE 754 The standard for floating point that almost every CPU follows float, double
Single precision IEEE 754 32-bit format: 1 / 8 / 23, bias 127 C float
Double precision IEEE 754 64-bit format: 1 / 11 / 52, bias 1023 C double, Python float
Precision How many correct significant digits a type carries float ≈ 7, double ≈ 15-16
Exponent bias A constant added to the real exponent so the stored value is never negative 2 + 127 = 129
Normalized number Mantissa written as 1.xxx; exponent bits not all 0s or all 1s 1.101₂ × 2²
Hidden bit The leading 1 of a normalized mantissa, not stored 23 stored bits → 24 bits precision
Denormalized (subnormal) number Exponent all 0s, hidden bit 0: values below the smallest normal FLT_MIN / 2 = 5.87747e-39
Underflow A result too small to store as a normal number 2⁻¹⁵⁰ in a float
Gradual underflow Denormals let values fade to zero step by step 2⁻¹²⁶ → 2⁻¹²⁷ → ... → 2⁻¹⁴⁹
Overflow (floating point) A result too large for the type; becomes infinity 1e308 × 10 = inf
Positive / negative zero Zero with sign bit 0 / 1; equal when compared 0x80000000 = -0.0
Infinity Exponent all 1s, mantissa 0; result of overflow or x ÷ 0 1.0 / 0.0 = inf
NaN Not a Number: exponent all 1s, mantissa ≠ 0; not equal to anything, even itself 0.0 / 0.0
Special values Zero, infinity and NaN, encoded with the extreme exponent patterns see the table in Problem 3
Precision loss Digits beyond the mantissa's capacity are lost 2²⁴ + 1 becomes 2²⁴ in a float
Rounding error The difference between the true value and the stored value 0.1 stored as 0.1000000000000000055...
0.1 + 0.2 != 0.3 Both inputs and 0.3 are rounded; the sums differ in the last bit 0.30000000000000004
Machine epsilon Gap between 1.0 and the next representable number DBL_EPSILON = 2.22045e-16
Tolerance (epsilon in code) The largest difference you accept as "equal" abs(a - b) < 1e-9
Decimal type A number type that stores decimal digits exactly Python Decimal, Java BigDecimal
Character set A chart giving every character a number ASCII, Unicode
Character encoding The rule that turns character numbers into bytes UTF-8
ASCII 7-bit character set of 128 characters 'A' = 65, '0' = 48
Control code An invisible ASCII command character newline = 10, tab = 9
Extended ASCII 8-bit sets that add 128 characters (128-255) on top of ASCII Windows-1252
Code page One particular extended-ASCII chart 0xE9 = é in cp1252, Θ in cp437
Mojibake Garbled text from decoding with the wrong encoding नम...
Replacement character � (U+FFFD), shown for bytes that can't be decoded byte 0x8D in cp1252
Unicode One universal character set for every script 159,801 characters in Unicode 17.0
Code point A character's number in Unicode, written U+hex न = U+0928
UTF-32 Fixed 4 bytes per code point ₹ = 00 00 20 b9
Variable-length encoding Different characters take different numbers of bytes UTF-8: 1 to 4 bytes
UTF-8 Variable-length, 1-4 bytes, ASCII-compatible encoding of Unicode ₹ = e2 82 b9
Continuation byte A UTF-8 follow-up byte, always starting with 10 82, b9 in ₹
UTF-16 2 bytes per code point up to U+FFFF, 4 bytes (surrogate pair) above न = 09 28
Surrogate pair Two 16-bit units (D800-DBFF, DC00-DFFF) for one code point above U+FFFF 😀 = D83D DE00
BOM Byte order mark: U+FEFF at the start of a file, shows the byte order FE FF = big-endian UTF-16
Memory address The position number of a byte in memory 0x100, 0x101 ...
Endianness The order in which a multi-byte value's bytes sit in memory big or little
Big-endian Most significant byte at the lowest address 12 34 56 78
Little-endian Least significant byte at the lowest address (x86-64) 78 56 34 12
Host byte order The byte order of the machine you are on little-endian on x86-64
Network byte order Big-endian, the agreed order on the network port 8080 sent as 1F 90
htonl / htons Host to network, 32-bit / 16-bit htons(8080) = 0x901F on little-endian
ntohl / ntohs Network to host, 32-bit / 16-bit ntohs(0x901F) = 8080
struct (Python) Module that packs numbers into raw bytes and back struct.pack('>f', 5.75)
<float.h> C header with the limits of float and double FLT_MAX, DBL_EPSILON

Summary

  • Fixed point stores an integer with an agreed point: exact for money, but one precision can't cover tiny and huge values.
  • Floating point stores sign × 1.mantissa × 2^(exponent − bias). IEEE 754 float: 1 / 8 / 23, bias 127, ≈ 7 digits, up to ≈ 3.4 × 10³⁸. double: 1 / 11 / 52, bias 1023, ≈ 15-16 digits, up to ≈ 1.8 × 10³⁰⁸.
  • Normalized numbers have a hidden bit 1. Exponent all 0s → zero or denormals (gradual underflow); all 1s → infinity or NaN (NaN != NaN).
  • 0.1 repeats in binary, so it is rounded: 0.1 + 0.2 != 0.3. Errors accumulate (the float clock was 1.16 s off after an hour; the Patriot's was 0.3433 s after 100 hours). Compare with a tolerance; count money and ticks as integers.
  • ASCII: 7 bits, 'A' = 65, 'a' = 97, '0' = 48. Extended ASCII code pages disagree → mojibake.
  • Unicode gives every character a code point; UTF-8 encodes it in 1-4 bytes and is ASCII-compatible; UTF-16 uses 2 or 4 bytes (surrogate pairs); UTF-32 uses 4.
  • Big-endian puts the most significant byte first, little-endian the least. The network uses big-endian (network byte order): htons/htonl before sending, ntohs/ntohl after receiving.

Next episode: Logic gates, Boolean algebra and digital circuits. AND, OR, NOT, NAND, Boolean algebra, K-maps, adders, multiplexers and flip-flops: how wires and switches actually add two binary numbers.