SyntaxPilot
Courses/Data Analytics with Python/Lesson 8

Data Analytics with Python · Lesson 8 of 90 · 19 min

NumPy 1: arrays and vectorised maths

NumPyndarraynp.arraydtypeshapendimsizenp.arangenp.linspacenp.zerosnp.onesnp.fullvectorisationelement-wise operation

These are the notes for Data Analytics with Python, episode 8. Python lists are great at holding things, but they can't do maths on a column of numbers, and looping over a million values is slow. Here we invent the fix: NumPy and its ndarray. You'll learn vectorisation and element-wise operations, the array's dtype, shape, ndim and size, quick builders (np.arange, np.linspace, np.zeros, np.ones, np.full), universal functions (ufuncs), and exactly why arrays beat lists. Every term is defined the first time it appears and collected in the Glossary. Every program below was run for real (Python 3.13, NumPy 2.x); outputs are unedited. Timings change from machine to machine and run to run.

The problem: lists can't do maths

Ramesh's menu is a list: chai Rs 12, samosa Rs 15, bun maska Rs 25. Milk got costlier, so he wants to raise every price by 10%, which means multiplying each one by 1.1. On paper you'd write prices * 1.1. Python disagrees.

prices = [12, 15, 25]
print(prices * 2)
print(prices + 5)
[12, 15, 25, 12, 15, 25]
Traceback (most recent call last):
  File "main.py", line 3, in <module>
    print(prices + 5)
          ~~~~~~~^~~
TypeError: can only concatenate list (not "int") to list
  • For a list, * means repeat the list, not multiply the numbers.
  • For a list, + means concatenate: join two lists end to end. A number isn't a list, so it fails.
prices = [12, 15, 25]
print(prices * 1.1)
Traceback (most recent call last):
  File "main.py", line 2, in <module>
    print(prices * 1.1)
          ~~~~~~~^~~~~
TypeError: can't multiply sequence by non-int of type 'float'

A sequence is an ordered collection like a list or a string. You can repeat a list 2 times, but not 1.1 times. Lists were built to hold things, not to do maths.

The workaround from episode 6 is a comprehension, one price at a time:

prices = [12, 15, 25]
new = [p * 1.1 for p in prices]
print(new)
[13.200000000000001, 16.5, 27.500000000000004]

It works. The long tails like 13.200000000000001 are normal: computers store most decimals as a very close floating-point approximation. But Python walks the list one item at a time, which is slow for a million values.

NumPy and the ndarray

Real-life example: a list is a cloth jhola bag — anything goes in (eggs, keys, a phone), but to wash every egg you take them out one by one. An array is an egg tray — same-sized slots side by side, and you can wash the whole tray in one go.

  • A library is a package of ready-made code you bring in with import.
  • NumPy (Numerical Python) is the library for fast maths on arrays of numbers.
  • Its main object is the ndarray ("N-dimensional array"): a grid of values that all have the same type.
  • np.array(...) turns a list (or a list of lists) into an ndarray.
  • import numpy as np: as gives the library a short nickname; np is the name everyone uses.
import numpy as np
prices = np.array([12, 15, 25])
print(prices)
print(type(prices))
print(prices * 2)
print(prices + 5)
print(prices * 1.1)
[12 15 25]
<class 'numpy.ndarray'>
[24 30 50]
[17 20 30]
[13.2 16.5 27.5]

An array prints without commas. Now * 2 doubles every price and + 5 adds 5 to every price. The 10% rise is one line, with no loop. (The tiny decimal error is still stored; NumPy just prints fewer digits.)

Vectorisation and element-wise operations

  • Vectorisation: writing one operation for the whole array, with no loop written by you. NumPy runs the loop internally.
  • Element-wise: the operation is applied to each element (one value inside the array) on its own: [12, 15, 25] * 2 → [24, 30, 50] means 12×2, 15×2, 25×2.

Loop vs NumPy, timed

time.perf_counter() is a stopwatch: read it before and after, and subtract. 1_000_000 is one million; underscores in a number are only for readability.

import time
import numpy as np
bills = list(range(1_000_000))
start = time.perf_counter()
new = [b * 1.1 for b in bills]
list_time = time.perf_counter() - start
arr = np.array(bills)
start = time.perf_counter()
new_arr = arr * 1.1
np_time = time.perf_counter() - start
print(f"list loop: {list_time:.4f} s")
print(f"NumPy:     {np_time:.4f} s")
print(f"NumPy is about {list_time / np_time:.0f}x faster")

One real run:

list loop: 0.0857 s
NumPy:     0.0055 s
NumPy is about 15x faster

Two more runs on the same machine gave 22x and 16x. Your numbers will differ (they depend on your computer and change slightly every run), but NumPy wins every time, and the gap grows with heavier maths.

Why arrays beat lists

Real life NumPy idea What it means
Train coaches coupled together Contiguous memory The numbers sit side by side in one unbroken block of memory. A list stores each value as a separate Python object somewhere in memory and only keeps a pointer (an address) to each one.
Every tray slot fits one egg One dtype Every element has the same type, so the computer never checks "is this a number or text?" per item. A list must check every item.
A machine vs doing it by hand Loops in C The loop still happens, inside NumPy, in code written in C, a compiled language (translated to machine code ahead of time), which runs far faster than Python stepping through items.

You can see the tidy storage:

import numpy as np
week = np.array([320, 280, 410, 300, 350, 390, 450])
print(week.itemsize, "bytes per number")
print(week.nbytes, "bytes in total")
mixed = [320, "N/A", 4.5, True]
print([type(x).__name__ for x in mixed])
8 bytes per number
56 bytes in total
['int', 'str', 'float', 'bool']
  • A byte is a small unit of computer memory (8 bits; enough for one English letter). A bit is a single 0 or 1.
  • itemsize is the bytes per element; nbytes the bytes for the whole array: 7 × 8 = 56, nothing wasted.
  • A list happily mixes an int, a str, a float and a bool. Flexible, but that flexibility is what makes it slow for maths.

dtype: the array's one data type

An attribute is a value stored on an object, read with a dot and no brackets: a.dtype, a.shape. (A method is a function on an object, called with a dot and brackets: a.sum().)

The dtype (data type) is the one type every element shares.

import numpy as np
a = np.array([320, 280, 410])
b = np.array([1, 2.5])
c = np.array([320, "N/A", 410])
print(a.dtype)
print(b, b.dtype)
print(c, c.dtype)
int64
[1.  2.5] float64
['320' 'N/A' '410'] <U21
  • int64: whole numbers stored in 64 bits (8 bytes) — that's where the 8 bytes came from.
  • float64: decimal (floating-point) numbers in 64 bits. Mixing 1 and 2.5 upcasts (upgrades) everything to float64, so 1 prints as 1..
  • <U21: Unicode text, up to 21 characters each. Unicode is the standard that gives every character in every language a number. One "N/A" turned the whole array into text: 320 became the string '320'.

Maths on that text array fails:

import numpy as np
c = np.array([320, "N/A", 410])
print(c + 5)
Traceback (most recent call last):
  File "main.py", line 3, in <module>
    print(c + 5)
          ~~^~~
numpy._core._exceptions._UFuncNoLoopError: ufunc 'add' did not contain a loop with signature matching types (dtype('<U21'), dtype('int64')) -> None

In plain words: NumPy has no routine for adding a number to text. Always check .dtype after loading data: text where you expected numbers is the first sign of dirty data (cleaned properly in episodes 19-21).

You can also choose the dtype yourself:

import numpy as np
print(np.array([True, 2, 3.5]).dtype, np.array([1,2,3], dtype=float))
print(np.array([10, 20]).astype(float))
float64 [1. 2. 3.]
[10. 20.]

dtype=float sets the type when creating; .astype(float) makes a converted copy of an existing array.

shape, ndim and size

Ramesh has a second stall in Bandra. Two separate arrays would drift apart (like the parallel lists in episode 5). Better: one 2-D (two-dimensional) array — one row per stall, one column per day. A dimension (or axis) is a direction you can move along: along a row, or down a column.

  • shape: how many elements along each dimension, as a tuple. (7,) = 7 in one dimension; (2, 7) = 2 rows × 7 columns.
  • ndim: the number of dimensions.
  • size: the total number of elements.
import numpy as np
week = np.array([320, 280, 410, 300, 350, 390, 450])
print(week.shape, week.ndim, week.size)
stalls = np.array([[320, 280, 410, 300, 350, 390, 450],
                   [210, 190, 260, 240, 230, 280, 300]])
print(stalls)
print(stalls.shape, stalls.ndim, stalls.size)
print(stalls.dtype)
(7,) 1 7
[[320 280 410 300 350 390 450]
 [210 190 260 240 230 280 300]]
(2, 7) 2 14
int64

(7,) is a tuple with one number (the comma makes it a tuple). A 2-D array is built from a list of lists, one inner list per row. Every row must be the same length:

import numpy as np
bad = np.array([[320, 280, 410],
                [210, 190]])
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

Inhomogeneous means "not all the same": the rows have different lengths.

Building arrays fast: arange, linspace, zeros, ones, full

Real-life examples: token numbers at a hospital counter (start, stop, step) → np.arange; the marks on a ruler, evenly spaced with both ends included → np.linspace; a blank register page, or one pre-filled with a target → np.zeros, np.ones, np.full.

import numpy as np
print(np.arange(5))
print(np.arange(1, 8))
print(np.arange(0, 50, 10))
print(np.arange(0, 1, 0.25))
print(np.linspace(0, 1, 5))
print(np.linspace(10, 20, 3))
[0 1 2 3 4]
[1 2 3 4 5 6 7]
[ 0 10 20 30 40]
[0.   0.25 0.5  0.75]
[0.   0.25 0.5  0.75 1.  ]
[10. 15. 20.]
  • np.arange(start, stop, step): like range(), but returns an array, and accepts decimal steps. The stop is excluded.
  • np.linspace(start, stop, num) ("linear space"): num evenly spaced points, both ends included, always float.
  • Rule of thumb: with arange you choose the step; with linspace you choose how many points. For decimal steps prefer linspace, since floating-point steps in arange can give one point more or fewer than you expect.
import numpy as np
print(np.zeros(4))
print(np.ones(3, dtype=int))
print(np.full(7, 300))
print(np.zeros((2, 3)))
target = np.full(7, 300)
week = np.array([320, 280, 410, 300, 350, 390, 450])
print(week - target)
[0. 0. 0. 0.]
[1 1 1]
[300 300 300 300 300 300 300]
[[0. 0. 0.]
 [0. 0. 0.]]
[ 20 -20 110   0  50  90 150]
  • np.zeros(shape): all 0, float64 by default. np.ones(shape): all 1. np.full(shape, value): all value.
  • shape is a number for 1-D, or a tuple like (2, 3) for 2-D.
  • week - target shows how far each day was from the 300-cup target, element by element.

Element-wise maths between two arrays

import numpy as np
cups = np.array([320, 280, 410])
price = np.array([12, 12, 15])
print(cups * price)
print(cups + cups)
print(cups > 300)
[3840 3360 6150]
[640 560 820]
[ True False  True]

Elements pair up by position: first with first, second with second. Revenue (money taken in) per day is cups * price. A comparison gives a boolean array of True/False — the basis of filtering in episode 9.

Shapes must match:

import numpy as np
cups = np.array([320, 280, 410])
price = np.array([12, 12, 15, 15])
print(cups * price)
Traceback (most recent call last):
  File "main.py", line 4, in <module>
    print(cups * price)
          ~~~~~^~~~~~~
ValueError: operands could not be broadcast together with shapes (3,) (4,) 

Operands are the values an operator works on (cups and price). The fourth price has no partner. Broadcasting is NumPy's rule for stretching a smaller array (like a single number) across a bigger one; that's why prices * 2 works. Its full rules come in episode 10.

Universal functions (ufuncs)

Real-life example: a calculator's √ button works on one number; imagine a √ button that works on a whole column at once. That's a universal function (ufunc): a NumPy function that runs element-wise over an entire array, in fast C. Even + and * on arrays call ufuncs (np.add, np.multiply).

import numpy as np
area = np.array([16.0, 25.0, 36.0])
print(np.sqrt(area))
growth = np.array([1.1234, 0.987, 1.05])
print(np.round(growth, 2))
print(np.maximum(np.array([320, 280, 410]), 300))
print(np.abs(np.array([20, -20, 110])))
print(type(np.sqrt))
[4. 5. 6.]
[1.12 0.99 1.05]
[320 300 410]
[ 20  20 110]
<class 'numpy.ufunc'>
Function Does Example result
np.sqrt square root of each element [4. 5. 6.]
np.round(a, 2) round each element to 2 decimals [1.12 0.99 1.05]
np.maximum(a, 300) the bigger of each element and 300 (a floor: a minimum allowed value) [320 300 410]
np.abs absolute value (drops the minus sign) [ 20 20 110]
np.add, np.multiply what + and * call

Python's math.sqrt takes one number only, so it fails on an array:

TypeError: only 0-dimensional arrays can be converted to Python scalars

With arrays, use the np. version.

Mini project: two stalls' weekly revenue

import numpy as np
cups = np.array([[320, 280, 410, 300, 350, 390, 450],
                 [210, 190, 260, 240, 230, 280, 300]])
price = np.full((2, 7), 12)
revenue = cups * price
print(revenue)
new_price = np.round(price * 1.1)
print(new_price[0][:3])
new_revenue = cups * new_price
print("Old total:", revenue.sum())
print("New total:", new_revenue.sum())
print("Extra:", new_revenue.sum() - revenue.sum())
[[3840 3360 4920 3600 4200 4680 5400]
 [2520 2280 3120 2880 2760 3360 3600]]
[13. 13. 13.]
Old total: 50520
New total: 54730.0
Extra: 4210.0
  • 14 revenues at once, no loop. 12 * 1.1 = 13.2 rounds to 13.
  • .sum() is a method that adds up every element.
  • 54730.0 has .0 because np.round returned float64 — the dtype lesson, live. (new_price[0][:3] peeks at the first 3 values of row 0; indexing 2-D arrays properly is episode 9.)
  • About Rs 4,210 extra a week, if sales stay the same after the price rise — a real analyst would check that next.

Common mistakes

Code Real error Why Fix
[12, 15, 25] + 5 TypeError: can only concatenate list (not "int") to list For lists, + joins lists np.array([12, 15, 25]) + 5
[12, 15, 25] * 1.1 TypeError: can't multiply sequence by non-int of type 'float' For lists, * repeats np.array(prices) * 1.1
[12, 15, 25] * 2 expecting doubling (no error: [12, 15, 25, 12, 15, 25]) List repetition Use an array
np.array([320, "N/A", 410]) + 5 numpy._core._exceptions._UFuncNoLoopError: ufunc 'add' did not contain a loop with signature matching types (dtype('<U21'), dtype('int64')) -> None One string made the dtype text Check .dtype; clean the data first
cups (3,) * price (4,) ValueError: operands could not be broadcast together with shapes (3,) (4,) Element-wise needs matching shapes Make the lengths match
Rows of different lengths ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. ... Every row must be the same length Fix the data, or fill gaps
np.array(1, 2, 3) TypeError: array() takes from 1 to 2 positional arguments but 3 were given np.array takes one list np.array([1, 2, 3])
np.zeros(2, 3) TypeError: Cannot interpret '3' as a data type The shape must be one tuple np.zeros((2, 3))
math.sqrt(array) TypeError: only 0-dimensional arrays can be converted to Python scalars math works on one number np.sqrt(array)

Practice

1. Put a week of screen time in hours, [3.5, 5.2, 4.1, 2.0, 6.3, 4.8, 3.0], in an array and convert it to minutes in one line.

Solution
import numpy as np
screen = np.array([3.5, 5.2, 4.1, 2.0, 6.3, 4.8, 3.0])
print(screen * 60)
[210. 312. 246. 120. 378. 288. 180.]

2. Build 0 to 100 in steps of 5 with arange, and 11 evenly spaced points from 0 to 100 with linspace.

Solution
print(np.arange(0, 101, 5))
print(np.linspace(0, 100, 11))
[  0   5  10  15  20  25  30  35  40  45  50  55  60  65  70  75  80  85
  90  95 100]
[  0.  10.  20.  30.  40.  50.  60.  70.  80.  90. 100.]

The stop of arange is excluded, so use 101 to include 100. Long arrays wrap onto a second line when printed.

3. Time a list comprehension against NumPy for squaring 1,000,000 numbers.

Solution
import time
import numpy as np
nums = list(range(1_000_000))
start = time.perf_counter()
sq = [n ** 2 for n in nums]
t1 = time.perf_counter() - start
arr = np.array(nums)
start = time.perf_counter()
sq2 = arr ** 2
t2 = time.perf_counter() - start
print(f"loop {t1:.4f} s, NumPy {t2:.4f} s")
print(sq[:3], sq2[:3])

One real run:

loop 0.0846 s, NumPy 0.0041 s
[0, 1, 4] [0 1 4]

4. Convert Mumbai temperatures [31.5, 33.0, 29.8, 35.2] °C to °F (F = C × 9/5 + 32), then round.

Solution
temps_c = np.array([31.5, 33.0, 29.8, 35.2])
temps_f = temps_c * 9 / 5 + 32
print(temps_f)
print(np.round(temps_f))
[88.7  91.4  85.64 95.36]
[89. 91. 86. 95.]

5. Prices [120, 85, 240, 60] with discounts [0.10, 0.05, 0.20, 0.0]. Find the sale prices, then make sure nothing sells below Rs 80.

Solution
mrp = np.array([120, 85, 240, 60])
discount = np.array([0.10, 0.05, 0.20, 0.0])
print(mrp * (1 - discount))
print(np.maximum(mrp * (1 - discount) , 80))
[108.    80.75 192.    60.  ]
[108.    80.75 192.    80.  ]

6. Make a 3 × 4 array of integer zeros and print its shape, ndim and size.

Solution
grid = np.zeros((3, 4), dtype=int)
print(grid, grid.shape, grid.ndim, grid.size)
[[0 0 0 0]
 [0 0 0 0]
 [0 0 0 0]] (3, 4) 2 12

Quick quiz

import numpy as np
a = np.array([2, 4, 6])
print(a * 2, [2, 4, 6] * 2)

A) [4 8 12] [4, 8, 12] B) [ 4 8 12] [2, 4, 6, 2, 4, 6] C) [2 4 6 2 4 6] [4, 8, 12] D) TypeError

Answer

B. The real output is [ 4 8 12] [2, 4, 6, 2, 4, 6]. The array multiplies element-wise; the list repeats itself. (NumPy pads numbers with spaces so columns line up.)

Interview questions

  1. Why is NumPy faster than a Python list for numeric work? Contiguous memory, one fixed dtype (no per-item type checks), and loops that run in compiled C instead of the Python interpreter.
  2. What is vectorisation? Expressing an operation on a whole array at once, letting NumPy's compiled loop do the work instead of a Python for loop.
  3. What does [1, 2] * 3 give vs np.array([1, 2]) * 3? [1, 2, 1, 2, 1, 2] (repetition) vs [3 6] (element-wise).
  4. What happens if you create an array from [1, 2.5, "x"]? Everything becomes a string dtype (<U32 or similar); maths on it fails.
  5. Difference between np.arange and np.linspace? arange takes a step and excludes the stop; linspace takes a number of points and includes both ends.
  6. What are shape, ndim and size? Elements per dimension (tuple), number of dimensions, total element count.
  7. What is a ufunc? A universal function: a NumPy function that operates element-wise on arrays in C (np.sqrt, np.add, np.maximum).
  8. When does operands could not be broadcast together appear? When element-wise maths is done on arrays whose shapes are incompatible, like (3,) and (4,).

Glossary

Term Meaning Example
Library A package of ready-made code you import import numpy as np
NumPy Numerical Python: the library for fast maths on arrays np.array([1, 2])
as (import alias) A short nickname for an imported module np for numpy
ndarray NumPy's N-dimensional array: a grid of same-type values <class 'numpy.ndarray'>
np.array Creates an ndarray from a list (or list of lists) np.array([12, 15, 25])
Element One value inside an array 15 in [12 15 25]
Sequence An ordered collection like a list or string [12, 15, 25]
Concatenate Join end to end [1] + [2] → [1, 2]
Vectorisation One operation on a whole array, no loop written by you prices * 1.1
Element-wise operation Applied to each element separately, pairing by position [1 2] * [3 4] → [3 8]
Floating-point approximation How computers store most decimals: very close, not exact 13.200000000000001
time.perf_counter() A precise stopwatch reading, in seconds end - start
Contiguous memory Values stored side by side in one unbroken block an array's numbers
Pointer A stored memory address of an object what a list keeps per item
C / compiled language A language translated to machine code ahead of time; fast NumPy's loops
Bit / byte A bit is one 0 or 1; a byte is 8 bits itemsize = 8 bytes
itemsize / nbytes Bytes per element / bytes for the whole array 8, 56
Attribute A value on an object, read with a dot, no brackets a.dtype
Method A function on an object, called with a dot and brackets a.sum()
dtype The one data type all elements share int64
int64 Whole numbers in 64 bits 320
float64 Decimal numbers in 64 bits 2.5
Upcasting Converting all values to a type that fits every one [1, 2.5] → float64
Unicode / <U21 The standard numbering every character; <U21 = text up to 21 characters 'N/A'
astype Makes a converted copy with another dtype a.astype(float)
Dimension / axis A direction you can move along in an array rows, columns
2-D array An array with rows and columns (2, 7)
shape Elements per dimension, as a tuple (2, 7)
ndim Number of dimensions 2
size Total number of elements 14
Inhomogeneous Not all the same (rows of different length) [[1, 2], [3]]
np.arange Like range() but an array; start, stop (excluded), step np.arange(0, 50, 10)
np.linspace N evenly spaced points, both ends included np.linspace(0, 1, 5)
np.zeros / np.ones Arrays filled with 0 / 1 np.zeros((2, 3))
np.full Array filled with a chosen value np.full(7, 300)
Revenue Money taken in: quantity × price cups * price
Boolean array An array of True/False from a comparison cups > 300
Operand A value an operator works on cups in cups * price
Broadcasting NumPy's rule for stretching a smaller array across a bigger one (episode 10) prices * 2
Universal function (ufunc) A NumPy function applied element-wise in C np.sqrt, np.add
Floor (minimum) The lowest allowed value np.maximum(a, 300)
Absolute value A number without its minus sign np.abs(-20) → 20
Why arrays beat lists One dtype, contiguous memory, loops in C many times faster than a list loop

Summary

  • Lists can't do maths: * repeats and + concatenates; * 1.1 is a TypeError.
  • NumPy's ndarray holds one dtype in contiguous memory; vectorised, element-wise operations do the maths in one line with loops in C.
  • dtype: int64, float64, and one stray string turns the array into text (<U21) — check it after loading data.
  • shape, ndim, size describe an array; 2-D arrays have rows and columns of equal length.
  • np.arange (step, stop excluded), np.linspace (count, both ends), np.zeros / ones / full build arrays fast.
  • Element-wise maths needs matching shapes; ufuncs like np.sqrt, np.round, np.maximum and np.abs work on every element.
  • Next episode: NumPy 2: indexing, masks, reshaping and axes.