Tested
12 Programming Beliefs, Measured
The numbers
Twelve programming performance beliefs, measured instead of argued about: eight timed live with Python's timeit, four cross-checked against the author's other measured books. Six were busted outright, three confirmed, two came down to "it depends," and one was called outdated - the widest swing was a set beating a list at membership tests by about 204,000×.
204,000×
set vs list membership speedup
8
beliefs timed live with timeit
6 of 12
beliefs busted outright
What's inside
- Eight Python performance folklore claims timed live with timeit: string concatenation, list comprehensions, set membership, memoization, exceptions, regex, linear scan vs bisect, and range's memory footprint.
- Four systems-level claims cross-cited from the author's other measured books: LSM-tree write amplification, gzip vs Zstd, HNSW vs IVF, and iptables service-rule scaling.
- Every card gets a verdict - Confirmed, Busted, Outdated, or It depends - plus the actual measured ratio, not a guess.
- A stated methodology line: timeit best-of-repeated-runs for the coding claims, reproducible harnesses for the systems claims.
- A closing note pointing each 'it depends' verdict to the full measured book behind it.
- One-page, print-ready sheet built to counter unverified performance folklore.
Everything included
- It depends: "s += x in a loop is O(n²) and catastrophic" - measured over 20,000 appends, ''.join(...) was only 1.3× faster than +=. Modern CPython optimises string += to near-linear, so the old warning is mostly outdated; use join for clarity, not because += is a disaster.
- Confirmed: "List comprehensions beat a for-loop with .append()" - measured over 20,000 items, the comprehension was 1.4× faster. Real, but modest; use comprehensions for readability first, the speed is a small bonus.
- Confirmed: "Use a set, not a list, for membership tests" - measured x in data over 1,000,000 items, the set was 204,000× faster. A list checks every element; a set hashes straight to the answer.
- Confirmed: "Memoization is a nice-to-have" - @lru_cache made recursive fib(30) about 1,000,000× faster (0.13s to 0.12µs). For overlapping subproblems it's the difference between usable and not.
- Busted: "Exceptions are slow; avoid try/except in hot code" - a try block that doesn't raise costs about 9ns, effectively free. Raising is about 9× more, but still tiny. Fear the raise, not the try.
- Busted: "Regex is fine for finding a literal substring" - 'lazy' in s was 4.3× faster than re.search('lazy', s). For a fixed string, skip the regex engine; use in or str.find.
- Busted: "For repeated lookups, just scan the list" - measured 1,000 lookups against 100,000 sorted items, sort-once-and-bisect was 53× faster than a linear scan.
- Busted: "range(n) builds a list of n numbers in memory" - range(1_000_000) is 48 bytes; the equivalent list is about 8MB, roughly 167,000× larger. range is a lazy sequence, not a list.
- Busted (from The Write Path): "LSM-trees always have worse write amplification than B-trees" - measured over 1M random inserts, the copy-on-write B-tree wrote 44× its data; the LSM-tree wrote only 2.5×.
- Outdated (from The Compression Trade-off): "gzip is a fine default for compression" - Zstd level 3 matched gzip's ratio while compressing about 5× faster and decompressing about 2× faster.
- It depends (from Nearest and Fastest): "HNSW simply beats IVF for vector search" - measured on 100,000 vectors, both reached about 0.9997 recall. HNSW answered queries faster; IVF built about 5× faster and used less memory.
- Busted (from Packets in the Mesh): "kube-proxy in iptables mode scales fine" - installing the service rules took 0.03s at 100 services and 0.63s at 10,000: linear, and paid on every change.
- How it was measured: the eight coding beliefs were timed with Python 3's timeit (best of repeated runs, via timeit.Timer.autorange); the four systems findings are cross-cited from the named books and their own reproducible harnesses. Absolute timings are machine-specific - read the ratios and the direction, not the exact numbers.
Frequently asked
Is this really free?
Yes. It's a free one-page give-away sheet by Md Nasim Sheikh (www.nasimstg.dev), not a sample chapter of something paid.
Were all 12 claims actually measured, or are some just cited?
Eight of them, the Python coding claims, were timed live in this book's own timeit harness (measure.py). The other four - LSM-tree write amplification, gzip vs Zstd, HNSW vs IVF, and iptables service-rule scaling - are cross-cited from the author's other measured books rather than re-run here; each of those has its own reproducible harness in its source book.
Will I get the same numbers if I run the script myself?
Not exactly. The sheet states outright that absolute timings depend on the machine, so it asks you to read the ratios and the direction rather than the exact figures. The harness uses Python's timeit with autorange and takes the best of 5 repeats, so re-running it should reproduce the same relative story even if the raw numbers shift.
What's the verdict system?
Each of the 12 myth cards is labeled Confirmed, Busted, Outdated, or It depends, color-coded, followed by the measured number and a one-line takeaway. Of the 12, six were busted outright, three confirmed, two came down to 'it depends,' and one (gzip as a default) was called outdated.
Do I need to know Python to use this?
You need enough Python to read snippets like ''.join(x for x in ...) or @lru_cache, since the eight coding cards show real code, but no benchmarking background is assumed. Each card states its result and verdict directly.
Is there a paid version with more detail?
The sheet itself doesn't link to a paid edition, but four of its claims point back to full paid books (The Write Path, The Compression Trade-off, Nearest and Fastest, Packets in the Mesh) that go deeper into the underlying benchmark for that one claim, each with its own reproducible harness.