Optimize from evidence¶

The investigation has found repeated punctuation replacements inside a frequently called normalizer. We can now make one focused change, prove that its results agree with the baseline, and remeasure the same workload. That closed loop—not the chosen trick—is the transferable skill.

Objectives

Form an optimization hypothesis from profile evidence.
Validate an alternative implementation against the correctness oracle.
Report a measured comparison with limitations and next steps.

Instructor note

10 min teaching
15 min type-along
10 min exercise and knowledge check

State the hypothesis¶

normalize_slow() loops over every punctuation character and creates repeated intermediate strings. The alternative constructs one translation table and delegates the removal to str.translate():

import string

TRANSLATION_TABLE = str.maketrans("", "", string.punctuation)

def normalize_fast(token: str) -> str:
    return token.lower().translate(TRANSLATION_TABLE)

normalize_fast("Python;")

The hypothesis is specific: reducing Python-level calls and intermediate strings in the measured hot path should reduce elapsed time for the fixed workload.

It also has a falsifiable prediction: the candidate profile should contain fewer Python-level replacement calls while producing the same counter. If the call path changes but elapsed time does not improve, the profile can still be correct—the removed work may simply be too small relative to the rest of the application.

Type-along: verify, benchmark, profile¶

From the repository root:

$ pytest -q
$ python content/episodes/code/benchmark_wordcount.py
$ cd content/episodes/code
$ python profile_wordcount.py --implementation both

The test compares complete counters produced by both implementations. The benchmark prints the raw observations and compares the same input, repeats, and statistic. The profiling command creates wordcount-slow.prof and wordcount-fast.prof, making it possible to check whether the predicted call-path change occurred. Exact ratios are deliberately not embedded in the lesson because they depend on the machine and Python version.

Run a compact version of that loop in the notebook:

from pathlib import Path
import sys
from statistics import median

code_dir = next(
    path.resolve()
    for path in (Path("content/episodes/code"), Path("../content/episodes/code"))
    if path.is_dir()
)
if str(code_dir) not in sys.path:
    sys.path.insert(0, str(code_dir))

from benchmark_wordcount import benchmark
from wordcount import count_words, make_workload, normalize_fast, normalize_slow

text = make_workload(1_000)
baseline = count_words(text, normalize_slow)
candidate = count_words(text, normalize_fast)
assert candidate == baseline

observations = benchmark(repeats=5, number=2, workload_repeats=1_000)
for name, samples in observations.items():
    print(name, "median:", f"{median(samples):.6f} s/call", "raw:", samples)

Treat the output as local evidence. A speed ratio is useful only together with the underlying times, raw observations, and stated workload; otherwise a very small absolute change can sound more consequential than it is.

If the alternative is not faster, keep the evidence. Possible explanations include a workload too small to expose the cost, a different dominant function, or an optimization whose overhead exceeds its benefit. Do not select only favourable samples.

Decide whether to keep the change¶

Performance is only one acceptance criterion. Also consider readability, portability, dependency cost, memory use, and the risk of changing behaviour for inputs outside the teaching fixture. A fast candidate that is difficult to verify or maintain may be the wrong engineering choice. Record rejected experiments briefly; they prevent the next person from repeating the same investigation without context.

Write the performance note¶

A compact report can use this shape:

Question: Does one translation reduce normalization time?
Revision/environment: <commit>, Python <version>, <node/CPU>, <allocation>
Workload: SAMPLE_SENTENCE repeated 20,000 times
Correctness: both implementations returned identical Counter objects
Method: timeit, 5 repeats × 3 calls, median seconds per call
Observations: baseline <samples>; candidate <samples>
Interpretation: <what changed, variability, profiler support>
Limitations/next step: <representativeness, memory, larger application>

The best next optimization may be algorithmic, a library call, compiled code, parallel execution, or no change at all. Complexity has a maintenance cost. Stop when the relevant requirement is met and the evidence no longer justifies another change.

Exercise

Complete the performance-note template with your own observations. Then exchange it with a partner: can they identify the workload, correctness check, comparison boundary, and uncertainty without asking you?

Solution

A complete answer contains local measurements rather than course-wide expected numbers. It names the environment and useful work, confirms equal results, states repeats and the chosen statistic, reports observations honestly, and limits the conclusion to the measured case.

Finish with the knowledge check.

Keypoints

Optimize a measured cause, not a guess.
Verify correctness before accepting performance results.
A credible claim includes workload, environment, method, observations, and limitations.