Reference for learners

The reusable workflow

  1. Reproduce the problem with the smallest representative input.

  2. State the expected result and protect it with a test.

  3. Choose a benchmark boundary and repeat the fixed useful work.

  4. Profile broadly enough to locate a costly call path.

  5. Narrow the investigation to functions, lines, memory, or system activity.

  6. Form one hypothesis and make one focused change.

  7. Rerun correctness checks, then the original benchmark.

  8. Report evidence, variability, limitations, and the next decision.

Command map

Goal

Terminal or Python

IPython/Jupyter

Debug interactively

python -m pdb script.py or breakpoint()

%debug after an exception

Time an application region

time.perf_counter()

%time expression

Repeat a short benchmark

python -m timeit ... or timeit.repeat()

%timeit expression

Profile calls

python -m cProfile -o run.prof script.py

%prun -s cumulative expression

Read profile data

python -m pstats run.prof

pstats.Stats(...)

Profile selected lines

kernprof -l -v script.py

%lprun -f function expression

Visualize a call profile

snakeviz run.prof

same, subject to site policy

Profiler fields

  • call count (ncalls): how often a function ran;

  • internal time (tottime): time in the body, excluding callees;

  • cumulative time (cumtime): time in the body and all descendants;

  • hits: executions of a profiled line;

  • time per hit: measured line time divided by hits.

Profiler timings include profiler overhead. Use their ranking to decide where to investigate, then use a benchmark to compare the final variants.

Measurement checklist

  • What useful work and input are fixed?

  • How is correctness checked?

  • What begins and ends the timed region?

  • Were setup, I/O, compilation, or transfer costs included intentionally?

  • What machine, allocation, Python environment, and thread settings were used?

  • Were warm-up and repetitions appropriate for the run length?

  • Are raw observations retained, and which statistic summarizes them?

  • What uncertainty or limitation constrains the conclusion?

Diagnostic safety

Logs, tracebacks, ic() output, and profile artifacts can reveal tokens, file paths, user names, hostnames, arguments, or input-derived data. Prefer synthetic reproductions. Log metadata such as shape and range rather than complete data, inspect artifacts before sharing, and follow the site’s retention and transfer policy.

Glossary

Benchmark : A controlled measurement used to compare performance for fixed useful work.

Correctness oracle : A check that decides whether an observed result is acceptable.

Deterministic profiler : A profiler that records call events, usually with more perturbation than sampling.

Hot path : A frequently executed or costly path supported by measurement evidence.

Instrumentation overhead : Extra work introduced by observation itself.

Sampling profiler : A profiler that periodically observes execution rather than recording every event.

Further reading