wro.cpp
verification · part 05

The hottest line is often not the one worth optimizing

· english· audience: working-cpp· AI-generated, reviewed by Filip Sajdak

Every other episode in this series has ended with a link you can click. This one cannot, and the reason is worth stating plainly: a profiler measures a running process on real hardware over real time. Compiler Explorer compiles and runs short programs in a shared sandbox, which is the wrong instrument entirely. Everything below runs on your machine.

That is also why the previous episode is worth doing first. If the question is “why is this loop slow”, the compiler’s own output often answers it for free. Reach for a profiler when the question is “which of these ten thousand lines should I even look at.”

The question a normal profiler does not answer

A sampling profiler interrupts your program many times a second and records the stack. Aggregate that and you get where time is spent. It is the right first tool and it is often enough.

But “where time is spent” is not the question you actually have. You want to know where making things faster would make the program faster, and in any concurrent program those two can come apart badly. A thread that burns 40% of samples while blocked behind a lock is not a 40% opportunity. Speeding up a function on a path that is not the critical one changes nothing at all. Optimize the hottest line, measure, discover you have won nothing: everyone who has profiled a threaded program has had this afternoon.

Coz (plasma-umass/coz) attacks that gap directly with an idea called causal profiling. It cannot make a line faster to see what would happen, so it does the next best thing: it slows down everything else by a set amount, which is observationally equivalent to speeding up the line under test. Run that experiment across the program and you get a graph, per line, of predicted whole-program speedup against hypothetical local speedup. A flat line means optimizing there is worthless no matter how much faster you make it. The technique was published at SOSP 2015 and the paper is short and worth reading; its whole point is that the answers frequently disagree with what a conventional profiler suggests.

Coz is Linux-only and needs debug info, and you insert a progress point to tell it what “faster” means for your workload.

The rest of the toolbox

  • perf, with Hotspot as a GUI, is the Linux baseline: sampling, flamegraphs, off-CPU analysis, hardware counters. If you learn one profiler, learn this one.
  • Tracy is the frame profiler, built for anything with a loop that must finish on time: games, audio, robotics. You instrument scopes by hand and get a nanosecond-resolution timeline with lock contention, allocations, and GPU work on it, streamed live to a viewer while the program runs. When your problem is “the 1% frame is bad” rather than “the average is bad”, a timeline beats an aggregate.
  • samply is the low-friction modern on-ramp: install it, prefix your command with it, and it opens the profile in the Firefox Profiler UI in your browser. Cross-platform, no build changes, good enough for most “where does the time go” questions.
  • poop (“Performance Optimizer Observation Platform”) answers a different and very practical question: is version B actually faster than version A? It runs both, repeatedly, and reports cycles, branch misses, cache references, and peak RSS with confidence intervals. It is the honest way to compare two binaries, and a good companion to the microbenchmarks from the Google Benchmark episode. Linux-only.
  • Callgrind (with KCachegrind) simulates rather than samples, so it is slow but gives exact, reproducible call counts, which is occasionally what you want. gperftools, Intel VTune, and Orbit round out the list.

How to use them together

The layers of this series compose into a routine rather than a menu. Tests tell you the code is correct on the cases you wrote. Sanitizers tell you it is correct on the paths you ran. Fuzzing looks for the cases you did not write. Benchmarks tell you whether a change helped, in the small. Then a profiler tells you where to aim in the large, and the assembly tells you why the aim was right.

The one habit worth taking from this episode: when a profiler tells you a line is hot, treat that as a hypothesis rather than an instruction. Ask what would happen if that line were free. Sometimes the honest answer is “nothing”, and knowing it before you spend a week is the whole value of the tool.

Sources: Coz and the causal-profiling paper, “Coz: Finding Code that Counts with Causal Profiling” (SOSP 2015) · Tracy · samply · poop · Hotspot.